MigratingToNetplan
Differences between revisions 22 and 23
|
Size: 15669
Comment:
|
Size: 5044
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 6: | Line 6: |
| * '''Packages affected''': bridge-utils cloud-init juju maas netcfg nplan net-tools ifenslave ifupdown systemd vlan | * '''Packages affected''': netcfg nplan ifupdown systemd |
| Line 12: | Line 12: |
| Deprecate ifupdown and previous-generation networking tools in Ubuntu for the 17.10 release. Ambitiously target a unified network stack of netplan, networkd, network-manager, and iputils2 across all 17.10 images in preparation for the 18.04 LTS. | Deprecate ifupdown in Ubuntu for the 17.10 release. Ambitiously target a unified network stack of netplan and networkd/network-manager across all 17.10 images in preparation for the 18.04 LTS. |
| Line 14: | Line 14: |
| This plan includes the phasing out of outdated tools such as ifupdown and net-tools and the packages which are used as extensions to ifupdown and net-tools; in favor of the preferred, modern alternatives from iproute2: using the 'ip' command, for example. | |
| Line 18: | Line 17: |
| In this release, several networking tools have been modernized or replaced; the table below outlines the most common replacement for well-known commands: | === For netplan === |
| Line 20: | Line 19: |
| /!\ Include table here; e.g ifconfig == ip address / bridgectl addif eth0 br0 == bridge link set ... | Ubuntu 17.10 introduces a new default configuration method for network devices. It now uses netplan instead of ifupdown. As such, configuration is now written as YAML files to /etc/netplan. See [[Netplan|Netplan]] for more information about how to configure your network devices. |
| Line 42: | Line 41: |
| Migration to netplan and phasing out of old network tools will happen in several phases, which may overlap: | Migration to netplan: |
| Line 44: | Line 43: |
| * Phase I: Identify critical-path features for netplan, implement critical-path. * Phase II: Identify all code change targets, prioritize and fix critical pieces (packages in main, infrastructure) * Phase III: Implement migration tools for existing installs from ifupdown to an equivalent network YAML v2. * Phase IV: Identify and implement missing features for MaaS and other specialist users. * Phase V: Any cleanup, as necessary (package demotions, removals, etc.) |
* Phase I: Switch to using netplan configuration by default, stop seeding ifupdown; initial feedback. * Phase II: Implement migration tools for existing installs from ifupdown to an equivalent network YAML v2. |
| Line 50: | Line 46: |
| ==== Phase I: Critical-path netplan features ==== | ==== Phase I: Switch to using netplan; stop seeding ifupdown ==== |
| Line 52: | Line 48: |
| {{{#!wiki warning deadline ubuntu-17.06 }}} |
Early on, we switch to using netplan by default; this is a rather straight-forward stop of stopping to seed ifupdown. |
| Line 56: | Line 50: |
| To begin with, we'll need to identify those features we already know are abolutely required as a bare minimum to use netplan /instead of/ ifupdown on all systems. | Some pieces will require code changes: debian-installer (netcfg) will need to be writing the new config format. |
| Line 58: | Line 52: |
| These should be reviewed and agreed on between Foundations, Desktop and Server teams and carefully selected; it should be limited to critical-path features, not nice-to-haves coming from bias from using ifupdown. We should not aim to duplicate exactly what ifupdown did, but rather take the opportunity to improve on it. | ==== Phase II: Migration tools ==== |
| Line 60: | Line 54: |
| Netplan should already be considered as relatively mature before the deadline; as it is already shipped on systems. Possibly, provide an initial image without ifupdown for testing. Also set up transition tracker for packages that are obviously affected by the migration efforts, and start scanning the archive / default installs for other users that may not be declaring a Depends/Recommends/Suggest on affected packages. ==== Phase II: Code change targets, fixing main and infrastructure ==== {{{#!wiki warning deadline ubuntu-17.07 }}} There are multiple packages depending on one of the affected packages selected for replacement: http://paste.ubuntu.com/24500232/: || aranym || 1.0.2-2 || artful/universe || source, amd64, arm64, armhf, i386, ppc64el, s390x|| || grub-legacy-ec2 || 0.7.9-113-g513e99e0-0ubuntu1 || artful || all|| || controlaula || 1.8.0-3.1 || artful/universe || source, all|| || ltsp-controlaula || 1.8.0-3.1 || artful/universe || all|| || core-network-daemon || 4.8-3 || artful/universe || amd64, arm64, armhf, i386, ppc64el, s390x|| || freedombox-setup || 0.8ubuntu1 || artful/universe || source, all|| || ganeti-2.15 || 2.15.2-6build3 || artful/universe || all|| || grml-btnet || 0.04ubuntu3 || artful/universe || source, all|| || ifenslave || 2.7ubuntu1 || artful || source, all|| || ifenslave-2.6 || 2.7ubuntu1 || artful/universe || all|| || ifenslave-2.6 || 2.7ubuntu1 || artful/universe || all|| || ifplugd || 0.28-19.2 || artful/universe || source, amd64, arm64, armhf, i386, ppc64el, s390x|| || ifscheme || 1.7-5 || artful/universe || source, all|| || ifupdown-scripts-zg2 || 0.6-1 || artful/universe || source, all|| || [[/libvirt|libvirt-daemon-system]] || 2.5.0-3ubuntu7 || artful || amd64, arm64, armhf, i386, ppc64el, s390x|| || lxc1 || 2.0.7-0ubuntu2 || artful || amd64, arm64, armhf, i386, ppc64el, s390x|| || lxctl || 0.3.1+debian-3 || artful/universe || source, all|| || [[/netcf|netcf]] || || || || || netscript-2.4 || 5.4.11ubuntu1 || artful/universe || source, all|| || netscript-2.4-upstart || 5.4.11ubuntu1 || artful/universe || all|| || netscript-ipfilter || 5.4.11ubuntu1 || artful/universe || all|| || neutron-linuxbridge-agent || 2:10.0.0-0ubuntu5.1 || artful || all|| || nova-compute || 2:15.0.2-0ubuntu1 || artful || all|| || nova-compute-libvirt || 2:15.0.2-0ubuntu1 || artful || all|| || nova-network || 2:15.0.2-0ubuntu1 || artful || all|| || pppoeconf || 1.21ubuntu1 || artful || source, all|| || python-nemu || 0.3.1-1 || artful/universe || source, all|| || [[/qemu|qemu-system-common]] || 1:2.8+dfsg-3ubuntu2.1 || artful || amd64, arm64, armhf, i386, ppc64el, s390x || || resolvconf || 1.79ubuntu4 || artful || source, all|| || slapos-node-unofficial || 1.3.18-1 || artful/universe || all|| || ubuntu-fan || 0.12.2 || artful || source, all|| || ubuntu-minimal || 1.380 || artful || amd64, arm64, armhf, i386, ppc64el, s390x|| || ubuntu-server || 1.380 || artful || amd64, arm64, armhf, i386, ppc64el, s390x|| || ucarp || 1.5.2-2 || artful/universe || source, amd64, arm64, armhf, i386, ppc64el, s390x|| || xen-utils-4.8 || 4.8.0-1ubuntu2 || artful/universe || amd64, arm64, armhf, i386|| These, where possible, will need to be modified to use the "newer tools"; iproute2, etc. Packages for which Canonical is upstream will need to be ported (or dropped if it makes sense to do so), and packages where the upstream is elsewhere will need to be checked. Upstreams should be contacted with proposed patches to support the right commands for future-proofing. When necessary, implement autopkgtests for the affected packages to ensure previous behavior is maintained. Testing should be ongoing from that point. ==== Phase III: Implement migration tooling ==== {{{#!wiki warning deadline ubuntu-17.08 -- feature freeze }}} Old installs still using ifupdown need to be migrated to allow for upgrades from pre-17.10 to the next LTS. Validate the migration code (some is already available in netplan) to ensure the most common upgrade scenarios are supported. Make sure we can leave ifupdown behind and forcefully warn the user, or otherwise gracefully leave the system half-broken. /!\ Go/no-go decision by then? ==== Phase IV: Specialist features ==== {{{#!wiki warning deadline ubuntu-17.09 }}} Identify and implement any "specialist" features. Identification should happen in parallel to the previous phases in collaboration with the MaaS, Juju teams and others; then proceed to implementing anything missing. ==== Phase V: Polish ==== {{{#!wiki warning deadline ubuntu-17.10 }}} Proceed with demotions, cleanups, last efforts for polish. At this point the migration and new implementations should be well tested and deemed stable. |
In this phase; implement better migration tooling for netplan to be able to read ifupdown config and write the proper netplan YAML. This is also where further design elements from the original netplan design will be implemented: handling addition/removal of devices "live", by capturing existing configuration or cloning of an existing device. |
| Line 146: | Line 60: |
| /!\ This section only includes GUI changes. | Not applicable. |
| Line 148: | Line 62: |
| * Drop gnome-nettools | |
| Line 152: | Line 65: |
| ==== Dropped packages ==== | |
| Line 154: | Line 66: |
| ifenslave, vlan, bridge-utils may have reverse-depends in main; these need to be reviewed and fixed/demoted/nuked. | ==== debian-installer / netcfg ==== |
| Line 156: | Line 68: |
| ==== netplan ==== Some features are still missing from netplan; such as proper support for setting the MAC on devices (blocked on some systemd bugs), or setting IPv6 MTU. The relevant missing pieces have bugs open in Launchpad. ==== systemd ==== Systemd may have bugs that would block proper use of the new tooling, especially when it comes to netplan generating systemd-networkd configuration -- this configuration is usually based on what the documentation expresses as being available, but actual implementation might fail (cf. MAC address issues in progress). ==== MaaS ==== MaaS needs to generate network YAML v2. ==== cloud-init ==== Generating/parsing network YAML v2 to apply configuration. ==== netcfg ==== netcfg needs to generate network YAML v2 instead of ifupdown configuration. It will need to copy that configuration to the installed system. ==== ubuntu-fan ==== Replace uses of bridge-utils / other "old tools" with the new. |
Netcfg needs to write network V2 YAML as well as e/n/i configuration. |
| Line 181: | Line 71: |
| === Infrastructure changes === ==== Autopkgtests ==== Many packages may be using net-tools, bridge-utils or vlan to set up their environment for autopkgtest. Review this usage and apply fixes as appropriate. ==== Image build scripts ==== Any use of the "old commands" should be ported to the "new commands". ==== Other infrastructure ==== Review the use of old tools in Ubuntu infrastructure. |
|
| Line 198: | Line 75: |
| /!\ This impacts tools commonly used in user scripts, they would be broken unless we provide sufficient wrappers/redirections. | There is a '''netplan ifupdown-migrate''' command that exists, it should be extended to understand and migrate more complex configurations. |
| Line 200: | Line 77: |
| * The changes should be well documented, and a clear migration path identified in Release Notes for users. * Replace some of the most commonly used tools with stubs or wrappers to point to the new way. * Document changes in slideshow? MOTD? * Implement a conversion tools in netplan for existing ifupdown configurations. |
|
| Line 207: | Line 81: |
| * Add autopkgtests as an ongoing step while porting things to not use ifupdown/legacy tools. * Testing should be ongoing as ifupdown users are being ported to make sure there are no regressions. |
* Testing should be ongoing to make sure there are no regressions. Expect feedback at any step of the migration. |
| Line 222: | Line 95: |
| Proposed workitems for blueprint: {{{ Work items for ubuntu-17.06: [cyphermox] blog about netplan and its awesomeness: INPROGRESS [cyphermox] netplan cheerleading: TODO [cyphermox] document pre-planned out of scope work: TODO [cyphermox] document/discuss/announce plan on ubuntu-devel@: TODO [cyphermox] review systemd issues with applying MAC addresses: TODO [cyphermox] merge MAC address code: TODO [cyphermox] set up tracker for netplan migration: TODO [cyphermox] scan on default installs for undocumented uses of ifupdown: TODO review and fix netcf usage of ifupdown / add support for netplan: TODO [rharper] IPv6 MTU code: TODO [rharper] cloud-init pass-through config for netplan v2 yaml: TODO [cyphermox] speak to mpontillo for netplan crit-path features for MaaS: TODO [cyphermox] speak to dpb for netplan crit-path features for server: TODO [cyphermox] speak to tbaumann / balloons / sinzui for netplan crit-path features for Juju: TODO [cyphermox] speak to willcooke for netplan crit-path features for desktop: TODO provide an easy way (preseed?) to bring up an ifupdown-less system: TODO schedule feature work in milestones: TODO [cyphermox] implement Device Add?: TODO [cyphermox] implement Device Removal?: TODO [cyphermox] implement "show current config": TODO [cyphermox] netplan integration when libvirt/lxd are installed: TODO [cyphermox] teach netcfg to write netplan config on install: TODO [cyphermox] teach ubiquity to write netplan config on install: TODO Work items for ubuntu-17.07: review and fix grub-legacy-ec2 usage of old tools: TODO review and fix ifenslave usage of old tools: TODO review and fix libvirt-daemon-system usage of old tools: TODO review and fix lxc1 usage of old tools: TODO review and fix neutron-linuxbridge-agent usage of old tools: TODO review and fix nova-compute usage of old tools: TODO review and fix nova-compute-libvirt usage of old tools: TODO review and fix nova-network usage of old tools: TODO review and fix pppoeconf usage of old tools: TODO review and fix qemu usage of old tools: TODO review and fix resolvconf usage of old tools: TODO review and fix ubuntu-fan usage of old tools: TODO autopkgtests for ifupdown-less install: TODO review and fix infrastructure for using new tools: TODO implement MOTD/command-not-found documentation for new commands: TODO implement wrappers for critical-path tools / warnings: TODO Work items for ubuntu-17.08: update seeds to not install ifupdown: TODO review and postpone or crack down on remaining feature work: TODO test migration code existing in netplan: TODO fix/update migration code: TODO proceed with ifupdown migration by default on install of netplan: TODO get attention from the proper people for go/no-go decision: TODO announce changes again / reminder on ubuntu-devel@: TODO Work items for ubuntu-17.09: check with MaaS team for extra features still missing: TODO check with Juju team for extra features still missing: TODO check with other teams for extra features still missing: TODO review postponed features and re-target for 18.04: TODO bug fixes for release-critical bugs in netplan/affected packages: TODO Work items for ubuntu-17.10: release note the changes: TODO cleanup remaining affected packages: TODO last-minute testing and bugfix: TODO }}} |
Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: foundations-aa-migrating-to-netplan
Packages affected: netcfg nplan ifupdown systemd
Summary
Migrate default installs from ifupdown to netplan as a preferred method of describing network configuration for users, and leaving the implementation details to netplan itself. Make sure people can easily describe complex network configurations without worrying about the syntax required by the software that will implement the configuration.
Deprecate ifupdown in Ubuntu for the 17.10 release. Ambitiously target a unified network stack of netplan and networkd/network-manager across all 17.10 images in preparation for the 18.04 LTS.
Release Note
For netplan
Ubuntu 17.10 introduces a new default configuration method for network devices. It now uses netplan instead of ifupdown. As such, configuration is now written as YAML files to /etc/netplan. See Netplan for more information about how to configure your network devices.
Rationale
Increased demand on being able to represent complex network configurations simply as well as some current limitations of ifupdown have lead to the implementation of netplan, which provides a simple configuration syntax and hides the details of syntax of the tools that will support and implement the configuration while still providing feedback on the feasibility of the requested configuration. With new issues such as potentially long network interface names which are not well supported by ifupdown, layered devices and telco requirements; it is time to overhaul the default network configuration tool on Ubuntu and netplan appears to be the perfect solution.
We've also been carrying old tools that have been deprecated upstream for a while. Some of them are unmaintained or not well maintained at all, and many alternatives already do exist. Furthermore, given increasing demand for complex networking scenarios (large cloud uses often require complex layering of different features, such as bridges over bonds over VLANs, etc.), it has shown to be important to improve in the ease of configuring the network.
Use Cases
ACME wants to set up a new hyper-speed cloud for their clients, and need to configure VLANs on multiple bonded devices, bridged to the various networks. They can use netplan to easily define the network, and let netplan generate and apply the configuration to systemd-networkd or NetworkManager without having to worry about different syntaxes.
- John has been using brctl for a while to add interfaces for new virtual machines to his VM network; but wonders if there is a better way to do it.
- Mulan needs to list interfaces and their IPv4 and IPv6 addresses; she uses 'ip link' and 'ip addr' or 'ip -6 addr' to do so.
Implementation
This section should describe a plan of action to implement the changes discussed. The general summary is below, followed by some notes on specific changes that will be required. The summary includes a rough outline of the implementation plan.
Summary
Migration to netplan:
- Phase I: Switch to using netplan configuration by default, stop seeding ifupdown; initial feedback.
- Phase II: Implement migration tools for existing installs from ifupdown to an equivalent network YAML v2.
Phase I: Switch to using netplan; stop seeding ifupdown
Early on, we switch to using netplan by default; this is a rather straight-forward stop of stopping to seed ifupdown.
Some pieces will require code changes: debian-installer (netcfg) will need to be writing the new config format.
Phase II: Migration tools
In this phase; implement better migration tooling for netplan to be able to read ifupdown config and write the proper netplan YAML. This is also where further design elements from the original netplan design will be implemented: handling addition/removal of devices "live", by capturing existing configuration or cloning of an existing device.
UI Changes
Not applicable.
Code Changes
debian-installer / netcfg
Netcfg needs to write network V2 YAML as well as e/n/i configuration.
Migration
There is a netplan ifupdown-migrate command that exists, it should be extended to understand and migrate more complex configurations.
Test/Demo Plan
- Testing should be ongoing to make sure there are no regressions. Expect feedback at any step of the migration.
- Spin up instances of all supported clouds to make sure they work in a world without ifupdown, using netplan instead.
Outstanding Issues
Systemd bugs (to be identified precisely).
BoF agenda and discussion
BoF Notes
MigratingToNetplan (last edited 2017-06-19 14:05:09 by raharper)