AutomaticUpgrade
|
Size: 6263
Comment: adressing scotts comments
|
Size: 11418
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from UbuntuDevel/PackageDependencyManagement = Package Dependency Management = |
|
| Line 7: | Line 3: |
| * '''Launchpad Entry''': https://launchpad.net/distros/ubuntu/+spec/dependency-removal * '''Created''': [[Date(2005-11-01T18:24:04Z)]] by MichaelVogt * '''Contributors''': MichaelVogt, DanielBurrows * '''Packages affected''': apt, aptitude |
* '''Launchpad entry:''' https://launchpad.net/distros/ubuntu/+spec/release-upgrades * '''Created:''' [[Date(2005-10-06T19:41:30Z)]] by JaneWeideman * '''Contributors:''' * '''Packages affected:''' `update-manager` |
| Line 12: | Line 8: |
| == Introduction == | == Summary == |
| Line 14: | Line 10: |
| This specification describes an enhancement to the information the system stores about packages that are installed locally. The goal is to improve the ability of systems to evolve over time to continue to reflect the current Ubuntu release. These primary improvements are envisaged: 1. The system needs to have some idea '''why''' a package was installed. Some of the higher-level package management tools, such as `Aptitude`, already do something similar to this. They track, for example, whether or not a package was installed because of a specific request by the system administrator, or to satisfy the dependencies of a package that was being installed. This allows them to offer to remove those dependency packages when the chosen package is removed later. 1. Over time, it is inevitable that the Ubuntu team may choose to change the open source products preferred for specific functionality in an Ubuntu system. For example, the Ubuntu developers might choose to migrate from `Postfix` to `Nullmailer`, or from `Esound` to `Polyp``Audio`. For new installations that presents no problem, someone installing the new release will simply get the new apps. However, for someone upgrading, it will be necessary to identify whether or not the user has modified any configuration files for the relevant app, and if not, to migrate the system to the new app. |
This is a proposal for an application capable of easing the transition from one version of Ubuntu to the next. |
| Line 27: | Line 14: |
| Over time, a Linux system accumulates a substantial amount of cruft. Traditionally, installing a package requires installing its dependencies. Later, when that package is upgraded, new dependencies might be introduced and older ones dropped, but in most systems the old package dependencies are never uninstalled. The end result is a plethora of unwanted and un-needed libraries and supporting packages which take up disk space and potentially also slow down the day to day operation of the computer. |
At the moment people who want to upgrade from one version of Ubuntu to the new release have to update their sources.list manually, and then do a dist-upgrade. For most people not experienced with Ubuntu, using the command-line is a big step (''see'' CommandLineDisintegration), especially where it concerns something as important as an upgrade. Aside from this, many people may not know that a new version of Ubuntu is available. |
| Line 35: | Line 16: |
| These improvements will make it easier to keep Ubuntu systems lean-and-mean over a series of releases, leaving only those packages which the Ubuntu team recommends, the packages explicitly selected by users, and the necessary dependencies required to support those packages. |
The Starter Guide in Ubuntu 5.10 contains instructions on how to upgrade ... to Ubuntu 5.10. Instructions on how to upgrade to Ubuntu 6.04ish could not be included because they depended on knowing the codename for that release, and the codename was not announced in time. |
| Line 40: | Line 18: |
| == Scope and Use Cases == | One of the difficulties of actually doing an upgrade is being confronted with unexpected questions about how to deal with config files during the upgrade itself. These come at a bad time, since the user can't use the machine that is being upgraded to search the web for help. So it would also help if the system could simulate a "dry run" of the upgrade or look for changed configuration packages or somehow provide upgrade information tailored to the set of packages currently installed. |
| Line 42: | Line 23: |
| The following use cases illustrate the ideas in this spec: | == Use cases == |
| Line 44: | Line 25: |
| 1. Jimmy installed Breezy, and has now updated to the next release. During the update, no matter which package management tool he uses, old dependencies will be removed and new ones installed. There should be no old libraries left on his system once the upgrade is complete. | Fred is running Breezy. He is using a fast Internet connection. When Dapper is released, his Update Manager notices that there's a new release available and that his system will have to be rebooted. Fred accepts the upgrade at the end of his work day, goes home, and his machine is running Dapper when he comes back next morning. |
| Line 46: | Line 27: |
| 1. Annabel has installed openldap, and all of its dependencies were automatically installed on her system. Now she is going to remove it from this system using a different package manager. That package manager identifies packages that were installed purely in support of openldap, and offers to remove those too. | Jane does not have an Internet connection, but she got a new Dapper CD-ROM from Shipit. She puts the disk in her drive, and the same thing happens. |
| Line 48: | Line 29: |
| 1. Jack is upgrading from Breezy to the next Ubuntu release. His system has postfix installed, because that was the default mail server in Breezy, but he has never modified its configuration and so when he does the update, postfix is removed and replaced with newmailer, the Ubuntu team's chosen replacement for postfix in the new release. | Joe has also installed Breezy by CD-ROM. He does have a low-bandwidth Internet connection. When Dapper is released, his system notices and asks him whether he wants to update. He declines, because his Internet connection is too slow. After a month, he too gets a new Dapper CD-ROM and upgrades his system just like Jane did. * s/Dapper/Dapper+1/ s/Breezy/Dapper/ ? == Scope == We are concerned with helping the user update from one release of a distribution to the next one, via either the network or a CD/DVD-ROM with .deb files. Any other update, such as from Debian to Ubuntu, from $release to $release+2, or via the live / UbuntuExpress CD (which does not have .deb files), is out of scope. |
| Line 52: | Line 43: |
| For the use case 1 we are going to mark each package `automatic` if it was installed only to satisfy a dependency. If a package is removed later a `mark-and-sweep` algorithm is run that marks any package that is part of a dependency of a given root-set (essential+manually installed). Any package that is notm marked here has no reverse dependencies and can be safely removed. | There are three steps that needs to be taken: 1) The user must be informed about the new available distro (possibly release notes as well) and run the upgrade tool 2) The upgrade tool must be able to download updated information how to perform the upgrade (e.g. additional steps like upgrading certain libs first) 3) The upgrade tools runs and installs/remove packages and does some additional steps like post-release cleanup |
| Line 54: | Line 52: |
| The use case 2 is adressed by the upgrade tool discussed at https://wiki.ubuntu.com/AutomaticUpgrade. It will identify changes in meta-packages based on the information we provide and suggest the removal of the superseded package. | === Inform the user === |
| Line 56: | Line 54: |
| All packages installed by the installer are installed via aptitudes task matcher right now. That means that the auto-flag is set for a lot of the libraries but not for the installed applications. That means that removing e.g. ubuntu-desktop will *not* result in the suggestion to automatically remove all it's dependencies. | The user is informed about new releases with the meta-release file that is downloaded from the net. It contains information like codename, version number and date of the release. The release file fetching is part of the update-manager application. When a new distro release is found it adds some button to the gui that allows a dist-upgrade. |
| Line 58: | Line 56: |
| == Implementation Plan == | === Get the upgrade information === |
| Line 60: | Line 58: |
| For the use cases Number 1 the automark feature of aptitude needs to be ported to libapt. It should be easily accessible to all the frontends (apt-get.cc, synaptic, aptitude, python-apt). | A upgrade may involves more steps than to upgrade the packages. It may be required to upgrade certain packages first (like libgtk2.0-0 for the hoary->breezy upgrade because of a bug that made gtk apps crash) or do other stuff that may involve changing files or other scary stuff. |
| Line 62: | Line 60: |
| === Data Preservation and Migration === | Because we do not know the exact steps at the time the tools is written this upgrade information must be totally flexible. It also must not be part of the regular packaging system because the user may not have run "apt-get update" for a long time and therefore apt does not know about the existance of the package. |
| Line 64: | Line 62: |
| The current flags in aptitude will be converted automatically on the first run of aptitude. | The information must not be in a package included in the distribution because we want to be able to solve new problems as they are discovered. |
| Line 66: | Line 64: |
| === Packages Affected === | A mechanism is proposed that involves downloading a gpg signed tar.gz from a URL. That tar.gz will contain a executable that is run and may contain additional stuff that may be required (even packages that needs to be installed). The executable "is" the upgrade information. It will do all the needed steps automatically that are usually written in the release notes. The update tool downloads the tarball, unpacks it, runs the script and make use of possible additonal stuff inside the tarball. Any actions that are taken need to communicate with a frontend that displays a nice and user-friendly screen. |
| Line 68: | Line 67: |
| The `apt` package needs to be improved to provide support for the marking of automatic dependencies. This feature needs to be exported so that frontends like synaptic, python-apt can use it too. | The mechanism needs to be flexible enough to support cd upgrades as well without any network sources. If the network is available, it should still check there for a new version of the upgrade tarball (embarrassing bus may still happen...) |
| Line 70: | Line 70: |
| === User Interface Requirements === | === The actual Upgrade === |
| Line 72: | Line 72: |
| A new "apt auto-remove" commands will be added that suggests to remove the automatically installed packages. Synaptic needs to be updated to use this feature too. Python-apt needs to support it too. Aptitude should make use of the feature as well (use the version in libapt instead of the version in aptitude). | Before the actual upgrade, some sanity checks needs to be performed. For instance, checking that the cache has a BrokenCount == 0 or that no unusal amount of 3rd party packages is installed. |
| Line 74: | Line 74: |
| Applications are usually save from being suggested to automatically removed because they are usually in the top of the depedency chain and will not be pulled in as a part of a dependecny. Aptitude will work as before, synaptic will present packages that can be removed automatically but it will not mark them for removal automatically. | The steps that needs to be taken in order to guarantee a upgrade are run next. That includes things like installing/upgrading packages before the actual dist-upgrade, modify sources.list (with a /etc/apt/sources.list.d/ entry) or run random scripts. Because a upgrade always involves removing packages there must be some sanity checks to ensure that nothing is removed that shouldn't be. Checking if e.g. no Kernel, X, or meta-packages are removed needs to be checked. For meta-packages a additional flag (MetaPackage) should be used in the Packages file. Additionaly it should be possible to have a whitelist in the upgrade-package that knows about save removals. We should remove packages that are obsolete and were available in the old release only (or at least display a list of those packages). That requires to save the Origin of the installed packages before the upgrade. We should having different GUIs for the upgrade tool (console,gtk). People should be able to get information about the new distro even when they don't run X. These goals (provide a server upgrade) may not make it into the first version and that will probably be superseded by smart because it contains a "abstract" gui already. == Implementation == === Step 2, getting the upgrade information === We should publish it through the launchpad archive infrastructure and use the ubuntu-archive key for it. The tarball should be called dist-upgrade.tar.gz and should be in dist/$distro release context and it should contain a executable with the same name. It is uncompressed into a temp directory. See the CustomUploadFormats spec also. If any packages are contained in the tarball we must include the source (because of the GPL). It also means that it would be arch-specific (something we dont't want to do). Daniel Silverstone needs a Python function which installs the tarball into the dists/ tree, adds any relevant symlinks, etc., The changes file needs to be in section "raw-dist-upgrade". We should also add upload validation (another Python callable which checks "is this a tarball, does it install all of its files in the right place, does it include an upgrade script, ..."). === Step 3, install the upgrade === An extension to UpdateManager written in python-gtk/python-apt (with a nice abstraction for the frontend) that is able to perform dist-upgrades. It should be able to To find cruft we have to generate a list of obsolete packages: 1) unknown origin (dpkg -i, whatever) 2) obsolete (in new release) 3) obsolete obselete (eg obsoleted in warty -> breezy upgrades) we can do this by pre-calculating what obsoletes we had in the previous upgrades and put it in the upgrade script Check for space constraints. Preservation of user package selection (e.g., via metapackages) -> daniels new problem-resolver into python-apt / libapt and make sure that the problemresolver never removes them Warnings about unsupported packages? -> if anything from universe is installed warn the user that the upgrade may not be as smooth as it should be Do something sane with old kernel(s) -> trying to keep not too many kernels around. |
| Line 77: | Line 116: |
| == Outstanding Issues == | === Code === |
| Line 79: | Line 118: |
| A implementation (that needs testing) is available in the michael.vogt@ubuntu.com--2005/apt--auto-mark--0 branch. Close cooperation with Daniel Burrows (the aptitude author/maintainer) is taking place, he branched from the initial port and added the missing bits that he needed for aptitude. python-apt code is written as well. All the code needs testing. | === Data preservation and migration === |
| Line 81: | Line 120: |
| == Outstanding issues == == BoF agenda and discussion == == See also == See also the UDU discussion https://wiki.ubuntu.com/SystemUpgradeTool == Discussion before the BOF == My proposed solution is to make a small application that will depend on the desktop metapackages (ubuntu-desktop, kubuntu-desktop, xubuntu-desktop), and will have a menu entry in System->Administration (and its KDE/XFCE equivalents).[[BR]] This application would only become part of the ubuntu-desktop package once the next release has been released, at which time it would be automatically uploaded to everybody who hasn't removed the ubuntu-desktop meta-package.[[BR]] After this had happened, there could be a notification similar to what now happens after you upgrade the kernel, only stating that there is a new version and that the user might want to look at the Automatic Upgrade application.[[BR]] The application itself would only contain several buttons:[[BR]] * [Upgrade to Breezy] - would change the sources.list, and do a dist-upgrade. * [Show me the release notes] - would open the release notes in the user's browser. Preferably a local copy that came with the application and also contains links to where the CDs can be downloaded. * [Where can I download the CD?] - would open up the page with the CDs that can be downloaded for those without the necessary connection, but able to download them elsewhere. This might be overkill as the links are already present in the release notes as well. * [Send me the CD] - would open up the [https://shipit.ubuntu.com/ ShipIt page], or a simple form that already takes the information from About Me. * [Cancel] - for those who don't want to upgrade now. * [Remind me one month from now] - a checkbox for periodic reminders. === Pros === * This solution would let users know that a new version is available, and offer them an easy way to upgrade to this version. * Users who do not want to update only have an unobtrusive entry in their Administration menu, and could in theory remove that. Although doing that through synaptic would break the ubuntu-desktop meta-package. === Cons === * This solution only works for users who still have the (x|k)ubuntu-desktop metapackage installed. On the other hand, the users who no longer have that are most likely the ones who know how and when to do an upgrade anyway. * There would be a high clutter factor for in having an entire Administration menu item for something that, on average, is going to be used less than once in six months (since some people will upgrade using a CD or not at all). |
Launchpad entry: https://launchpad.net/distros/ubuntu/+spec/release-upgrades
Created: Date(2005-10-06T19:41:30Z) by JaneWeideman
Contributors:
Packages affected: update-manager
Summary
This is a proposal for an application capable of easing the transition from one version of Ubuntu to the next.
Rationale
At the moment people who want to upgrade from one version of Ubuntu to the new release have to update their sources.list manually, and then do a dist-upgrade. For most people not experienced with Ubuntu, using the command-line is a big step (see CommandLineDisintegration), especially where it concerns something as important as an upgrade. Aside from this, many people may not know that a new version of Ubuntu is available.
The Starter Guide in Ubuntu 5.10 contains instructions on how to upgrade ... to Ubuntu 5.10. Instructions on how to upgrade to Ubuntu 6.04ish could not be included because they depended on knowing the codename for that release, and the codename was not announced in time.
One of the difficulties of actually doing an upgrade is being confronted with unexpected questions about how to deal with config files during the upgrade itself. These come at a bad time, since the user can't use the machine that is being upgraded to search the web for help. So it would also help if the system could simulate a "dry run" of the upgrade or look for changed configuration packages or somehow provide upgrade information tailored to the set of packages currently installed.
Use cases
Fred is running Breezy. He is using a fast Internet connection. When Dapper is released, his Update Manager notices that there's a new release available and that his system will have to be rebooted. Fred accepts the upgrade at the end of his work day, goes home, and his machine is running Dapper when he comes back next morning.
Jane does not have an Internet connection, but she got a new Dapper CD-ROM from Shipit. She puts the disk in her drive, and the same thing happens.
Joe has also installed Breezy by CD-ROM. He does have a low-bandwidth Internet connection. When Dapper is released, his system notices and asks him whether he wants to update. He declines, because his Internet connection is too slow. After a month, he too gets a new Dapper CD-ROM and upgrades his system just like Jane did.
- s/Dapper/Dapper+1/ s/Breezy/Dapper/ ?
Scope
We are concerned with helping the user update from one release of a distribution to the next one, via either the network or a CD/DVD-ROM with .deb files.
Any other update, such as from Debian to Ubuntu, from $release to $release+2, or via the live / UbuntuExpress CD (which does not have .deb files), is out of scope.
Design
There are three steps that needs to be taken:
- 1) The user must be informed about the new available distro
- (possibly release notes as well) and run the upgrade tool
- how to perform the upgrade (e.g. additional steps like upgrading certain libs first)
- some additional steps like post-release cleanup
Inform the user
The user is informed about new releases with the meta-release file that is downloaded from the net. It contains information like codename, version number and date of the release. The release file fetching is part of the update-manager application. When a new distro release is found it adds some button to the gui that allows a dist-upgrade.
Get the upgrade information
A upgrade may involves more steps than to upgrade the packages. It may be required to upgrade certain packages first (like libgtk2.0-0 for the hoary->breezy upgrade because of a bug that made gtk apps crash) or do other stuff that may involve changing files or other scary stuff.
Because we do not know the exact steps at the time the tools is written this upgrade information must be totally flexible. It also must not be part of the regular packaging system because the user may not have run "apt-get update" for a long time and therefore apt does not know about the existance of the package.
The information must not be in a package included in the distribution because we want to be able to solve new problems as they are discovered.
A mechanism is proposed that involves downloading a gpg signed tar.gz from a URL. That tar.gz will contain a executable that is run and may contain additional stuff that may be required (even packages that needs to be installed). The executable "is" the upgrade information. It will do all the needed steps automatically that are usually written in the release notes. The update tool downloads the tarball, unpacks it, runs the script and make use of possible additonal stuff inside the tarball. Any actions that are taken need to communicate with a frontend that displays a nice and user-friendly screen.
The mechanism needs to be flexible enough to support cd upgrades as well without any network sources. If the network is available, it should still check there for a new version of the upgrade tarball (embarrassing bus may still happen...)
The actual Upgrade
Before the actual upgrade, some sanity checks needs to be performed. For instance, checking that the cache has a BrokenCount == 0 or that no unusal amount of 3rd party packages is installed.
The steps that needs to be taken in order to guarantee a upgrade are run next. That includes things like installing/upgrading packages before the actual dist-upgrade, modify sources.list (with a /etc/apt/sources.list.d/ entry) or run random scripts.
Because a upgrade always involves removing packages there must be some sanity checks to ensure that nothing is removed that shouldn't be. Checking if e.g. no Kernel, X, or meta-packages are removed needs to be checked. For meta-packages a additional flag (MetaPackage) should be used in the Packages file. Additionaly it should be possible to have a whitelist in the upgrade-package that knows about save removals.
We should remove packages that are obsolete and were available in the old release only (or at least display a list of those packages). That requires to save the Origin of the installed packages before the upgrade.
We should having different GUIs for the upgrade tool (console,gtk). People should be able to get information about the new distro even when they don't run X. These goals (provide a server upgrade) may not make it into the first version and that will probably be superseded by smart because it contains a "abstract" gui already.
Implementation
Step 2, getting the upgrade information
We should publish it through the launchpad archive infrastructure and use the ubuntu-archive key for it. The tarball should be called dist-upgrade.tar.gz and should be in dist/$distro release context and it should contain a executable with the same name. It is uncompressed into a temp directory. See the CustomUploadFormats spec also.
If any packages are contained in the tarball we must include the source (because of the GPL). It also means that it would be arch-specific (something we dont't want to do).
Daniel Silverstone needs a Python function which installs the tarball into the dists/ tree, adds any relevant symlinks, etc., The changes file needs to be in section "raw-dist-upgrade". We should also add upload validation (another Python callable which checks "is this a tarball, does it install all of its files in the right place, does it include an upgrade script, ...").
Step 3, install the upgrade
An extension to UpdateManager written in python-gtk/python-apt (with a nice abstraction for the frontend) that is able to perform dist-upgrades. It should be able to
To find cruft we have to generate a list of obsolete packages:
- 1) unknown origin (dpkg -i, whatever) 2) obsolete (in new release)
3) obsolete obselete (eg obsoleted in warty -> breezy upgrades) we can do this by pre-calculating what obsoletes we had in the previous upgrades and put it in the upgrade script
Check for space constraints.
Preservation of user package selection (e.g., via metapackages) -> daniels new problem-resolver into python-apt / libapt and make
- sure that the problemresolver never removes them
Warnings about unsupported packages? -> if anything from universe is installed warn the user that the upgrade
- may not be as smooth as it should be
Do something sane with old kernel(s) -> trying to keep not too many kernels around.
Code
Data preservation and migration
Outstanding issues
BoF agenda and discussion
See also
See also the UDU discussion https://wiki.ubuntu.com/SystemUpgradeTool
Discussion before the BOF
My proposed solution is to make a small application that will depend on the desktop metapackages (ubuntu-desktop, kubuntu-desktop, xubuntu-desktop), and will have a menu entry in System->Administration (and its KDE/XFCE equivalents).BR This application would only become part of the ubuntu-desktop package once the next release has been released, at which time it would be automatically uploaded to everybody who hasn't removed the ubuntu-desktop meta-package.BR After this had happened, there could be a notification similar to what now happens after you upgrade the kernel, only stating that there is a new version and that the user might want to look at the Automatic Upgrade application.BR The application itself would only contain several buttons:BR
- [Upgrade to Breezy] - would change the sources.list, and do a dist-upgrade.
- [Show me the release notes] - would open the release notes in the user's browser. Preferably a local copy that came with the application and also contains links to where the CDs can be downloaded.
- [Where can I download the CD?] - would open up the page with the CDs that can be downloaded for those without the necessary connection, but able to download them elsewhere. This might be overkill as the links are already present in the release notes as well.
[Send me the CD] - would open up the [https://shipit.ubuntu.com/ ShipIt page], or a simple form that already takes the information from About Me.
- [Cancel] - for those who don't want to upgrade now.
- [Remind me one month from now] - a checkbox for periodic reminders.
Pros
- This solution would let users know that a new version is available, and offer them an easy way to upgrade to this version.
- Users who do not want to update only have an unobtrusive entry in their Administration menu, and could in theory remove that. Although doing that through synaptic would break the ubuntu-desktop meta-package.
Cons
- This solution only works for users who still have the (x|k)ubuntu-desktop metapackage installed. On the other hand, the users who no longer have that are most likely the ones who know how and when to do an upgrade anyway.
- There would be a high clutter factor for in having an entire Administration menu item for something that, on average, is going to be used less than once in six months (since some people will upgrade using a CD or not at all).
AutomaticUpgrade (last edited 2008-08-06 16:19:16 by localhost)