Ubuntu Wiki

• Launchpad Entry: https://launchpad.net/distros/ubuntu/+spec/offline-updater

• Created: 2006-05-01 by BaishampayanGhose

• Contributors: BaishampayanGhose

• Packages affected: A new package to be created (offline-updater)

• APTonCD implements most of this spec

• SynapticPackageDownloadScript does not fully implements this spec.

  • The synaptic script eases downloading packages on online machines and installing them on offline machines later. But no package lists will be updated and thus no security updates propagate to the offline machines this way. Offline updates to the apt system would be necessary in order to get updates. To do this apt can be configured at run-time to use package lists and cache on a portable medium. This way the apt package lists and cache on the medium can be updated on online machines and used to install packages on the offline machines. Apt-medium is a simple shell script tool that does this and a little multi-machine management too. (orphaned preliminary development status)

• http://wiki.debian.org/AptMedium

Summary

We need a way to update Ubuntu installations which are not connected to the Internet. In many developing countries most of the people don't have access to the Internet. Though the Ubuntu CD contains many useful applications, they are very basic in nature and don't contain many useful applications. Also the security updates which are made available by Ubuntu aren't useful to people without Internet. This tool will enable a user to create Service Packs by downloading updates and required applications with dependencies automatically. Those Service Packs then can be installed with ease on other machines using this tool.

Rationale

The basic difference between Ubuntu and other distributions is that Ubuntu being a single CD distribution, can only put in a limited number of applications in the CD. While it's very easy to install more packages via Synaptic, people who don't have access to Internet find it very difficult to install more packages and updates. Also multiple installations which are not connected to each other have to download and install the same packages multiple times to install them. This not only is sub-optimal, it also wastes a lot of bandwidth.

Use cases

• Joe has Ubuntu installed in both University and Home. He wants to install the LAMP stack in his home but he doesn't have Internet connection there. He uses the tool to download and create a Update Pack and takes it home in his USB Thumdrive. He then installs the LAMP stack on his Home machine using the tool.

• Jane has multiple PCs in her office which use Ubuntu but are not connected to each other. She wants to install the security updates to all her PCs but doesn't want to download the updates on all the PCs. She uses the tool to create a Service Pack and burns the pack to a CD. She then installs the Service Pack to all her PCs from the CD. She also uses the same CD for updating her PC at Home.

• John's mother uses a standalone Ubuntu PC. Each time security updates are released, he creates Service Packs using the tool and mails them on a CD to her.

Scope

• Installation and upgrading of packages

Related

• https://wiki.ubuntu.com/APTonCD

• http://wiki.debian.org/AptMedium - for offline updates with any apt-tool.

• http://wiki.debian.org/ListInstalledPackages

Design

• The tool should calculate the dependencies of the package to be downloaded and download them

• It will create an archive from the downloaded packages by using apt-ftparchive and create an archive from it

• It will also add some Metadata to the archive if needed

• The archive will be used as the Service Pack when installing

• When installing, the tool will first decompress the archive and then will install all the packages in the directory
• Optionally, it should be able to roll-back to a previous state in case some error occurs
• Some sort of heuristics should be added so that the tool doesn't download all the dependencies. The specifics are yet to be decided.
• As for the above heuristics, just excluding the packages included in the first CD seems feasible enough for the time being

• Integration with Synaptic may be considered once this tool is useful and stable

Preliminary mockups of the GUI

Web version

Optionally, the web version is specially usefull when the user only has the Internet connection in a non-Ubuntu computer.

• On any connected computer, the user would upload the package list to the web (packages installed and packages to install - this includes broken packages -). Package details would include : packagename, release (stable, testing, unstable, other), version number, status (installed, desired or broken)
• The web server should calculate the dependencies of the package to be downloaded and download them
• The user would install the packages in the Ubuntu computer.

This avoids the use of Cygwin and other problems.

Implementation

The tool will be written using Python,GTk+ & python-apt. An Ubuntu package will also be made.

• BaishampayanGhose will do the programming

• BaishampayanGhose will do the packaging

The discussion about the design of the tool is being summarised in OfflineUpdateSpec/DesignDiscussion

Comments

mvo (2006-05-03): I love the idea and the spec. Some minor things: I think we should use "apt-ftparchive" these days instead of "dpkg-scanpackages". But they are similar enough. I'm not sure that the archives needs to be compressed again (because debs don't compress very well). If we don't compress them the user does not have to unpack them on the local machine but can install directly from the cd and can use apt-cdrom add/synaptic cdrom support to get the packages from it. We may consider putting some additional meta-data on the cd to have some sort of "recommended install" (e.g. a add-on-cd with gnustep could recommend a gnustep meta-package or a add-on with some lean office apps could recommend gnumeric, abiword, scribus). Those recommended installs can then be displayed by e.g. update-notifier that detects when a cd is inserted (that is what it can do already).

Unfortunately roll-back is not easy (but would be very cool to have). The problem with rollback is that package downgrade don't work in the general case because postinst scripts can to stuff that works only one way (e.g. upgrading a datafile to a newer format, remove old symlinks etc).

For the heuristics, I think we might try to assume that at least "ubuntu-minimal" is installed so that we dont need packages that are dependencies of this package. We need to be careful about the versions because a user may need a newer version from e.g. dapper-updates for a particular tool. We may even add a "[ ] has ubuntu-desktop" installed to minize the download even further.

BaishampayanGhose (2006-05-03): I have added the above suggestions from mvo to the spec above.

LaszloPandy (2006-05-05): First off, I absolutely love this idea. Here are some questions/problems I came across while thinking about how this could be implemented:

• Many Service Packs would most likely only assume that the ubuntu-desktop or ubuntu-minimal sets of packages are installed. But for users creating their own Service Packs, this would make it include more dependencies then necessary, and make the size much bigger. Would there be a way to export a list of installed packages (including versions) from the target machine which would be taken to the download machine to intelligently reduce the bundle size?
• If the target machine has also installed many other applications in addition to the base Ubuntu system, and the user would like to update those as well, what would be the best way to transfer the required package list to the computer with internet access?
• Should the user be allowed to do selective upgrading (i.e. only upgrading some applications but not all of them)? If so, this would require a more complex user interface, as well a list like in the first point.

BaishampayanGhose (2006-05-05): Laszlo, thanks for your comments, my replies are here --

• I agree with that. Probably we will create some diagnostic tool which will basically do a dpkg -l and generate a report on the client system, and on the basis of that the deps will be calculated.

• Again, I guess the diagnostic tool might help in this regard.
• Possibly. That'd include the use of APT-Pinning. The UI will certainly be more complicated, IMHO we would be better off integrating this feature to Synaptic itself.

Bill Cairns (30-05-2006): I think that this is a great idea and will greatly improve the useability of Ubuntu to those of us who have no, or only slow, network connections. (I have machines with both!) I know that this is absolute heresy - but this tool would be extremely useful if it was also available under Windows. Many of us only have broadband access on Windows platforms. It would be great to be able to create an update pack under Windows and take it to the Ubuntu machine. If the tool is to be programmed in Python this may not be too difficult.

Rene Rattur (01-06-2006): I have been wanting for this feature for a long time. You should definetly add support for Windows platform. How about the users run some tool on their ubuntu boxes, which generates metadata file saying that the user has the default ubuntu installation + some extra packages. Then the user could take this file to some other machine having a internet connection and feed this file to your application.

Jakob Petsovits (2006/06/08): It would also be a good idea to seperate the gui/gtk+ part and the worker part, making a library for the latter one which is then used by the gui. Maybe you had this in mind anyway. This approach makes it easier to build a similar KDE tool (or, if you want, a native Windows one) on the same code base and can only benefit the viability of this project.

Lukas Sabota (6/21/2006): I concur with Jakob's above statement. Also, a command-line tool would be beneficial to many.

CypherBios (8/26/2006): See this project, has been started, and maybe we can work together - APTonCD

Bisho (9/05/2006): It would be really nice to generate a "machine spec" with the packages & versions installed and use it from the update pack generator. Also if you could maintain the history would be really nice. Something like this:

• Home computer profile [ Import updated spec ] <-- this will clear the list and start with a fresh list of pkgs from the machine.

  • Initial profile spec

    Update pack 1 (06/06/2006) [ ] <-- Mark this to consider the update as "applied"

    Update pack 2 (12/06/2006) [ ] <-- Mark this to consider the update as "applied"

  [ See list of packages and versions of this machine ] <-- button, will list packages consdering the updates selected.

  [ Generate new Update pack ] <-- Initiate a new update pack with and upgrade of all pkts and being able to add new pkgs.

Warbo: Maybe a meta-package could be added to each service pack's generated repository, which depends on all of the packages included in that service pack, so when someone makes a service pack then it creates a meta-package called "security-updates-september06" or "graphics-applications-1" or whatever they have called that service pack (maybe even the much coveted "all-codecs"). That way it would at least help rolling back in terms of newly installed packages, because while it would be hard to implement package downgrades when "security-updates-september06" is removed, at least removing "graphics-applications-1" would remove whatever extra packages were put into that particular service pack, since aptitude-like dependency removal is scheduled for whatever package manager Ubuntu is going to use for Edgy and beyond. This would also address LaszloPandy's idea of selective upgrading, since the service pack's local repository can be used by APT normally, and if someone doesn't want the whole lot then they just don't need to install the overall meta-package. If APT is set up to use whatever medium the service pack comes on (mvo says above that CDs are autodetected by update manager, but I assume this is only when that drive is already in sources.list?) then the meta-package can be seperated from the rest of the packages, making the service pack's hierarchy contain simply "service-pack-name.deb" and a folder containing the repository (and by using a ".hidden" file Nautilus, and I think Konqueror, will not even display that). Then not only is the installation of a service pack basically fool-proof (because there is only one file visible, which just needs to be double clicked) but Gdebi can do all of the work to install it!

Additionally, with the idea of installing many service packs for periodic updates over a length of time then the tool can maintain a snapshot of the installed package lists at the time each service pack is created and compare them afterwards (I think Bisho is talking about this above, but it isn't very clear to me, sorry ), so the service pack including "security-updates-september06" can be made with the assumption that a previous "security-updates-may05", for example, is already installed. This would be a pretty simple implementation to keep service pack sizes down, since it uses dpkg's current dependency system to make "security-updates-may05" a dependency of "security-updates-september06"

Here is a gui mockup showing what I mean about depending on other service packs.

Bisho: Yeah, I mean that. It would be nice to maintaing several "profiles" for the machines you generate updates for. First time you add a new computer to generate updates for you will be able to: * import a list of installed pkgs (generated first on the destine computer) * Set it as the standar install (perhaps selecting the ubuntu CD version used) * Generate updates for all the posible pkgs installed from CD. Afterwards you will have the list of generated updates, and you could mark them as already applied or not. I have added a Mockup. See above.

Pedrom: I would add something similar to apt-zip, to create in the unconnected computer (we can also call it the target computer) a list of packages to download. In the connected computer, the user would open the file and it would automatically to download the needed packages for the unconnected computer.

SamTygier: Some packages require downloading of files to work, eg flashplayer-nonfree, flashplayer-nonfree and b43-fwcutter. would there be a way to handle these? maybe it is more something that should be handled by the package, if they cannot download the files they need, they should ask the user to download the file, and show a file chooser.

CategorySpec