ImprovedPortsMirroring
Launchpad Entry: ubuntutheproject-foundations-n-improved-ports-mirroring
Created: 2010-10-22
Contributors: Emmet Hikory
Packages affected: ubumirror, lmirror, python-apt
Summary
The current mirror structure is split between mirrors of archive.ubuntu.com and ports.ubuntu.com, and can be confusing for some users who find an insufficient number of mirrors for their architecture, or receive incorrect advice in support fora.
The decision to split the mirror should be reinvestigated, and considered in light of newer, more advanced mirroring technologies that have become available, to ease the means by which individual mirror administrators may select which architectures they prefer to mirror.
Release Note
Ubuntu mirrors may now choose to mirror Ubuntu for as many or as few architectures as they prefer.
Rationale
- ports.ubuntu.com is *really* slow
- There's no good way to advertise ports mirrors
- There's no good way to mirror ports without lots of duplication to the regular archive (arch:all, sources, etc.)
- architectures keep switching back and forth between archive and ports, confusingly (python-apt has fairly complete history now)
- Users get different, sometimes conflicting instructions in different fora as to how to set sources.list
User stories
- Alice is a mirror admin and finds that 80% of her users use amd64 or armel, and wants to reduce the disk footprint of her mirror.
- Bob works for a silicon vendor, and wants to mirror only the architectures his firm builds locally.
- Chris wants to update a powerpc laptop, and finds it frustrating that downloads take so very long. Checking for local mirrors allows a faster update.
- Dora wants to install on her armel board, and wants to be able to use a local mirror, if one exists.
- Everett would like to use Soyuz chroots for pbuilder and doesn't want to mangle sources.list. By setting an alias for ftp.internal, he can build all architectures without fuss.
Assumptions
TBD
Design
Mirroring
- Mirrors will use lmirror to mirror selected architectures.
- Mirrors will make available a machine-readable list of mirrored architectures
- Mirrors will register supported architectures
python-apt
- python-apt's mirror database will be adjusted to choose preferred mirrors on a per-architecture basis
Implementation
Code Changes
- Need to refactor the python-apt mirror selection API: this probably involves having structured data showing which mirrors support which architectures, and then using this to determine which to set as the local mirror.
- Need to adjust recommended mirror scripts to use lmirror and select only desired architectures
Migration
- Round-trip discussions with each mirror maintainer to ensure comfort with data size
- Disk-space analysis to determine how to handle in-DC archive.ubuntu.com mirroring (especially around releases)
- Likely need to completely deploy new mirroring solution to all mirrors prior to cutover
Test/Demo Plan
There's really no good way to test the mirror side of this, except against experimental mirrors. This just involves setting up an experimental mirror, and mirroring various things.
Once there are some mirrors that have odd architecture mixes, python-apt modifications may begin to be tested.
Unresolved issues
- Branding issues related to mirror names
BoF agenda and discussion
- Is lmirror the way forward, and can it do this?
- How can we represent architecture selections and keep them up-to-date?
- What other code needs to change?
- What operational effects does this have on mirrors internal to the Canonical DC?
- What level of mirroring do we expect for current ports architectures?
- What sort of migration is least disruptive?
Specs/ImprovedPortsMirroring (last edited 2010-10-22 21:38:26 by softbank126012031121)