ImprovedPortsMirroring

Summary

The current mirror structure is split between mirrors of archive.ubuntu.com and ports.ubuntu.com, and can be confusing for some users who find an insufficient number of mirrors for their architecture, or receive incorrect advice in support fora.

The decision to split the mirror should be reinvestigated, and considered in light of newer, more advanced mirroring technologies that have become available, to ease the means by which individual mirror administrators may select which architectures they prefer to mirror.

Release Note

Ubuntu mirrors may now choose to mirror Ubuntu for as many or as few architectures as they prefer.

Rationale

  • ports.ubuntu.com is *really* slow
  • There's no good way to advertise ports mirrors
  • There's no good way to mirror ports without lots of duplication to the regular archive (arch:all, sources, etc.)
  • architectures keep switching back and forth between archive and ports, confusingly (python-apt has fairly complete history now)
  • Users get different, sometimes conflicting instructions in different fora as to how to set sources.list

User stories

  • Alice is a mirror admin and finds that 80% of her users use amd64 or armel, and wants to reduce the disk footprint of her mirror.
  • Bob works for a silicon vendor, and wants to mirror only the architectures his firm builds locally.
  • Chris wants to update a powerpc laptop, and finds it frustrating that downloads take so very long. Checking for local mirrors allows a faster update.
  • Dora wants to install on her armel board, and wants to be able to use a local mirror, if one exists.
  • Everett would like to use Soyuz chroots for pbuilder and doesn't want to mangle sources.list. By setting an alias for ftp.internal, he can build all architectures without fuss.

Assumptions

TBD

Design

Mirroring

  • Mirrors will use lmirror to mirror selected architectures.
  • Mirrors will make available a machine-readable list of mirrored architectures
  • Mirrors will register supported architectures

python-apt

  • python-apt's mirror database will be adjusted to choose preferred mirrors on a per-architecture basis

Implementation

Code Changes

  • Need to refactor the python-apt mirror selection API: this probably involves having structured data showing which mirrors support which architectures, and then using this to determine which to set as the local mirror.
  • Need to adjust recommended mirror scripts to use lmirror and select only desired architectures

Migration

  • Round-trip discussions with each mirror maintainer to ensure comfort with data size
  • Disk-space analysis to determine how to handle in-DC archive.ubuntu.com mirroring (especially around releases)
  • Likely need to completely deploy new mirroring solution to all mirrors prior to cutover

Test/Demo Plan

There's really no good way to test the mirror side of this, except against experimental mirrors. This just involves setting up an experimental mirror, and mirroring various things.

Once there are some mirrors that have odd architecture mixes, python-apt modifications may begin to be tested.

Unresolved issues

  • Branding issues related to mirror names

BoF agenda and discussion

  • Is lmirror the way forward, and can it do this?
  • How can we represent architecture selections and keep them up-to-date?
  • What other code needs to change?
  • What operational effects does this have on mirrors internal to the Canonical DC?
  • What level of mirroring do we expect for current ports architectures?
  • What sort of migration is least disruptive?


CategorySpec

Specs/ImprovedPortsMirroring (last edited 2010-10-22 21:38:26 by softbank126012031121)