Ubuntu Wiki

NOTE: This page is part of the Ubuntu Specification process. Please check the status and details in Launchpad before editing. If the spec is Approved then you should contact the Assignee, or another knowledgeable person, before making changes.

• Launchpad entry: [https://features.launchpad.net/distros/ubuntu/+spec/metalink-iso-downloads metalink-iso-downloads]

• Created: 2007-02-26 by [https://launchpad.net/~albryan Ant]

• Contributors: [https://launchpad.net/~albryan Ant]

• Packages affected: None

Summary

Using Metalink for Ubuntu CD ISO downloads could increase availability, efficiency, and eliminate errors during download and repair corrupted files.

Rationale

Metalink can list FTP/HTTP/P2P sources for files and provide error free downloads. When dealing with large downloads, normally if there is an error and you are using FTP or HTTP then the download needs to be restarted from the beginning (doubling the bandwidth, a 600 MB transfer is now takes 1.2 GB). Most users do not know how to repair a download.

A simple .metalink file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/">
  
  <files>  
   <file name="ubuntu-6.10-desktop-i386.iso">
    <os>Linux-x86</os>
    <size>732293120</size>
    <verification>
     <hash type="md5">b950a4d7cf3151e5f213843e2ad77fe3</hash>
    </verification>
    <resources>     
      <url type="http"
           location="ro"
           preference="90">
       http://ftp.iasi.roedu.net/mirrors/ubuntulinux.org/releases/.pool/ubuntu-6.10-desktop-i386.iso
      </url>
      <url type="http"
           location="jp"
           preference="100">
       http://ftp.yz.yamagata-u.ac.jp/pub/linux/ubuntu/releases/.pool/ubuntu-6.10-desktop-i386.iso
      </url>
      <url type="http"
           location="us"
           preference="90">
       http://ftp.osuosl.org/pub/ubuntu/.pool/ubuntu-6.10-desktop-i386.iso
      </url>      
    </resources>
   </file>
  </files> 
</metalink>

Metalink has the following advantages:

• List multiple FTP/HTTP/P2P sources for a download. A .metalink for an ISO will list all mirrors that it is available from. (This allows for faster downloading with segmented multi-source downloads, if you want).
• Some ISPs and organizations don't allow P2P, so it's good to have the FTP/HTTP sources to fall back on.
• If one link or server fails, download automatically continues using another.
• Automatic full file checksum verification at end of transfer.
• Automatic error correction for downloads in progress and repairing of corrupted downloads (using optional chunk checksums/repair info/segment hashes).
• Ability to give certain mirrors or P2P sources priority.
• List location information (by country) for mirrors, so downloads can be filtered by location.
• Simplifies download page layout. (The current download page has 14 screens of mirrors. This can be reduced to 3 .metalinks).
• No server side changes or tracker necessary.
• GPLed Clients for Mac, Unix, and Windows, along with non-Free clients. (aria2 included in feisty).

More information is available here:

• http://www.metalinker.org/

• http://en.wikipedia.org/wiki/Metalink

Use cases

1. John is a non-technical Windows/Mac/Linux user. He wants to download an Ubuntu CD image from Ubuntu.com, all he needs to choose is his architecture and CD type (Desktop/Alternate/Server) and use the small Metalink client linked to from that page.

2. Harriet is using Firefox with a Metalink client extension, this downloads an Ubuntu CD image for her from the given metalink files.

3. Eric has an incredibly fast Internet connection. His computer uses multiple sources from the Ubuntu Metalink file to maximise his bandwidth usage.

4. Mary has an unreliable wireless connection. Downloading an Ubuntu CD image is a major time investment, and she does not want to keep restarting it on error. Metalink checks the image for her as it is downloaded and only redownloads what is needed, leaving Mary with an error-free Ubuntu CD image.

Scope

Reducing bandwidth costs in distributing Ubuntu online by using P2P and current mirror network more efficiently. Reduce support costs of dealing with corrupted and error downloads. Simplify download pages. Simplify download process for happier users.

Design

Implementation

• Add script to generate metalink files to the ISO building process. Some such scripts are available in [http://metalinks.sourceforge.net/ metalink tools]:

  • [http://metamirrors.nl/node/60 mtealink]

  • [http://download.packages.ro/perl-foundry/ RoPkg::Metalink] (Perl).

  • [http://wiki.osuosl.org/display/Bouncer/Home Bouncer].

• Host the generated files on the official website: http://www.ubuntu.com/metalinks might be a good place

As an example see [http://download.packages.ro/metalink/testing/ Metalink @ Package Resources] (which uses [http://download.packages.ro/perl-foundry/ RoPkg::Metalink]) and now has .metalinks with chunk checksums/repair information for Ubuntu, Edubuntu, and Kubuntu ISOs (in testing). OpenOffice.org, openSUSE, and others use the .metalinks generated there.

Code

N/A

Data preservation and migration

N/A

Unresolved issues

aria2 0.9.0 is in the feisty repos. aria2 0.10.0 or higher would be best suited for this, as earlier versions do not support chunk checksums.

BoF agenda and discussion

CategorySpec