This spec describes the plan for dealing with kernel updates on EC2, both in terms of how users will be notified of available upgrades and how they will apply them.

Release Note

Ubuntu on EC2 now alerts users about available kernel security updates.


EC2 deals quite differently with kernels compared to other platforms. Upgrades are difficult, because the kernel is external from the machine image. Kernel modules are difficult, because they need to match the booted kernel, but reside in the machine image. These are old challenges that need to be resolved.

User stories

  1. George is running the final Ubuntu Karmic release AMI on EC2. On the Intarwebs, he learns about a critical vulnerability in the kernels he's using, so he wants to upgrade. He logs into his instance, at which point he's alerted that there's a new AMI available with a fix. This AMI is based on the /exact/ same filesystem image, but with a new default AKI+ARI setting.
  2. Hamish is running a rebundled AMI, based off of the Ubuntu Karmic release AMI on EC2. He learns about a critical vulnerability in the corresponding kernel and wants to upgrade. The USN references a wiki page explaining the steps he needs to take to switch to the new kernel.



Query interface

old info (2010-02-10)

We need a way to determine the following:

  1. Is the current AMI an original Ubuntu one or is it rebundled?
  2. Is the current AMI up-to-date (in terms of security fixes)?

$BASE_URL/all_amis will return a list of every AMI we ever published, one AMI per line.

$BASE_URL/current/$region/$arch/$series/ami contains the current AMI for the given region (one of "us-east-1" or "eu-west-1"), arch (one of "i386" or "x86_64"), and $series (one of "hardy", "intrepid", or "jaunty"). Replace "ami" with "aki" or "ari" to get the most up-to-date kernel or ramdisk ID's, respectively.

A preliminary version can be found at:

update-motd hook

Pseudo code:

running_ami = ec2_metadata_service.get_running_ami()
all_ubuntu_amis = ec2_version_query_interface.all()

up_to_date_ami = ec2_version_query_interface.current(ec2_metadata_service.get_current_region, os.uname()[4], current_series())

if running_ami not in all_ubuntu_amis:
        for ami in ec2_metadata_service.get_ancestors():
                if ami in all_ubuntu_amis and ami != up_to_date_ami:
            print "One of the AMI's upon which this AMI is based has a critical security update available. For upgrade instructions, see"
    if running_ami != up_to_date_ami:
        print "There's a critical security update available for your AMI. The new current AMI is %s. For upgrade instructions, see" % up_to_date_ami

HowToUpgrade wiki page

For AMIs that have not been rebundled, upgrading is simply a matter of changing one's AMI reference to the new one.

For rebundled AMI's, upgrading involves changing the default kernel during rebundling or changing one's ec2-run-instances invocation to use the new kernel.

This needs to be explained in great detail.

Keep modules in filesystem in sync with booted kernel

For the ramdisks on EC2, we'll put all the modules into the ramdisk, and export it to userspace from within the ramdisk, somewhat like udev does it.

So, in init-bottom (which is run after the root filesystem is mounted), we'll

  1. source $rootmnt/etc/default/ec2-init
  2. if not $EC2_EXPORT_MODULES_FROM_INITRAMFS == 1, bail out.
  3. create a new tmpfs, /modules
  4. bind mount the $rootmnt/lib/modules/* into /modules/
  5. if it exists, rename /modules/uname -r to /modules/uname -r.ec2-init-moved

  6. copy /lib/modules/uname -r (in the initramfs) to /modules/

  7. "move mount" /modules onto $rootmnt/lib/modules

The fallback plan, if this makes the ramdisk too large, is to make sure there's enough stuff in the initramfs to get to the network and then publish the modules as a tarball named after the AKI id, and have ec2-init download that.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.


EC2UpgradesSpec (last edited 2010-02-10 18:11:59 by smoser)