MobileKarmicArmCloudBuilds

Archive rebuilds in the cloud for armel

Summary

Creation of a tool to build packages for armel in the cloud, allowing optimized binaries with different toolchains, QA, and perhaps PPAs.

Release Note

Ubuntu now provides a framework for doing armel rebuilds in the cloud.

Rationale

  • Creation of optimized binaries for OEM
  • Testing of compiler flags to determine what we want to enable/disable
  • Possibly used in implementation for ARM PPA
  • Also can be used to add builds for older ARM architecture variants

User stories

Assumptions

Design

  • Based around an EC2 API (either EC2 itself, or private cloud)
  • One build master and build daemons, with control point being in the cloud.
    • - Want to be able to use the subset between EC2 and Eucalyptus
  • build around the standard debian build tools (sbuild)
  • For wanna-build database with current debian wanna-build, or wb_python.
    • - Debian buildd system is email based - Being able to easy track and (if necessary) import new sources; perhaps
      • based on quinn-diff
  • Input is a list of APT sources
  • Archive software
    • mini-dak, dpkg-scanpackages, apt-ftparchive?
    • needs to preserve components
    • Backend? filesystem? Should be in S3
  • Multiple passes? Using binaries from the old or currently built archive?
    • Probably an option
    • Pass for required + base + build-essential packages? Check with infinity
  • Keep build logs
    • Useful to debug things if you do a second pass
    • Maybe base around wanna-persue
  • Command-line with a config file
  • Optionally: import the archive into the cloud to save bandwidth

Implementation

UI Changes

  • No UI changes, new framework being written.
  • New UI to likely be command-line based
  • wanna-build/buildd should be modified not to use email if possible
  • wanna-persue (if used for web interface) should have a summary view added to it

Code Changes

  • Creation of an rebuild-driver which handles all the implementation steps above
    • Creates w-b databases for each stage of the rebuild
      • Stage 0 - rebuild of base system
      • Stage 1 - first pass of entire system
      • Stage 2 - rebuild of the entire system to ensure sane end result (optional)
  • buildd should be modified to be able to upload unsigned packages assuming success
  • Entire framework should have a turn-key like setup

Migration

  • No migration, new functionality

Test/Demo Plan

  • The rebuilder should be tested by doing a full archive build, installing the result, and making sure the end results work

Unresolved issues

  • How to handle blacklisting of packages (i.e., taking a package from a local source, and confirming it always superseeds any other package.)


CategorySpec

Specs/MobileKarmicArmCloudBuilds (last edited 2009-06-18 17:10:43 by cpe-72-226-207-134)