Ubuntu Wiki

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

• Launchpad Entry: iso-test-tracker

• Packages affected:

Summary

A web-based test reporting system for release testing of ISO images.

Release Note

Contributing to the CD testing effort in Ubuntu has been made easier with a purpose-built test tracker.

Rationale

For the Feisty release process we used first an improvised system in Malone and then a prototype version of a dedicated web-based tracker. This system should be matured and extended to become an efficient and reliable tool in the release process.

Use Cases

• Community tester
• Release Manager
• Test admin

Assumptions

Design

New features

• Tabbed browsing: Organise the different distro flavours by tabs to reduce the length of the ISO list
• Trusted submitters: Create a category for users that are known to produce reliable test results. A pass or fail report from a trusted user will propagate up to the test and ISO summaries, reducing work for the administrators.
• Bulk email: New registrants should receive an email with a link to on-line documentation of how the system works. Allow the administrators to send email notifications to registered testers about new ISO images, etc. Alow users to opt-out of receiving messages.
• Gather Launchpad bug info: Hovering over a bug should show its title. The status of a bug should also be reflected on the test site: Is it confirmed, serious, milestoned, etc.?
• Automation of test posting: Connect the script used to generate the ISO images with the tracker so that building milestone test images also posts the tracker entries. Realistically - add a single command line step to post the test entries after the builds are done.
• Improve navigation efficiency: Explore ways to reduce the number of page levels. Most likely make the test cases expandable so that the list of submitted tests can appear below it (click to show and hide).
• Archives: As we grow more archive information we need to organise it better, probably by milestone.
• Look and feel: Create a theme that is attractive and efficient. Add the new testing logo, use icons that are easier to spot and use the screen space more efficiently.
• Download link: Provide an easy link to download each ISO image. Also provide md5sums and a sample rsync command that can be used.
• Add checkbox for the release team to set a bug as a blocker.
• Report generation post release to show which bugs were fixed for the release and which are still open issues.

Hardening

• Backup systems
• DB integrity and restore, unit tests?
• Flood control - efficiencies and option for restricting access

Implementation

Extend and polish the [https://www.stgraber.org/ubuntu/isotesting/ existing site].

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

Hosting

• Run on the stgraber server for the first few Gutsy alphas to let the code mature and then move it to Canonical Data Centre servers.

UI Changes

Build on existing UI with some enhancements:

• Use tabs for the different distro flavours
• Use the screen realestate more efficiently
• Add some colour and use better icons

Code Changes

• Switch from MySQL to Postgres

Migration

• Do not migrate data, start with fresh test cases

Test/Demo Plan

Test changes on the stgraber community server for the first few Alphas then move to Canonical hosting.

Outstanding Issues

CategorySpec