Launchpad Entry: karmic-qa-testing-weather-report
Created: 08/04/2009
Contributors: MuratGüneş
Summary
A web application similar to the Ubuntu release weather report oriented towards Ubuntu pre-release testers that will display various bits of relevant information on the state of the development branch presented in a single location in an attractive format.
(archive integrity, ongoing library transitions, high-impact bugs, critical or high importance bugs in core components, daily image health and build status, new packages, updates in components related to release goals, calls for testing by developers, etc.)
Rationale
Currently the testing information is very scattered and confusing for new users and testers. A testing weather report will provide a service to the testing community, helping them target their work better and will encourage more contributors to test early releases and report results using appropriate channels.
Use Cases
Jono is an inexperienced Ubuntu tester, and wants to know whether it's a good idea to jump into the development branch today. Among the things he's likely to care about are whether today's daily CD is healthy, whether there are any grave bugs affecting his configuration that will make it meaningless to start testing, and whether there's any activity regarding the design goals he's interested in.
Ara has been seeing calls for urgent testing of certain components during the development cycle in various places such as Planet Ubuntu, ubuntu-devel, the Ubuntu Forums, personal blogs, and would like a one-stop shop where she can follow them.
Efrain is a would-be tester who wants to get started in contributing to free software through testing, and seeing that a particular prominent free software distribution has a pretty "dashboard" that displays its development branch status, feels slightly more eager to get started with that project.
Assumptions
The metrics listed in "Design" can all be pulled from their respective sources.
Design
Design Goals
- Come up with a list of most relevant metrics and other data that will help testers
- Display the data in a manner in which the most relevant elements can be evaluated at a single glance, on a single page, and the slightly less relevant can still be easy to reach
- Present it all on an attractive web page that will hopefully present a small bit of extra incentive for people to do testing for Ubuntu.
Provide links to documentation about the appropriate ways of reporting results including ReportingBugs and #ubuntu-testing
Data
Possible non-exhaustive list of data to include:
General
- Current milestone, release notes / technical overview, next milestone name, how many days are left
- Notices of significant ongoing work, e.g. library transitions
- Critical bugs in core packages (kernel, gcc, glibc, etc.) that may have high impact on a broad range of testers
- Bugs targeted for the next milestone
- Significant updates in components related to release goals
Archive
- Archive integrity
- New packages in the archive
- Held back packages
- Freeze status (freezes we are on and upcoming ones)
- Archive mirror status
- Build daemon status
ISO images
- Daily image build status
- Daily image health check (oversized or not)
- ISO package diff (how up-to-date are the current ISO images with the archive?)
Test results
- Upgrade situation (possibly with automatic upgrade test results)
- Calls for testing of specific packages (possibly not yet pushed to the development branch) by developers
Implementation
- A single static web page in Python and Django updated every 5 minutes on a cron