• Launchpad entry: https://launchpad.net/distros/ubuntu/+spec/edgy-testing

• Created: 2006-06-06 by SimonLaw

• Contributors: SimonLaw

• Packages affected: Everything in Ubuntu proper

Summary

This specification outlines our plan for getting Edgy tested properly.

Rationale

Previous releases have been tested in an ad-hoc fashion. Apparently, slightly more organization would be useful, although this has to take into account the organic and volunteer nature of most of our testing infrastructure.

User stories

• Alexandria is a release manager for Edgy and wants to know if a feature should be kicked out of the release. She consults the testing results offered by the QA efforts and finds the right information to make a decision.
• Barry is an Ubuntu user who wants to help test Ubuntu, but doesn't know where to start.

• Carolyn is a Ubuntu user who is on the LaptopTestingTeam. She wants to help test Ubuntu and to report her results so that they'll be acted upon.

Scope

This specification outlines how testing is going to approached for Edgy. This is not meant to be a general specification stretching until the end of time.

Plan

We will harness the efforts of the LaptopTestingTeam as well as the BugSquad to provide continuous testing throughout Edgy Eft's development cycle. In this plan, these awesome volunteers will be referred to as the "testing team".

• We begin by examining the feature specifications that are scheduled for Edgy. These each need test plans, even preliminary ones, that will ensure basic sanity. We will have created a test plan for each approved specification by 20 July. This is meant to:

  • Ensure that each specfication has some form of measurable objective.
• As each specification is worked upon, the testing team will periodically run tests on the partial implementations. Test plan development will continue in this stage, adapting as more test cases are thought up. New test cases should also be spawned whenever a new bug is discovered. At this stage, quick sanity checks on the installer will be performed at every milestone. Preliminary test results should be available by 17 August. This is meant to:
  • Measure the progress of each implementation in an objective way.
  • Catch regressions early in the development cycle.
  • Ensures that hugs are distributed early and often.

• During the DistroSprint, the release managers will examine the test results and decide on which specifications to continue developing. And which to defer. This is meant to:

  • Refine which specifications we should concentrate our effort upon.
  • Concentrate developer effort on specifications that will complete successfully.

• After the DistroSprint, the testing team will continue to refine and run the test plans for the remaining specifications. These test should provide firm data about these specs by FeatureFreeze on 7 September. The testing team will still run test plans all the way until FinalRelease. We expect that only bug fixes should be released into the archive, so test results should look more and more promising every day. This is meant to:

  • Track the progress of bug fixes.

• After FeatureFreeze, a test plan for the installation will be written. Installer testing will be our focus on 28 September, which coincides with BetaRelease. We've done this kind of testing before for Dapper Drake, so this should be familiar to many people. Both the testing team and the DistroTeam will perform installer testing. This is meant to:

  • Ensure that the installer works on as many platforms as possible.

  • Test the installer on configurations that are unavailable to the DistroTeam.

• Specification and installer test results for all architectures should be up-to-date for the ReleaseCandidate, on 19 October. This is meant to:

  • Give the release managers enough information to decide whether to perform the FinalRelease.

Timeline

See EdgyReleaseSchedule, which this is mirroring a bit.

Unresolved issues

• We need to write up templates for the Wiki:

  • TestPlanTemplate

  • TestResultsTemplate

BoF agenda and discussion

---

CategorySpec