Status
Created: 2005-04-25 by JaneW
Priority: HighPriority
People: JeffBaileyLead, AdamConradSecond
Contributors: JaneW, MattZimmerman
Interested: MattZimmerman, MarkShuttleworth, MauricioHernandez
Introduction
This spec addresses the need for a formal test plan to use when testing.
Unlike most specifications, this is one that will exist forever and be steadily improved over time.
Rationale
We need to provide a test procedure, and a sequence for testing new releases of Ubuntu and derivatives on various platforms.
It should include checklists for the testers, which make it easy to test different parts of the distribution step by step. Longterm plan is to store this information in Launchpad, where a sign-off list makes sure that it's as easy as possible to preserve information on how the tests went.
The document will form the base of this implementation.
Scope and Use Cases
- A member of the Ubuntu community wishes to help test a development snapshot of Ubuntu, and would benefit from a rigorous, formal procedure in order to ensure a thorough test and effective feedback to the developers
- An Ubuntu developer wishes to regression-test the system after introducing an intrusive change in a development environment
Implementation Plan
The end result of this project should include:
- Documents under revision control, suitable for collaborative maintenance and improvement
- The same documents, converted and packaged in formats suitable for default installation with Ubuntu
- The same documents, published in a format suitable for publishing on the web
The test plan refers to generic tasks and applications ('the email application') in order to be applicable to all versions of Ubuntu. If a derivative wants to offer a test plan of its own, these generic parts have to be specified.
There should be multiple versions of the test plan. For example, a long and short version should be available. The short version should test core functionality, while the longer one could test things that are less obvious or take longer to perform.
At the end of the Breezy release cycle, we started off with a short "Desktop test" (BreezyTestPlan), which usually followed an installation (or the use of the Live CD). This is the basis for the short test.
The test document can be found here: TestPlans
Outstanding Issues
It may make more sense to integrate AutomatedTesting as an early component of FormalTestPlans: Manual testing should never find bugs which trivial automated testing could have uncovered.