Ubuntu Wiki

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

• Launchpad Entry: automatedtesting

• Packages affected: None

Summary

This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec.

FAUmachine is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on Testing with a minimal administrative overhead.

Release Note

Having widespread testing ready will reduce the chances of regressions.

Rationale

Checking for regressions during the development cycle by hand is a burdensome task. Hence performing these tasks automatically seems a plausible goal.

Use Cases

Release manager Andrea wants to know which regressions have been introduced during the development cycle of Ubuntu. Thus, she looks up the web-page containing the results of the most recent automated test run, which shows that 9 of 10 tests have succeeded. She looks at the failed and sees a screenshot of what went wrong.

Assumptions

Design

Implementation

• The test cases listed at Testing need to be analyzed if these can be automated.

• Experiment scripts for the applicable test-cases need to be written in the first place.
• Some framework needs to be written to present the results of different test runs.
• Ideally, these tests are run regularly, picking up the latest alpha/beta/daily images from the development version.
• Over time, some test cases need to be modified (e.g. adapted to UI changes of the installer), but this should be a fairly easy task (merely replacing a screenshot or a string in the test-script).

Outstanding Issues

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

CategorySpec