AutomatedTestingUsingFAUmachine
Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: foo
Packages affected: None
Summary
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec.
[http://www.faumachine.org FAUmachine] is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on [:Testing] with a minimal administrative overhead.
Release Note
Having widespread testing ready will reduce the chances of regressions.
Rationale
Checking for regressions during the development cycle by hand is a burdensome task. Hence performing these tasks automatically seems a plausible goal.
Use Cases
Release manager Andrea wants to know which regressions have been introduced during the development cycle of Ubuntu. Thus, she looks up the web-page containing the results of the most recent automated test run, which shows that 9 of 10 tests have succeeded. She looks at the failed and sees a screenshot of what went wrong.
Assumptions
Design
Implementation
- The test cases listed at [:Testing] need to be analyzed if these can be automated.
- Experiment scripts for the applicable test-cases need to be written in the first place.
- Some framework needs to be written to present the results of different test runs.
- Ideally, these tests are run regularly, picking up the latest alpha/beta/daily images from the development version.
- Over time, some test cases need to be modified (e.g. adapted to UI changes of the installer), but this should be a fairly easy task (merely replacing a screenshot or a string in the test-script).
Outstanding Issues
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.