AutomatedTestingUsingFAUmachine
2345
Comment: change spec link
|
← Revision 4 as of 2008-08-06 16:23:15 ⇥
2349
converted to 1.6 markup
|
Deletions are marked like this. | Additions are marked like this. |
Line 12: | Line 12: |
[http://www.faumachine.org FAUmachine] is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on [:Testing] with a minimal administrative overhead. | [[http://www.faumachine.org|FAUmachine]] is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on [[Testing]] with a minimal administrative overhead. |
Line 32: | Line 32: |
* The test cases listed at [:Testing] need to be analyzed if these can be automated. | * The test cases listed at [[Testing]] need to be analyzed if these can be automated. |
Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: automatedtesting
Packages affected: None
Summary
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec.
FAUmachine is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on Testing with a minimal administrative overhead.
Release Note
Having widespread testing ready will reduce the chances of regressions.
Rationale
Checking for regressions during the development cycle by hand is a burdensome task. Hence performing these tasks automatically seems a plausible goal.
Use Cases
Release manager Andrea wants to know which regressions have been introduced during the development cycle of Ubuntu. Thus, she looks up the web-page containing the results of the most recent automated test run, which shows that 9 of 10 tests have succeeded. She looks at the failed and sees a screenshot of what went wrong.
Assumptions
Design
Implementation
The test cases listed at Testing need to be analyzed if these can be automated.
- Experiment scripts for the applicable test-cases need to be written in the first place.
- Some framework needs to be written to present the results of different test runs.
- Ideally, these tests are run regularly, picking up the latest alpha/beta/daily images from the development version.
- Over time, some test cases need to be modified (e.g. adapted to UI changes of the installer), but this should be a fairly easy task (merely replacing a screenshot or a string in the test-script).
Outstanding Issues
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.
AutomatedTestingUsingFAUmachine (last edited 2008-08-06 16:23:15 by localhost)