Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: self-testing-desktop
Packages affected:
Summary
Investigate the possibility of including the automatic desktop testing framework in main and on the installation CD, to make it easier for users to participate in testing. The accessibility infrastructure and python are already included and the test scripts themselves are small.
Release Note
The Live CD will include a script, to be run from the command line, which runs a self-test on the system. The same self-test may also be installed on the target system, and run there.
Rationale
Including the test suite on the Live CD, and making it possible to install on a system via normal package management, makes automating QA easier, since the tested image or system is as identical as possible to the one users use. There is no need to install additional packages in the Live CD environment after it has booted, or onto the installed system. The end-user impact is minimal.
Use Cases
- The Live with a self-contained test system can be tested more efficiently on multiple machines by a single person and testing can be initiated simply in a virtual environment.
- A standardised test scheme can be run by multiple users on a range of different hardware and with differently configured -- real life -- software configurations, and yet provide data in a consistent and useful format.
Assumptions
The test suite and test runner have already been implemented according to the DesktopAutomatedTests specification.
- There will be one or two additional (small) packages included in main for this. One for the test runner, one for the test suite.
- Apart from these packages, the Live CD should require no additional packages installed.
- The packages may or may not be installed on the target system by default, depending on which is easier to implement.
Design and Implementation
The test suite and test runner are packaged as .debs, and included on the Live CD. To start the test suite, the user opens a terminal and executes the runner manually. (Some day in the future there may be a "TESTME" button that starts a GUI version of the runner.)
The result is saved in an XML file that can be attached to bug reports or uploaded to a (future) test tracking system.