Launchpad Entry: qa-m-checkbox
Created: 2010-04-20
Contributors: Marc Legris
Packages affected: checkbox, checkbox-editor
Summary
This spec is for testing improvements to make checkbox more robust and easier to use.
Release Note
End-users would have a testing adapted for different situations of laptop, netbook, nettop, and desktop testing with checkbox.
Rationale
Current users of checkbox are frustrated with some limitations in checkbox. The main drive behind this spec is to make a testers job easier and less time consuming.
UI Changes
Left hand pane addition
- contains a tree-like structure for displaying all test hierarchy (ie same as checkbox-editor)
- tree item has indicator (pre-test): valid check (will the test be run/skipped? have all dependencies been met?)
- tree item has indicator (post-test): test status (pass, fail, N/A)
Menu additions
- import project plan
- bzr sync test cases, media and scripts from PPAs
- test cases would be combined with the default checkbox testcases
- open test cases, media and script directories
Expanding Checkbox
Tester Environment
- Addition of tester added variables (IE "desktop/laptop")
- Ability to use these variables for dependency checks in testcases
Pretest
- All automated test scripts that do not need user action
- After each test case, results would be saved
- Tester would be shown which automated test is running (not just "information gathering")
- After each test case, results would be saved
Test results
- After each test case, results would be saved
- If checkbox is restarted after a partial run, it should remember where the tester left off
Result submission
- Testers can review test cases, and log bugs at this point
- Offline option given (if user does not have connectivity, save results separately)
- On subsequent runs, if offline results are detected, give an option to send these results
Unresolved issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarizing what was discussed and note any options that were rejected.
== UDS notes ==
offline reporting improve algorithm n^2 for tests command output to checkbox (overview of information gathering to user) comments on bugs enhanced