Ubuntu Wiki

• Launchpad Entry: qa-m-checkbox

• Created: 2010-04-20

• Contributors: Marc Legris

• Packages affected: checkbox, checkbox-editor

Summary

This spec is for testing improvements to make checkbox more robust and easier to use.

Release Note

End-users would have a testing adapted for different situations of laptop, netbook, nettop, and desktop testing with checkbox.

Rationale

Current users of checkbox are frustrated with some limitations in checkbox. The main drive behind this spec is to make a testers job easier and less time consuming.

UI Changes

Left hand pane addition

• contains a tree-like structure for displaying all test hierarchy (ie same as checkbox-editor)
  • tree item has indicator (pre-test): valid check (will the test be run/skipped? have all dependencies been met?)
  • tree item has indicator (post-test): test status (pass, fail, N/A)

Menu additions

• import project plan
• bzr sync test cases, media and scripts from PPAs
  • test cases would be combined with the default checkbox testcases
• open test cases, media and script directories

Expanding Checkbox

Tester Environment

• Addition of tester added variables (IE "desktop/laptop")
  • Ability to use these variables for dependency checks in testcases

Pretest

• All automated test scripts that do not need user action
  • After each test case, results would be saved
    • Tester would be shown which automated test is running (not just "information gathering")

Test results

• After each test case, results would be saved
  • If checkbox is restarted after a partial run, it should remember where the tester left off

Result submission

• Testers can review test cases, and log bugs at this point
• Offline option given (if user does not have connectivity, save results separately)
  • On subsequent runs, if offline results are detected, give an option to send these results

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarizing what was discussed and note any options that were rejected.

== UDS notes ==

offline reporting improve algorithm n^2 for tests command output to checkbox (overview of information gathering to user) comments on bugs enhanced

CategorySpec