CheckboxWishlist

Summary

Further modifications to Checkbox that are low priority or not fully defined. Although they are targeted for Karmic, they will only be tackled if there is time.

Release Note

Release Notes are specific to each enhancement.

Rationale

Checkbox requires various enhancements to improve the user experience, or provide functionality for teams using Checkbox. These enhancements vary in size and complexity, but none are significant enough to require their own blueprint. Therefore a decision was made to combine them into a single blueprint for tracking purposes.

The enhancements within this blueprint are either a low priority, or are not fully defined, and as such will only be addressed during the karmic cycle if there is enough time.

Enhancements

Each enhancement is dealt with individually.

State

Checkbox should remember how far through the testing process it is, and resume from this point when restarted. If the last test run is incomplete, give the option to skip/restart that test.

Long Running Tests

As long running tests are introduced to Checkbox (e.g. suspend/resume or mago suites) we need to cater for the following:

  • Cancelling tests - users may wish to cancel a long running test
  • Failures - a failure during a long running test could be handled in two ways:
    • Skipping to the next part of test (e.g. mago suite)
    • Failing the whole test

Long running tests should include test output/status in the results, and decisions made on how to handle failures on a case-by-case basis.

NB Some of this functionality may be handled by filters.

Multiple Tests

As part of stress/performance testing, we should cater for running a single test n times, with the result being stored as overall of all runs.

Should there be a facility for acceptable ranges? E.g. if 4/5 tests pass, the overall result is a pass.

Random/Stress Testing

Needs definition

The ability to trigger a random selection of tests over a) a defined period or b) a defined number of tests.

E.g. Run tests for 30 mins, or run 100 tests.

CategorySpec

QATeam/Specs/CheckboxWishlist (last edited 2009-06-12 16:05:34 by genld-217-080)