CheckboxDesktopExperienceTests

Summary

The desktop experience team would benefit from having Checkbox run some tests on a regular basis to evaluate the performance of the desktop.

Rationale

All the software we're developing must be as efficient as possible in terms of cpu, memory and I/O usage. It should not impact boot performance negatively and even try to improve things. To ensure that we meet this goal and go in the right direction during its development, we need constant performance metrics, generated every day, or each time a trunk commit is approved.

User stories

  • Ted merges his branch on trunk; the build system generates a new build in the daily PPA; the test infrastructure picks up that new build an triggers a new custom ISO run on reference machines; it gathers the bootchart metrics and publishes them; our build system picks up the metrics and aggregates that to update performance graphs
  • Mirco lands some new GL code; the build system generates new build which are then taken into account by the test infrastructure; the GL specific tests are run reference machines and results published; our build system similarly aggregates results and updates graphs

Assumptions

We want to re-use:

  • our continuous build system
  • the automated test infrastructure and set of reference machines of the QA team

Design

We'd like to add:

  • a custom ISO (or rebooting with new packages) to produce custom bootcharts
  • a set of extra GL tests for our netbook software
  • a publishing mechanism to be able to access test results / metrics with a simple HTTP GET operation
  • trigger mechanisms to automate the generate / run / retrieval / consolidation of indicators
  • modules to generate performance graphs for different domains (bootchart, custom tests, other)

BoF agenda and discussion


CategorySpec

Specs/CheckboxDesktopExperienceTests (last edited 2009-11-17 14:29:58 by 63)