RegressionImmunisation

Summary

Put in place procedures to discover major upstream infrastructure changes early and chart the regression potential inherent in these for our users. Use this information at around Feature Freeze to decide whether to push out the new changes or roll back.

Rationale

We need a better idea of new features and invasive changes that are coming in the development cycle that will affect our users, and we need to be able to do risk assessment of whether these features will negatively or positively impact our users' experience based on how complete the feature is and the quality of the implementation.

User stories

  • Leann informs us that the upstream kernel is rewriting its USB stack. We want to be able to decide based on some evidence whether to move to the new kernel that includes this rewritten stack, or stay with the prior version; also, if we move forward, and then decide the USB stack is too unstable to inflict upon our users (based on some measure of impact), we need to know what needs to roll back with the kernel.

Implementation

Process for tracking

  • Each UDS, grab an early plenary session to request input
  • Identify key components with major changes
  • Explore those component's communities to determine what's likely to land soon
    • items discussed in UDS sessions are likely candidates
  • Record/collate information and develop test plans
    • Can be automated tests but also a measure for estimating how much testing has been provided by the user community
  • Review the state of the software in question at FeatureFreeze for deviations of upcoming features

    • additional new features that came in
    • remove features that didn't make it
    • examine status of each feature
  • QA-driven testing days with emphasis on those features
  • Look for bug volume increase when feature lands
  • If we raise issues, the wider team can allocate resources after feature freeze to address them
  • Integrate in the release cycle at alpha 6 a go/no-go point for these features
  • Produce release notes for features that were kept but had significant issues
    • Work with upstream to address those issues

Information to capture in the table

  • Impact
    • Installed-by-default vs supported in main vs universe
    • What are likely breakage scenarios, and for what percentage of users.
  • Dependencies
    • what needs to move forward/roll back in lockstep
  • Maintenance issues (primarily for LTS releases)

We should review at the end of the Karmic cycle to see whether adjustments should be made to this process.

Initial Info Capture

Desktop:

[taken from https://wiki.ubuntu.com/DesktopTeam/ReleaseStatus]

Foundations:

  • grub -> grub2 transition


CategorySpec

QATeam/Specs/RegressionImmunisation (last edited 2009-06-09 18:19:33 by 208-151-246-43)