• Launchpad Entry: lucid-qa-iso-tracker

• Created: 2009-11-06

• Contributors: AraPulido

• Packages affected:

Summary

The ISO tracker is a very important tool for Ubuntu's release process. There are some historical bugs that need to be fixed as well as some new features that will improve the experience. This specification document summarize what it is going to be implemented.

Release Note

TBA

Rationale

The ISO tracker has proved to be of great utility to track milestone ISO testing and, therefore, we are going to keep using it. However, there are things that need to be improved, in order to make its usability better and to match current needs of the QA and the release teams.

User stories

• Megan wants to add a new test case to the ISO tracker, but she does not think that it is needed for release, as the targeted architecture is not supported yet. She marks the testcase as "optional for release" and it does not appear in the release report.
• Earl runs one of the test cases against Lucid Alpha 2. He finds one major bug that prevents the installation to be completed correctly. He cannot mark the test as "Failed" until he has provided a bug number. He files a bug in Launchpad and marks the test as "Failed".

Assumptions

• The ISO tracker will continue being the main tool to track ISO milestone testing.

Design

You can have subsections that better describe specific parts of the issue.

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

• data migration, if any
• redirects from old URLs to new ones, if any
• how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

List of opened bugs in the ubuntu-qa-website project with tag: qa-tracker

436450 iso.qa. needs a working admin interface High Confirmed 456325 The ISO tracker does not generate UMR or UNR download links properly High Fix Committed 176558 New icons required Medium Triaged 291066 [iso.qa] Add an "I'm starting this test" flag Medium Triaged 220378 Testing report shows duplicate bugs Low Confirmed 220379 Testing report shows Invalid bugs Low Confirmed 291076 [iso.qa] "Suggest a case for me" Wishlist Confirmed 291089 [iso.qa] "Not complete" filter Wishlist Triaged 365060 allow launchpad IDs to login to *.qa.ubuntu.com Wishlist Confirmed 457964 [iso] We need a way to mark a product as "optional" for release purposes Wishlist Triaged 221066 E-mail notification of new build should say why Undecided New 363042 Can't sign up for Iso Testing Tracker. Undecided Incomplete ??? No requirement to add bugs to failed test cases

== Bug Discussion ==

• 436450 - Scope of Admin Interface
  • slangasek -- less of an issue now than it was, as several people have DB access
  • should change from 'High' to 'Wishlist' -- Ara
• 456325 The ISO tracker does not generate UMR or UNR download links properly
  • fixed in trunk but not applied to production
  • need a process to pull from trunk and put in production
  • probably should request this 3, 3.5 weeks out and be explicit about deadline
  • needs to be escalated by marjo to get done
• 176558 New icons required
  • is this really a 'Medium' importance?
  • perhaps this should be closed 'wontfix'?
• 291066 [iso.qa] Add an "I'm starting this test" flag
  • this is very important -- should this be 'High' or 'Critical'?
  • should contain a timestamp of some sort, possibly expire
  • how about an indication that someone is downloading a particular image?
• 220378 Testing report shows duplicate bugs
  • should bump priority to 'Medium'?
• 220379 Testing report shows Invalid bugs
  • can likely be fixed at the same time as 220378
  • arguably not a real bug; we may not want to do this
• 291076 [iso.qa] "Suggest a case for me"
  • tell users what to test based on the architecture, etc. that they have
  • should take into account that some test cases largely overlap, and should prioritize untested cases
  • this suggests a need to have priorities/importances associated with test cases
  • perhaps a generic product that test cases can be associated with; not required for release, known to be lower priority
    • in this way we can have a single area to capture lower-priority test cases rather than having them spread among many products
• 365060 allow launchpad IDs to login to *.qa.ubuntu.com
  • should note somewhere that it is no longer the case that this is a shared database with brainstorm
  • should skip this for now
• 457964 [iso] We need a way to mark a product as "optional" for release purposes
  • currently 'Wishlist' -- should this be 'High'?
• 221066 E-mail notification of new build should say why
  • show purpose of rebuild
  • should be 'Low' priority
• 363042 Can't sign up for Iso Testing Tracker.

Improve link between ISO tracker and Launchpad

• add link to test case on bug report
• put comments in ISO tracker in bug report

Action Items

• notify marjo that we need to have IS pull from trunk to production (stgraber)
• change 436450 to 'Wishlist' (ara)
• 'wontfix' 176558 (??)
• file a bug about failures requiring a bug report (??)
• Create a generic product to add testcases that need to be run just once per release (slangasek)

Priorities from the release team

457964 [iso] We need a way to mark a product as "optional" for release purposes Wishlist Triaged ??? No requirement to add bugs and serious to failed test cases

---

CategorySpec