Summary

Goal

Maximize the amount of bug fixing that occurs per unit time invested in bugs management by desktop team engineers.

Rationale

The current situation is:

The goal state is:

User stories

End User

A user encounters a bug in program Foo. Based on previous experience they go to launchpad.net to log the bug. When click the link to add a bug, they are shown instructions to use ubuntu-bug. They run ubuntu-bug which gathers data relevant to their symptoms and logs the bug for them. When the bug is logged they see a message telling them that the bug may be triaged, but may not be fixed, and also tells them where to go for support.

Triaging Tool

seb128 starts work in the morning. He fires up an app that chugs away on launchpad loading bugs into a list in the areas of GNOME that he works on. He gets some coffee and works on email. When the list is finished loading, he is able to look over the list and select bugs in batches. He can click buttons to set the bugs into different states, such needs info, won't fix, etc.... When he is done, he knows that he has triaged all the new bugs in his area.

Doing an Upload

asac uploads a firefox security fix. An hour later he starts getting bug mails about a regression some users experience. The mails are marked as being related to his recent upload (reason=upload watch). He triages the bugs. He gets similar bug reports for the next four days. After four days, the emails stop coming related to the upload, though bug mails are sent in the normal fashion.

Assumptions

  1. Being more or less forced to use ubuntu-bugs will decrease the number of incoming bugs, and will increase the quality of the reports.

Design

A multi-pronged approach is being used to make progress towards this goal state:

  1. The launchpad team is changing launchpad so that it helps users use ubuntu-bug to report bugs. This should result in fewer but better quality bug reports.
  2. pitti is enhancing ubuntu-bug to be "symptom based" (see desktop-karmic-symptom-based-bug-reporting). This should result in even better bug reports.

  3. Work with the launchpad team to add verbiage to launchpad that could help set the users' expectations regarding bug reports.
  4. The launchpad team will add an "expired" state to older bugs, reducing the noise in the system.
  5. The desktop team engineers will limit bugs assigned to them to bugs which they can and will realistically work on, preferably in the current cycle
  6. Craft a tool that dramatically speeds up the bug triaging process.
  7. Craft a tool that "watches" for bugs in new uploads, allowing engineers to respond to regressions more quickly.

Implementation

Triaging Tool

This will likely be added to or derived from the pm-dashboard code:

This will require:

  1. a way to define a set of packages to query for bugs to triage
  2. A list view with enough information for assessing the impact of the bug
  3. A way to quickly select batches of bugs
  4. A needs info button that can edit bugs in bulk to need info
  5. A needs to be worked on button
  6. A would be nice to get worked on button

Needs Info

Clicking this button would set the status to "needs info". It would inject boiler plate "needs info" text into the comments.

Fix

Will set the bug priority to High and the bug status to Triaged, and will assign to the canonical-desktop-team.

Community

Will set the bug priority to Medium. Will not assign the bug. Will set the bug to triaged. Will add a comment that the canonical-desktop-team will not work on it, but would welcome help on the bug.

Triaging a Single Bug Task

Notice that the description is visible. The buttons "Need Info", "Community", and "Fix" will set the selected bug to the appropriate state.

Bulk Triaging

While multi-selected, the selected bug titles can be read, and the triaging buttons will apply to all selected bug tasks.

UI Changes

Code Changes

Upload Watch

There is no UI for this per se. This is essentially a cron job that will run on rookery, and detect recent uploads, and direct bug mail appropriately. The key idea here is that after doing an upload, engineers must monitor their email for regressions related to the uploaded packages, and must sort those bug mails out from the bulk of other bug mails. This system will separate bug mails out as potential regressions if they are related to recently uploaded packages.

Code Changes

  1. a cron job that detects recently uploads
  2. a way to create a special subscription to bugs on packages related to those uploads
  3. a way to change the headers for those bug mails so that engineers can sort them seperately
  4. a way to unsubscribe after a set period of time, when the upload is no longer "hot"

Test/Demo Plan

Unresolved issues

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

UDS Discussion Notes

pitti: Rick, would you mind drafting this? If not, I'm fine with doing that.

Note:

"Bug workflow for desktop engineers"

Problem according to Rick: "Too many freaking bugs (not code defects, just noise) with culture of addressing all reports (not feasible). This is causing burn out or supressing time to work on bug fixes"

How do you mark a bug that I looked at it, but I want it off the new list?

How can QA drive to 0?

Seb's dream:

Rick has a desktop tool for queuing multiple bugs for processing

Should also have a way that users can communicate with upstream quickly - then need to observe what is going on with communication

Bryce - several workflows

Pitti wants a way to have "gravity" that means that you can see key bugs quickly Workflow - regression watch after an upload (for particular version) Important that users use apport to make this work asac wants automatic temporary (3 days) package bug subscription for New bugs only

Bug upload-watch flow:

Upstreaming process of bugs:

How engineer's should prioritize bug work:

  1. Watch out/work on major regressions
  2. Work on bugs that are already assigned to me
  3. Already triaged bugs, that need to be worked on, upstreamed, etc.
  4. Triage incoming bugs

Must be honest about only assigning bugs to self that will be worked on; keep /+assignedbugs list clean Almost all x bugs are regressions - matter of degree needs to be determined

Need a "responded" state as opposed to Incomplete w/ Response - since this is not visibile anywhere in the web interface for launchpad

If you don't have a triaging team, then the bug work flow related to state changes doesn't work for you

New: Not being looked at yet Incomplete: Waiting for more information from reporter(s) Invalid: Wont Fix: Triaged = enough information for a developer to understand the root cause and fix the issue (i.e. no more user discussion required) Confirmed = user has asked for all the information / another user also has the problem / Triager has been able to reproduce the issue

In Progress: Fix Committed: Fix Released:

rickspencer3 thinks higher quality lower quantity bugs will be better

Apport and apport collect:

Launchpad:

Can we mass close a large amount of bugs

Will add an "expired" state: LP bug janitor disaster

Roman Friesen input:


CategorySpec

DesktopTeam/Specs/Karmic/BugWorkflow (last edited 2009-06-17 16:08:12 by rick-rickspencer3)