Ubuntu Wiki

Summary

Identify additional bug statistics that will be useful to the Ubuntu team during the cycle to drive quality improvements and assist during release periods.

Rationale

We should use bug metrics not only to see where we are but also drive the development and QA effort. Existing bug statistics can be expanded and new ones created to take advantage of new data points being tracked in Launchpad.

The bug tracking system of Launchpad has been expanded to record some additional date information regarding bug tasks. The information currently being recored includes the following:

• date-created (the date the bug task was created)
• date-left-new (the date the bug task's status was first changed)
• date-incomplete (the date the bug task's status was last set to Incomplete)
• date-confirmed (the date the bug task's status was last set to Confirmed)
• date-triaged (the date the bug task's status was last set to Triaged)
• date-inprogress (the date the bug task's status was last set to In Progress)
• date-fixcommitted (the date the bug task's status was last set to Fix Committed)
• date-fixreleased (the date the bug task's status was last set to Fix Released)
• date-closed (the date the bug task's status was set to a end status)
• date-assigned (the last date the bug was assigned to someone)

Currently, this information is only available via the +text interface for a bug report and is not particularly useful by itself. Rather, these dates become more useful when considering the difference between any two dates and over a package as a whole. Subsequently, a new tool should be developed to find the differences and aggregate the information for packages or teams.

Use Cases

• The Mozilla team wants to analyse their bug workflow and throughput from New to Fixed to identify the bottlenecks where they should focus their efforts. The useful bug metrics displayed on a Mozilla summary page lets them target their effort and monitor their progress.
• Pedro is planning a bug day to focus on the bugs that have been left in the New and Incomplete states for the longest times. The new bughelper tools let's him produce the appropriate wiki pages.

Assumptions

Design

Currently, information is being gathered on the quantities of bugs with a specific status or importance. It would be useful to expand this out to include status and importance. Additionally, the Incomplete should be more granular by breaking those numbers down into Incomplete w/ Response and Incomplete w/o Response. Another useful quantity are bugs reported via apport - those tagged as 'apport-bug' and 'apport-crash'.

The bughelper suite of applications complements the bug tracking system by providing statistics regarding quantities of bug reports and specific lists of bug reports. The bugnumbers tool has the ability to determine the quantity of bug reports matching specific criteria. However, bugnumbers has an overwhelming number of options and is primarily used to create lists of bug numbers matching specific criteria. Subsequently, the statistics options should become a separate tool and be expanded to include new options. The new bug lifecycle analysis tool should also be included in the bughelper package.

Implementation

• A new lifecycle tool will be added to the bughelper suite which will also take over some of bugnumber's current features (leaving a simpler bugnumbers)
• A range of new throughput statistics will be produced with the bug-lifecycle tool for various slices of time (a given week, month, from the start of the project). Examples:

  • Time in New: Sum(date-left-new - date-created) / Count(bugs_in_slice)

  • Time to confirmed Sum(date-confirmed - date-created) / Count(bugs_in_slice)

  • Time to triage: Sum(date-triaged - date-created) / Count(bugs_in_slice)

  • Time to assigned: Sum(date-assigned - date-created) / Count(bugs_in_slice)

  • Time to fixed: Sum(date-fixreleased - date-created) / Count(bugs_in_slice)

  • Time from Triaged to fixed: Sum(date-fixreleased - date-triaged) / Count(bugs_in_slice)

  • Time worked on: Sum(date-fixcommitted - date-inprogress) / Count(bugs_in_slice)

  • Oldest New/Triaged bug - display as a link to the current oldest and graphically as a function of time

• Key statistics will be displayed for select packages (or rather package groups) See: PackageStatusPages.

Code Changes

The statistics features of bugnumbers will become a separate tool from bugnumbers. This new tool's functionality will be expanded to return quantities of bugs in a state and importance. Quantities of Incomplete bug reports will also be broken down into Incomplete with and without a response.

The bughelper package relies on python-launchpad-bugs for the parsing of bug pages and accessing attributes of a bug report. Therefore python-launchpad-bugs will need to be extended to make these new bug task dates accessible.

After these new dates are accessible it will be possible to create a new application in the bughelper package to calculate the time for a change to occur for a bug task. This tool should accept any list of bugs, whether it be a package's bugs, a team's bugs, a project's bugs or a whole distribution's bugs, and calculate the time to any change.

After this functionality has been added the most useful statistics should be made publicly available for packages and teams with a substantial quantity of bug reports.

BoF agenda and discussion

MarkusKorn:

• Implementation details for bughelper and python-launchpad-bugs

  • the new version of the +text-parser is able to parse all needed dates. This branch needs some final testing before being merged with the .main branch. branch in launchpad

  • during UDS I started a bughelper branch which adds a 'buggraphs' tool to the bughelper suite. This tool is supposed to be a first draft of the implementation of this spec in bughelper. There are some brief informations on how to use this tool at the top of the sourcecode of 'buggraphs'.

CategorySpec