Why are bugs being filed?

• About 100 new bugs three weeks ago
• About 300 new bugs now.
• New users are joining Ubuntu every day, and they want to contribute
• New users bring new configurations (software, hardware, etc.)
• Difficult to find duplicates (~60% noise, duplicates)
• Filed bugs instead of support cases
• "Noise bugs" being filed: changing defaults, request features, translation er rors, ...
• Ubuntu may suck more as time goes on, because we introduce more bugs with mor e software
• Community encourages contributions in the form of bug filing

Workflows

• Permissions are used to prevent people from damaging Malone

Volunteer work

• Have really good processes to triage bugs
• Managable, predictable units of works

Malone

• Present the UI such that helpful work is obvious
• Searching should have shared filters which people can build upon
• Better data-mining and reporting needs to work better
• Design Malone to encourage bug triaging
  • Looking at a package suggest bugs to triag.
  • Encourage upstream to use Malone (xml-rpc)
  • Amazon-style "stalking" of bugs
  • Getting stats on Untriaged to Need Info
    • Response times
    • Bugs filed per user
    • What we think are the root causes
    • How do we locate good submitters? reporters
• How do we turn people in better bug reporters?

  • Karma & metrics to encourage good behavior

  • Multi-stage bug filing for duplicates
  • Bug reporting tool (automated info)
  • Finger printing bugs.
  • Hardware matching

Workflow

• Encourage support tracker
  • Turn support into bug
  • Education about differences
  • Find help in the tracker
• Feature requests need to be turned into specs
  • User choosable "wishlist"
  • Turn them into specs
• How to file bugs?

Future

What should bug squad do? developers do? various interfaces? workflow?

Support tracker

• The support tracker is not mature yet
• Have volunteers prioritize bugs(?)
  • proper instructions
  • highlight important bugs
• Can we integrate support tracker in forums?
• Having response templates would be nice
  • Standard comment when marking a duplicate by example
• The forums and the support tracker have the same function
  • Transform the forums into a support tracker
  • Should we encourage people to use the forum instead of the support tracker for now until the support tracker is ready?

Comments

JohnMoser: I'm currently looking at grouping and sorting stuff from AutomatedProblemReports; perhaps finishing that first and looking back at this afterwards would be best. If you can manage to group AutomatedProblemReports in a heuristic manner, then encouraging users to not report crashes may be an option. Some heuristics include:

• crash occurs at the same point (i.e. function, line of code), in the same module (library or program)

  • i.e. we found a stack smash because it called __stack_chk_fail(), the calling function was some_vuln_function()

• list all crashes with SOME_NUMBER of common back trace from the crash point

  • i.e. we found a double-free(), determined some_stupid_function() called free(), and the last 3 up to there were good_function(), fine_function(), some_stupid_function().

• Crashes fault on the same problem

  • i.e. SIGILL crash.

  • SIGSEGV detected attempting to execute unmapped memory

You can simply tag this information into the bugs and then display it based on different search criteria. For example, you may select a crash, and search for "similar crashes." You may then enter (through check boxes) various similarities to locate. Let's say this particular fault is a SIGSEGV attempting to execute the stack; the below options may be available to you:

• Similar fault

  • SIGSEGV

  • Attempt to execute
    • Attempt to execute non-executable area
• Same module

  • Previous executing function was very_broken_function(), so we assume we are in module very_broken_lib.so

  • The version was 0.1.19 (i.e. very_broken_lib.so.0.1.19)

• Same program

  • Running /usr/bin/program_that_crashed

• Similar backtrace

  • Fault occurred in very_broken_function()

  • Previous N calls (you enter a value for N)
• Related (as manually tagged by the developers)
  • Yes
  • No
  • Unknown

You may decide now to search for SIGSEGV in /usr/bin/program_that_crashed in module very_broken_lib.so (any version) apparently from very_broken_function(); but not worry about matching any part of the tail of the backtrace, or that the attempt was to execute.

You could instead decide to search for SIGSEGV on attempt to execute related to very_broken_lib.so.

You could also search for any fault in very_broken_lib.so.

The list goes on but the point is you now have a powerful tool for taking characteristics of an automatically detected problem and matching it with other problems. These other problems may or may not be related; you can manually tag them as being the same bug, group them in the same group, and have quick and easy future reference to all the different reports.

AutomatedProblemReports describes the crash reporting mechanism. Working from there, AutomatedProblemReportsTaggingForSecurity was written to show a use for this kind of mechanism running on top of AutomatedProblemReports; but does not go into great detail on how exactly to implement a tagging and ordering system. Someone should spec that functionality out as well.

Michaeljt: A number of semi-random thoughts on the subject.

• The current "flat" bug tracker model does not necessarily encourage effective bug reporting and handling. In particular for packages for which a large number of bugs are filed, it can be hard to find relevant bugs and duplicates. An alternative might be a bug tracker with a browsable tree structure (say Packages/KDE/Konqueror/UserInterface or Packages/Services/OpenSSH/SupportRequests). People with bug tracker administrator privileges could create and remove tree branches as needed and move bugs between branches. Not all branches need relate directly to packages. It would also be good for some way of collapsing duplicates into a single bug (or perhaps just creating tree branches to house sets of duplicate bugs).
• Often bug reports are used as forums where people experiencing a problem discuss how to solve it or work around it. This can also be a useful part of solving the problem.
• One reason that co-ordination with upstream may be difficult is that distribution packages are usually maintained separately from upstream packages. This includes a set of patches which make the package different (and potentially exhibit different behaviour) to the upstream version, and using a version which may no longer be supported upstream. It is partly to do with the fact that upstream maintainers are reluctant to get involved in the tricky process of packaging for lots of different distributions. This situation could be improved by
  • Packaging a version of the package which upstream intends to support for a while
  • Setting up some sort of automated packaging system à la Gentoo, whereby upstream could upload a tar.gz with bug fixes, and a new .deb would automatically be generated. It would also be good to have a couple of extra repositories per package to which the upstream maintainer could upload respectively bug-fixed versions to be tested and backports of newer versions for people who badly need a fix which is too large to be backported to the stable version.
• Automated bug reports are nice, but it would also be nice to be able to generate them for running (non-crashed) applications. As in, say, an application produces an error message which looks like something has gone badly wrong, or an application has become unresponsive. The automated report tool is started, the user clicks on the application, and a stack trace and (optionally) stack dump is generated. As an alternative, the user could enter a PID or the name of an executable into the tool. The report automatically generated could be uploaded as part of a bug report.