SymptomBasedBugReporting

Differences between revisions 2 and 3
⇤ ← Revision 2 as of 2009-05-25 15:46:21 →
Size: 3433
Editor: 80
Comment: BoF notes
← Revision 3 as of 2009-06-01 09:36:05 → ⇥
Size: 5315
Editor: pD9EB404B
Comment: spec it out
Deletions are marked like this. Additions are marked like this.
Line 9: Line 9:
Line 23: Line 22:
For a lot of debugging scenarios, the user needs to do an action during the debugging process, or needs to answer further questions interactively. This currently needs to happen in the bug report trail, but it would be better to ask these questions during the initial data collection to get better structured bug reports.
Line 25: Line 26:
== Assumptions ==  * Martin maintains hal in Ubuntu. In bug reports about automount failures he referred reporters to the DebuggingRemovableDevices wiki page. Now he writes an interactive hook to ask the reporter about the type of device, then set up hal in debug mode, ask the user to plug in the problematic device, collect debug logs, and then reset the system back to normal.

 * Chris maintains OpenOffice.org. He writes an Apport hook to ask the user for a document file which reproduces the problem.

 * Joe installs Ubuntu on his notebook and tries to play a video. He does not hear any sound. He runs `ubuntu-bug sound`, gets asked a few questions. The apport hook does a few system checks, and together with the answers it determines that the Linux and PulseAudio portions work, and it is most probably a problem in the ALSA codec quirks. The bug gets filed against "alsa-lib" with a "codec-quirk" tag.
Line 29: Line 34:
You can have subsections that better describe specific parts of the issue.  * ''Interactivity'': Add standard UI methods to abstract user interface, implement it for GTK/Qt/CLI, and pass UI object to package hooks so that they can call methods on it.

 * ''Run commands as root:'' Add function to the `hookutils` library, so that hooks do not need to duplicate detection for running desktop environment and availability of gksu/kdesudo/etc.

 * ''Symptoms:'' These are provided as Python scripts, very similar to Apport package hooks. Their primary purpose is to reliably detect the affected package, and enrich the report with helpful information such as tag classification or standard titles.
Line 33: Line 42:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: === Interactivity ===
Line 35: Line 44:
=== UI Changes === Extend `apport.ui.UserInterface` API with new methods which hooks can call:
Line 37: Line 46:
Should cover changes required to the UI, or specific UI that is required to implement this  || ''Method'' || 'Return value'' || ''Description'' ||
 || `ui_yesno_message(self, title, text)` || Boolean || Yes/No question ||
 || `ui_file_selector(self, title, text)` || File path, or `None` when cancelled || File selector ||
 || ui_choice(self, title, text, options, multiple=False) || List of choices, or `None` when cancelled || Single or multiple choice question ||
Line 39: Line 51:
=== Code Changes === In addition, hooks can call the already existing `ui_info_message(self, title, text)` or `ui_error_message(self, title, text)`.
Line 41: Line 53:
Code changes should include an overview of what needs to change, and in some cases even the specific details. Pass current UserInterface object to hooks by extending the current API: `add_info(report, ui)`. For backwards compatibility, it should still be possible to run hooks with the `add_info(report)` API.
Line 43: Line 55:
=== Migration === === Running commands as root ===
Line 45: Line 57:
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.		 Add new function `apport.hookutils.root_command_output()` which runs a command as root and returns its output, similar to the current `command_output()`. This will detect the currently used desktop environment and call gksu/kdesudo/sudo accordingly.

=== Symptoms ===

Symptom scripts are put into `/usr/share/apport/symptoms/` and need to define a
method `run(report, ui)`. By a combination of interactive questions and
automatic system tests this method needs to determine the affected package and
return it.

It can optionally add information to the passed `report` object, such as tags.
After that, Apport adds package related information and calls the package hooks
as usual. Before `run()` is called, Apport already added the OS and user
information to the `report` object.

=== Abort ===

If the user cancelled an interactive question for which the hook requires an answer, the hook should `raise StopIteration`, which will stop the bug reporting process.

=== Documentation ===

Fully document hooks, symptoms, and interactivity in doc/package-hooks.txt. Also explain how to set tags.
Line 52: Line 81:
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

== Unresolved issues ==

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

== BoF agenda and discussion ==
 * types of interactivity

 * xorg: crashes, freezes, resolution, font size (EDID)
 * list of common options, no other information
 * checkboxes

 * "now do this" info boxes -> debug hal

 * report against package -> also use interactivity, reassign

 * Q&A game could lead to non-bugs -> redirect to answers

 * LP +filebug -> also show answer tracker dupes

 * tag based on symptom (already possible) -> documentation

 * info box, yes/no, check boxes, radio buttons

 * abstraction for running external program as root

 * consolidate documentation

 * file bug as private
 TBD when beta available.

Summary

We provide an extension of the notion of hooks to be symptom based and interactive. "ubuntu-bug" without parameters would then show the available symptoms, and the symptom hooks can do some (limited) interactivity to get more information from the user, and finally figure out which package to file it against and which information to collect.

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.

Rationale

Apport package hooks help tremendously to collect specific information for a bug report. However, in some cases it is quite hard for the reporter to figure out which package to file a bug against, especially for functionality which spans multiple packages. For example, sound problems are divided between the kernel, alsa, pulseaudio, and gstreamer.

For a lot of debugging scenarios, the user needs to do an action during the debugging process, or needs to answer further questions interactively. This currently needs to happen in the bug report trail, but it would be better to ask these questions during the initial data collection to get better structured bug reports.

User stories

  • Martin maintains hal in Ubuntu. In bug reports about automount failures he referred reporters to the DebuggingRemovableDevices wiki page. Now he writes an interactive hook to ask the reporter about the type of device, then set up hal in debug mode, ask the user to plug in the problematic device, collect debug logs, and then reset the system back to normal.

  • Chris maintains OpenOffice.org. He writes an Apport hook to ask the user for a document file which reproduces the problem.

  • Joe installs Ubuntu on his notebook and tries to play a video. He does not hear any sound. He runs ubuntu-bug sound, gets asked a few questions. The apport hook does a few system checks, and together with the answers it determines that the Linux and PulseAudio portions work, and it is most probably a problem in the ALSA codec quirks. The bug gets filed against "alsa-lib" with a "codec-quirk" tag.

Design

  • Interactivity: Add standard UI methods to abstract user interface, implement it for GTK/Qt/CLI, and pass UI object to package hooks so that they can call methods on it.

  • Run commands as root: Add function to the hookutils library, so that hooks do not need to duplicate detection for running desktop environment and availability of gksu/kdesudo/etc.

  • Symptoms: These are provided as Python scripts, very similar to Apport package hooks. Their primary purpose is to reliably detect the affected package, and enrich the report with helpful information such as tag classification or standard titles.

Implementation

Interactivity

Extend apport.ui.UserInterface API with new methods which hooks can call:

  • Method

    'Return value

    Description

    ui_yesno_message(self, title, text)

    Boolean

    Yes/No question

    ui_file_selector(self, title, text)

    File path, or None when cancelled

    File selector

    ui_choice(self, title, text, options, multiple=False)

    List of choices, or None when cancelled

    Single or multiple choice question

In addition, hooks can call the already existing ui_info_message(self, title, text) or ui_error_message(self, title, text).

Pass current UserInterface object to hooks by extending the current API: add_info(report, ui). For backwards compatibility, it should still be possible to run hooks with the add_info(report) API.

Running commands as root

Add new function apport.hookutils.root_command_output() which runs a command as root and returns its output, similar to the current command_output(). This will detect the currently used desktop environment and call gksu/kdesudo/sudo accordingly.

Symptoms

Symptom scripts are put into /usr/share/apport/symptoms/ and need to define a method run(report, ui). By a combination of interactive questions and automatic system tests this method needs to determine the affected package and return it.

It can optionally add information to the passed report object, such as tags. After that, Apport adds package related information and calls the package hooks as usual. Before run() is called, Apport already added the OS and user information to the report object.

Abort

If the user cancelled an interactive question for which the hook requires an answer, the hook should raise StopIteration, which will stop the bug reporting process.

Documentation

Fully document hooks, symptoms, and interactivity in doc/package-hooks.txt. Also explain how to set tags.

Test/Demo Plan

TBD when beta available.

CategorySpec

DesktopTeam/Specs/Karmic/SymptomBasedBugReporting (last edited 2009-08-31 09:06:04 by pD9EB58DE)