CommunityBugReporting
|
Size: 2583
Comment:
|
← Revision 11 as of 2008-08-06 16:22:05 ⇥
Size: 7903
Comment: converted to 1.6 markup
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| [[Include(LoCoMenuHeader)]] | ''Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.'' |
| Line 5: | Line 5: |
| Go back to the main [:LoCoTeamsUDSMVSpecs] page | * '''Launchpad entry''': https://launchpad.net/distros/ubuntu/+spec/community-bug-reporting * '''Products affected''': malone, ubuntu-website |
| Line 7: | Line 8: |
| Effective bug reporting is central to the success of Ubuntu or any other project. Right now some people do indeed submit bug reports, but many enthusiasts and users still don't. The aim of this spec is to explore why users don't submit bug reports and try to develop a culture where bug reporting is a natural process that enthusiasts can do. We are unlikely to get regular users (such as the average dad) to submit bugs, but the huge community of Ubuntu enthusiasts are ideally placed to test and submit bugs. | <<Include(LoCoMenuHeader)>> [[LoCoTeamsUDSMVSpecs]] |
| Line 9: | Line 11: |
| Testing is the key here. If we want future versions of Ubuntu to work as best as possible, we need users to test hardware support ad features, and submit bugs when things don't work. This needs to happen ''before'' release where possible.. | == Summary == |
| Line 11: | Line 13: |
| This spec is intended to discuss ideas that can make this process easier. For this to happen we should work with the documentation, bug and marketing teams where possible. I think it would make sense to put together a team of people comprising people from these teams to coordinate how this works. | To increase the effectiveness of Ubuntu's bugtracking, we should get ForumAmbassadors to encourage forum participants in creating useful bug reports; establish "bug parties" for Lo``Co teams and user groups to help out with QA; provide a "Report a Bug..." menu item for programs in Ubuntu alphas and betas; and make various changes to Launchpad's bug reporting and report pages to encourage useful bug reports and other QA work. |
| Line 13: | Line 15: |
| Things to think about: | == Rationale == |
| Line 15: | Line 17: |
| == Better documentation == | Effective bug reporting and tracking is central to the success of software projects. Daniel Holbach expressed concern that Ubuntu does not have enough QA volunteers to keep up with reported bugs. So encouraging more bug reporters, without a corresponding increase in QA volunteers, would likely result in newly-reported bugs going for weeks or months without a response, which would give the overall bugtracking process a bad reputation. |
| Line 17: | Line 19: |
| Bug reports are useful, but well written bug reports are really what we want. We want users to know what to test, where to go and what kind of information to put into a bug report. We need two types of documentation here: | Jono Bacon and Simon Law agreed that we should, therefore, concentrate on training new QA volunteers before taking any major steps to encourage new bug reporters. However, we should still make it easier to report bugs for those people whose bug reports are already likely to be immediately useful. |
| Line 19: | Line 21: |
| * Quickshot guides - simple, concise, attractive guides to doing things. These guides should not only be easy to read and step by step, but be attractive and easy to find. * General manual style docs - details of the bug reporting process. |
== Scope == |
| Line 22: | Line 23: |
| Right now we have the latter, and partially the former, but we need to get the quickshot guides to ''everyone'' in the community. | We should target three groups of people: forum participants, Lo``Co team members, and the sort of enthusiasts who install pre-release versions. |
| Line 24: | Line 25: |
| == Awareness Campaign == | Out of scope are the establishment of a testing "group", standardized test scripts, or a testing checklist; these are all good ideas, but a different topic. |
| Line 26: | Line 27: |
| We need to market the fact that bug reporting is a fairly simple process and it ''really does help the community''. For this we really need an awareness campaign. | == Design == |
| Line 28: | Line 29: |
| == Making it happen == | === Forum participants === |
| Line 30: | Line 31: |
| Lets make this happen. To do this I recommend we take a representative from each team to coordinate how this project is run. Each representative can consult their team to satisfy their part of the project. | Many forum threads are created for troubleshooting something that turns out to be a bug, and sometimes these bugs don't get reported. (Simon Law estimates that 5~10% of bug reports currently originate in the forums.) |
| Line 32: | Line 33: |
| == Comments == | To get these bugs reported, the ForumAmbassadors "job description" should include watching for bug-related forum threads, and encouraging the originator to report a bug containing relevant debugging information (or report it themselves, if the originator doesn't want to and no further information is required). These bug reports should link back to the original forum thread, as it often contains useful debugging information. If this system works, forum contributors eventually should be encouraging each other to report bugs, without prompting by the forum ambassadors. |
| Line 34: | Line 35: |
| UbuntuDemon : IMHO an integrated ubuntuforums / launchpad login would be very nice. Often users with problems go to the forums. When we (the forum users) think the problem is a bug the user who reported the problem should be able to report this bug on launchpad/malone with as little effort as possible. | We considered having a forum subsection dedicated to reporting/triaging bugs, but UbuntuDemon disagreed with that idea. Often people posting about a problem will not initially realize that it is a bug. There may, however, be a forum section for forum users to contact forum ambassadors, and this may include asking for help in reporting a bug. === LoCo team members === Lo``Co teams and user groups should be encouraged to get involved with bug tracking. One way to do this is to organize "bug parties", where team members get together to eat snacks and to clean up bug reports (taking care not to [[http://eat.bees.net/|confuse the two]]). These would be analogous to Debian's [[http://wiki.debian.org/BSP|Bug Squashing Parties]]. Activities at bug parties could include: * Resuscitating initially-useless bug reports, by asking the reporter for the necessary information. * Finding and reporting (or fixing) translation bugs, for Lo``Cos in countries where any language other than US English is widely used. * Collaborating on reporting bugs, between English non-writers who experience bugs and English writers who can report them. * Passing around live CDs (or VM images?) of the latest development alpha/beta, so people can test all their hardware and report bugs if necessary. === Enthusiasts === Most Ubuntu users use release versions. For bug reporting purposes, these are less useful than the development version, for two reasons: * a bug in the release version may have been fixed in the development version * it is better for a bug to be reported and fixed before a release than after it. Therefore we should encourage those using the development version to report bugs. Ubuntu's Launchpad integration should be altered so that Help menus include a "Report a Bug..." item during the development process, with this item should disappear shortly before the final release. (Perhaps in future those using final releases should be provided with easy access to the support tracker). The Hug Days programme should also be continued, and expanded if possible, to get more enthusiasts involved in QA work. === Everyone === Some changes should be made to Launchpad to make bug reporting and tracking more effective. * On the bug reporting page, ask for the version number of the reporter's software. (Simon Law says this is his most common initial question on new bug reports.) * Manage expectations about bug reports ([[https://launchpad.net/bugs/37926|bug 37926]]), by providing an indication (either during, or immediately after, the bug is reported) of how likely it is to be fixed. For example: * a bug reported about an Ubuntu 6.06 Universe package probably won't be fixed * even a bad bug on a Restricted package ''can't'' be fixed. * Let developers/packagers specify what is "on their plate" right now. * Technically this is handled by the "In Progress" state, but maybe this can be made easier to set. * Provide reassurance, and reduce frustration, by representing activity in a bug report (such as changes in state) [[https://launchpad.net/products/malone/+bugs?field.searchtext=activity|more obviously and in more detail]]. * Make it easier to forward a bug report upstream -- for example, by linking from relevant Launchpad pages to the upstream product's bugtracker. * Make it more obvious when a bug is waiting for a fix from upstream, including introduction of the "Not For Us" status or an equivalent Importance level (''see'' [[https://launchpad.net/products/malone/+spec/bug-workflow|Bug workflow]]). * When someone reports a bug, invite them to help out further by getting involved in other parts of the QA process. * Provide more prominent feedback (such as statistics) about how your bug reports have helped Ubuntu. We should also encourage problem reports that do not have to be processed as intensively as bug reports do: ''see'' IncreaseHardwareDatabaseParticipation and CrashReporting. == Implementation == === Process === * Jono Bacon should work with Lo``Co team leaders to create and refine plans for Lo``Co bug parties. * Daniel Holbach should investigate the possibility of holding more Hug Days. === Code === * `launchpad-integration` should be adjusted to include the "Report a Bug..." item in non-final versions. * Launchpad's bug reporting form and bug page should be changed as described. == Alternative approaches == If we had enough QA volunteers, so that we could concentrate on increasing the number of bug reports, we could also set up an attractive "How to report a bug" page on ubuntu.com. This could: * provide a walkthrough on identifying bugs, emphasizing how to reproduce them * include anecdotes about how someone reporting a bug helped improve the software * perhaps have specific instructions on reporting (or even fixing!) translation problems * be very pretty, with help from the Art Team * be localized. We could also launch an awareness campaign, to market the fact that bug reporting is a fairly simple process and it really does help improve Ubuntu. |
Community Bug Reporting
Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad entry: https://launchpad.net/distros/ubuntu/+spec/community-bug-reporting
Products affected: malone, ubuntu-website
|
NEXT LOCO MEETING:
Summary
To increase the effectiveness of Ubuntu's bugtracking, we should get ForumAmbassadors to encourage forum participants in creating useful bug reports; establish "bug parties" for LoCo teams and user groups to help out with QA; provide a "Report a Bug..." menu item for programs in Ubuntu alphas and betas; and make various changes to Launchpad's bug reporting and report pages to encourage useful bug reports and other QA work.
Rationale
Effective bug reporting and tracking is central to the success of software projects. Daniel Holbach expressed concern that Ubuntu does not have enough QA volunteers to keep up with reported bugs. So encouraging more bug reporters, without a corresponding increase in QA volunteers, would likely result in newly-reported bugs going for weeks or months without a response, which would give the overall bugtracking process a bad reputation.
Jono Bacon and Simon Law agreed that we should, therefore, concentrate on training new QA volunteers before taking any major steps to encourage new bug reporters. However, we should still make it easier to report bugs for those people whose bug reports are already likely to be immediately useful.
Scope
We should target three groups of people: forum participants, LoCo team members, and the sort of enthusiasts who install pre-release versions.
Out of scope are the establishment of a testing "group", standardized test scripts, or a testing checklist; these are all good ideas, but a different topic.
Design
Forum participants
Many forum threads are created for troubleshooting something that turns out to be a bug, and sometimes these bugs don't get reported. (Simon Law estimates that 5~10% of bug reports currently originate in the forums.)
To get these bugs reported, the ForumAmbassadors "job description" should include watching for bug-related forum threads, and encouraging the originator to report a bug containing relevant debugging information (or report it themselves, if the originator doesn't want to and no further information is required). These bug reports should link back to the original forum thread, as it often contains useful debugging information. If this system works, forum contributors eventually should be encouraging each other to report bugs, without prompting by the forum ambassadors.
We considered having a forum subsection dedicated to reporting/triaging bugs, but UbuntuDemon disagreed with that idea. Often people posting about a problem will not initially realize that it is a bug. There may, however, be a forum section for forum users to contact forum ambassadors, and this may include asking for help in reporting a bug.
LoCo team members
LoCo teams and user groups should be encouraged to get involved with bug tracking. One way to do this is to organize "bug parties", where team members get together to eat snacks and to clean up bug reports (taking care not to confuse the two). These would be analogous to Debian's Bug Squashing Parties.
Activities at bug parties could include:
- Resuscitating initially-useless bug reports, by asking the reporter for the necessary information.
Finding and reporting (or fixing) translation bugs, for LoCos in countries where any language other than US English is widely used.
- Collaborating on reporting bugs, between English non-writers who experience bugs and English writers who can report them.
- Passing around live CDs (or VM images?) of the latest development alpha/beta, so people can test all their hardware and report bugs if necessary.
Enthusiasts
Most Ubuntu users use release versions. For bug reporting purposes, these are less useful than the development version, for two reasons:
- a bug in the release version may have been fixed in the development version
- it is better for a bug to be reported and fixed before a release than after it.
Therefore we should encourage those using the development version to report bugs. Ubuntu's Launchpad integration should be altered so that Help menus include a "Report a Bug..." item during the development process, with this item should disappear shortly before the final release. (Perhaps in future those using final releases should be provided with easy access to the support tracker).
The Hug Days programme should also be continued, and expanded if possible, to get more enthusiasts involved in QA work.
Everyone
Some changes should be made to Launchpad to make bug reporting and tracking more effective.
- On the bug reporting page, ask for the version number of the reporter's software. (Simon Law says this is his most common initial question on new bug reports.)
Manage expectations about bug reports (bug 37926), by providing an indication (either during, or immediately after, the bug is reported) of how likely it is to be fixed. For example:
- a bug reported about an Ubuntu 6.06 Universe package probably won't be fixed
even a bad bug on a Restricted package can't be fixed.
- Let developers/packagers specify what is "on their plate" right now.
- Technically this is handled by the "In Progress" state, but maybe this can be made easier to set.
Provide reassurance, and reduce frustration, by representing activity in a bug report (such as changes in state) more obviously and in more detail.
- Make it easier to forward a bug report upstream -- for example, by linking from relevant Launchpad pages to the upstream product's bugtracker.
Make it more obvious when a bug is waiting for a fix from upstream, including introduction of the "Not For Us" status or an equivalent Importance level (see Bug workflow).
- When someone reports a bug, invite them to help out further by getting involved in other parts of the QA process.
- Provide more prominent feedback (such as statistics) about how your bug reports have helped Ubuntu.
We should also encourage problem reports that do not have to be processed as intensively as bug reports do: see IncreaseHardwareDatabaseParticipation and CrashReporting.
Implementation
Process
Jono Bacon should work with LoCo team leaders to create and refine plans for LoCo bug parties.
- Daniel Holbach should investigate the possibility of holding more Hug Days.
Code
launchpad-integration should be adjusted to include the "Report a Bug..." item in non-final versions.
- Launchpad's bug reporting form and bug page should be changed as described.
Alternative approaches
If we had enough QA volunteers, so that we could concentrate on increasing the number of bug reports, we could also set up an attractive "How to report a bug" page on ubuntu.com. This could:
- provide a walkthrough on identifying bugs, emphasizing how to reproduce them
- include anecdotes about how someone reporting a bug helped improve the software
- perhaps have specific instructions on reporting (or even fixing!) translation problems
- be very pretty, with help from the Art Team
- be localized.
We could also launch an awareness campaign, to market the fact that bug reporting is a fairly simple process and it really does help improve Ubuntu.
LoCoTeamsUDSMVSpecs/CommunityBugReporting (last edited 2008-08-06 16:22:05 by localhost)