What makes for a good bug (report)? The usual responses might refer to the need for supporting information like software version, operating system, error messages, backtraces, etc. While I agree, my question is more basic: what’s a good problem to file as a bug?
I think it makes sense to work backward from the possible outcomes that your bug system provides. My system (a version of bugzilla) has these:
FIXED – A fix for this bug is checked into the tree and tested.
INVALID – The problem described is not a bug
WONTFIX – The problem described is a bug which will never be fixed.
LATER – The problem described is a bug which will not be fixed in this version of the product.
REMIND – The problem described is a bug which will probably not be fixed in this version of the product, but might still be.
DUPLICATE – The problem is a duplicate of an existing bug. [..]
WORKSFORME – All attempts at reproducing this bug were futile [..]
PUSHED – [..] Another form of FIXED. In the development context, FIXED means that the change has been commited.
If these are the possible outcomes, then one possible definition of a good bug is something that:
1) Might ultimately be resolved into one and only one outcome (otherwise why file in to this system?), and
2) (at the time of filing) is not clearly destined for one and only one outcome (otherwise, why ask for an investigation?)
So say we have the following bug:
Bug 666666: Peace on Earth and good will toward men.
Hmm. Currently, it’s definitely not FIXED, not PUSHED, and not INVALID. WONTFIX? A bit pessimistic that, in the long term. DUPLICATE? Undoubtedly, but let’s assume we’re dealing with the first one. WORKSFORME? Not at all.
What’s left? LATER, and REMIND. Now, these are actually good responses. Yeah, that’s it — we’re going to get to that one LATER, and if by any chance we forget, then do REMIND us. But you get the feeling that these categories are a bit weaselly, and in fact may have been added _because_ people weren’t filing good bugs. So let’s set them aside, and decide that bug 66666 is still open.
OK, if a good bug is one that might be resolved into one of the non-weaselly categories, then what would it mean to close bug 666666? I guess that there would be no disturbances of the peace (at all) and no failures of good will (at all). You see the problem. Even if we were very very pleased indeed with progress on this front (harmony in the Middle East, etc.), we couldn’t really close it.
So this one we’re likely to leave open for a while. But this is discouraging, isn’t it? Every day you come in to work, and you see your bug list (“Good will toward men”, “Cancer cure”, “Affordable anti-gravity device”), and it’s really hard to see which one you’re going to knock off before lunch. And then there are the ones that seem achievable, but annoyingly vague (“Less spam in search results”). Where to begin?
So it isn’t just that some bugs fit the paradigm less well than others – it’s that bugs that don’t fit the paradigm begin to kind of pollute the workflow and drag down morale. The problem isn’t with quality control on the code; it’s with quality control on the _bugs_. Bugs need to be fully-baked, crisply-defined, bite-sized, toasty little nuggets of unitary progress to be useful.
So it seems like resolutions should include more filer feedback beyond the somewhat surly and uninformative alternatives of WONTFIX and LATER. How about these:
BREAKITDOWN – This bug seems to be multiple bugs in one — refile individually
SCALEITBACK – This bug is breathtaking in its ambition and scope — try a version for humans
DEFINESUCCESS – This bug doesn’t seem to have an endpoint. Refile with a description of what it would mean to close the bug
(The last of these (DEFINESUCCESS) might also be a good entry field for all bugs filed, even if freetext and optional: describe what it would mean to you (and only to you) if this bug were fixed. What would a test look like?)