There’s been a bit of confusion about the point of the process for auto-resolving old UNCONFIRMED bugs. This is mostly my fault, as is what people see as the lack of notice – we’d been discussing this for so long that when it finally happened, I didn’t lift my eyes up enough to see that I should have made more noise about it than just posting on my blog. Apologies for that.
So, why are we doing it? The logic goes as follows:
- We don’t have enough manpower to triage all the bugs in Bugzilla. This is abundantly clear from the charts. So, we have to concentrate our efforts on a subset.
- Bugs rot. They become fixed by some other change, the product changes so the enhancement is no longer relevant, the reporter discovers it was his configuration, people change their minds about an enhancement, etc. etc.
- Statistical analysis showed that if a bug is not confirmed after three months, it’s highly unlikely to end up FIXED. We’ve got 96% of the value out of the bug pile by this point. So the remaining bugs, as a group, are the least useful bugs in the system.
- Given point 1, the bugs from point 3 are therefore the ones QA should not be spending time on. To triage them by hand is an inefficient use of time.
- However, we want to try and extract some value from those bugs also. So, the plan was to offload the work by getting the reporters to tell us if the bugs are still valid. Hence the auto-UNCO proposal.
So what does this mean?
- There are currently 18,000 UNCO bugs. We are doing the process on 12,500. If, say, 11,000 of them get closed, we’ve focussed QA effort in on what is roughly the best 2/5 of the UNCOs.
- If you are a longtime contributor or module owner, and you’ve got a lot of bugmail from this process, it’s not good to spend a lot of time carefully triaging each bug. The entire point of the process is to avoid that. If your pile is manageable and you want to whiz through them quickly, go for it. But don’t let this land you with a load of extra work.
- People will comment to say “this bug is still good”. If this happens, it’s a good thing, because it means that bug is far more worth spending triage time on than either other auto-UNCO bugs or even new UNCO bugs. It helps focus QA effort.
- Yes, the process may result in several bugs being eventually closed which are good reports of actual problems. But, given the lack of manpower, and the benefits of being able to focus on a small and better quality bug set, I think that’s a good trade-off. After all, without this, those bugs would have just sat there unlooked-at anyway.
- Once the process is complete, QA should spend time looking at the bugs people have commented on; because these are much more likely to be good than the ones people haven’t commented on.
One quick clarification: the bugs will be RESOLVED EXPIRED, so they are easy to find if you need to.
As a side effect, the implementation of the first part of the plan has (re)thrown up a need for workflow improvements to Bugzilla. Ideas for this are being tracked in the wiki. It’s now part of my day job to make developers’ lives easier, and so I’m going to be looking at those suggestions very carefully indeed, with a view to doing something about them. Please contribute to that page if you have ideas.
I want to have a good go at convincing skeptic module owners that this is a good thing. So email me if you think it isn’t, and I will. However, if I can’t convince you, then I will exclude your module(s) from the second part of the plan. After all, they are your modules.