Page MenuHomePhabricator

Introduce bug life cycle documentation.
ClosedPublic

Authored by kristof.beyls on Oct 25 2018, 2:26 AM.

Details

Summary

Document what is expected during:

  • triaging
  • actively working on a bug
  • closing

Also document how we maintain:

  • product/component breakdown
  • default-cc lists per component

Diff Detail

Repository
rL LLVM

Event Timeline

kristof.beyls created this revision.Oct 25 2018, 2:26 AM
rnk added inline comments.Oct 25 2018, 11:31 AM
docs/BugLifeCycle.rst
68 ↗(On Diff #171053)

These seem to render as smart quotes. As long as it's UTF-8, this is probably fine, but they scare me. :) Up to you if you want to keep them.

103–104 ↗(On Diff #171053)

What about this scenario:

  • User reports bug with vague reproduction instructions
  • Bug languishes for one year
  • Developer does a pass over open bugs, tries to reproduce, but is unable to because there never was a clear, confirmed repro in the older version of llvm.

I think our policy should encourage us to close such bugs with some sort of "stale / instructions unclear / needsinfo" status along with a request for more information. The point is to put the bug into a state that says that no further action will be taken by LLVM developers unless the original reporter does something. For old bugs, chances are they won't.

I think as long as we encourage reporters to reopen if they have more info and can still reproduce the bug, this won't be discouraging to reporters. If the bug languished for a year, it's best to acknowledge that nothing happened and nothing is likely to happen without more information.

I believe this would be a better policy than asking developers to ping stale bugs for more info from the OP while leaving the bug open. That just leaves the bug open and means it will come up during the next bug scrub, requiring more developer time to close it out if there was no response.

Looking at our current resolution statuses, I guess I'd close bugs that we were never able to confirm were bugs with "WORKSFORME".

tstellar added inline comments.Oct 25 2018, 12:10 PM
docs/BugLifeCycle.rst
87–88 ↗(On Diff #171053)

The alternative would be using the "Assigned" and "New" bug states to indicate that a bug is assigned or not assigned. The downside of this is that it's an extra step that people might forget when assigning bugs to themselves, but the advantage is that we wouldn't need the pseudo accounts for unassigned bugs which make the bugs more invisible than if they have a real assignee.

If we do want to keep using the pseudo accounts, I think we should just pick one to use (unassignedbugs@nondot.org) and document that here.

rnk added inline comments.Oct 25 2018, 1:42 PM
docs/BugLifeCycle.rst
103–104 ↗(On Diff #171053)

My previous comment is my opinion, and I don't think it reflects the consensus of the last llvm-dev discussion we had about this, so don't let it block the document, which I think is important. We can keep discussing policy after we have a doc.

kristof.beyls added inline comments.Oct 26 2018, 7:42 AM
docs/BugLifeCycle.rst
68 ↗(On Diff #171053)

D'Oh - I'll fix that to use ASCII quotes.

87–88 ↗(On Diff #171053)

I think we could only remove the pseudo accounts and introduce real assignees if we could find volunteers for every single component that exists to become the default assignee. It seems uncertain to me we'll always have volunteers for that for all components, given the state we are in today.
Therefore, I think I want to opt for continuing to use the pseudo account for now. If we start having evidence that we can assume there will always be a volunteer to be default-assigned to for every component, we can reconsider this.
I agree that we only need a single pseudo account; not two (unassignedclangbugs@nondot.org is the other one I've seen. Not sure if there are more).

103–104 ↗(On Diff #171053)

Looking back at the last llvm-dev discussion, my impression is actually that most people were in favour of something along the lines of what you're proposing. Let me have a stab at concisely writing something up that hopefully encodes consensus.

RKSimon added inline comments.Oct 27 2018, 1:16 AM
docs/BugLifeCycle.rst
96 ↗(On Diff #171053)

Do we need to distinguish between resolving and closing a bug? We have verified/closed states but AFAICT these are barely used.

kristof.beyls added inline comments.Oct 28 2018, 1:26 AM
docs/BugLifeCycle.rst
96 ↗(On Diff #171053)

Good point.
As far as I can see, we don't really attach much difference in meaning to the "RESOLVED", "VERIFIED" and "CLOSED" states.
Therefore, my opinion right now is that we're probably best of to just remove the "RESOLVED" and "VERIFIED" states from our bugzilla instance and only keep the "CLOSED" state, making the workflows simpler.
I do think that this change can be made mostly orthogonally to completing this documentation. If/When we remove the "VERIFIED" and "RESOLVED" states, the documentation becomes less open for interpretation.
What do others think about this?

jhenderson added inline comments.
docs/BugLifeCycle.rst
46–47 ↗(On Diff #171053)

Maybe this should contain instructions on how to identify a good person to add as a CC? I imagine a good basis is the same as the people you'd add as reviewers for a change in the area.

96 ↗(On Diff #171053)

This is purely a personal opinion, but I would prefer to keep RESOLVED, above VERIFIED or CLOSED, partly because that's how I'm used to using bugzilla (I've always used RESOLVED). However, if consensus is against me on this, I don't mind. If we went down this route, you should probably change "closing" to "resolving" here and elsewhere.

kristof.beyls added inline comments.Oct 29 2018, 8:18 AM
docs/BugLifeCycle.rst
46–47 ↗(On Diff #171053)

I think we should aim to have at least one and ideally a few people by default added on CC for each component, as I expect that users that are not developers may find it hard to follow the guidelines we use for finding code reviewers (look at CODE_OWNERs; use svn/git blame on the code area affected).
Nonetheless, it may indeed be useful to say something along the lines of "If you know the area of LLVM code the root cause of the bug is in, good candidates to add as cc on the bug report may be the same people you'd ask for code review on that area. See Phabricator.html#requesting-a-review-via-the-web-interface for more details."

96 ↗(On Diff #171053)

I have no strong preference myself on RESOLVED vs CLOSED.
Having looked at the Bugzilla documentation for a few minutes just now on how to remove possible values for field "status", it seems that the "RESOLVED" value cannot be deleted, whereas "CLOSED" and "VERIFIED" can be deleted. Therefore, I think there is a strong technical argument to just decide that "RESOLVED" is the final bug status we retain, if we decide to only have 1, not 3 final bug statuses.

kristof.beyls marked 13 inline comments as done.Oct 29 2018, 8:59 AM
kristof.beyls added inline comments.
docs/BugLifeCycle.rst
96 ↗(On Diff #171053)

I have now updated the proposed documentation here as if we've decided to remove the VERIFIED and CLOSED states.
After looking into what it'd take to do so, it seems pretty straightforward for a bugzilla admin to remove the VERIFIED and CLOSED states.

103–104 ↗(On Diff #171053)

I've made an attempt at concisely writing up something the conveys the intent of what I believe the consensus is.

LGTM. Thanks for writting this up.

kristof.beyls marked 2 inline comments as done.

Add a sentence on statuses involved when triaging.

jhenderson accepted this revision.Oct 31 2018, 6:33 AM

LGTM too.

This revision is now accepted and ready to land.Oct 31 2018, 6:33 AM
Closed by commit rL346299: Introduce bug life cycle documentation. (authored by kbeyls, committed by ). · Explain WhyNov 7 2018, 12:52 AM
This revision was automatically updated to reflect the committed changes.