Update developer policy on potentially breaking changes

[aaron.ballman]

Summary

We've recently had issues appropriately notifying users and stakeholders of changes to projects that may be potentially disruptive when upgrading. This led to discussion on Discourse about how to improve the situation, which can be found at: https://discourse.llvm.org/t/rfc-add-new-discourse-channel-for-potentially-breaking-disruptive-changes-for-clang/65251

Ultimately, it sounds like we want to encourage three things:

1. Alert vendors during the code review so they can provide early feedback on potentially breaking changes that would be unacceptable for them.
2. Alert vendors and users after committing the changes so they can perform pre-release testing on a completed change to determine if it causes unreasonable problems for them.
3. Alert users more clearly through the release notes so that it's easier to determine how disruptive an upgrade might be.

This updates the developer policy accordingly.

[aaron.ballman]

Note: I would love to add a vendors group to Phabricator for LLVM and lldb (and any other project we think has vendors), but this requires admin privileges to do. @mehdi_amini, is this something you could help out with assuming folks are in favor of the developer policy changes?

[aaron.ballman]

Adding someone from lldb as a review for awareness.

[ldionne]

Thanks for working on this! FWIW, this more or less standardizes what we've been doing in libc++ for the past ~1.5 years and it's been pretty low effort for us to do. I'm saying this just in case some folks are worried about "additional development overhead" -- in our experience there's basically none. And putting my vendor hat on, it's been extremely useful for me to track down potential issues when trying to ship a new version of libc++. This LGTM, although I'm somewhat neutral on whether to post on Discourse as well as having the vendor groups on Phabricator. I'm not sure I understand the benefit of doing both, but I will happily conform if folks see value in it.

llvm/docs/DeveloperPolicy.rst

140

I wonder whether Announcements is truly a lower-traffic alternative to vendors groups, if we end up posting each potentially breaking change to the list and tagging vendors on each such review. I'm not against posting on Discourse, however it seems to me like basically another equivalent channel of communication for these changes (which might be beneficial, I'm neutral on that).

[aaron.ballman]

llvm/docs/DeveloperPolicy.rst

140

The reason we have a split like that is for timing and chattiness. If you're a downstream like Intel has with ICX, you might want to be in clang-vendors so that you are involved in conversations about potentially breaking changes. You'll be getting emails for all review comments on that review. But if you're a downstream like a Gentoo package maintainer, you might not want to be *that* involved in the development of the compiler, but still want to know when changes are coming down the pipeline to do early pre-release testing while there's still time to put the brakes on before a release goes out.

[Mordante]

Thanks a lot for working on this! A few small nits, otherwise LGTM.

llvm/docs/DeveloperPolicy.rst

112

129

150

[aaron.ballman]

Applying review feedback.

llvm/docs/DeveloperPolicy.rst

129

I didn't apply this one because I think it's grammatically correct without the comma.

[Mordante]

llvm/docs/DeveloperPolicy.rst

129

I looks slightly odd to me, but I'm not a native speaker.

[ldionne]

llvm/docs/DeveloperPolicy.rst

140

Okay, makes sense. Let's go for it.

[aaron.ballman]

Pinging reviewers from projects other than libcxx (I'm hoping to get buy-in from someone on the LLVM side of things; lldb would be nice as well).

[aaron.ballman]

In D134878#3836840, @aaron.ballman wrote:

Pinging reviewers from projects other than libcxx (I'm hoping to get buy-in from someone on the LLVM side of things; lldb would be nice as well).

I'll be landing this later today on the assumption that folks are comfortable with the changes here, so if you have concerns, bringing them up now would be appreciated.

[mehdi_amini]

From what I saw when you posted the discourse thread initially, I understand you're targeting user-visible features, and mostly from the "toolchain" side of the project.
However I find the language here to be potentially confusing for the API surface of the libraries: that is how much of this applies to the LLVM C++ libraries API surface?
If this is out-of-scope, can this be called out more explicitly?

[aaron.ballman]

In D134878#3869947, @mehdi_amini wrote:

From what I saw when you posted the discourse thread initially, I understand you're targeting user-visible features, and mostly from the "toolchain" side of the project.
However I find the language here to be potentially confusing for the API surface of the libraries: that is how much of this applies to the LLVM C++ libraries API surface?
If this is out-of-scope, can this be called out more explicitly?

Sure! Would it help to add a paragraph before the bulleted list in breaking that says something along the lines of:

The C++ interfaces of individual library components of projects like LLVM or Clang are not intended to be stable interfaces. Potentially disruptive changes to such C++ interfaces do not constitute a breaking change unless the disruption is exposed via another mechanism such as a stable C API.

(Feel free to wordsmith it!)

llvm/docs/DeveloperPolicy.rst

 
 Code Reviews
 ------------
 
 LLVM has a code-review policy. Code review is one way to increase the quality of
 software. Please see :doc:`CodeReview` for more information on LLVM's code-review
 process.
 
+.. _breaking:
+
+Making Potentially Breaking Changes
+-----------------------------------
+
+Please help notify users and vendors of potential disruptions when upgrading to
+a newer version of a tool. For example, deprecating a feature that is expected
+to be removed in the future, removing an already-deprecated feature, upgrading a
+diagnostic from a warning to an error, switching important default behavior, or
+any other potentially disruptive situation thought to be worth raising
+awareness of. For such changes, the following should be done:
+
+* When performing the code review for the change, please add any applicable
+  "vendors" group to the review for their awareness. The purpose of these
+  groups is to give vendors early notice that potentially disruptive changes
+  are being considered but have not yet been accepted. Vendors can give early
+  testing feedback on the changes to alert us to unacceptable breakages. The
+  current list of vendor groups is:
+
+  * `Clang vendors <https://reviews.llvm.org/project/members/113/>`_
+  * `libc++ vendors <https://reviews.llvm.org/project/members/109/>`_
+
+  People interested in joining the vendors group can do so by clicking the
+  "Join Project" link on the vendor's "Members" page in Phabricator.
+
+* When committing the change to the repository, add appropriate information
+  about the potentially breaking changes to the ``Potentially Breaking Changes``
+  section of the project's release notes. The release note should have
+  information about what the change is, what is potentially disruptive about
+  it, as well as any code examples, links, and motivation that is appropriate
+  to share with users. This helps users to learn about potential issues with
+  upgrading to that release.
+
+* After the change has been committed to the repository, the potentially
+  disruptive changes described in the release notes should be posted to the
+  `Announcements <https://discourse.llvm.org/c/announce/>`_ channel on
+  Discourse. The post should be tagged with the ``potentially-breaking`` label
+  and a label specific to the project (such as ``clang``, ``llvm``, etc). This
+  is another mechanism by which we can give pre-release notice to users about
+  potentially disruptive changes. It is a lower-traffic alternative to the
+  joining "vendors" group. To automatically be notified of new announcements
+  with the ``potentially-breaking`` label, go to your user preferences page in
+  Discourse, and add the label to one of the watch categories under
+  ``Notifications->Tags``.
+
 .. _code owners:
 
 Code Owners
 -----------
 
 The LLVM Project relies on two features of its process to maintain rapid
 development in addition to the high quality of its source base: the combination
 of code review plus post-commit review for trusted maintainers.  Having both is

 * Adding, removing, or modifying command-line options.
 * Adding, removing, or regrouping a diagnostic.
 * Fixing a bug that potentially has significant user-facing impact (please link
   to the issue fixed in the bug database).
 * Adding or removing optimizations that have widespread impact or enables new
   programming paradigms.
 * Modifying a C stable API.
 * Notifying users about a potentially disruptive change expected to be made in
-  a future release, such as removal of a deprecated feature.
+  a future release, such as removal of a deprecated feature. In this case, the
+  release note should be added to a ``Potentially Breaking Changes`` section of
+  the notes with sufficient information and examples to demonstrate the
+  potential disruption. Additionally, any new entries to this section should be
+  announced in the `Announcements <https://discourse.llvm.org/c/announce/>`_
+  channel on Discourse. See :ref:`breaking` for more details.
 
 Code reviewers are encouraged to request a release note if they think one is
 warranted when performing a code review.
 
 Quality
 -------
 
 The minimum quality standards that any change must satisfy before being