[libc++][doc] Updates the tasks to do for a release.

[Mordante]

Summary

This is a followup of the review comments in D144499.

[philnik]

Thanks for working on improving this!

libcxx/docs/ReleaseProcedure.rst

12

Maybe also check that GCC is on the latest version?

85–108

I don't think we want to list the removals here. We already have the release notes for that.

88–95

Where is this coming from?

[Mordante]

libcxx/docs/ReleaseProcedure.rst

12

I considered that, but left it out on purpose. GCC's release schedule is between releases so there is no nice match with our releases.

Reconsidering it might make sense to do that in the preparation. For Unicode I also don't intend to wait until we prepare the LLVM release, instead I intend to update once I'm aware of the release. That way we at least won't forget about it.

85–108

We have, but somehow things from the list are removed by the release managers after branching. Based on your char_traits question and D150580 we need to have a better way to keep track of this. I'm open to other suggestions.

88–95

https://releases.llvm.org/16.0.0/projects/libcxx/docs/ReleaseNotes.html#deprecations-and-removals

[philnik]

libcxx/docs/ReleaseProcedure.rst

12

Yeah, I think it's fine to keep the release preparation phase a bit hand-wavy.

72

Could we also define a single TODO LLVM xy style to make it easier to grep for that?

85–108

Wouldn't it be enough to communicate to the release managers that they shouldn't touch the Upcoming Deprecations and Removals list? Or maybe move that section into a different file that is included there? (IDK if that's possible with Sphinx) I don't think we had that section when _LIBCPP_ABI_OLD_LOGNORMAL_DISTRIBUTION was introduced, and I think the section already massively helps with removing stuff in-time.

88–95

Could you re-add this to the release notes?

[ldionne]

Thanks for working on this!

libcxx/docs/ReleaseProcedure.rst

34

43

If we have annotations like LLVM RELEASE bump, does it make sense to carry that list around? Duplicating the information IMO is going to create confusion and they'll get out of sync, so I think we should pick one.

CC @tstellar

72

I like that! So the documentation could say "Search for TODO(LLVM-<version>)" maybe? I don't care about how it's spelled, but we could document it right here and start doing it right away.

88–95

Instead, I think what we should do here is have version-specific release notes and keep the latest N versions in the repo (probably N = 1). That would actually solve other problems as well, such as merge conflicts during the early branching when we cherry-pick stuff to the release branch.

Then we could easily obtain this information without having to copy-paste anything, which is going to be error-prone.

[Mordante]

Addresses review comments and use new style TODO for existing items.

[philnik]

LGTM % nit! Leaving final approval to Louis.

libcxx/utils/libcxx/test/features.py

29

Same below.

[ldionne]

libcxx/docs/ReleaseProcedure.rst

80–89

I am not sure that we should keep this section now.

[Mordante]

libcxx/docs/ReleaseProcedure.rst

12

I added a note, but more generic for all our compilers.

43

I agree if we have that we can link that, just to make it easier to find for us.

I originally copied this from the contribution page.

72

I used your suggestion and adjusted the existing TODOs.

80–89

This is indeed an unwanted left over.

85–108

For now I just added an item for us to copy it back.

88–95

For now I moved it to the release notes under a LLVM 18 section. I think it would make sense to do

Upcoming Deprecations and Removals
----------------------------------

LLVM 18
~~~~~~~

... list of items

LLVM 19

... list of items

I think we can just copy that after the release managers removed it then we don't need to keep N files.

libcxx/docs/Contributing.rst

 mention it and provide your ``Name <email@domain>`` for us to attribute the commit properly.
 
 Note that the rule for accepting as the "libc++" review group is to wait for two members
 of the group to have approved the patch, excluding the patch author. This is not a hard
 rule -- for very simple patches, use your judgement. The `"libc++" review group <https://reviews.llvm.org/project/members/64/>`__
 consists of frequent libc++ contributors with a good understanding of the project's
 guidelines -- if you would like to be added to it, please reach out on Discord.
 
-Post-release check list
-=======================
-
-After branching for an LLVM release:
-
-1. Update ``_LIBCPP_VERSION`` in ``libcxx/include/__config``
-2. Update the version number in ``libcxx/docs/conf.py``
-3. Update ``_LIBCPPABI_VERSION`` in ``libcxxabi/include/cxxabi.h``
-4. Update ``_LIBUNWIND_VERSION`` in ``libunwind/include/__libunwind_config.h``
-5. Update the list of supported clang versions in ``libcxx/docs/index.rst``
-6. Remove the in-progress warning from ``libcxx/docs/ReleaseNotes.rst``
-
 Exporting new symbols from the library
 ======================================
 
 When exporting new symbols from libc++, you must update the ABI lists located in ``lib/abi``.
 To test whether the lists are up-to-date, please run the target ``check-cxx-abilist``.
 To regenerate the lists, use the target ``generate-cxx-abilist``.
 The ABI lists must be updated for all supported platforms; currently Linux and
 Apple.  If you don't have access to one of these platforms, you can download an

libcxx/docs/ReleaseNotes.rst

- Libc++ used to provide some C++11 tag type global variables in C++03 as an extension, which are removed in
   which led to ODR violations when mixed with the C++03 definition. Cleaning this up is required for libc++ to
   make progress on support for C++20 modules.
 
 - The ``_LIBCPP_ABI_OLD_LOGNORMAL_DISTRIBUTION`` macro has been removed.
 
 Upcoming Deprecations and Removals
 ----------------------------------
 
+LLVM 18
+~~~~~~~
+
+- The base template for ``std::char_traits`` has been marked as deprecated and
+  will be removed in LLVM 18. If you are using ``std::char_traits`` with types
+  other than ``char``, ``wchar_t``, ``char8_t``, ``char16_t``, ``char32_t`` or
+  a custom character type for which you specialized ``std::char_traits``, your code
+  will stop working when we remove the base template. The Standard does not
+  mandate that a base template is provided, and such a base template is bound
+  to be incorrect for some types, which could currently cause unexpected
+  behavior while going undetected.
+
 - The ``_LIBCPP_AVAILABILITY_CUSTOM_VERBOSE_ABORT_PROVIDED`` macro will not be honored anymore in LLVM 18.
   Please see the updated documentation about the safe libc++ mode and in particular the ``_LIBCPP_VERBOSE_ABORT``
   macro for details.
 
 - The headers ``<experimental/deque>``, ``<experimental/forward_list>``, ``<experimental/list>``,
   ``<experimental/map>``, ``<experimental/memory_resource>``, ``<experimental/regex>``, ``<experimental/set>``,
   ``<experimental/string>``, ``<experimental/unordered_map>``, ``<experimental/unordered_set>``,
   and ``<experimental/vector>`` will be removed in LLVM 18, as all their contents will have been implemented in

libcxx/docs/ReleaseProcedure.rst

+.. _ReleaseProcedure:
+
+=================
+Release procedure
+=================
+
+The LLVM project creates a new release twice a year following a fixed
+`schedule <https://llvm.org/docs/HowToReleaseLLVM.html#annual-release-schedule>`__.
+This page describes the libc++ procedure for that release.
+
+Prepare the release
+===================
+
+This is done by the libc++ developers.
+
+It should be finished before the Release managers start branching the new
+release:
+
+* Make sure the ``libcxx/docs/ReleaseNotes.rst`` is up to date. Typically
+  this file is updated when contributing patches. Still there might be some
+  information added regarding the general improvements of larger projects.
+
+* Make sure the deprecated features on this page are up to date. Typically a
+  new deprecated feature should be added to the release notes and this page.
+  However this should be verified so removals won't get forgotten.
+
+* Make sure the latest Unicode version is used. The C++ Standard
+  `refers to the Unicode Standard <https://wg21.link/intro.refs#1.10>`__
+
+  ``The Unicode Consortium. The Unicode Standard. Available from: https://www.unicode.org/versions/latest/``
+
+  Typically the Unicode Consortium has one release per year. The libc++
+  format library uses the Unicode Standard. Libc++ should be updated to the
+  latest Unicode version. Updating means using the latest data files and, if
+  needed, adapting the code to changes in the Unicode Standard.
+
+* Make sure all libc++ supported compilers in the CI are updated to their
+  latest release.
+
+Branching
+=========
+
+This is done by the LLVM Release managers.
+
+After branching for an LLVM release:
+
+1. Update ``_LIBCPP_VERSION`` in ``libcxx/include/__config``
+2. Update the version number in ``libcxx/docs/conf.py``
+3. Update ``_LIBCPPABI_VERSION`` in ``libcxxabi/include/cxxabi.h``
+4. Update ``_LIBUNWIND_VERSION`` in ``libunwind/include/__libunwind_config.h``
+5. Remove the in-progress warning from ``libcxx/docs/ReleaseNotes.rst``
+
+Post branching
+==============
+
+This is done by the libc++ developers.
+
+After branching it takes a couple of days before the new LLVM ToT version is
+available on `<https://apt.llvm.org>`_. Once it is available the pre-commit CI
+can start using the new ToT version. In order to make sure patches can be
+backported to the release branch the oldest compiler is not removed yet.
+
+The section ``Upcoming Deprecations and Removals`` is cleared by the release
+managers. Copy back the items that were in this section.
+
+The items that need changing are marked with ``LLVM POST-BRANCH``.
+
+Post release
+============
+
+This is done by the libc++ developers.
+
+Support for the ToT - 3 version is removed:
+
+- Search for ``LLVM RELEASE`` and address their comments
+- Search for test that have ``UNSUPPORTED`` or ``XFAIL`` for the no longer supported version
+- Search for ``TODO(LLVM-<ToT>)`` and address their comments

libcxx/docs/index.rst

 .. toctree::
    :maxdepth: 1
 
    ReleaseNotes
    UsingLibcxx
    BuildingLibcxx
    TestingLibcxx
    Contributing
+   ReleaseProcedure
    Status/Cxx14
    Status/Cxx17
    Status/Cxx20
    Status/Cxx23
    Status/Format
    Status/Parallelism
    Status/Ranges
    Status/Spaceship
* STLport and the Apache libstdcxx library are two other popular
   experience of libstdc++ developers) is that adding support for C++11 (in
   particular rvalue references and move-only types) requires changes to
   almost every class and function, essentially amounting to a rewrite.
   Faced with a rewrite, we decided to start from scratch and evaluate every
   design decision from first principles based on experience.
   Further, both projects are apparently abandoned: STLport 5.2.1 was
   released in Oct'08, and STDCXX 4.2.1 in May'08.
 
+..
+  LLVM RELEASE bump version
+
 .. _SupportedPlatforms:
 
 Platform and Compiler Support
 =============================
 
 Libc++ aims to support common compilers that implement the C++11 Standard. In order to strike a
 good balance between stability for users and maintenance cost, testing coverage and development
 velocity, libc++ drops support for older compilers as newer ones are released.

libcxx/include/__memory/allocator_traits.h

 #endif
 
 #ifdef _LIBCPP_HAS_ASAN_CONTAINER_ANNOTATIONS_FOR_ALL_ALLOCATORS
 template <class _Alloc>
 struct __asan_annotate_container_with_allocator
 #   if defined(_LIBCPP_CLANG_VER) && _LIBCPP_CLANG_VER >= 1600
       : true_type {};
 #   else
-      // TODO LLVM18: Remove the special-casing
+      // TODO(LLVM-18): Remove the special-casing
       : false_type {};
 #   endif
 
 template <class _Tp>
 struct __asan_annotate_container_with_allocator<allocator<_Tp> > : true_type {};
 #endif
 
 #undef _LIBCPP_ALLOCATOR_TRAITS_HAS_XXX
 
 _LIBCPP_END_NAMESPACE_STD
 
 _LIBCPP_POP_MACROS
 
 #endif // _LIBCPP___MEMORY_ALLOCATOR_TRAITS_H

libcxx/include/__string/char_traits.h

exposition-only to document what members a char_traits specialization should provide:
     static int_type  to_int_type(char_type);
     static bool      eq_int_type(int_type, int_type);
     static int_type  eof();
 };
 */
 
 //
 // Temporary extension to provide a base template for std::char_traits.
-// TODO: Remove in LLVM 18.
+// TODO(LLVM-18): Remove this class.
 //
 template <class _CharT>
 struct _LIBCPP_DEPRECATED_("char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it.")
     char_traits
 {
     using char_type  = _CharT;
     using int_type   = int;
     using off_type   = streamoff;

libcxx/test/libcxx/containers/sequences/vector/asan.pass.cpp

 
 void do_exit() {
   exit(0);
 }
 
 int main(int, char**)
 {
 #if TEST_STD_VER >= 11 && TEST_CLANG_VER >= 1600
-  // TODO LLVM18: Remove the special-casing
+  // TODO(LLVM-18): Remove the special-casing
   {
     typedef int T;
     typedef cpp17_input_iterator<T*> MyInputIter;
     std::vector<T, min_allocator<T>> v;
     v.reserve(1);
     int i[] = {42};
     v.insert(v.begin(), MyInputIter(i), MyInputIter(i + 1));
     assert(v[0] == 42);

libcxx/utils/ci/Dockerfile

 
 # Install Clang <latest>, <latest-1> and ToT, which are the ones we support.
 # We also install <latest-2> because we need to support the "latest-1" of the
 # current LLVM release branch, which is effectively the <latest-2> of the
 # tip-of-trunk LLVM. For example, after branching LLVM 14 but before branching
 # LLVM 15, we still need to have Clang 12 in this Docker image because the LLVM
 # 14 release branch CI uses it. The tip-of-trunk CI will never use Clang 12,
 # though.
+# LLVM POST-BRANCH bump version
 ENV LLVM_HEAD_VERSION=17
 RUN apt-get update && apt-get install -y lsb-release wget software-properties-common
 RUN wget https://apt.llvm.org/llvm.sh -O /tmp/llvm.sh
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 3)) # for CI transitions
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 2)) # previous release
 RUN bash /tmp/llvm.sh $(($LLVM_HEAD_VERSION - 1)) # latest release
 RUN bash /tmp/llvm.sh $LLVM_HEAD_VERSION          # current ToT
 
 # Install clang-scan-deps, which is required to build modules; always all supported versions.
 RUN apt-get update && apt-get install -y clang-tools-$(($LLVM_HEAD_VERSION - 3)) \
                                          clang-tools-$(($LLVM_HEAD_VERSION - 2)) \
                                          clang-tools-$(($LLVM_HEAD_VERSION - 1)) \
                                          clang-tools-$LLVM_HEAD_VERSION
 
 # Install clang-format; always use the lastest stable branch.
 RUN apt-get update && apt-get install -y clang-format-$(($LLVM_HEAD_VERSION - 2)) clang-format-$(($LLVM_HEAD_VERSION - 1))
 
 # Install clang-tidy
-# TODO LLVM 17 revert D148831 to only install $(($LLVM_HEAD_VERSION - 1)) and $LLVM_HEAD_VERSION
+# TODO(LLVM-17) revert D148831 to only install $(($LLVM_HEAD_VERSION - 1)) and $LLVM_HEAD_VERSION
 # The usage of the ToT version is needed due to module issues with Clang 16
 RUN apt-get update && apt-get install -y clang-tidy-$(($LLVM_HEAD_VERSION - 2)) clang-tidy-$(($LLVM_HEAD_VERSION - 1)) clang-tidy-$LLVM_HEAD_VERSION
 
 # Install llvm-dev and libclang-dev to compile custom clang-tidy checks
-# TODO LLVM 17 revert D148831 to only install $(($LLVM_HEAD_VERSION - 1)) and $LLVM_HEAD_VERSION
+# TODO(LLVM-17) revert D148831 to only install $(($LLVM_HEAD_VERSION - 1)) and $LLVM_HEAD_VERSION
 # The usage of the ToT version is needed due to module issues with Clang 16
 RUN apt-get update && apt-get install -y llvm-$(($LLVM_HEAD_VERSION - 2))-dev llvm-$(($LLVM_HEAD_VERSION - 1))-dev     llvm-$LLVM_HEAD_VERSION-dev \
                                      libclang-$(($LLVM_HEAD_VERSION - 2))-dev libclang-$(($LLVM_HEAD_VERSION - 1))-dev libclang-$LLVM_HEAD_VERSION-dev
 
 # Install the most recent GCC, like clang install the previous version as a transition.
 ENV GCC_LATEST_VERSION=13
 RUN add-apt-repository ppa:ubuntu-toolchain-r/test
 RUN apt-get update && apt install -y gcc-$((GCC_LATEST_VERSION - 1)) g++-$((GCC_LATEST_VERSION - 1))

libcxx/utils/ci/buildkite-pipeline-clang.yml

 #===----------------------------------------------------------------------===##
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 #===----------------------------------------------------------------------===##
 
 #
 # This file describes the various pre-commit CI bots used to test Clang against
 # libc++ under various configurations. Unlike the usual libc++ CI pipeline,
 # which aims to test libc++ itself, this pipeline aims to test Clang by
 # compiling libc++ and running its test suite against the just-built Clang,
 # in various configurations.
 #
 env:
+    # LLVM RELEASE bump version
     LLVM_HEAD_VERSION: "17"
 steps:
   - label: "Building clang"
     commands:
       - "mkdir install"
       # We use Release here to avoid including debug information. Otherwise, the clang binary is very large, which
       # is problematic because we need to upload the artifacts for other jobs to use. This may seem like nothing,
       # but with the number of jobs we run daily, this can result in thousands of GB of network I/O.

libcxx/utils/ci/buildkite-pipeline.yml

 #
 
 # The Linux CI runners use the nightly ToT build provided by the Docker image.
 # (Note the image isn't updated daily.) The LLVM_HEAD_VERSION contains that
 # version number. The Linux CI runners for GCC use the latest stable version.
 # Theses numbers are available in all runners, making it easier to update the
 # version number.
 env:
+    # LLVM POST-BRANCH bump version
+    # LLVM POST-BRANCH add compiler test for ToT - 1, e.g. "Clang 17"
+    # LLVM RELEASE bump remove compiler ToT - 3, e.g. "Clang 15"
     LLVM_STABLE_VERSION: "16"
     LLVM_HEAD_VERSION: "17"
     GCC_STABLE_VERSION: "12"
 steps:
   #
   # Light pre-commit tests for things like formatting or when people forget
   # to update generated files.
   #
steps:
   - label: "Clang 15"
     command: "libcxx/utils/ci/run-buildbot generic-cxx23"
     artifact_paths:
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-15"
         CXX: "clang++-15"
-        # TODO LLVM18: Enable clang-tidy
+        # TODO(LLVM-18): Enable clang-tidy
         # ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2

libcxx/utils/generate_header_tests.py

header_restrictions = {
     "mutex": "!defined(_LIBCPP_HAS_NO_THREADS)",
     "semaphore": "!defined(_LIBCPP_HAS_NO_THREADS)",
     "shared_mutex": "!defined(_LIBCPP_HAS_NO_THREADS)",
     "stdatomic.h": "__cplusplus > 202002L && !defined(_LIBCPP_HAS_NO_THREADS)",
     "thread": "!defined(_LIBCPP_HAS_NO_THREADS)",
 
     "filesystem": "!defined(_LIBCPP_HAS_NO_FILESYSTEM_LIBRARY)",
 
-    # TODO LLVM17: simplify this to __cplusplus >= 202002L
+    # TODO(LLVM-17): simplify this to __cplusplus >= 202002L
     "coroutine": "(defined(__cpp_impl_coroutine) && __cpp_impl_coroutine >= 201902L) || (defined(__cpp_coroutines) && __cpp_coroutines >= 201703L)",
 
     "clocale": "!defined(_LIBCPP_HAS_NO_LOCALIZATION)",
     "codecvt": "!defined(_LIBCPP_HAS_NO_LOCALIZATION)",
     "fstream": "!defined(_LIBCPP_HAS_NO_LOCALIZATION) && !defined(_LIBCPP_HAS_NO_FSTREAM)",
     "iomanip": "!defined(_LIBCPP_HAS_NO_LOCALIZATION)",
     "ios": "!defined(_LIBCPP_HAS_NO_LOCALIZATION)",
     "iostream": "!defined(_LIBCPP_HAS_NO_LOCALIZATION)",

libcxx/utils/libcxx/test/features.py

 
 def _getSuitableClangTidy(cfg):
   try:
     # If we didn't build the libcxx-tidy plugin via CMake, we can't run the clang-tidy tests.
     if runScriptExitCode(cfg, ['stat %{test-tools}/clang_tidy_checks/libcxx-tidy.plugin']) != 0:
       return None
 
     # TODO This should be the last stable release.
+    # LLVM RELEASE bump to latest stable version
     if runScriptExitCode(cfg, ['clang-tidy-16 --version']) == 0:
       return 'clang-tidy-16'
 
+    # LLVM RELEASE bump version
     if int(re.search('[0-9]+', commandOutput(cfg, ['clang-tidy --version'])).group()) >= 16:
       return 'clang-tidy'
 
   except ConfigurationRuntimeError:
     return None
 
 DEFAULT_FEATURES = [
   Feature(name='thread-safety',