[libc++][C++20 modules] Enabling in C++20.

[Mordante]

Summary

The vendors of the MSVC STL, libstdc++ and libc++ have agreed [1] to
make the C++23 modules std and std.compat available in C++20. This
provides the std module; libc++ has not implemented the std.compat
module yet.

[1] https://github.com/microsoft/STL/issues/3945

Depends on D158357

[Mordante]

Rebased and minor rework to match other patches.

[Zingam]

@Mordante This is really nice. Is it landing soon? The ranges patches and maybe some others will need to be updated accordingly after this lands.

[Mordante]

In D158358#4619359, @Zingam wrote:

@Mordante This is really nice. Is it landing soon? The ranges patches and maybe some others will need to be updated accordingly after this lands.

I hope next week, but it depends on when it gets reviewed. If the ranges patches lands first I'll adjust this patch.

[ldionne]

LGTM with nits, thanks!

libcxx/docs/UsingLibcxx.rst

526–527

We should only discuss *our* implementation here. I don't think this documentation is the right place to make claims about other implementations, even though we've all agreed on this. For example, if they changed their minds, this documentation would become erroneous.

You could say something more vague like "Like other major implementations, libc++ provides C++23 modules in C++20 as an extension". Let's just not be specific.

libcxx/modules/std/vector.inc

33

IMO many of these #endif // comments are not really helpful since they are very short blocks. I find it adds clutter to the code more than anything. Non-blocking, but I would consider removing the comments for short blocks.

libcxx/docs/Modules.rst

* The following "parts disabled" configuration options are supported
 
    * ``LIBCXX_ENABLE_LOCALIZATION``
    * ``LIBCXX_ENABLE_WIDE_CHARACTERS``
    * ``LIBCXX_ENABLE_THREADS``
    * ``LIBCXX_ENABLE_FILESYSTEM``
    * ``LIBCXX_ENABLE_RANDOM_DEVICE``
    * ``LIBCXX_ENABLE_UNICODE``
 
+ * A C++20 based extension
+
 Some of the current limitations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
  * There is no official build system support, libc++ has experimental CMake support
+ * Requires CMake 3.26 for C++20 support
  * Requires CMake 3.26 for C++23 support
  * Requires CMake 3.27 for C++26 support
  * Requires Ninja 1.11
  * Requires a recent Clang 17
  * The path to the compiler may not be a symlink, ``clang-scan-deps`` does
    not handle that case properly
  * Only C++23 and C++26 are tested
  * Libc++ is not tested with modules instead of headers

libcxx/docs/UsingLibcxx.rst

 Standard. Libc++ retroactively updates the Unicode Standard in older C++
 versions. This allows the library to have better estimates for newly introduced
 Unicode code points, without requiring the user to use the latest C++ version
 in their code base.
 
 In C++26 formatting pointers gained a type ``P`` and allows to use
 zero-padding. These options have been retroactively applied to C++20.
 
+Extensions to the C++23 modules ``std`` and ``std.compat``
+----------------------------------------------------------
+
+Like other major implementations, libc++ provides C++23 modules ``std`` and
+``std.compat`` in C++20 as an extension"
+
 .. _turning-off-asan:
 
 Turning off ASan annotation in containers
 -----------------------------------------
 
 ``__asan_annotate_container_with_allocator`` is a customization point to allow users to disable
 `Address Sanitizer annotations for containers <https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow>`_ for specific allocators. This may be necessary for allocators that access allocated memory.
 This customization point exists only when ``_LIBCPP_HAS_ASAN_CONTAINER_ANNOTATIONS_FOR_ALL_ALLOCATORS`` Feature Test Macro is defined.

libcxx/modules/std/algorithm.inc

#endif
 
   using std::search_n;
 
   namespace ranges {
     using std::ranges::search_n;
   }
 
   namespace ranges {
+#if _LIBCPP_STD_VER >= 23
     // [alg.starts.with], starts with
     using std::ranges::starts_with;
 
-#if 0
+#  if 0
     // [alg.ends.with], ends with
     using std::ranges::ends_with;
 
     // [alg.fold], fold
     using std::ranges::fold_left;
     using std::ranges::fold_left_first;
     using std::ranges::fold_right;
     using std::ranges::fold_right_last;
     using std::ranges::fold_left_with_iter;
     using std::ranges::fold_left_with_iter_result;
     using std::ranges::fold_left_with_iter;
     using std::ranges::fold_left_first_with_iter;
     using std::ranges::fold_left_first_with_iter;
-#endif
-  } // namespace ranges
+#  endif
+#endif // _LIBCPP_STD_VER >= 23
+  }    // namespace ranges
 
   // [alg.modifying.operations], mutating sequence operations
   // [alg.copy], copy
   using std::copy;
 
   namespace ranges {
     using std::ranges::copy;
     using std::ranges::copy_result;

libcxx/modules/std/bit.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
   // [bit.cast], bit_cast
   using std::bit_cast;
 
+#if _LIBCPP_STD_VER >= 23
   // [bit.byteswap], byteswap
   using std::byteswap;
+#endif
 
   // [bit.pow.two], integral powers of 2
   using std::bit_ceil;
   using std::bit_floor;
   using std::bit_width;
   using std::has_single_bit;
 
   // [bit.rotate], rotating

libcxx/modules/std/expected.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
+#if _LIBCPP_STD_VER >= 23
   // [expected.unexpected], class template unexpected
   using std::unexpected;
 
   // [expected.bad], class template bad_expected_access
   using std::bad_expected_access;
 
   // in-place construction of unexpected values
   using std::unexpect;
   using std::unexpect_t;
 
   // [expected.expected], class template expected
   using std::expected;
+#endif // _LIBCPP_STD_VER >= 23
 } // namespace std

libcxx/modules/std/format.inc

#endif
 
   using std::format_to_n;
   using std::format_to_n_result;
   using std::formatted_size;
 
   // [format.formatter], formatter
   using std::formatter;
 
+#if _LIBCPP_STD_VER >= 23
   // [format.formattable], concept formattable
   using std::formattable;
+#endif
 
   // [format.parse.ctx], class template basic_format_parse_context
   using std::basic_format_parse_context;
   using std::format_parse_context;
 #ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
   using std::wformat_parse_context;
 #endif
 
+#if _LIBCPP_STD_VER >= 23
   // [format.range], formatting of ranges
   // [format.range.fmtkind], variable template format_kind
   using std::format_kind;
   using std::range_format;
 
   // [format.range.formatter], class template range_formatter
   using std::range_formatter;
+#endif // _LIBCPP_STD_VER >= 23
 
   // [format.arg], class template basic_format_arg
   using std::basic_format_arg;
   using std::visit_format_arg;
 
   // [format.arg.store], class template format-arg-store
   using std::make_format_args;
 #ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
   using std::make_wformat_args;
 #endif
 
   // [format.error], class format_error
   using std::format_error;
 } // namespace std

libcxx/modules/std/functional.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
   // [func.invoke], invoke
   using std::invoke;
+#if _LIBCPP_STD_VER >= 23
   using std::invoke_r;
+#endif
 
   // [refwrap], reference_wrapper
   using std::reference_wrapper;
 
   using std::cref;
   using std::ref;
 
   // [arithmetic.operations], arithmetic operations

libcxx/modules/std/mdspan.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
+#if _LIBCPP_STD_VER >= 23
   // [mdspan.extents], class template extents
   using std::extents;
 
   // [mdspan.extents.dextents], alias template dextents
   using std::dextents;
 
   // [mdspan.layout], layout mapping
   using std::layout_left;
   using std::layout_right;
   // using std::layout_stride;
 
   // [mdspan.accessor.default], class template default_accessor
   using std::default_accessor;
 
   // [mdspan.mdspan], class template mdspan
   using std::mdspan;
+#endif // _LIBCPP_STD_VER >= 23
 } // namespace std

libcxx/modules/std/memory.inc

export namespace std {
   using std::uses_allocator_construction_args;
 
   using std::make_obj_using_allocator;
   using std::uninitialized_construct_using_allocator;
 
   // [allocator.traits], allocator traits
   using std::allocator_traits;
 
+#if _LIBCPP_STD_VER >= 23
   using std::allocation_result;
 
   using std::allocate_at_least;
+#endif
 
   // [default.allocator], the default allocator
   using std::allocator;
   using std::operator==;
 
   // [specialized.addressof], addressof
   using std::addressof;
 

libcxx/modules/std/print.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
+#if _LIBCPP_STD_VER >= 23
   // [print.fun], print functions
   using std::print;
   using std::println;
 
   using std::vprint_nonunicode;
-#ifndef _LIBCPP_HAS_NO_UNICODE
+#  ifndef _LIBCPP_HAS_NO_UNICODE
   using std::vprint_unicode;
-#endif // _LIBCPP_HAS_NO_UNICODE
+#  endif // _LIBCPP_HAS_NO_UNICODE
+#endif   // _LIBCPP_STD_VER >= 23
 } // namespace std

libcxx/modules/std/ranges.inc

namespace ranges {
 
     // [range.elementsof], class template elements_­of
     // using std::ranges::elements_of;
 
     using std::ranges::borrowed_iterator_t;
 
     using std::ranges::borrowed_subrange_t;
 
+#if _LIBCPP_STD_VER >= 23
     // [range.utility.conv], range conversions
     using std::ranges::to;
+#endif
 
     // [range.empty], empty view
     using std::ranges::empty_view;
 
     namespace views {
       using std::ranges::views::empty;
     }
 
     // [range.single], single view
     using std::ranges::single_view;
 
     namespace views {
       using std::ranges::views::single;
     } // namespace views
 
     // [range.iota], iota view
     using std::ranges::iota_view;
 
     namespace views {
       using std::ranges::views::iota;
     } // namespace views
 
+#if _LIBCPP_STD_VER >= 23
     // [range.repeat], repeat view
     using std::ranges::repeat_view;
 
     namespace views {
       using std::ranges::views::repeat;
     } // namespace views
+#endif // _LIBCPP_STD_VER >= 23
 
 #ifndef _LIBCPP_HAS_NO_LOCALIZATION
     // [range.istream], istream view
     using std::ranges::basic_istream_view;
     using std::ranges::istream_view;
 #  ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
     using std::ranges::wistream_view;
 #  endif
#endif // _LIBCPP_HAS_NO_LOCALIZATION
     } // namespace views
 
     // [range.ref.view], ref view
     using std::ranges::ref_view;
 
     // [range.owning.view], owning view
     using std::ranges::owning_view;
 
+#if _LIBCPP_STD_VER >= 23
     // [range.as.rvalue], as rvalue view
     using std::ranges::as_rvalue_view;
 
     namespace views {
       using std::ranges::views::as_rvalue;
     } // namespace views
+#endif // _LIBCPP_STD_VER >= 23
 
     // [range.filter], filter view
     using std::ranges::filter_view;
 
     namespace views {
       using std::ranges::views::filter;
     } // namespace views
 
#endif
     using std::ranges::values_view;
 
     namespace views {
       using std::ranges::views::elements;
       using std::ranges::views::keys;
       using std::ranges::views::values;
     } // namespace views
 
+#if _LIBCPP_STD_VER >= 23
     // [range.zip], zip view
     using std::ranges::zip_view;
 
     namespace views {
       using std::ranges::views::zip;
     } // namespace views
+#endif // _LIBCPP_STD_VER >= 23
 
 #if 0
     // [range.zip.transform], zip transform view
     using std::ranges::zip_transform_view;
 
     namespace views {
       using std::ranges::views::zip_transform;
     }

 #endif
   } // namespace ranges
 
   namespace views = ranges::views;
 
   using std::tuple_element;
   using std::tuple_size;
 
+#if _LIBCPP_STD_VER >= 23
   using std::from_range;
   using std::from_range_t;
+#endif // _LIBCPP_STD_VER >= 23
 } // namespace std

libcxx/modules/std/string.inc

#ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
     using std::pmr::wstring;
 #endif
   } // namespace pmr
 
   // [basic.string.hash], hash support
   using std::hash;
 
   // TODO MODULES is this a bug?
-#if 1
+#if _LIBCPP_STD_VER >= 23
   using std::operator""s;
 #else
   inline namespace literals {
     inline namespace string_literals {
       // [basic.string.literals], suffix for basic_string literals
       using std::literals::string_literals::operator""s;
     } // namespace string_literals
   }   // namespace literals
 #endif
 } // namespace std

libcxx/modules/std/thread.inc

#ifndef _LIBCPP_HAS_NO_THREADS
 
   // [thread.thread.id]
   using std::operator==;
   using std::operator<=>;
 #  ifndef _LIBCPP_HAS_NO_LOCALIZATION
   using std::operator<<;
 #  endif // _LIBCPP_HAS_NO_LOCALIZATION
 
+#  if _LIBCPP_STD_VER >= 23
   using std::formatter;
+#  endif
 
   using std::hash;
 #endif // _LIBCPP_HAS_NO_THREADS
 } // namespace std

libcxx/modules/std/tuple.inc

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 export namespace std {
   // [tuple.tuple], class template tuple
   using std::tuple;
 
   // [tuple.like], concept tuple-like
 
+#if _LIBCPP_STD_VER >= 23
   // [tuple.common.ref], common_reference related specializations
   using std::basic_common_reference;
   using std::common_type;
+#endif
 
   // [tuple.creation], tuple creation functions
   using std::ignore;
 
   using std::forward_as_tuple;
   using std::make_tuple;
   using std::tie;
   using std::tuple_cat;

libcxx/modules/std/type_traits.inc

export namespace std {
   using std::is_final;
   using std::is_polymorphic;
   using std::is_standard_layout;
   using std::is_trivial;
   using std::is_trivially_copyable;
   using std::is_volatile;
 
   using std::is_bounded_array;
+#if _LIBCPP_STD_VER >= 23
   using std::is_scoped_enum;
+#endif
   using std::is_signed;
   using std::is_unbounded_array;
   using std::is_unsigned;
 
   using std::is_constructible;
   using std::is_copy_constructible;
   using std::is_default_constructible;
   using std::is_move_constructible;
#endif
   using std::is_nothrow_copy_constructible_v;
   using std::is_nothrow_default_constructible_v;
   using std::is_nothrow_destructible_v;
   using std::is_nothrow_move_assignable_v;
   using std::is_nothrow_move_constructible_v;
   using std::is_nothrow_swappable_v;
   using std::is_nothrow_swappable_with_v;
   using std::is_polymorphic_v;
+#if _LIBCPP_STD_VER >= 23
   using std::is_scoped_enum_v;
+#endif
   using std::is_signed_v;
   using std::is_standard_layout_v;
   using std::is_swappable_v;
   using std::is_swappable_with_v;
   using std::is_trivial_v;
   using std::is_trivially_assignable_v;
   using std::is_trivially_constructible_v;
   using std::is_trivially_copy_assignable_v;

libcxx/modules/std/utility.inc

export namespace std {
   // [utility.swap], swap
   using std::swap;
 
   // [utility.exchange], exchange
   using std::exchange;
 
   // [forward], forward/move
   using std::forward;
+#if _LIBCPP_STD_VER >= 23
   using std::forward_like;
+#endif
   using std::move;
   using std::move_if_noexcept;
 
   // [utility.as.const], as_const
   using std::as_const;
 
   // [declval], declval
   using std::declval;
 
   // [utility.intcmp], integer comparison functions
   using std::cmp_equal;
   using std::cmp_not_equal;
 
   using std::cmp_greater;
   using std::cmp_greater_equal;
   using std::cmp_less;
   using std::cmp_less_equal;
 
   using std::in_range;
 
+#if _LIBCPP_STD_VER >= 23
   // [utility.underlying], to_underlying
   using std::to_underlying;
 
   // [utility.unreachable], unreachable
   using std::unreachable;
+#endif // _LIBCPP_STD_VER >= 23
 
   // [intseq], compile-time integer sequences
   using std::index_sequence;
   using std::integer_sequence;
 
   using std::make_index_sequence;
   using std::make_integer_sequence;
 
   using std::index_sequence_for;
 
   // [pairs], class template pair
   using std::pair;
 
+#if _LIBCPP_STD_VER >= 23
   using std::basic_common_reference;
   using std::common_type;
+#endif
   // [pairs.spec], pair specialized algorithms
   using std::operator==;
   using std::operator<=>;
 
   using std::make_pair;
 
   // [pair.astuple], tuple-like access to pair
   using std::tuple_element;

libcxx/modules/std/vector.inc

export namespace std {
 
   namespace pmr {
     using std::pmr::vector;
   }
 
   // hash support
   using std::hash;
 
+#if _LIBCPP_STD_VER >= 23
   // [vector.bool.fmt], formatter specialization for vector<bool>
   using std::formatter;
+#endif
 } // namespace std

libcxx/test/libcxx/module_std.gen.py

 
 # newline needs to be escaped for the module partition output.
 nl = '\\\\n'
 
 # Create empty file with all parts.
 print(
     f"""\
 //--- module_std.sh.cpp
-// UNSUPPORTED{BLOCKLIT}: c++03, c++11, c++14, c++17, c++20
+// UNSUPPORTED{BLOCKLIT}: c++03, c++11, c++14, c++17
 // UNSUPPORTED{BLOCKLIT}: libcpp-has-no-std-modules
 // UNSUPPORTED{BLOCKLIT}: modules-build
 
 // REQUIRES{BLOCKLIT}: has-clang-tidy
 
 // The GCC compiler flags are not always compatible with clang-tidy.
 // UNSUPPORTED{BLOCKLIT}: gcc
 

libcxx/test/lit.local.cfg

def appendToSubstitution(substitutions, key, value):
     return [(k, v + " " + value) if k == key else (k, v) for (k, v) in substitutions]
 
 
 std = getSubstitution("%{cxx_std}", config)
 if std == "cxx26":
     std = "26"
 elif std == "cxx23":
     std = "23"
+elif std == "cxx20":
+    std = "20"
 else:
     std = ""
 
 if (
     std
     and not "libcpp-has-no-std-modules" in config.available_features
     and not "modules-build" in config.available_features
 ):

libcxx/utils/ci/buildkite-pipeline.yml

- label: "C++23"
     timeout_in_minutes: 120
 
   - label: "C++20"
     command: "libcxx/utils/ci/run-buildbot generic-cxx20"
     artifact_paths:
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
-        CC: "clang-${LLVM_HEAD_VERSION}"
-        CXX: "clang++-${LLVM_HEAD_VERSION}"
+        # Note: Modules require and absolute path for clang-scan-deps
+        # https://github.com/llvm/llvm-project/issues/61006
+        CC: "/usr/lib/llvm-${LLVM_HEAD_VERSION}/bin/clang"
+        CXX: "/usr/lib/llvm-${LLVM_HEAD_VERSION}/bin/clang++"
+        CMAKE: "/opt/bin/cmake"
         ENABLE_CLANG_TIDY: "On"
+        ENABLE_STD_MODULES: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120