[libc++] Add fallback standard flags and normalize corresponding feature.

[curdeius]

Summary

Using std=c++2a is now equivalent to std=c++20 (if both flags are supported by the compiler).
Features use non-fallback standard names (c++20 instead of c++2a).
E.g. using --param=std=c++2a will detect feature c++20.
Note: you still cannot pass --param=std=c++23 if the compiler doesn't support -std=c++23.

[• Quuxplusone]

libcxx/utils/libcxx/test/params.py

11–13

I have two weak rationales here:

• Technically, it's not "C++23" until it's released. I don't want to get into a situation where we have to remove the string c++23 from this codepath. So we should leave it as c++2b until 2023.
• I'd like to see us develop a "deploy plan" for moving from one standardization-epoch to the next, which we execute atomically every 3 years. I described my ideal here: [SADLY I CANNOT LOCATE THIS REVIEW ANYMORE maybe @Mordante remembers our discussion and can dig it up?] Therefore I'd like this patch to get us into the "C++20 has been released, C++2b has not yet been released" state. Three years from now, we can atomically move into the "C++23 has been released, C++2c has not yet been released" state. I don't want us to ever be in some halfway state like the "C++23 has been released sorta not really" state described in this patch right now.

D93383 was related. D65043 was loosely related.

I think (but am not 100% sure) that all I'm asking for here is to replace 'c++23' with 'c++2b' in the one place it appears. Everything else can stay the same, right?

[curdeius]

I'm ok with removing c++23. I was actually hesitant when adding it.

[Mordante]

I like this patch a lot. This makes it a lot easier to switch to newer C++ versions shortly after their release without the need to wait for the compilers to update.
Please update or rebase the patch to trigger the CI.

libcxx/utils/libcxx/test/params.py

11–13

I recall the discussion, but also can't find it anymore. I'm also slightly in favour to use c++2b instead of c++23. I get the impression the members in the committee feel more comfortable with the 3 year plan so I would be very surprised when c++23 becomes c++24. Still I think using c++2b safer and give a nice statement the standard isn't released.

[curdeius]

• Don't use c++23.
• Rebase.

libcxx/utils/libcxx/test/params.py

11–13

@Quuxplusone, have you had in mind the discussion from https://reviews.llvm.org/D97365#2603777, https://reviews.llvm.org/D97365#2604643?

Also, https://reviews.llvm.org/D93383 mentioned this a bit too.

[curdeius]

@ldionne, I'd like to hear you input on the changes in params.py. At least just to check that I haven't done anything stupid (given my lack of knowledge of lit and related stuff).

Apart from that, I start thinking of the plan for adding new standard, if anyone sees other points, please LMK.

Example: adding c++2c, c++23 (before: c++2b) released:

• In libcxx/utils/libcxx/test/params.py:
  • Change c++2b to c++23, add c++2c to _allStandards.
  • Add 'c++23': 'c++2b' to _standardFallback dictionary.
• Change c++2b to c++23 and add c++2c to std_dialects list in libcxx/utils/generate_feature_test_macro_components.py.
• Add C++2c CI job to libcxx/utils/ci/buildkite-pipeline.yml.
• Add generic-cxx2c to libcxx/utils/ci/run-buildbot.
• Add libcxx/cmake/caches/Generic-cxx2c.cmake CMake cache file.
• [Optionally] Add new feature-test macros libcxx/utils/generate_feature_test_macro_components.py.

• In libcxx/include/__config, bump _LIBCPP_STD_VER for C++2b and add a new value for C++2c.
• In libcxx/test/support/test_macros.h, set TEST_STD_VER to 23 for C++2b and to 99 for C++2c.
• In libcxx/test/support/msvc_stdlib_force_include.h, bump TEST_STD_VER for MSVC.

• In libcxx/CMakeLists.txt, use CXX_STANDARD 23 (or higher?).

[curdeius]

Can anyone help me with the CI failure (Runtimes build)?
https://buildkite.com/llvm-project/libcxx-ci/builds/2347#91e9025e-e17d-41eb-a9ba-3bd5fd28c086
(TBH, I'm looking for a confirmation that it's unrelated to this patch, as I've never seen a failure in this test, test_demangle.pass.cpp, and the message libc++abi: terminating is not very informative)

[mstorsjo]

In D99789#2671013, @curdeius wrote:

Can anyone help me with the CI failure (Runtimes build)?
https://buildkite.com/llvm-project/libcxx-ci/builds/2347#91e9025e-e17d-41eb-a9ba-3bd5fd28c086
(TBH, I'm looking for a confirmation that it's unrelated to this patch, as I've never seen a failure in this test, test_demangle.pass.cpp, and the message libc++abi: terminating is not very informative)

I'm seeing that in my recent CI runs too - so it's most probably unrelated to your patch, but no idea what triggered it...

[curdeius]

OK, right, it seems that it's a non-libc++ change that provoked this failure (Runtimes build uses just-built ToT clang).
FYI.
There are few possible culprits out there, some commit between today (6th April 2021) 8:00 (green https://buildkite.com/llvm-project/libcxx-ci/builds/2342#7a94194a-b986-449a-8c6c-4952a1ec8763, commit b7ef804807855e607da3eba221c1fc59e27f778e) and 10:00 UTC (this patch, red: https://buildkite.com/llvm-project/libcxx-ci/builds/2347#91e9025e-e17d-41eb-a9ba-3bd5fd28c086, based on main's commit f8f4d8f87ba4c1fbb18a4e7f4a5ea03a8b8ec061).

[• Quuxplusone]

Looks acceptable to me, but I think my comment on sized_delete11.pass.cpp is worth addressing.

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/sized_delete11.pass.cpp

14

Here and above, it looks like the intent is "REQUIRES: c++03 OR c++11". I don't know if there's any way to express that in llvm-lit. However, I predict that these tests fail in c++2b mode; am I right?
It might be better to duplicate these files into sized_delete11.pass.cpp and sized_delete03.pass.cpp, add an appropriate REQUIRES: to each of them, and then never touch them again.

libcxx/utils/libcxx/test/params.py

11–13

Yes, thank you! Specifically https://reviews.llvm.org/D97365#2604643 was the comment I was looking for.

14

I believe this line could be

_allStandardsAndFallbacks = _allStandards + list(_standardFallback.values())

Orthogonally, how about s/_standardFallbackReverse/_undoStandardFallback/? I think that would make line 62 read a little clearer.

[curdeius]

Apply review comments.

• Use REQUIRES.
• Improving my Python-fu.

[curdeius]

Thank you for your comments, Arthur.

[curdeius]

• Use REQUIRES once more.

[ldionne]

The patch looks good to me, but I think we can simplify the implementation in params.py a bit IMO. I left some comments to that effect.

I also agree with the list of things to do when a new standard comes out - do you think it would be possible to document it somewhere? Maybe in Contributing.rst?

libcxx/utils/libcxx/test/params.py

11–13

43–51

55–59

[curdeius]

libcxx/utils/libcxx/test/params.py

55–59

Unless I'm mistaken, we still need to call AddFeature, no? @ldionne

[curdeius]

libcxx/utils/libcxx/test/params.py

55–59

Also, in actions, I don't have cfg, only std...

[curdeius]

libcxx/utils/libcxx/test/params.py

55–59

WDYT about:

AddFeature(next((s for s, f in reversed(_allStandards) if f == std), std)),

?

[ldionne]

libcxx/utils/libcxx/test/params.py

55–59

Yeah, it seems I really botched my code change suggestion :-).

Basically anything that works is fine by me, but I found the logic with multiple variables at the beginning of the file to be hard to follow.

[curdeius]

• Apply Louis' comments.

[curdeius]

libcxx/utils/libcxx/test/params.py

11–13

I've kept c++1z as I sometimes use oldish gcc (e.g. cross-compiling for AVR). Hope that doesn't bother.

55

We need to have both keys and values here to allow fallbacks.

I'll land it with this suggestion but I don't want to trigger yet another CI only for this (given that the choices is visible only when the user provides a bad argument to --param=std, it isn't really tested anyway).

57

The default needs to be [1] because we don't want to use -std=c++20 on the compiler that supports only -std=c++2a.

[• Quuxplusone]

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/sized_delete11.pass.cpp

14

Nice! I did not know || would work here. :)

libcxx/utils/libcxx/test/params.py

11–13

I like this. Is it worth adding c++0x, c++1y as fallbacks for those two? I assume the logic of leaving them out is simply that we don't officially support any compiler old enough to lack a c++14 mode. But it might be useful-to-someone and/or encyclopedic to keep them in the script, I dunno.

55–59

I think all we're actually doing here is saying, "If the user gave us c++2a, turn it into c++20." Right? So that would be, like,

def what_were_doing(flag):
  if flag in _allStandards.values():
    AddFeature(next(s for s, f in _allStandards.items() if f == flag))
  else:
    AddFeature(flag)

which is like... exactly what you wrote, except without the reversed, right? The reversed would make a difference only if _allStandards.items() were in some relevant order to begin with.

I'm not thrilled by the readability of this expression, but I have no concrete improvements to suggest.

So I think I'm just asking to remove the reversed(...).

[curdeius]

libcxx/utils/libcxx/test/params.py

11–13

I can add these if Louis is OK. Louis removed them in his suggested edit.

55–59

Ok, I'll remove reversed.

[ldionne]

libcxx/utils/libcxx/test/params.py

57

It seems that we don't ever use getLatestStdFlag(cfg)[0] anymore, do we?

59

I don't understand the if f == std part -- isn't that always False?

[curdeius]

• Fix choices.
• Remove reversed.

[curdeius]

libcxx/utils/libcxx/test/params.py

57

We don't indeed. Should I remove?

59

std here can be the a fallback flag.

[curdeius]

• Clean up.

[ldionne]

Sorry it took me so long to understand what this patch was doing, however IMO we don't want to allow people to pass --param std=c++2a to Lit. We want to use --param std=c++20, and then have that query the compiler for either -std=c++20 or -std=c++2a if that's the only thing it supports. I think doing that is simpler, and it leaves the "legacy" -std=XXX parameter names to be handled as implementation details of the test suite without making them prominent when using the test suite.

[• Quuxplusone]

@ldionne wrote:

IMO we don't want to allow people to pass --param std=c++2a to Lit. We want to use --param std=c++20

Unfortunately that doesn't scale. Today we force people to use --param std=c++2a (because this patch hasn't been committed); and we also force people to use --param std=c++2b (because C++2b hasn't got an official number yet). We don't want to force everyone to have a "flag day" where they all simultaneously get forced over from std=c++2a to std=c++20; we want them to be able to adjust their scripts and stuff on their own time. So, since we need to support std=c++2a as a synonym for std=c++20 here anyway at the compiler level, I think it's quite correct to just reuse that list of synonyms for argument parsing as well. Which is what this patch does.

TLDR: This morning I ran ./bin/llvm-lit --param std=c++2a ../libcxx/test/. I want to run that same command tomorrow morning.

[ldionne]

In D99789#2679958, @Quuxplusone wrote:

@ldionne wrote:

IMO we don't want to allow people to pass --param std=c++2a to Lit. We want to use --param std=c++20

Unfortunately that doesn't scale. Today we force people to use --param std=c++2a (because this patch hasn't been committed); and we also force people to use --param std=c++2b (because C++2b hasn't got an official number yet). We don't want to force everyone to have a "flag day" where they all simultaneously get forced over from std=c++2a to std=c++20; we want them to be able to adjust their scripts and stuff on their own time. So, since we need to support std=c++2a as a synonym for std=c++20 here anyway at the compiler level, I think it's quite correct to just reuse that list of synonyms for argument parsing as well. Which is what this patch does.

TLDR: This morning I ran ./bin/llvm-lit --param std=c++2a ../libcxx/test/. I want to run that same command tomorrow morning.

I disagree - this is just the interface for running Lit from the command line, I think it's entirely fine if there's a "flag" day, and even if it's not official. There are some things that we need to create engineering solutions for in order not to break people, but I don't think that's one. I think pretty much the only places where there's --param std=c++2a is going to be in the CI scripts (which can be updated in lockstep with this patch), and in the libc++ developers' shell command history or local helper functions. I think it's fine if they just need to change it to --param std=c++20 at some point.

[curdeius]

Abandoning in favour of D100210.

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/sized_delete_array11.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // test sized operator delete[] replacement.
 
 // Note that sized delete operator definitions below are simply ignored
 // when sized deallocation is not supported, e.g., prior to C++14.
 
-// UNSUPPORTED: c++14, c++17, c++2a
+// REQUIRES: c++03 || c++11
 // UNSUPPORTED: sanitizer-new-delete
 
 #include <new>
 #include <cstddef>
 #include <cstdlib>
 #include <cassert>
 
 #include "test_macros.h"

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/sized_delete11.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // test sized operator delete replacement.
 
 // Note that sized delete operator definitions below are simply ignored
 // when sized deallocation is not supported, e.g., prior to C++14.
 
-// UNSUPPORTED: c++14, c++17, c++2a
+// REQUIRES: c++03 || c++11
 // UNSUPPORTED: sanitizer-new-delete
 
 #include <new>
 #include <cstddef>
 #include <cstdlib>
 #include <cassert>
 
 #include "test_macros.h"

libcxx/test/std/strings/basic.string/string.contains/contains.char.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string>
 
 //   constexpr bool contains(charT x) const noexcept;
 
 #include <string>
 #include <cassert>
 

libcxx/test/std/strings/basic.string/string.contains/contains.ptr.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string>
 
 //   constexpr bool contains(const CharT *x) const;
 
 #include <string>
 #include <cassert>
 

libcxx/test/std/strings/basic.string/string.contains/contains.string_view.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string>
 
 //   constexpr bool contains(basic_string_view x) const noexcept;
 
 #include <string>
 #include <cassert>
 

libcxx/test/std/strings/string.view/string.view.template/contains.char.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string_view>
 
 //   constexpr bool contains(charT x) const noexcept;
 
 #include <string_view>
 #include <cassert>
 

libcxx/test/std/strings/string.view/string.view.template/contains.ptr.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string_view>
 
 //   constexpr bool contains(const CharT *x) const;
 
 #include <string_view>
 #include <cassert>
 

libcxx/test/std/strings/string.view/string.view.template/contains.string_view.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // <string_view>
 
 //   constexpr bool contains(string_view x) const noexcept;
 
 #include <string_view>
 #include <cassert>
 

libcxx/test/std/utilities/meta/meta.unary/meta.unary.prop/is_scoped_enum.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // type_traits
 
 // is_scoped_enum // C++2b
 
 #include <type_traits>
 #include <cstddef> // for std::nullptr_t
 #include "test_macros.h"

libcxx/test/std/utilities/utility/utility.underlying/to_underlying.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // [utility.underlying], to_underlying
 // template <class T>
 //     constexpr underlying_type_t<T> to_underlying( T value ) noexcept; // C++2b
 
 #include <utility>
 #include <cassert>
 #include <limits>

libcxx/test/std/utilities/utility/utility.underlying/to_underlying.verify.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-// UNSUPPORTED: c++03, c++11, c++14, c++17, c++2a
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
 
 // [utility.underlying], to_underlying
 // template <class T>
 //     constexpr underlying_type_t<T> to_underlying( T value ) noexcept; // C++2b
 
 #include <utility>
 
 struct S {};
 
 int main(int, char**) {
   std::to_underlying(125); // expected-error {{no matching function for call}}
   std::to_underlying(S{}); // expected-error {{no matching function for call}}
 
   return 0;
 }

libcxx/utils/libcxx/test/params.py

 #===----------------------------------------------------------------------===##
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 #===----------------------------------------------------------------------===##
 
 from libcxx.test.dsl import *
 
-_allStandards = ['c++03', 'c++11', 'c++14', 'c++17', 'c++2a', 'c++2b']
+_allStandards = {'c++03': None, 'c++11': 'c++0x', 'c++14': 'c++1y',
+                 'c++17': 'c++1z', 'c++20': 'c++2a', 'c++2b': None}
 _warningFlags = [
   '-Werror',
   '-Wall',
   '-Wextra',
   '-Wshadow',
   '-Wundef',
   '-Wno-unused-command-line-argument',
   '-Wno-attributes',
   '-Wno-pessimizing-move',
   '-Wno-c++11-extensions',
_warningFlags = [
   # expect the test suite to be clean.
   '-Wsign-compare',
   '-Wunused-variable',
   '-Wunused-parameter',
   '-Wunreachable-code',
   '-Wno-unused-local-typedef',
 ]
 
+
+def getLatestStdFlag(cfg):
+  standards = sorted(_allStandards.items(), key=lambda sf: sf[0])
+  for std, fallback in reversed(standards):
+    if hasCompileFlag(cfg, '-std='+std):
+      return std
+    if fallback and hasCompileFlag(cfg, '-std='+fallback):
+      return fallback
+  raise RuntimeError("The compiler doesn't seem to support any standard flag")
+
+
 DEFAULT_PARAMETERS = [
   # Core parameters of the test suite
-  Parameter(name='std', choices=_allStandards, type=str,
+  Parameter(name='std', choices=sorted(list(_allStandards.keys()) + [v for v in _allStandards.values() if v is not None]), type=str,
             help="The version of the standard to compile the test suite with.",
-            default=lambda cfg: next(s for s in reversed(_allStandards) if hasCompileFlag(cfg, '-std='+s)),
+            default=lambda cfg: getLatestStdFlag(cfg),
             actions=lambda std: [
-              AddFeature(std),
+              AddFeature(next((s for s, f in _allStandards.items() if s == std or f == std), std)),
               AddCompileFlag('-std={}'.format(std)),
             ]),
 
   Parameter(name='enable_exceptions', choices=[True, False], type=bool, default=True,
             help="Whether to enable exceptions when compiling the test suite.",
             actions=lambda exceptions: [] if exceptions else [
               AddFeature('no-exceptions'),
               AddCompileFlag('-fno-exceptions')