[libc++][RFC] Guard against libc++ macros in tests.

[Mordante]

Summary

D117992 removed an libc++ specific macro in our tests. This example lit
tests guards against reintroducing this macro in future commits.

Is there an interest in a script like this? If so I'll look at the other
libc++ macros we want to guard against.

Do we want to guard both test/std and test/libcxx or only the
former? (The latter is already libc++ specific, but using the TEST
macros there seems good practice.) Currently the script only tests for
test/std.

NOTE the test should fail since we don't depend on D117992.

[• Quuxplusone]

Interesting. Is there some way to generalize this to catch all but a whitelist of _LIBCPP macros? Here's the current tally; can we think of any way to eliminate more of these?

$ git grep -h '#if.*[ (]_LIBCPP' ../libcxx/test/std/ | sort | uniq -c | sort -rn
 110 #ifndef _LIBCPP_HAS_NO_INT128
  86 #if defined(_LIBCPP_VERSION)
  84 #ifndef _LIBCPP_HAS_NO_UNICODE_CHARS
  26 #ifndef _LIBCPP_HAS_NO_CHAR8_T
  14 #ifdef _LIBCPP_VERSION
   9 #ifndef _LIBCPP_HAS_NO_THREADS
   6 #ifndef _LIBCPP_HAS_NO_LOCALIZATION
   4 //#ifndef _LIBCPP_HAS_NO_UNICODE_CHARS
   4 #ifndef _LIBCPP_HAS_NO_INCOMPLETE_RANGES
   4 #ifndef _LIBCPP_HAS_NO_FGETPOS_FSETPOS
   4 #if TEST_STD_VER < 11 && defined(_LIBCPP_VERSION)
   3 #if defined(_LIBCPP_VERSION) && 0 // FIXME these extensions are currently disabled.
   3 #if !defined(_LIBCPP_DEPRECATED_ABI_DISABLE_PAIR_TRIVIAL_COPY_CTOR)
   2 #ifndef _LIBCPP_HAS_NO_FILESYSTEM_LIBRARY
   2 #ifndef _LIBCPP_ABI_MICROSOFT
   2 #ifdef _LIBCPP_VERSION // extension
   2 #ifdef _LIBCPP_VERSION // Don't violate precondition [atomics.flag]/6
   2 #if defined(_LIBCPP_VERSION) && TEST_STD_VER >= 11
   2 #if TEST_STD_VER == 11 && defined(_LIBCPP_VERSION)
   2 #if !defined(_LIBCPP_HAS_NO_INT128) && !defined(_LIBCPP_HAS_NO_STRONG_ENUMS)
   1 #ifndef _LIBCPP_UTILITY
   1 #ifndef _LIBCPP_HAS_NO_VECTOR_EXTENSION
   1 #ifndef _LIBCPP_DEBUG
   1 #ifdef _LIBCPP_VERSION // assignment from std::array is a libc++ extension
   1 #ifdef _LIBCPP_SAFE_STATIC
   1 #ifdef _LIBCPP_HAS_NO_STRONG_ENUMS
   1 #ifdef _LIBCPP_HAS_NO_NULLPTR
   1 #ifdef _LIBCPP_ABI_REGEX_CONSTANTS_NONZERO  // https://llvm.org/PR35967
   1 #if defined(_LIBCPP_VERSION) && !defined(_LIBCPP_HAS_NO_LOCALIZATION)
   1 #if defined(_LIBCPP_USING_ARC4_RANDOM)
   1 #if __has_attribute(vector_size) && defined(_LIBCPP_VERSION)
   1 #if TEST_STD_VER > 17 && defined(__cpp_char8_t) && defined(_LIBCPP_VERSION) && \
   1 #if TEST_STD_VER > 17 && defined(_LIBCPP_VERSION)
   1 #if TEST_STD_VER > 17 && !defined(_LIBCPP_HAS_NO_RANGES)
   1 #if TEST_HAS_BUILTIN(__make_integer_seq) && !defined(_LIBCPP_TESTING_FALLBACK_MAKE_INTEGER_SEQUENCE)
   1 #if !defined(_LIBCPP_HAS_NO_LOCALIZATION)
   1 #if !defined(TEST_HAS_NO_EXCEPTIONS) && defined(_LIBCPP_USING_DEV_RANDOM)

[Mordante]

Fix the temporary file name. Since the review now depends on D117992 the CI should pass.

[Mordante]

I hope we're able to remove all but _LIBCPP_VERSION from the test/std directory. Hopefully that makes it easier for others to use our tests.
Especially since MSVC STL is using our tests.

[ldionne]

Yes, I think there is definitely interest in something like this. To echo Arthur, I think we should generalize to all macros starting with _LIBCPP except for a list of allowed macros. At first the list can contain a lot of stuff, and as we move them towards TEST_HAS_NO_XXXXX macros instead, we can remove them from the list.

[Mordante]

Updated the script. Most positives have been fixed upstream.
Some remain what to do with them should be discussed on Discord.

libcxx/test/libcxx/lint/lint_libcpp_macros_in_tests.sh.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// Tests in test/std should work with all Standard library implementations.
+// This test greps for the usage of some libc++ internal defines that have a
+// TEST macro that achieves the same in a portable fashion.
+//
+// Upon failure the found results are part of the test's failure output. This
+// makes finding the occurrences easy.
+
+// Some items are whitelisted
+// - _LIBCPP_VERSION is used to test whether we're testing against libc++
+// - The language.support directory uses internal macros by generated scripts
+// - The experimental directory is a testing ground
+
+RUN: not grep -R --exclude-dir language.support --exclude-dir experimental '#if.*[ (]_LIBCPP' %S/../../std |grep -v ':.*_LIBCPP_VERSION'|grep -v ':.*_LIBCPP[A-Z_]*_ABI_'