[libc++] Use .gen.py tests for the transitive inclusion tests

[ldionne]

Summary

This finishes the transition of tests covered in generate_header_tests.py
to the new .gen.py format.

[philnik]

I think this makes sense, but I'd rather have @Mordante have a look, since he is more familiar with the interns of this test, and especially the transitive_includes_to_csv.py script.

[Mordante]

In general the approach seems fine, but I want to do some testing with this patch.
Especially to see what happens when the regenerate_expected_results flag is toggled.

libcxx/test/libcxx/transitive_includes.gen.py

28–31

These instructions seem outdated an no longer working. The regeneration happen when I try to run this test. Unfortunately running the test fails.

<build>/bin/llvm-lit --param std=c++03 test/libcxx/transitive_includes.gen.py gives

<llvm>/llvm/utils/lit/lit/discovery.py:189: error: 'llvm-libc++-shared.cfg.in :: libcxx/transitive_includes.gen.py' would not be run indirectly: change name or LIT config(e.g. suffixes or standalone_tests variables)
1 errors, exiting

Running the generated test manually
<build>/bin/llvm-lit <build>/build/test/libcxx/transitive_includes.gen.py/algorithm.sh.cpp
gives

llvm-lit: <llvm>/llvm/utils/lit/lit/discovery.py:137: warning: unable to find test suite for '<build>/test/libcxx/transitive_includes.gen.py/algorithm.sh.cpp'
llvm-lit: <llvm>/llvm/utils/lit/lit/discovery.py:314: warning: input '<build>/test/libcxx/transitive_includes.gen.py/algorithm.sh.cpp' contained no tests
error: did not discover any tests for provided path(s)

How can we run this test and update the transitive includes?

37

This file is generated when regenerate_expected_results is set to True but not removed when set to False again. I would like to test whether this generated test will be executed, but I can't seem to run this test. (See comment above.)

89

It seems we don't use awk in our tests yet. It's already a dependency since it's on of the POSIX tools. However I wonder why not using grep instead?

[ldionne]

libcxx/test/libcxx/transitive_includes.gen.py

28–31

This would be fixed by D151664.

89

That's a good question. I was using grep first.

grep returns non-zero when there are no matches, which fails this test for headers that have no transitive dependencies.

[Mordante]

LGTM modulo one nit.

libcxx/test/libcxx/transitive_includes.gen.py

28–29

28–31

Great! That patch was also very useful to test the new style tests for D151814.

37

I tested this with D151664 and only the proper branch of regenerate_expected_results was executed. So this works as expected and the stale files are not an issue.

89

Ah interesting.

[Mordante]

libcxx/test/libcxx/transitive_includes.gen.py

28–31

s/D151814/D144994/

[ldionne]

I'm going to ship this now despite D151664 not being landed because this is blocking other work.

[Mordante]

In D152008#4395783, @ldionne wrote:

I'm going to ship this now despite D151664 not being landed because this is blocking other work.

No objections. I just needed to be able to test the patch locally to see whether the stale tests were an issue.

libcxx/test/libcxx/transitive_includes.gen.py

+#===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+#===----------------------------------------------------------------------===##
+
+# Test that we don't remove transitive includes of public C++ headers in the library accidentally.
+# When we remove a transitive public include, clients tend to break because they don't always
+# properly include what they use. Note that we don't check which system (C) headers are
+# included transitively, because that is too unstable across platforms, and hence difficult
+# to test for.
+#
+# This is not meant to block libc++ from removing unused transitive includes
+# forever, however we do try to group removals for a couple of releases
+# to avoid breaking users at every release.
+
+# RUN: %{python} %s %{libcxx}/utils
+
+import sys
+sys.path.append(sys.argv[1])
+from libcxx.test.header_information import header_restrictions, public_headers
+
+import re
+
+# To re-generate the list of expected headers, temporarily set this to True, and run this test.
+# Note that this needs to be done for all supported language versions of libc++:
+# for std in c++03 c++11 c++14 c++17 c++20 c++23 c++26; do <build>/bin/llvm-lit --param std=$std ${path_to_this_file}; done
+regenerate_expected_results = False
+
+BLOCKLIT = '' # block Lit from interpreting a RUN/XFAIL/etc inside the generation script
+if regenerate_expected_results:
+  print(f"""\
+//--- generate-transitive-includes.sh.cpp
+// RUN{BLOCKLIT}: mkdir %t
+""")
+
+  all_traces = []
+  for header in sorted(public_headers):
+    if header.endswith('.h'): # Skip C compatibility or detail headers
+      continue
+
+    normalized_header = re.sub('/', '_', header)
+    print(f"""\
+// RUN{BLOCKLIT}: echo "#include <{header}>" | %{{cxx}} -xc++ - %{{flags}} %{{compile_flags}} --trace-includes -fshow-skipped-includes --preprocess > /dev/null 2> %t/trace-includes.{normalized_header}.txt
+""")
+    all_traces.append(f'%t/trace-includes.{normalized_header}.txt')
+
+  print(f"""\
+// RUN{BLOCKLIT}: %{{python}} %{{libcxx}}/test/libcxx/transitive_includes_to_csv.py {' '.join(all_traces)} > %{{libcxx}}/test/libcxx/transitive_includes/%{{cxx_std}}.csv
+""")
+
+else:
+  for header in public_headers:
+    if header.endswith('.h'): # Skip C compatibility or detail headers
+      continue
+
+    # Escape slashes for the awk command below
+    escaped_header = header.replace('/', '\/')
+
+    print(f"""\
+//--- {header}.sh.cpp
+
+// This test doesn't support being run when some headers are not available, since we
+// would need to add significant complexity to make that work.
+// UNSUPPORTED{BLOCKLIT}: no-localization, no-threads, no-wide-characters, no-filesystem
+
+// When built with modules, this test doesn't work because --trace-includes doesn't
+// report the stack of includes correctly.
+// UNSUPPORTED{BLOCKLIT}: modules-build
+
+// This test uses --trace-includes, which is not supported by GCC.
+// UNSUPPORTED{BLOCKLIT}: gcc
+
+// This test is not supported when we remove the transitive includes provided for backwards
+// compatibility. When we bulk-remove them, we'll adjust the includes that are expected by
+// this test instead.
+// UNSUPPORTED{BLOCKLIT}: transitive-includes-disabled
+
+// TODO: Figure out why <stdatomic.h> doesn't work on FreeBSD
+// UNSUPPORTED{BLOCKLIT}: LIBCXX-FREEBSD-FIXME
+
+// RUN{BLOCKLIT}: mkdir %t
+// RUN{BLOCKLIT}: %{{cxx}} %s %{{flags}} %{{compile_flags}} --trace-includes -fshow-skipped-includes --preprocess > /dev/null 2> %t/trace-includes.txt
+// RUN{BLOCKLIT}: %{{python}} %{{libcxx}}/test/libcxx/transitive_includes_to_csv.py %t/trace-includes.txt > %t/actual_transitive_includes.csv
+// RUN{BLOCKLIT}: cat %{{libcxx}}/test/libcxx/transitive_includes/%{{cxx_std}}.csv | awk '/^{escaped_header} / {{ print }}' > %t/expected_transitive_includes.csv
+// RUN{BLOCKLIT}: diff -w %t/expected_transitive_includes.csv %t/actual_transitive_includes.csv
+#include <{header}>
+""")

libcxx/test/libcxx/transitive_includes.sh.cpp

-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// Test that we don't remove transitive includes of public C++ headers in the library accidentally.
-// When we remove a transitive public include, clients tend to break because they don't always
-// properly include what they use. Note that we don't check which system (C) headers are
-// included transitively, because that is too unstable across platforms, and hence difficult
-// to test for.
-//
-// This is not meant to block libc++ from removing unused transitive includes
-// forever, however we do try to group removals for a couple of releases
-// to avoid breaking users at every release.
-
-// This test doesn't support being run when some headers are not available, since we
-// would need to add significant complexity to make that work.
-// UNSUPPORTED: no-localization, no-threads, no-wide-characters, no-filesystem
-
-// When built with modules, this test doesn't work because --trace-includes doesn't
-// report the stack of includes correctly.
-// UNSUPPORTED: modules-build
-
-// This test uses --trace-includes, which is not supported by GCC.
-// UNSUPPORTED: gcc
-
-// This test is not supported when we remove the transitive includes provided for backwards
-// compatibility. When we bulk-remove them, we'll adjust the includes that are expected by
-// this test instead.
-// UNSUPPORTED: transitive-includes-disabled
-
-/*
-BEGIN-SCRIPT
-
-import re
-
-# To re-generate the list of expected headers, temporarily set this to True, re-generate
-# the file and run this test.
-# Note that this needs to be done for all supported language versions of libc++:
-# for std in c++03 c++11 c++14 c++17 c++20 c++23 c++26; do <build>/bin/llvm-lit --param std=$std ${path_to_this_file}; done
-regenerate_expected_results = False
-# Used because the sequence of tokens RUN : can't appear anywhere or it'll confuse Lit.
-RUN = "RUN"
-
-print(f"// {RUN}: mkdir %t")
-
-for i, header in enumerate(public_headers):
-  if header.endswith('.h'): # Skip C compatibility or detail headers
-    continue
-
-  normalized_header = re.sub('/', '_', header)
-  print(f"// {RUN}: %{{cxx}} %s %{{flags}} %{{compile_flags}} --trace-includes -fshow-skipped-includes --preprocess -DTEST_{i} > /dev/null 2> %t/header.{normalized_header}")
-  print(f"#if defined(TEST_{i})")
-  print(f"#include <{header}>")
-  print("#endif")
-
-if regenerate_expected_results:
-  print(f"// {RUN}: %{{python}} %S/transitive_includes_to_csv.py %t > %S/transitive_includes/%{{cxx_std}}.csv")
-else:
-  print(f"// {RUN}: %{{python}} %S/transitive_includes_to_csv.py %t > %t/transitive_includes.csv")
-  print(f"// {RUN}: diff -w %S/transitive_includes/%{{cxx_std}}.csv %t/transitive_includes.csv")
-
-END-SCRIPT
-*/
-
-// DO NOT MANUALLY EDIT ANYTHING BETWEEN THE MARKERS BELOW
-// GENERATED-MARKER
-// RUN: mkdir %t
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_0 > /dev/null 2> %t/header.algorithm
-#if defined(TEST_0)
-#include <algorithm>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_1 > /dev/null 2> %t/header.any
-#if defined(TEST_1)
-#include <any>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_2 > /dev/null 2> %t/header.array
-#if defined(TEST_2)
-#include <array>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_3 > /dev/null 2> %t/header.atomic
-#if defined(TEST_3)
-#include <atomic>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_4 > /dev/null 2> %t/header.barrier
-#if defined(TEST_4)
-#include <barrier>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_5 > /dev/null 2> %t/header.bit
-#if defined(TEST_5)
-#include <bit>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_6 > /dev/null 2> %t/header.bitset
-#if defined(TEST_6)
-#include <bitset>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_7 > /dev/null 2> %t/header.cassert
-#if defined(TEST_7)
-#include <cassert>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_8 > /dev/null 2> %t/header.ccomplex
-#if defined(TEST_8)
-#include <ccomplex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_9 > /dev/null 2> %t/header.cctype
-#if defined(TEST_9)
-#include <cctype>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_10 > /dev/null 2> %t/header.cerrno
-#if defined(TEST_10)
-#include <cerrno>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_11 > /dev/null 2> %t/header.cfenv
-#if defined(TEST_11)
-#include <cfenv>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_12 > /dev/null 2> %t/header.cfloat
-#if defined(TEST_12)
-#include <cfloat>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_13 > /dev/null 2> %t/header.charconv
-#if defined(TEST_13)
-#include <charconv>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_14 > /dev/null 2> %t/header.chrono
-#if defined(TEST_14)
-#include <chrono>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_15 > /dev/null 2> %t/header.cinttypes
-#if defined(TEST_15)
-#include <cinttypes>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_16 > /dev/null 2> %t/header.ciso646
-#if defined(TEST_16)
-#include <ciso646>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_17 > /dev/null 2> %t/header.climits
-#if defined(TEST_17)
-#include <climits>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_18 > /dev/null 2> %t/header.clocale
-#if defined(TEST_18)
-#include <clocale>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_19 > /dev/null 2> %t/header.cmath
-#if defined(TEST_19)
-#include <cmath>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_20 > /dev/null 2> %t/header.codecvt
-#if defined(TEST_20)
-#include <codecvt>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_21 > /dev/null 2> %t/header.compare
-#if defined(TEST_21)
-#include <compare>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_22 > /dev/null 2> %t/header.complex
-#if defined(TEST_22)
-#include <complex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_24 > /dev/null 2> %t/header.concepts
-#if defined(TEST_24)
-#include <concepts>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_25 > /dev/null 2> %t/header.condition_variable
-#if defined(TEST_25)
-#include <condition_variable>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_26 > /dev/null 2> %t/header.coroutine
-#if defined(TEST_26)
-#include <coroutine>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_27 > /dev/null 2> %t/header.csetjmp
-#if defined(TEST_27)
-#include <csetjmp>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_28 > /dev/null 2> %t/header.csignal
-#if defined(TEST_28)
-#include <csignal>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_29 > /dev/null 2> %t/header.cstdarg
-#if defined(TEST_29)
-#include <cstdarg>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_30 > /dev/null 2> %t/header.cstdbool
-#if defined(TEST_30)
-#include <cstdbool>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_31 > /dev/null 2> %t/header.cstddef
-#if defined(TEST_31)
-#include <cstddef>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_32 > /dev/null 2> %t/header.cstdint
-#if defined(TEST_32)
-#include <cstdint>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_33 > /dev/null 2> %t/header.cstdio
-#if defined(TEST_33)
-#include <cstdio>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_34 > /dev/null 2> %t/header.cstdlib
-#if defined(TEST_34)
-#include <cstdlib>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_35 > /dev/null 2> %t/header.cstring
-#if defined(TEST_35)
-#include <cstring>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_36 > /dev/null 2> %t/header.ctgmath
-#if defined(TEST_36)
-#include <ctgmath>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_37 > /dev/null 2> %t/header.ctime
-#if defined(TEST_37)
-#include <ctime>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_39 > /dev/null 2> %t/header.cuchar
-#if defined(TEST_39)
-#include <cuchar>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_40 > /dev/null 2> %t/header.cwchar
-#if defined(TEST_40)
-#include <cwchar>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_41 > /dev/null 2> %t/header.cwctype
-#if defined(TEST_41)
-#include <cwctype>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_42 > /dev/null 2> %t/header.deque
-#if defined(TEST_42)
-#include <deque>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_44 > /dev/null 2> %t/header.exception
-#if defined(TEST_44)
-#include <exception>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_45 > /dev/null 2> %t/header.execution
-#if defined(TEST_45)
-#include <execution>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_46 > /dev/null 2> %t/header.expected
-#if defined(TEST_46)
-#include <expected>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_48 > /dev/null 2> %t/header.filesystem
-#if defined(TEST_48)
-#include <filesystem>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_50 > /dev/null 2> %t/header.format
-#if defined(TEST_50)
-#include <format>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_51 > /dev/null 2> %t/header.forward_list
-#if defined(TEST_51)
-#include <forward_list>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_52 > /dev/null 2> %t/header.fstream
-#if defined(TEST_52)
-#include <fstream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_53 > /dev/null 2> %t/header.functional
-#if defined(TEST_53)
-#include <functional>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_54 > /dev/null 2> %t/header.future
-#if defined(TEST_54)
-#include <future>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_55 > /dev/null 2> %t/header.initializer_list
-#if defined(TEST_55)
-#include <initializer_list>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_57 > /dev/null 2> %t/header.iomanip
-#if defined(TEST_57)
-#include <iomanip>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_58 > /dev/null 2> %t/header.ios
-#if defined(TEST_58)
-#include <ios>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_59 > /dev/null 2> %t/header.iosfwd
-#if defined(TEST_59)
-#include <iosfwd>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_60 > /dev/null 2> %t/header.iostream
-#if defined(TEST_60)
-#include <iostream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_61 > /dev/null 2> %t/header.istream
-#if defined(TEST_61)
-#include <istream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_62 > /dev/null 2> %t/header.iterator
-#if defined(TEST_62)
-#include <iterator>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_63 > /dev/null 2> %t/header.latch
-#if defined(TEST_63)
-#include <latch>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_64 > /dev/null 2> %t/header.limits
-#if defined(TEST_64)
-#include <limits>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_66 > /dev/null 2> %t/header.list
-#if defined(TEST_66)
-#include <list>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_67 > /dev/null 2> %t/header.locale
-#if defined(TEST_67)
-#include <locale>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_69 > /dev/null 2> %t/header.map
-#if defined(TEST_69)
-#include <map>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_71 > /dev/null 2> %t/header.mdspan
-#if defined(TEST_71)
-#include <mdspan>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_72 > /dev/null 2> %t/header.memory
-#if defined(TEST_72)
-#include <memory>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_73 > /dev/null 2> %t/header.memory_resource
-#if defined(TEST_73)
-#include <memory_resource>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_74 > /dev/null 2> %t/header.mutex
-#if defined(TEST_74)
-#include <mutex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_75 > /dev/null 2> %t/header.new
-#if defined(TEST_75)
-#include <new>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_76 > /dev/null 2> %t/header.numbers
-#if defined(TEST_76)
-#include <numbers>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_77 > /dev/null 2> %t/header.numeric
-#if defined(TEST_77)
-#include <numeric>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_78 > /dev/null 2> %t/header.optional
-#if defined(TEST_78)
-#include <optional>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_79 > /dev/null 2> %t/header.ostream
-#if defined(TEST_79)
-#include <ostream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_80 > /dev/null 2> %t/header.queue
-#if defined(TEST_80)
-#include <queue>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_81 > /dev/null 2> %t/header.random
-#if defined(TEST_81)
-#include <random>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_82 > /dev/null 2> %t/header.ranges
-#if defined(TEST_82)
-#include <ranges>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_83 > /dev/null 2> %t/header.ratio
-#if defined(TEST_83)
-#include <ratio>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_84 > /dev/null 2> %t/header.regex
-#if defined(TEST_84)
-#include <regex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_85 > /dev/null 2> %t/header.scoped_allocator
-#if defined(TEST_85)
-#include <scoped_allocator>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_86 > /dev/null 2> %t/header.semaphore
-#if defined(TEST_86)
-#include <semaphore>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_87 > /dev/null 2> %t/header.set
-#if defined(TEST_87)
-#include <set>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_89 > /dev/null 2> %t/header.shared_mutex
-#if defined(TEST_89)
-#include <shared_mutex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_90 > /dev/null 2> %t/header.source_location
-#if defined(TEST_90)
-#include <source_location>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_91 > /dev/null 2> %t/header.span
-#if defined(TEST_91)
-#include <span>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_92 > /dev/null 2> %t/header.sstream
-#if defined(TEST_92)
-#include <sstream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_93 > /dev/null 2> %t/header.stack
-#if defined(TEST_93)
-#include <stack>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_97 > /dev/null 2> %t/header.stdexcept
-#if defined(TEST_97)
-#include <stdexcept>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_101 > /dev/null 2> %t/header.stop_token
-#if defined(TEST_101)
-#include <stop_token>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_102 > /dev/null 2> %t/header.streambuf
-#if defined(TEST_102)
-#include <streambuf>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_103 > /dev/null 2> %t/header.string
-#if defined(TEST_103)
-#include <string>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_105 > /dev/null 2> %t/header.string_view
-#if defined(TEST_105)
-#include <string_view>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_106 > /dev/null 2> %t/header.strstream
-#if defined(TEST_106)
-#include <strstream>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_107 > /dev/null 2> %t/header.system_error
-#if defined(TEST_107)
-#include <system_error>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_109 > /dev/null 2> %t/header.thread
-#if defined(TEST_109)
-#include <thread>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_110 > /dev/null 2> %t/header.tuple
-#if defined(TEST_110)
-#include <tuple>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_111 > /dev/null 2> %t/header.type_traits
-#if defined(TEST_111)
-#include <type_traits>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_112 > /dev/null 2> %t/header.typeindex
-#if defined(TEST_112)
-#include <typeindex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_113 > /dev/null 2> %t/header.typeinfo
-#if defined(TEST_113)
-#include <typeinfo>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_115 > /dev/null 2> %t/header.unordered_map
-#if defined(TEST_115)
-#include <unordered_map>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_116 > /dev/null 2> %t/header.unordered_set
-#if defined(TEST_116)
-#include <unordered_set>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_117 > /dev/null 2> %t/header.utility
-#if defined(TEST_117)
-#include <utility>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_118 > /dev/null 2> %t/header.valarray
-#if defined(TEST_118)
-#include <valarray>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_119 > /dev/null 2> %t/header.variant
-#if defined(TEST_119)
-#include <variant>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_120 > /dev/null 2> %t/header.vector
-#if defined(TEST_120)
-#include <vector>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_121 > /dev/null 2> %t/header.version
-#if defined(TEST_121)
-#include <version>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_124 > /dev/null 2> %t/header.experimental_deque
-#if defined(TEST_124)
-#include <experimental/deque>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_125 > /dev/null 2> %t/header.experimental_forward_list
-#if defined(TEST_125)
-#include <experimental/forward_list>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_126 > /dev/null 2> %t/header.experimental_iterator
-#if defined(TEST_126)
-#include <experimental/iterator>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_127 > /dev/null 2> %t/header.experimental_list
-#if defined(TEST_127)
-#include <experimental/list>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_128 > /dev/null 2> %t/header.experimental_map
-#if defined(TEST_128)
-#include <experimental/map>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_129 > /dev/null 2> %t/header.experimental_memory_resource
-#if defined(TEST_129)
-#include <experimental/memory_resource>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_130 > /dev/null 2> %t/header.experimental_propagate_const
-#if defined(TEST_130)
-#include <experimental/propagate_const>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_131 > /dev/null 2> %t/header.experimental_regex
-#if defined(TEST_131)
-#include <experimental/regex>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_132 > /dev/null 2> %t/header.experimental_set
-#if defined(TEST_132)
-#include <experimental/set>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_133 > /dev/null 2> %t/header.experimental_simd
-#if defined(TEST_133)
-#include <experimental/simd>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_134 > /dev/null 2> %t/header.experimental_string
-#if defined(TEST_134)
-#include <experimental/string>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_135 > /dev/null 2> %t/header.experimental_type_traits
-#if defined(TEST_135)
-#include <experimental/type_traits>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_136 > /dev/null 2> %t/header.experimental_unordered_map
-#if defined(TEST_136)
-#include <experimental/unordered_map>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_137 > /dev/null 2> %t/header.experimental_unordered_set
-#if defined(TEST_137)
-#include <experimental/unordered_set>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_138 > /dev/null 2> %t/header.experimental_utility
-#if defined(TEST_138)
-#include <experimental/utility>
-#endif
-// RUN: %{cxx} %s %{flags} %{compile_flags} --trace-includes -fshow-skipped-includes --preprocess -DTEST_139 > /dev/null 2> %t/header.experimental_vector
-#if defined(TEST_139)
-#include <experimental/vector>
-#endif
-// RUN: %{python} %S/transitive_includes_to_csv.py %t > %t/transitive_includes.csv
-// RUN: diff -w %S/transitive_includes/%{cxx_std}.csv %t/transitive_includes.csv
-// GENERATED-MARKER

libcxx/test/libcxx/transitive_includes_to_csv.py

 @dataclass
 class header:
     name: str = None
     level: int = -1
 
 
 def parse_line(line: str) -> header:
     """
-    Parse an output line from --trace-include into a `header`.
+    Parse an output line from --trace-includes into a `header`.
     """
     match = re.match(r"(\.+) (.+)", line)
     if not match:
         sys.exit(f"Line {line} contains invalid data.")
 
     # The number of periods in front of the header name is the nesting level of
     # that header.
     return header(match.group(2), len(match.group(1)))
 
 
 # On Windows, the path separators can either be forward slash or backslash.
 # If it is a backslash, Clang prints it escaped as two consecutive
 # backslashes, and they need to be escaped in the RE. (Use a raw string for
 # the pattern to avoid needing another level of escaping on the Python string
 # literal level.)
 LIBCXX_HEADER_REGEX = r".*c\+\+(?:/|\\\\)v[0-9]+(?:/|\\\\)(.+)"
 
-
-def is_libcxx_public_header(header: str) -> bool:
-    """
-    Returns whether a header is a C++ public header file.
-    """
-    # Only keep files in the c++/vN directory.
-    match = re.match(LIBCXX_HEADER_REGEX, header)
-    if not match:
-        return False
-
-    # Skip C compatibility headers.
-    if header.endswith(".h"):
-        return False
-
-    # Skip all other detail headers (headers starting with __ or in a subdirectory starting with __).
-    relative = match.group(1)
-    if relative.startswith("__") or re.search(r"(/|\\\\)__", relative):
-        return False
-
-    return True
-
-
 def is_libcxx_header(header: str) -> bool:
     """
     Returns whether a header is a libc++ header, excluding the C-compatibility headers.
     """
     # Only keep files in the c++/vN directory.
     match = re.match(LIBCXX_HEADER_REGEX, header)
     if not match:
         return False
with file.open(encoding="utf-8") as f:
                 continue
 
             # Add the non-detail libc++ header to the list.
             level = header.level
             result.append(header)
     return result
 
 
-def create_include_graph(path: pathlib.Path) -> List[str]:
+def create_include_graph(trace_includes: List[pathlib.Path]) -> List[str]:
     result = list()
-    for file in sorted(path.glob("header.*")):
+    for file in trace_includes:
         headers = parse_file(file)
 
         # Get actual filenames relative to libc++'s installation directory instead of full paths
         relative = lambda h: re.match(LIBCXX_HEADER_REGEX, h).group(1)
 
         top_level = relative(
             next(h.name for h in headers if h.level == 1)
         )  # There should be only one top-level header
for includes in graph:
             if header == include:
                 sys.exit(f"Cycle detected: header {header} includes itself.")
             print(f"{header} {include}")
 
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser(
         description="""Produce a dependency graph of libc++ headers, in CSV format.
-Typically this script is executed by libcxx/test/libcxx/transitive_includes.sh.cpp""",
+This script is normally executed by libcxx/test/libcxx/transitive_includes.gen.py""",
         formatter_class=argparse.RawDescriptionHelpFormatter,
     )
     parser.add_argument(
-        "input",
+        "inputs",
         default=None,
-        metavar="DIR",
-        help="The directory containing the transitive includes of the headers.",
+        metavar="FILE",
+        nargs='+',
+        help="One or more files containing the result of --trace-includes on the headers one wishes to graph.",
     )
     options = parser.parse_args()
 
-    root = pathlib.Path(options.input)
-    print_csv(create_include_graph(root))
+    print_csv(create_include_graph(map(pathlib.Path, options.inputs)))

libcxx/utils/CMakeLists.txt

-add_custom_target(libcxx-generate-public-header-tests
-    COMMAND "${Python3_EXECUTABLE}" "${LIBCXX_SOURCE_DIR}/utils/generate_header_tests.py"
-    COMMENT "Generate tests for including public headers.")
-
 add_custom_target(libcxx-generate-feature-test-macros
     COMMAND "${Python3_EXECUTABLE}" "${LIBCXX_SOURCE_DIR}/utils/generate_feature_test_macro_components.py"
     COMMENT "Generate the <version> header and tests for feature test macros.")
 
 add_custom_target(libcxx-generate-extended-grapheme-cluster-tables
     COMMAND
         "${Python3_EXECUTABLE}"
         "${LIBCXX_SOURCE_DIR}/utils/generate_extended_grapheme_cluster_table.py"

 
 add_custom_target(libcxx-generate-iwyu-mapping
     COMMAND
         "${Python3_EXECUTABLE}"
         "${LIBCXX_SOURCE_DIR}/utils/generate_iwyu_mapping.py"
     COMMENT "Generate the mapping file for include-what-you-use")
 
 add_custom_target(libcxx-generate-files
-    DEPENDS libcxx-generate-public-header-tests
-            libcxx-generate-feature-test-macros
+    DEPENDS libcxx-generate-feature-test-macros
             libcxx-generate-extended-grapheme-cluster-tables
             libcxx-generate-extended-grapheme-cluster-tests
             libcxx-generate-escaped-output-table
             libcxx-generate-width-estimation-table
             libcxx-generate-iwyu-mapping
     COMMENT "Create all the auto-generated files in libc++ and its tests.")

libcxx/utils/generate_header_tests.py

-#!/usr/bin/env python
-
-import contextlib
-import glob
-import io
-import os
-import pathlib
-import re
-
-import libcxx.test.header_information
-
-
-def find_script(file):
-    """Finds the script used to generate a file inside the file itself. The script is delimited by
-    BEGIN-SCRIPT and END-SCRIPT markers.
-    """
-    with open(file, "r") as f:
-        content = f.read()
-
-    match = re.search(
-        r"^BEGIN-SCRIPT$(.+)^END-SCRIPT$", content, flags=re.MULTILINE | re.DOTALL
-    )
-    if not match:
-        raise RuntimeError(
-            "Was unable to find a script delimited with BEGIN-SCRIPT/END-SCRIPT markers in {}".format(
-                test_file
-            )
-        )
-    return match.group(1)
-
-
-def execute_script(script, variables):
-    """Executes the provided Mako template with the given variables available during the
-    evaluation of the script, and returns the result.
-    """
-    code = compile(script, "fake-filename", "exec")
-    output = io.StringIO()
-    with contextlib.redirect_stdout(output):
-        exec(code, variables)
-        output = output.getvalue()
-    return output
-
-
-def generate_new_file(file, new_content):
-    """Generates the new content of the file by inserting the new content in-between
-    two '// GENERATED-MARKER' markers located in the file.
-    """
-    with open(file, "r") as f:
-        old_content = f.read()
-
-    try:
-        before, begin_marker, _, end_marker, after = re.split(
-            r"(// GENERATED-MARKER\n)", old_content, flags=re.MULTILINE | re.DOTALL
-        )
-    except ValueError:
-        raise RuntimeError(
-            "Failed to split {} based on markers, please make sure the file has exactly two '// GENERATED-MARKER' occurrences".format(
-                file
-            )
-        )
-
-    return before + begin_marker + new_content + end_marker + after
-
-
-def produce(test_file, variables):
-    script = find_script(test_file)
-    result = execute_script(script, variables)
-    new_content = generate_new_file(test_file, result)
-    with open(test_file, "w", newline="\n") as f:
-        f.write(new_content)
-
-
-def main():
-    monorepo_root = pathlib.Path(
-        os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
-    )
-    test = pathlib.Path(os.path.join(monorepo_root, "libcxx", "test"))
-    assert monorepo_root.exists()
-
-    produce(test.joinpath("libcxx/transitive_includes.sh.cpp"), libcxx.test.header_information.variables)
-
-
-if __name__ == "__main__":
-    main()

libcxx/utils/libcxx/test/header_information.py

experimental_headers = sorted(
     p.relative_to(include).as_posix() for p in include.glob("experimental/[a-z]*") if is_header(p)
 )
 public_headers = toplevel_headers + experimental_headers
 private_headers = sorted(
     p.relative_to(include).as_posix() for p in include.rglob("*") if is_header(p)
                                                                      and str(p.relative_to(include)).startswith("__")
                                                                      and not p.name.startswith("pstl")
 )
-variables = {
-    "toplevel_headers": toplevel_headers,
-    "experimental_headers": experimental_headers,
-    "public_headers": public_headers,
-    "private_headers": private_headers,
-    "header_restrictions": header_restrictions,
-    "private_headers_still_public_in_modules": private_headers_still_public_in_modules,
-}