[libc++] Add .fuzz.cpp tests and move the fuzzing tests to the normal locations

[ldionne]

libcxx/test/libcxx/fuzzing/unique.pass.cpp

1

From https://buildkite.com/llvm-project/libcxx-ci/builds/27285#018908b8-92d3-42af-9094-090e2ac9773b:

St11char_traitsIcESaIcEEE[_ZNK6fuzzer7Command12getFlagValueERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x2c9): undefined reference to `std::__throw_length_error(char const*)'
/usr/bin/ld: (.text._ZNK6fuzzer7Command12getFlagValueERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZNK6fuzzer7Command12getFlagValueERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x2e1): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
/usr/bin/ld: /usr/lib/llvm-17/lib/clang/17/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerFork.cpp.o): in function `std::vector<unsigned long, std::allocator<unsigned long> >::insert(__gnu_cxx::__normal_iterator<unsigned long const*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long const&)':
(.text._ZNSt6vectorImSaImEE6insertEN9__gnu_cxx17__normal_iteratorIPKmS1_EERS4_[_ZNSt6vectorImSaImEE6insertEN9__gnu_cxx17__normal_iteratorIPKmS1_EERS4_]+0x280): undefined reference to `std::__throw_length_error(char const*)'
/usr/bin/ld: /usr/lib/llvm-17/lib/clang/17/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerFork.cpp.o): in function `void std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile> >::_M_realloc_insert<fuzzer::SizedFile const&>(__gnu_cxx::__normal_iterator<fuzzer::SizedFile*, std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile> > >, fuzzer::SizedFile const&)':
(.text._ZNSt6vectorIN6fuzzer9SizedFileESaIS1_EE17_M_realloc_insertIJRKS1_EEEvN9__gnu_cxx17__normal_iteratorIPS1_S3_EEDpOT_[_ZNSt6vectorIN6fuzzer9SizedFileESaIS1_EE17_M_realloc_insertIJRKS1_EEEvN9__gnu_cxx17__normal_iteratorIPS1_S3_EEDpOT_]+0xe9): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: 
...

So it turns out that on our CI machine (which runs Linux), /usr/lib/llvm-17/lib/clang/17/lib/linux/libclang_rt.fuzzer-x86_64.a has been compiled against libstdc++. I guess that's because compiler-rt is compiled against libstdc++ by default in LLVM releases. However, the result is that we can't use that part of compiler-rt with -nostdlib++.

@kcc Did you folks think about a way that the stdlib itself could use fuzzing? Are we doing something fundamentally wrong, or do we need to basically build our own compiler-rt against libc++ in order to do this?

[ldionne]

libcxx/test/libcxx/fuzzing/unique.pass.cpp

1

Other options:

1. Only run the fuzzing tests when we're doing a bootstrapping build in the CI, since then we can (?) easily control which stdlib compiler-rt is built against (I think). I think COMPILER_RT_CXX_LIBRARY="libcxx" will do that.
2. Always build a version of compiler-rt against libc++ in our CI and pass that to the test suite so it can set the right --rtlib= flag in Clang. We'd only want to do that for fuzzing tests, since we definitely don't want to customize the compiler-rt in use for our general tests.

[philnik]

Address comments

[philnik]

libcxx/test/libcxx/fuzzing/unique.pass.cpp

1

I went for just running it in the bootstrapping build for now. We can try to build compiler-rt everywhere later.

[philnik]

Try to fix CI

[ldionne]

libcxx/utils/libcxx/test/features.py

103–111

libcxx/utils/libcxx/test/format.py

1

You should take a look at libcxx/utils/ci/oss-fuzz.sh, it will need to be fixed. LIBCPP_OSS_FUZZ won't be needed anymore.

220–225

Let's remove -O3.

221–223

377–380

Let's not hardcode -O3 in fuzz tests. We should instead run the test suite with optimizations enabled when we mean to. And we should also run these fuzzing tests under OSS fuzz.

379

Instead of introducing %{run_flags}, let's hardcode -max_total_time here in the test format. Eventually we might make it something we can customize via the lit config (globally) if there's a use case for it.

Otherwise we're adding complexity (RUN_FLAGS: support) for a very tiny use case.

[philnik]

Address comments

[ldionne]

Pretty much LGTM after applying feedback.

libcxx/utils/ci/run-buildbot

343

Let's add - "**/crash-" to the artifacts paths in buildkite-pipeline.yml. Also we probably want to use a different crash file pattern like libcxx-fuzz-crash-XXXXXXXXXXXXXXXXX if that's possible.

libcxx/utils/libcxx/test/dsl.py

208

Let's add a docstring explaining what this does.

208–213

This also means you're missing tests for this function.

libcxx/utils/libcxx/test/format.py

379

Let's also add a timeout for each function to execute, that way we won't blow up if calling e.g. std::regex with some crazy input leads to an infinite loop.

[ldionne]

libcxx/test/libcxx/selftest/fuzz.cpp/compile-error.fuzz.cpp

2

Not attached: we should mark the tests that are failing right now as UNSUPPORTED temporarily and then we can enable them again as we fix those issues. Otherwise this review might be blocked for a long time.

[Mordante]

Thanks for working on this! Should we mention fuzzing on the libc++ contributing page. Until this patch I wasn't even aware we had a fuzzing framework. I would say that format should have fuzzing too, @philnik already asked about it, and I will do it.

[philnik]

In D154140#4485554, @Mordante wrote:

Thanks for working on this! Should we mention fuzzing on the libc++ contributing page. Until this patch I wasn't even aware we had a fuzzing framework. I would say that format should have fuzzing too, @philnik already asked about it, and I will do it.

I think we should first find out what we actually want to fuzz and how to properly do it before we mention anything on the contributing page. After all, these should be just additional coverage, not the main thing we use for testing.

[Mordante]

In D154140#4485596, @philnik wrote:

In D154140#4485554, @Mordante wrote:

Thanks for working on this! Should we mention fuzzing on the libc++ contributing page. Until this patch I wasn't even aware we had a fuzzing framework. I would say that format should have fuzzing too, @philnik already asked about it, and I will do it.

I think we should first find out what we actually want to fuzz and how to properly do it before we mention anything on the contributing page. After all, these should be just additional coverage, not the main thing we use for testing.

Fair point. My general idea is to fuzz everything where the library acts on user defined input. I guess that is mainly parsers, where I consider format specifiers and regexes also parsing.

[philnik]

Address comments

[ldionne]

LGTM w/ CI and comments applied.

libcxx/test/libcxx/selftest/dsl/dsl.sh.py

243

Please add simple tests for compileAndRunSucceeds.

libcxx/test/std/re/regex.fuzz.cpp

13

14

[philnik]

Address comments

[mclow.lists]

OSS-Fuzz pulls these fuzzing tests and runs them all the time.
If you move them, then you should update the settings at OSS-Fuzz.

[philnik]

In D154140#4490396, @mclow.lists wrote:

OSS-Fuzz pulls these fuzzing tests and runs them all the time.
If you move them, then you should update the settings at OSS-Fuzz.

IIUC this is handled through the changes to libcxx/utils/ci/oss-fuzz.sh.

[philnik]

Try to fix CI

libcxx/test/libcxx/selftest/dsl/dsl.sh.py

def test_program_stderr_is_not_conflated_with_stdout(self):
             std::fprintf(stdout, "STDOUT-OUTPUT");
             std::fprintf(stderr, "STDERR-OUTPUT");
             return 0;
         }
         """
         self.assertEqual(dsl.programOutput(self.config, source), "STDOUT-OUTPUT")
 
 
-class TestProgramSucceeds(SetupConfigs):
+class TestRunSucceeds(SetupConfigs):
     """
-    Tests for libcxx.test.dsl.programSucceeds
+    Tests for libcxx.test.dsl.runSucceeds
     """
 
     def test_success(self):
         source = """
         int main(int, char**) { return 0; }
         """
-        self.assertTrue(dsl.programSucceeds(self.config, source))
+        self.assertTrue(dsl.runSucceeds(self.config, source))
 
     def test_failure(self):
         source = """
         int main(int, char**) { return 1; }
         """
-        self.assertFalse(dsl.programSucceeds(self.config, source))
+        self.assertFalse(dsl.runSucceeds(self.config, source))
 
     def test_compile_failure(self):
         source = """
         this does not compile
         """
         self.assertRaises(
             dsl.ConfigurationCompilationError,
-            lambda: dsl.programSucceeds(self.config, source),
+            lambda: dsl.runSucceeds(self.config, source),
         )
 
+class TestCompileAndRunSucceeds(SetupConfigs):
+    """
+    Tests for libcxx.test.dsl.compileAndRunSucceeds
+    """
+
+    def test_success(self):
+        source = """
+        int main(int, char**) { return 0; }
+        """
+        self.assertTrue(dsl.compileAndRunSucceeds(self.config, source))
+
+    def test_failure(self):
+        source = """
+        int main(int, char**) { return 1; }
+        """
+        self.assertFalse(dsl.compileAndRunSucceeds(self.config, source))
+
+    def test_compile_failure(self):
+        source = """
+        this does not compile
+        """
+        self.assertFalse(dsl.compileAndRunSucceeds(self.config, source))
 
 class TestHasLocale(SetupConfigs):
     """
     Tests for libcxx.test.dsl.hasLocale
     """
 
     def test_doesnt_explode(self):
         # It's really hard to test that a system has a given locale, so at least

libcxx/test/libcxx/selftest/fuzz.cpp/compile-error.fuzz.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// XFAIL: *
+
+// Make sure the test DOES NOT pass if it fails at compile-time
+
+#include <cstddef>
+#include <cstdint>
+
+struct Foo {};
+typedef Foo::x x;
+
+extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t*, std::size_t) { return 0; }

libcxx/test/libcxx/selftest/fuzz.cpp/link-error.fuzz.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// XFAIL: *
+
+// Make sure the test DOES NOT pass if it fails at link-time
+
+#include <cstddef>
+#include <cstdint>
+
+extern void this_is_an_undefined_symbol();
+
+extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t*, std::size_t) {
+  this_is_an_undefined_symbol();
+
+  return 0;
+}

libcxx/test/libcxx/selftest/fuzz.cpp/run-error.fuzz.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// XFAIL: *
+
+// Make sure the test DOES NOT pass if it fails at runtime.
+
+#include <cstddef>
+#include <cstdint>
+
+extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t*, std::size_t) {
+  volatile int* ptr = nullptr;
+  volatile int val  = *ptr;
+  (void)val;
+
+  return 0;
+}

libcxx/test/libcxx/selftest/fuzz.cpp/run-success.fuzz.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// Make sure the test passes pass if it succeeds at runtime.
+
+#include <cstddef>
+#include <cstdint>
+
+extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t*, std::size_t) { return 0; }

libcxx/test/std/algorithms/alg.modifying.operations/alg.unique/unique.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     std::vector<std::uint8_t> working(data, data + size);
     std::sort(working.begin(), working.end());
     std::vector<std::uint8_t> results = working;
     std::vector<std::uint8_t>::iterator new_end = std::unique(results.begin(), results.end());
     std::vector<std::uint8_t>::iterator it; // scratch iterator
 

libcxx/test/std/algorithms/alg.modifying.operations/alg.unique/unique_copy.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <iterator>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     std::vector<std::uint8_t> working(data, data + size);
     std::sort(working.begin(), working.end());
     std::vector<std::uint8_t> results;
     (void)std::unique_copy(working.begin(), working.end(),
                            std::back_inserter<std::vector<std::uint8_t>>(results));
     std::vector<std::uint8_t>::iterator it; // scratch iterator

libcxx/test/std/algorithms/alg.nonmodifying/alg.search/search.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cassert>
 #include <cstddef>
 #include <cstdint>
 #include <limits>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size < 2)
         return 0;
 
     const std::size_t pat_size = data[0] * (size - 1) / std::numeric_limits<uint8_t>::max();
     assert(pat_size <= size - 1);
     const std::uint8_t *pat_begin = data + 1;

libcxx/test/std/algorithms/alg.sorting/alg.heap.operations/make.heap/make_heap.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     std::vector<std::uint8_t> working(data, data + size);
     std::make_heap(working.begin(), working.end());
 
     if (!std::is_heap(working.begin(), working.end()))
         return 1;
     if (!fast_is_permutation(data, data + size, working.cbegin()))
         return 99;
     return 0;
 }

libcxx/test/std/algorithms/alg.sorting/alg.heap.operations/pop.heap/pop_heap.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size < 2)
         return 0;
     std::vector<std::uint8_t> working(data, data + size);
     std::make_heap(working.begin(), working.end());
 
     // Pop things off, one at a time

libcxx/test/std/algorithms/alg.sorting/alg.heap.operations/push.heap/push_heap.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size < 2)
         return 0;
 
     // Make a heap from the first half of the data
     std::vector<std::uint8_t> working(data, data + size);
     auto iter = working.begin() + (size / 2);

libcxx/test/std/algorithms/alg.sorting/alg.nth.element/nth_element.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 // Use the first element as a position into the data
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size <= 1) return 0;
     const std::size_t partition_point = data[0] % size;
     std::vector<std::uint8_t> working(data + 1, data + size);
     const auto partition_iter = working.begin() + partition_point;
     std::nth_element(working.begin(), partition_iter, working.end());

libcxx/test/std/algorithms/alg.sorting/alg.partitions/partition.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     auto is_even = [](auto x) { return x % 2 == 0; };
     std::vector<std::uint8_t> working(data, data + size);
     auto iter = std::partition(working.begin(), working.end(), is_even);
 
     if (!std::all_of(working.begin(), iter, is_even))
         return 1;
     if (!std::none_of(iter,   working.end(), is_even))
         return 2;
     if (!fast_is_permutation(data, data + size, working.cbegin()))
         return 99;
     return 0;
 }

libcxx/test/std/algorithms/alg.sorting/alg.partitions/partition_copy.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <iterator>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     auto is_even = [](auto t) {
         return t % 2 == 0;
     };
 
     std::vector<std::uint8_t> v1, v2;
     auto iter = std::partition_copy(data, data + size,

libcxx/test/std/algorithms/alg.sorting/alg.partitions/stable_partition.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     auto is_even = [](auto b) { return b.key % 2 == 0; };
 
     std::vector<ByteWithPayload> input;
     for (std::size_t i = 0; i < size; ++i)
         input.push_back(ByteWithPayload(data[i], i));
     std::vector<ByteWithPayload> working = input;

libcxx/test/std/algorithms/alg.sorting/alg.sort/partial.sort.copy/partial_sort_copy.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 // Use the first element as a count
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size <= 1)
         return 0;
     const std::size_t num_results = data[0] % size;
     std::vector<std::uint8_t> results(num_results);
     (void)std::partial_sort_copy(data + 1, data + size, results.begin(), results.end());

libcxx/test/std/algorithms/alg.sorting/alg.sort/partial.sort/partial_sort.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 // Use the first element as a position into the data
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     if (size <= 1)
         return 0;
     const std::size_t sort_point = data[0] % size;
     std::vector<std::uint8_t> working(data + 1, data + size);
     const auto sort_iter = working.begin() + sort_point;

libcxx/test/std/algorithms/alg.sorting/alg.sort/sort/sort.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     std::vector<std::uint8_t> working(data, data + size);
     std::sort(working.begin(), working.end());
 
     if (!std::is_sorted(working.begin(), working.end()))
         return 1;
     if (!fast_is_permutation(data, data + size, working.cbegin()))
         return 99;
     return 0;
 }

libcxx/test/std/algorithms/alg.sorting/alg.sort/stable.sort/stable_sort.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
 #include <algorithm>
 #include <cstddef>
 #include <cstdint>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *data, std::size_t size) {
     std::vector<ByteWithPayload> input;
     for (std::size_t i = 0; i < size; ++i)
         input.push_back(ByteWithPayload(data[i], i));
 
     std::vector<ByteWithPayload> working = input;
     std::stable_sort(working.begin(), working.end(), ByteWithPayload::key_less());

libcxx/test/std/numerics/rand/rand.dist/random.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 
+// TODO: This seems to get into an infinite loop with -seed=1412493276. Find out what the problem is.
+// UNSUPPORTED: stdlib=libc++
+
 #include <cassert>
 #include <cmath>
 #include <cstddef>
 #include <cstdint>
 #include <cstring>
 #include <random>
 #include <type_traits>
 #include <vector>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 template <class IntT>
 std::vector<IntT> GetValues(const std::uint8_t *data, std::size_t size) {
   std::vector<IntT> result;
   while (size >= sizeof(IntT)) {
     IntT tmp;
     std::memcpy(&tmp, data, sizeof(IntT));
     size -= sizeof(IntT);

libcxx/test/std/re/regex.fuzz.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11
 // UNSUPPORTED: no-exceptions
 // UNSUPPORTED: no-localization
 
+// TODO: regex crashes on a bunch of inputs. Fix them to allow the fuzzer to run at least 10 seconds without crashing.
+// UNSUPPORTED: stdlib=libc++
+
 #include <cstddef>
 #include <cstdint>
 #include <regex>
 #include <string>
 
-#include "fuzz.h"
+#include "fuzz_helper.h"
 
 template <std::regex_constants::syntax_option_type Syntax>
 static int regex_test(const std::uint8_t *data, std::size_t size) {
     if (size == 0)
         return 0;
 
     std::string s((const char *)data, size);
     std::regex re;

libcxx/test/support/fuzz_helper.h

bool fast_is_permutation(Iter1 first1, Iter1 last1, Iter2 first2) {
 
   for (std::size_t i = 0; i < 256; ++i)
     if (xBuckets[i] != yBuckets[i])
       return false;
 
   return true;
 }
 
-// When running inside OSS-Fuzz, we link against a fuzzing library that defines
-// main() and calls LLVMFuzzerTestOneInput.
-//
-// Otherwise, when e.g. running the Lit tests, we define main() to run fuzzing
-// tests on a few inputs.
-#if !defined(LIBCPP_OSS_FUZZ)
-extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t*, std::size_t);
-
-int main(int, char**) {
-  const char* test_cases[] = {
-    "",
-    "s",
-    "bac",
-    "bacasf",
-    "lkajseravea",
-    "adsfkajdsfjkas;lnc441324513,34535r34525234",
-    "b*c",
-    "ba?sf",
-    "lka*ea",
-    "adsf*kas;lnc441[0-9]1r34525234"
-  };
-
-  for (const char* tc : test_cases) {
-    const std::size_t size = std::strlen(tc);
-    const std::uint8_t* data = reinterpret_cast<const std::uint8_t*>(tc);
-    int result = LLVMFuzzerTestOneInput(data, size);
-    assert(result == 0);
-  }
-
-  return 0;
-}
-#endif // !LIBCPP_OSS_FUZZ
-
 #endif // TEST_LIBCXX_FUZZING_FUZZ_H

libcxx/utils/ci/buildkite-pipeline.yml

steps:
   # Standard modes, with Clang and GCC. This catches most issues upfront.
   # The goal of this step is to catch most issues while being very fast.
   #
   - wait
 
   - label: "GCC ${GCC_STABLE_VERSION} / C++latest"
     command: "libcxx/utils/ci/run-buildbot generic-gcc"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "gcc-${GCC_STABLE_VERSION}"
         CXX: "g++-${GCC_STABLE_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++26"
     command: "libcxx/utils/ci/run-buildbot generic-cxx26"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Modular build"
     command: "libcxx/utils/ci/run-buildbot generic-modules"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++23 Module std"
     command: "libcxx/utils/ci/run-buildbot generic-module-std-cxx23"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         # Note modules require and absolute path for clang-scan-deps
         # https://github.com/llvm/llvm-project/issues/61006
         CC: "/usr/lib/llvm-${LLVM_HEAD_VERSION}/bin/clang"
         CXX: "/usr/lib/llvm-${LLVM_HEAD_VERSION}/bin/clang++"
         CMAKE: "/opt/bin/cmake"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++11"
     command: "libcxx/utils/ci/run-buildbot generic-cxx11"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++03"
     command: "libcxx/utils/ci/run-buildbot generic-cxx03"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   #
   # All other supported configurations of libc++.
   #
   - wait
 
   - label: "C++23"
     command: "libcxx/utils/ci/run-buildbot generic-cxx23"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++20"
     command: "libcxx/utils/ci/run-buildbot generic-cxx20"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++17"
     command: "libcxx/utils/ci/run-buildbot generic-cxx17"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "C++14"
     command: "libcxx/utils/ci/run-buildbot generic-cxx14"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   # Tests with the supported compilers.
   - label: "GCC ${GCC_STABLE_VERSION} / C++11"
     command: "libcxx/utils/ci/run-buildbot generic-gcc-cxx11"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "gcc-${GCC_STABLE_VERSION}"
         CXX: "g++-${GCC_STABLE_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Clang 15"
     command: "libcxx/utils/ci/run-buildbot generic-cxx23"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-15"
         CXX: "clang++-15"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Clang 16"
     command: "libcxx/utils/ci/run-buildbot generic-cxx23"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-16"
         CXX: "clang++-16"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   # Tests with the sanitizers.
   - group: "Sanitizers"
     steps:
     - label: "ASAN"
       command: "libcxx/utils/ci/run-buildbot generic-asan"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "TSAN"
       command: "libcxx/utils/ci/run-buildbot generic-tsan"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "UBSAN"
       command: "libcxx/utils/ci/run-buildbot generic-ubsan"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "MSAN"
       command: "libcxx/utils/ci/run-buildbot generic-msan"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   # Tests with the various supported ways to build libc++.
   - label: "Bootstrapping build"
     command: "libcxx/utils/ci/run-buildbot bootstrapping-build"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
       - "**/crash_diagnostics/*"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         LLVM_SYMBOLIZER_PATH: "/usr/bin/llvm-symbolizer-${LLVM_HEAD_VERSION}"
         CLANG_CRASH_DIAGNOSTICS_DIR: "crash_diagnostics"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   # Tests with various build configurations.
   - label: "Static libraries"
     command: "libcxx/utils/ci/run-buildbot generic-static"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Shared library with merged ABI and unwinder libraries"
     command: "libcxx/utils/ci/run-buildbot generic-merged"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Assertions enabled"
     command: "libcxx/utils/ci/run-buildbot generic-assertions"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "With LLVM's libunwind"
     command: "libcxx/utils/ci/run-buildbot generic-with_llvm_unwinder"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - label: "Modular build with Local Submodule Visibility"
     command: "libcxx/utils/ci/run-buildbot generic-modules-lsv"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   - group: "Parts disabled"
     steps:
     - label: "No threads"
       command: "libcxx/utils/ci/run-buildbot generic-no-threads"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No filesystem"
       command: "libcxx/utils/ci/run-buildbot generic-no-filesystem"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No random device"
       command: "libcxx/utils/ci/run-buildbot generic-no-random_device"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No locale"
       command: "libcxx/utils/ci/run-buildbot generic-no-localization"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No Unicode"
       command: "libcxx/utils/ci/run-buildbot generic-no-unicode"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No wide characters"
       command: "libcxx/utils/ci/run-buildbot generic-no-wide-characters"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No experimental features"
       command: "libcxx/utils/ci/run-buildbot generic-no-experimental"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "No exceptions"
       command: "libcxx/utils/ci/run-buildbot generic-no-exceptions"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang-${LLVM_HEAD_VERSION}"
           CXX: "clang++-${LLVM_HEAD_VERSION}"
           ENABLE_CLANG_TIDY: "On"
       agents:
         queue: "libcxx-builders"
         os: "linux"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   - label: "Unstable ABI"
     command: "libcxx/utils/ci/run-buildbot generic-abi-unstable"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   # Other non-testing CI jobs
   - label: "Benchmarks"
     command: "libcxx/utils/ci/run-buildbot benchmarks"
     artifact_paths:
+      - "**/crash-"
+      - "**/oom-"
+      - "**/timeout-"
       - "**/test-results.xml"
       - "**/*.abilist"
     env:
         CC: "clang-${LLVM_HEAD_VERSION}"
         CXX: "clang++-${LLVM_HEAD_VERSION}"
         ENABLE_CLANG_TIDY: "On"
     agents:
       queue: "libcxx-builders"
       os: "linux"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
     timeout_in_minutes: 120
 
   # Tests on non-Unix platforms
   - group: ":windows: Windows"
     steps:
     - label: "Clang-cl (DLL)"
       command: "bash libcxx/utils/ci/run-buildbot clang-cl-dll"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Clang-cl (Static)"
       command: "bash libcxx/utils/ci/run-buildbot clang-cl-static"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Clang-cl (no vcruntime exceptions)"
       command: "bash libcxx/utils/ci/run-buildbot clang-cl-no-vcruntime"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
 
     - label: "MinGW (DLL, x86_64)"
       command: "bash libcxx/utils/ci/run-buildbot mingw-dll"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "MinGW (Static, x86_64)"
       command: "bash libcxx/utils/ci/run-buildbot mingw-static"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "MinGW (DLL, i686)"
       command: "bash libcxx/utils/ci/run-buildbot mingw-dll-i686"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "windows"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   - group: ":apple: Apple"
     steps:
     - label: "MacOS x86_64"
       command: "libcxx/utils/ci/run-buildbot generic-cxx20"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "x86_64"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "MacOS arm64"
       command: "libcxx/utils/ci/run-buildbot generic-cxx20"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "arm64"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "MacOS with Modules"
       command: "libcxx/utils/ci/run-buildbot generic-modules"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     # Build with the configuration we use to generate libc++.dylib on Apple platforms
     - label: "Apple system"
       command: "libcxx/utils/ci/run-buildbot apple-system"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "arm64" # This can technically run on any architecture, but we have more resources on arm64 so we pin this job to arm64
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     # Test back-deployment to older Apple platforms
     - label: "Apple back-deployment macosx10.13"
       command: "libcxx/utils/ci/run-buildbot apple-system-backdeployment-10.13"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "x86_64" # We need to use x86_64 for back-deployment CI on this target since macOS didn't support arm64 back then.
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Apple back-deployment macosx10.15"
       command: "libcxx/utils/ci/run-buildbot apple-system-backdeployment-10.15"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "x86_64" # We need to use x86_64 for back-deployment CI on this target since macOS didn't support arm64 back then.
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Apple back-deployment macosx11.0 arm64"
       command: "libcxx/utils/ci/run-buildbot apple-system-backdeployment-11.0"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
         arch: "arm64"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Apple back-deployment with assertions enabled"
       command: "libcxx/utils/ci/run-buildbot apple-system-backdeployment-assertions-11.0"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders"
         os: "macos"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   - group: "ARM"
     steps:
     - label: "AArch64"
       command: "libcxx/utils/ci/run-buildbot aarch64"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "aarch64"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "AArch64 -fno-exceptions"
       command: "libcxx/utils/ci/run-buildbot aarch64-no-exceptions"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "aarch64"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Armv8"
       command: "libcxx/utils/ci/run-buildbot armv8"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "armv8l"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Armv8 -fno-exceptions"
       command: "libcxx/utils/ci/run-buildbot armv8-no-exceptions"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "armv8l"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Armv7"
       command: "libcxx/utils/ci/run-buildbot armv7"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "armv8l" # Compiling for v7, running on v8 hardware
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "Armv7 -fno-exceptions"
       command: "libcxx/utils/ci/run-buildbot armv7-no-exceptions"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       agents:
         queue: "libcxx-builders-linaro-arm"
         arch: "armv8l" # Compiling for v7, running on v8 hardware
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   - group: "AIX"
     steps:
     - label: "AIX (32-bit)"
       command: "libcxx/utils/ci/run-buildbot aix"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang"
           CXX: "clang++"
           OBJECT_MODE: "32"
       agents:
           queue: libcxx-builders
           os: aix
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
     - label: "AIX (64-bit)"
       command: "libcxx/utils/ci/run-buildbot aix"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang"
           CXX: "clang++"
           OBJECT_MODE: "64"
       agents:
           queue: libcxx-builders
           os: aix
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120
 
   - group: "FreeBSD"
     steps:
     - label: "FreeBSD 13 amd64"
       command: "libcxx/utils/ci/run-buildbot generic-cxx23"
       artifact_paths:
+        - "**/crash-"
+        - "**/oom-"
+        - "**/timeout-"
         - "**/test-results.xml"
         - "**/*.abilist"
       env:
           CC: "clang15"
           CXX: "clang++15"
       agents:
         queue: "libcxx-builders"
         os: "freebsd"
       retry:
         automatic:
           - exit_status: -1  # Agent was lost
             limit: 2
       timeout_in_minutes: 120

libcxx/utils/ci/oss-fuzz.sh

 
 mkdir ${BUILD}
 cmake -S ${MONOREPO_ROOT}/runtimes -B ${BUILD} \
       -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" \
       -DCMAKE_BUILD_TYPE=RelWithDebInfo \
       -DCMAKE_INSTALL_PREFIX="${INSTALL}"
 cmake --build ${BUILD} --target install-cxx-headers
 
-for test in libcxx/test/libcxx/fuzzing/*.pass.cpp; do
+for test in $(find libcxx/test -name *.fuzz.cpp); do
     exe="$(basename ${test})"
     exe="${exe%.pass.cpp}"
     ${CXX} ${CXXFLAGS} \
-        -std=c++14 \
-        -DLIBCPP_OSS_FUZZ \
+        -std=c++2b \
         -D_LIBCPP_HAS_NO_VENDOR_AVAILABILITY_ANNOTATIONS \
         -nostdinc++ -cxx-isystem ${INSTALL}/include/c++/v1 \
         -lpthread -ldl \
         -o "${OUT}/${exe}" \
         ${test} \
         ${LIB_FUZZING_ENGINE}
 done

libcxx/utils/ci/run-buildbot

generic-tsan)
     generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-tsan.cmake"
     check-runtimes
 ;;
 generic-ubsan)
     clean
     generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-ubsan.cmake"
     check-runtimes
 ;;
 #
 # Various build configurations
 #
 bootstrapping-build)
     clean
 
     echo "--- Generating CMake"
     ${CMAKE} \
-          -S "${MONOREPO_ROOT}/llvm" \
-          -B "${BUILD_DIR}" \
-          -GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}" \
-          -DCMAKE_CXX_COMPILER_LAUNCHER="ccache" \
-          -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-          -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" \
-          -DLLVM_ENABLE_PROJECTS="clang" \
-          -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind" \
-          -DLLVM_RUNTIME_TARGETS="$(${CXX} --print-target-triple)" \
-          -DLLVM_TARGETS_TO_BUILD="host" \
-          -DRUNTIMES_BUILD_ALLOW_DARWIN=ON \
-          -DLLVM_ENABLE_ASSERTIONS=ON
+          -S "${MONOREPO_ROOT}/llvm"                                           \
+          -B "${BUILD_DIR}"                                                    \
+          -GNinja -DCMAKE_MAKE_PROGRAM="${NINJA}"                              \
+          -DCMAKE_CXX_COMPILER_LAUNCHER="ccache"                               \
+          -DCMAKE_BUILD_TYPE=RelWithDebInfo                                    \
+          -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}"                              \
+          -DLLVM_ENABLE_ASSERTIONS=ON                                          \
+          -DLLVM_ENABLE_PROJECTS="clang"                                       \
+          -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind"      \
+          -DLLVM_RUNTIME_TARGETS="$(${CXX} --print-target-triple)"             \
+          -DLLVM_TARGETS_TO_BUILD="host"                                       \
+          -DCOMPILER_RT_CXX_LIBRARY=libcxx                                     \
+          -DRUNTIMES_BUILD_ALLOW_DARWIN=ON                                     \
+
+    # TODO: This shouldn't be required but check-runtimes doesn't have the necessary dependency on compiler-rt
+    echo "+++ Build compiler-rt"
+    ${NINJA} -C "${BUILD_DIR}" compiler-rt
 
     echo "+++ Running the libc++ and libc++abi tests"
     ${NINJA} -C "${BUILD_DIR}" check-runtimes
 
     echo "--- Installing libc++ and libc++abi to a fake location"
     ${NINJA} -C "${BUILD_DIR}" install-runtimes
 
     ccache -s

libcxx/utils/libcxx/test/dsl.py

with _makeConfigTest(config) as test:
             sourceFile.write(source)
         _, _, exitCode, _, _ = _executeWithFakeConfig(
             test, ["%{{build}} {}".format(" ".join(additionalFlags))]
         )
         return exitCode == 0
 
 
 @_memoizeExpensiveOperation(
-    lambda c, p, args=None: (c.substitutions, c.environment, p, args)
+    lambda c, p, args=None, additional_build_args=None: (c.substitutions, c.environment, p, args, additional_build_args)
 )
-def programOutput(config, program, args=None):
+def programOutput(config, program, args=None, additional_build_args=None):
     """
     Compiles a program for the test target, run it on the test target and return
     the output.
 
     Note that execution of the program is done through the %{exec} substitution,
     which means that the program may be run on a remote host depending on what
     %{exec} does.
     """
     if args is None:
         args = []
+
+    if additional_build_args is None:
+        additional_build_args = []
+
     with _makeConfigTest(config) as test:
         with open(test.getSourcePath(), "w") as source:
             source.write(program)
-        _, err, exitCode, _, buildcmd = _executeWithFakeConfig(test, ["%{build}"])
+        _, err, exitCode, _, buildcmd = _executeWithFakeConfig(test, ["%{{build}} {}".format(" ".join(additional_build_args))])
         if exitCode != 0:
             raise ConfigurationCompilationError(
                 "Failed to build program, cmd:\n{}\nstderr is:\n{}".format(
                     buildcmd, err
                 )
             )
 
         out, err, exitCode, _, runcmd = _executeWithFakeConfig(
             test, ["%{{run}} {}".format(" ".join(args))]
         )
         if exitCode != 0:
             raise ConfigurationRuntimeError(
                 "Failed to run program, cmd:\n{}\nstderr is:\n{}".format(runcmd, err)
             )
 
         return libcxx.test.format._parseLitOutput(out)
 
 
 @_memoizeExpensiveOperation(
-    lambda c, p, args=None: (c.substitutions, c.environment, p, args)
+    lambda c, p, args=None, additional_build_args=None: (c.substitutions, c.environment, p, args, additional_build_args)
 )
-def programSucceeds(config, program, args=None):
+def runSucceeds(config, program, args=None, additional_build_args=None):
     """
     Compiles a program for the test target, run it on the test target and return
     whether it completed successfully.
 
     Note that execution of the program is done through the %{exec} substitution,
     which means that the program may be run on a remote host depending on what
     %{exec} does.
     """
     try:
-        programOutput(config, program, args)
+        programOutput(config, program, args, additional_build_args)
     except ConfigurationRuntimeError:
         return False
     return True
 
+def compileAndRunSucceeds(config, program, args=None, additional_build_args=None):
+    """
+    Tries to compile a program for the test target, run it on the test target
+    and return whether it compiled and ran successfully.
+
+    Note that execution of the program is done through the %{exec} substitution,
+    which means that the program may be run on a remote host depending on what
+    %{exec} does.
+    """
+    try:
+        return runSucceeds(config, program, args, additional_build_args)
+    except ConfigurationCompilationError:
+        return False
+
 
 @_memoizeExpensiveOperation(lambda c, f: (c.substitutions, c.environment, f))
 def hasCompileFlag(config, flag):
     """
     Return whether the compiler in the configuration supports a given compiler flag.
 
     This is done by executing the %{cxx} substitution with the given flag and
     checking whether that succeeds.
#else
           if (::setlocale(LC_ALL, argv[i]) != NULL) {
             return 0;
           }
         }
         return 1;
       }
     #endif
   """
-    return programSucceeds(config, program, args=[pipes.quote(l) for l in locales])
+    return runSucceeds(config, program, args=[pipes.quote(l) for l in locales])
 
 
 @_memoizeExpensiveOperation(lambda c, flags="": (c.substitutions, c.environment, flags))
 def compilerMacros(config, flags=""):
     """
     Return a dictionary of predefined compiler macros.
 
     The keys are strings representing macros, and the values are strings

libcxx/utils/libcxx/test/features.py

Feature(
         name="objective-c++",
         when=lambda cfg: hasCompileFlag(cfg, "-xobjective-c++ -fobjc-arc"),
     ),
     Feature(
         name="verify-support",
         when=lambda cfg: hasCompileFlag(cfg, "-Xclang -verify-ignore-unexpected"),
     ),
     Feature(
+        name="fuzzer-support",
+        when=lambda cfg: compileAndRunSucceeds(cfg, """
+            #include <cstddef>
+            #include <cstdint>
+
+            extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t *, std::size_t) {
+                return 0;
+            }
+        """, args=["-runs=1"], additional_build_args=["-fsanitize=fuzzer"])
+    ),
+    Feature(
         name="non-lockfree-atomics",
         when=lambda cfg: sourceBuilds(
             cfg,
             """
             #include <atomic>
             struct Large { int storage[100]; };
             std::atomic<Large> x;
             int main(int, char**) { (void)x.load(); return 0; }
Feature(
         ),
     ),
     # Check for a Windows UCRT bug (fixed in UCRT/Windows 10.0.20348.0):
     # https://developercommunity.visualstudio.com/t/utf-8-locales-break-ctype-functions-for-wchar-type/1653678
     Feature(
         name="win32-broken-utf8-wchar-ctype",
         when=lambda cfg: not "_LIBCPP_HAS_NO_LOCALIZATION" in compilerMacros(cfg)
         and "_WIN32" in compilerMacros(cfg)
-        and not programSucceeds(
+        and not runSucceeds(
             cfg,
             """
             #include <locale.h>
             #include <wctype.h>
             int main(int, char**) {
               setlocale(LC_ALL, "en_US.UTF-8");
               return towlower(L'\\xDA') != L'\\xFA';
             }
           """,
         ),
     ),
     # Check for a Windows UCRT bug (fixed in UCRT/Windows 10.0.19041.0).
     # https://developercommunity.visualstudio.com/t/printf-formatting-with-g-outputs-too/1660837
     Feature(
         name="win32-broken-printf-g-precision",
         when=lambda cfg: "_WIN32" in compilerMacros(cfg)
-        and not programSucceeds(
+        and not runSucceeds(
             cfg,
             """
             #include <stdio.h>
             #include <string.h>
             int main(int, char**) {
               char buf[100];
               snprintf(buf, sizeof(buf), "%#.*g", 0, 0.0);
               return strcmp(buf, "0.");
             }
           """,
         ),
     ),
     # Check for Glibc < 2.27, where the ru_RU.UTF-8 locale had
     # mon_decimal_point == ".", which our tests don't handle.
     Feature(
         name="glibc-old-ru_RU-decimal-point",
         when=lambda cfg: not "_LIBCPP_HAS_NO_LOCALIZATION" in compilerMacros(cfg)
-        and not programSucceeds(
+        and not runSucceeds(
             cfg,
             """
             #include <locale.h>
             #include <string.h>
             int main(int, char**) {
               setlocale(LC_ALL, "ru_RU.UTF-8");
               return strcmp(localeconv()->mon_decimal_point, ",");
             }
Feature(
         when=lambda cfg: "_WIN32" in compilerMacros(cfg)
         and sourceBuilds(
             cfg,
             """
             #include <iostream>
             int main(int, char**) { return 0; }
           """,
         )
-        and programSucceeds(
+        and runSucceeds(
             cfg,
             """
             #include <iostream>
             #include <windows.h>
             #include <winnt.h>
             int main(int, char**) {
               // Get a pointer to a data member that gets linked from the C++
               // library. This must be a data member (functions can get

libcxx/utils/libcxx/test/format.py

 # ===----------------------------------------------------------------------===##
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 # ===----------------------------------------------------------------------===##
 
 import contextlib
FOO.gen.<anything>      - A .sh test that generates one or more Lit tests on the
                               multiple Lit tests from a single source file, which is
                               useful for testing repetitive properties in the library.
                               Be careful not to abuse this since this is not a replacement
                               for usual code reuse techniques.
 
     FOO.verify.cpp          - Compiles with clang-verify. This type of test is
                               automatically marked as UNSUPPORTED if the compiler
                               does not support Clang-verify.
 
+    FOO.fuzz.cpp            - Compiles with -fsanitize=fuzzer. This is disabled
+                              if the compiler doesn't provide a libfuzzer, or
+                              one that is incompatible with the standard library
+                              being tested.
+
 
     Substitution requirements
     ===============================
     The test format operates by assuming that each test's configuration provides
     the following substitutions, which it will reuse in the shell scripts it
     constructs:
         %{cxx}           - A command that can be used to invoke the compiler
         %{compile_flags} - Flags to use when compiling a test case
def getTestsForPath(self, testSuite, pathInSuite, litConfig, localConfig):
             "[.]compile[.]fail[.]cpp$",
             "[.]link[.]pass[.]cpp$",
             "[.]link[.]pass[.]mm$",
             "[.]link[.]fail[.]cpp$",
             "[.]sh[.][^.]+$",
             "[.]gen[.][^.]+$",
             "[.]verify[.]cpp$",
             "[.]fail[.]cpp$",
+            "[.]fuzz[.]cpp$",
         ]
 
         sourcePath = testSuite.getSourcePath(pathInSuite)
         filename = os.path.basename(sourcePath)
 
         # Ignore dot files, excluded tests and tests with an unsupported suffix
         hasSupportedSuffix = lambda f: any([re.search(ext, f) for ext in SUPPORTED_SUFFIXES])
         if filename.startswith(".") or filename in localConfig.excludes or not hasSupportedSuffix(filename):
             return
 
         # If this is a generated test, run the generation step and add
         # as many Lit tests as necessary.
         if re.search('[.]gen[.][^.]+$', filename):
             for test in self._generateGenTest(testSuite, pathInSuite, litConfig, localConfig):
                 yield test
         else:
             yield lit.Test.Test(testSuite, pathInSuite, localConfig)
 
     def execute(self, test, litConfig):
         VERIFY_FLAGS = (
             "-Xclang -verify -Xclang -verify-ignore-unexpected=note -ferror-limit=0"
         )
         supportsVerify = "verify-support" in test.config.available_features
+        supportsFuzzer = "fuzzer-support" in test.config.available_features
         filename = test.path_in_suite[-1]
 
         if re.search("[.]sh[.][^.]+$", filename):
             steps = []  # The steps are already in the script
             return self._executeShTest(test, litConfig, steps)
         elif filename.endswith(".compile.pass.cpp") or filename.endswith(
             ".compile.pass.mm"
         ):
def execute(self, test, litConfig):
         # Make sure to check these ones last, since they will match other
         # suffixes above too.
         elif filename.endswith(".pass.cpp") or filename.endswith(".pass.mm"):
             steps = [
                 "%dbg(COMPILED WITH) %{cxx} %s %{flags} %{compile_flags} %{link_flags} -o %t.exe",
                 "%dbg(EXECUTED AS) %{exec} %t.exe",
             ]
             return self._executeShTest(test, litConfig, steps)
+        elif filename.endswith(".fuzz.cpp"):
+            if not supportsFuzzer:
+                return lit.Test.Result(
+                    lit.Test.UNSUPPORTED,
+                    "Test {} requires support for libFuzzer, which isn't supported by the compiler".format(test.getFullName()))
+            steps = [
+                "%dbg(COMPILED WITH) %{cxx} %s %{flags} %{compile_flags} %{link_flags} -fsanitize=fuzzer -o %t.exe",
+                "%dbg(EXECUTED AS) %{exec} %t.exe -max_total_time=10 -timeout=10",
+            ]
+            return self._executeShTest(test, litConfig, steps)
         else:
             return lit.Test.Result(
                 lit.Test.UNRESOLVED, "Unknown test suffix for '{}'".format(filename)
             )
 
     def _executeShTest(self, test, litConfig, steps):
         if test.config.unsupported:
             return lit.Test.Result(lit.Test.UNSUPPORTED, "Test is unsupported")