Add a buildkite for _LIBCPP_DEBUG=1.

[• Quuxplusone]

Summary

Add a debug-mode CI step.
My general roadmap is intended to be:

• Land the changes to Generic-debug-iterators.cmake, run-buildbot, and buildkite-pipeline.yml, plus marking all offending tests as UNSUPPORTED: LIBCXX-DEBUG-FIXME.
• Land the fix to support N3644 "Null Forward Iterators" in debug mode (D100881). (This is done.)
• Land a fix to <span> (D101003) and unmark those tests.
• Fix other failures, unmarking tests that pass, and changing others from UNSUPPORTED: LIBCXX-DEBUG-FIXME to UNSUPPORTED: debug_level_0, debug_level_1 as appropriate. (This is done, locally. I haven't submitted PRs for them yet though.)
• Don't leave any LIBCXX-DEBUG-FIXME tests in master at the end of this whole process.
• Remove the LIBCXX-DEBUG-FIXME feature from Generic-debug-iterators.cmake.

To run llvm-lit manually from the command line:

    
./bin/llvm-lit -sv --param std=c++2b --param cxx_under_test=`pwd`/bin/clang \
    --param debug_level=1 ../libcxx/test/

[Mordante]

Nice to see another improvement to our build pipeline, thanks for working on it! The general approach looks good to me.

libcxx/src/debug.cpp

441

This seems trivial, but I think it should land in a separate commit. (I don't mind to do that after accepting this patch.)
I assume this is what you mean with "Land the fix to support N3644 "Null Forward Iterators" in debug mode." correct?

Just curious how did you discover this?

libcxx/utils/ci/buildkite-pipeline.yml

68

I'm not to fond of "Debug mode", it sounds too generic for me. How do you feel about "Debug iterators"?

[• Quuxplusone]

s/Debug Mode/Debug iterators/
Fix quoting in CMake file (I hope).
Remove N3644 fix to see how many tests it affects.

[• Quuxplusone]

[libc++] [test] Stop using invalid iterators to insert into sets/maps. (Applies 4c80bfbd53caf consistently across associative and unord tests.)
[libc++] std::advance shouldn't use ADL >= on the _Distance type.

[• Quuxplusone]

[libc++] [test] Our __debug_less breaks some complexity guarantees.
[libc++] [test] Iterating a string::iterator "off the end" is UB.
[libc++] Paper over a bug in string::assign.
[libc++] <span>, like <string_view>, has no use for debug iterators.
[libc++] Constexpr char_traits::copy mustn't compare unrelated pointers.

[• Quuxplusone]

libcxx/src/debug.cpp

441

It all started with D100029. :) That led to looking at _LIBCPP_DEBUG == 2 assertions, which led to looking at all code under #if _LIBCPP_DEBUG == 2 (and incidentally D100737 too), which led to D100730, which revealed that <span> was completely broken in debug mode, we just didn't have any tests specifically for span-in-debug-mode. So, since the apparent intent is that everything should just work in debug mode... let's test it!

Running tests locally with -D_LIBCPP_DEBUG=1 turned up some failures due to N3644 immediately, so it just happened that I fixed this line before finishing the buildkite-related parts of the patch. Then I ran tests locally and marked the remaining offenders with XFAIL; but, having already fixed N3644, I didn't actually know how many tests are affected by it — just "at least one."

But now I know, and have filed D100881! :)

libcxx/utils/libcxx/test/config.py

484

I still haven't figured out why https://buildkite.com/llvm-project/libcxx-ci/builds/2676#7b1ae675-f9cd-4f9d-8b4a-6db9ec8f8e53 is so unhappy with this script now...

[curdeius]

libcxx/test/std/containers/unord/unord.map/unord.map.modifiers/insert_hint_const_lvalue.pass.cpp

38 ↗

(On Diff #339077)

What's the reason for this and similar changes?

libcxx/utils/libcxx/test/config.py

484

@Quuxplusone, I have a similar failure when cmake chooses gcc 9 (without -std=c++2b flag), but it doesn't happen with a newer compiler nor with std=c++20.

[• Quuxplusone]

Try changing Generic-debug.cmake as

 set(LIBCXX_TEST_PARAMS "std=c++2b" "debug_level=1" "additional_features=LIBCXX-DEBUG-FIXME" CACHE STRING "")
-set(LIBCXXABI_TEST_PARAMS "${LIBCXX_TEST_PARAMS}" CACHE STRING "")
+set(LIBCXXABI_TEST_PARAMS "std=c++2b" CACHE STRING "")

to see if that fixes the cryptic failure.

[• Quuxplusone]

Keep trying to solve this "flag None" issue in the Python.

[• Quuxplusone]

Figured out how to run this locally, and I think I've diagnosed the problem: If you set the lit parameter std=c++2b (not via llvm-lit --param but via the weird cmake technique used in the buildbot scripts), and the system compiler doesn't support -std=c++2b, then you get this error:

AssertionError: Trying to enable compile flag None, which is not supported

Changing to std=c++20 fixes it locally for me. I'm hoping that changing the compiler to clang-tot (copied from the generic-cxx2b section of the script) will fix it just as well on buildkite, without needing to abandon C++2b. If this still fails on buildkite, my backup plan is to change it to std=c++20 for now.

libcxx/test/std/containers/unord/unord.map/unord.map.modifiers/insert_hint_const_lvalue.pass.cpp

38 ↗

(On Diff #339077)

[libc++] [test] Stop using invalid iterators to insert into sets/maps. (Applies 4c80bfbd53caf consistently across associative and unord tests.)

[libc++] [test] Stop using invalid iterators to insert into sets/maps.

This simply applies Howard's commit 4c80bfbd53caf consistently
across all the associative and unordered container tests.

"unord.set/insert_hint_const_lvalue.pass.cpp" failed with `-D_LIBCPP_DEBUG=1`
before this patch; it was the only one that incorrectly reused
invalid iterator `e`. The others already used valid iterators
(generally `c.end()`); I'm just making them all match the same pattern
of usage: "e, then r, then c.end() for the rest."

libcxx/test/std/containers/unord/unord.set/insert_hint_const_lvalue.pass.cpp

38

(@curdeius : Here's the one where the test was actually using an invalidated iterator e.)

[• Quuxplusone]

Eliminate all my fixes from this review, now that buildkite is happy. Now this review consists only of what will be committed as a single commit, following my original "roadmap" idea. Tests that fail at the moment are marked XFAIL; I'll submit PRs to fix them individually.

[Mordante]

In D100866#2705673, @Quuxplusone wrote:

Eliminate all my fixes from this review, now that buildkite is happy. Now this review consists only of what will be committed as a single commit, following my original "roadmap" idea. Tests that fail at the moment are marked XFAIL; I'll submit PRs to fix them individually.

Seems you forgot to restore the removed builds from libcxx/utils/ci/buildkite-pipeline.yml
Can you update this file to give the patch a full CI run.

libcxx/src/debug.cpp

441

Thanks for the information and the new patch.

[ldionne]

This is indeed quite nice! Some comments, but I love that we're adding this.

libcxx/cmake/caches/Generic-debug.cmake

2

This shouldn't set std=c++2b explicitly, I believe we should be using the default here.

libcxx/utils/ci/run-buildbot

518

Should be clang and clang++, like the other jobs?

Also, please move close to the generic-no-debug) step.

libcxx/utils/libcxx/test/config.py

484

You can rebase on top of 3d8f2059b95d3e9d6785255949283b258d7ca121.

[• Quuxplusone]

libcxx/cmake/caches/Generic-debug.cmake

2

I want to make sure that we're testing new features as soon as they're pull-requested; testing only C++14 mode wouldn't have caught the issues with <span>. Of course we could add debug-mode testing for multiple versions, like std=c++03 AND std=c++2b and maybe something in between; I thought that was overkill, but I'd do it if you want it.

I don't think we want to be testing only C++14, though; I think we want at least C++20, and I see only upsides to testing C++2b (given that the goal here is to catch things at pull-request time, and people tend to pull-request mostly features that are enabled in '20 and '2b, not '14 anymore).

How about if I provide just generic-cxx03-debug and generic-cxx2b-debug, with those names? (This also helps to indicate that they're not the opposite of generic-no-debug, as brought up below.)

libcxx/utils/ci/run-buildbot

518

(A) clang-tot turned out to be the magic incantation that makes the std=c++2b parameter work. Compare to the generic-cxx2b) case. (If we stopped trying to test 2b pull requests and went to 20 or earlier, then this could indeed be just clang.)

(B) Will do. But notice that generic-no-debug is actually talking about a build configuration that doesn't build debug.cpp into the library; it's not the "opposite" of generic-debug. All of generic-no-{debug,filesystem,localization,random_device} have the same structure of "normal compilation, just don't link in some .cpp files to the library."

libcxx/utils/libcxx/test/config.py

484

I'll do so. But actually I'm pretty sure that all the trouble I had was completely due to writing clang/std=c++2b instead of (either clang-tot/std=c++2b or clang/std=c++20) above.

[• Quuxplusone]

Jobs for generic-cxx03-debug and generic-cxx2b-debug.
Rebase.

[• Quuxplusone]

Rebase, and change XFAIL to UNSUPPORTED.
(Many of these tests pass in C++03 mode but fail in C++11 mode, so XFAIL is wrong.)

[ldionne]

@Quuxplusone

I want to make sure that we're testing new features as soon as they're pull-requested; testing only C++14 mode wouldn't have caught the issues with <span>. Of course we could add debug-mode testing for multiple versions, like std=c++03 AND std=c++2b and maybe something in between; I thought that was overkill, but I'd do it if you want it.

We agree that testing only C++14 is bad by default, but that's not what we do. By default, the Lit configuration will test the latest supported standard anyway. I think we should have a single job with the debug mode enabled, and that job should *not* mention what standard version it's using (it should use the default one).

[• Quuxplusone]

Remove generic-cxx03-debug.
Rename generic-cxx2b-debug to generic-debug-iterators.
Use "the highest supported version" instead of hard-coding std=c++2b ... But leave this build using clang-tot instead of plain old clang, so that our "highest supported version" actually is c++2b, instead of c++20.

[• Quuxplusone]

Poke buildkite.
@ldionne ping! (assuming buildkite is still happy with this)

[ldionne]

LGTM and I don't want to be unproductive so I'll let it slip, but it still seems to me that it would be more consistent to use

export CC=clang
export CXX=clang++

And if we really want to be using ToT Clang here, then perhaps we should consider using ToT everywhere instead, and adding a single job that tests on an "older" compiler, as a means of testing that we still support those older compilers. Actually, after writing that, I think it would make sense to do it as part of the patch that will explain our compiler support policy, which I still need to write.

libcxx/cmake/caches/Generic-debug.cmake

+set(LIBCXX_TEST_PARAMS "std=c++2b debug_level=1 additional_features=LIBCXX-DEBUG-FIXME" CACHE STRING "")
+set(LIBCXXABI_TEST_PARAMS "${LIBCXX_TEST_PARAMS}" CACHE STRING "")

libcxx/src/debug.cpp

 {
 #ifndef _LIBCPP_HAS_NO_THREADS
     RLock _(mut());
 #endif
     __i_node* i = __find_iterator(__i);
     __i_node* j = __find_iterator(__j);
     __c_node* ci = i != nullptr ? i->__c_ : nullptr;
     __c_node* cj = j != nullptr ? j->__c_ : nullptr;
-    return ci != nullptr && ci == cj;
+    return ci == cj;  // N3644
 }
 
 void
 __libcpp_db::swap(void* c1, void* c2)
 {
 #ifndef _LIBCPP_HAS_NO_THREADS
     WLock _(mut());
 #endif

libcxx/test/libcxx/containers/sequences/vector/robust_against_adl.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03
+// XFAIL: debug_level_0, debug_level_1
 
 // <vector>
 
 #include <vector>
 
 #include "test_macros.h"
 
 struct Incomplete;

libcxx/test/libcxx/diagnostics/nodiscard_extensions.pass.cpp

 // This test intentionally leaks memory, so it is unsupported under ASAN.
 // UNSUPPORTED: asan
 
 // AppleClang9 and GCC 5 don't support C++17's implicitly synthesized
 // deduction guides from existing ctors, needed by default_searcher() below.
 // UNSUPPORTED: apple-clang-9
 // UNSUPPORTED: gcc-5
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // All entities to which libc++ applies [[nodiscard]] as an extension should
 // be tested here and in nodiscard_extensions.fail.cpp. They should also
 // be listed in `UsingLibcxx.rst` in the documentation for the extension.
 
 #include <algorithm>
 #include <cstddef> // to_integer
 #include <functional> // identity
 #include <iterator>

libcxx/test/std/algorithms/alg.sorting/alg.merge/inplace_merge_comp.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <algorithm>
 
 // template<BidirectionalIterator Iter, StrictWeakOrder<auto, Iter::value_type> Compare>
 //   requires ShuffleIterator<Iter>
 //         && CopyConstructible<Compare>
 //   void
 //   inplace_merge(Iter first, Iter middle, Iter last, Compare comp);
 

libcxx/test/std/containers/unord/unord.set/insert_hint_const_lvalue.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <unordered_set>
 
 // template <class Value, class Hash = hash<Value>, class Pred = equal_to<Value>,
 //           class Alloc = allocator<Value>>
 // class unordered_set
 
 // iterator insert(const_iterator p, const value_type& x);
 
void do_insert_hint_const_lvalue_test()
     typedef typename C::value_type VT;
     C c;
     typename C::const_iterator e = c.end();
     const VT v1(3.5);
     R r = c.insert(e, v1);
     assert(c.size() == 1);
     assert(*r == 3.5);
 
     r = c.insert(e, v1);
     assert(c.size() == 1);
     assert(*r == 3.5);
 
     const VT v2(4.5);
     r = c.insert(e, v2);
     assert(c.size() == 2);
     assert(*r == 4.5);
 

libcxx/test/std/containers/views/span.cons/deduct.pass.cpp

 // -*- C++ -*-
 //===------------------------------ span ---------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 //   template<class T, size_t N>
 //     span(T (&)[N]) -> span<T, N>;
 //
 //   template<class T, size_t N>
 //     span(array<T, N>&) -> span<T, N>;

libcxx/test/std/containers/views/span.iterators/begin.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // constexpr       iterator  begin() const noexcept;
 // constexpr const_iterator cbegin() const noexcept;
 
 #include <span>
 #include <cassert>

libcxx/test/std/containers/views/span.iterators/end.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // constexpr       iterator  end() const noexcept;
 // constexpr const_iterator cend() const noexcept;
 
 #include <span>
 #include <cassert>

libcxx/test/std/containers/views/span.iterators/rbegin.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // constexpr       reverse_iterator  rbegin() const noexcept;
 // constexpr const_reverse_iterator crbegin() const noexcept;
 
 #include <span>
 #include <cassert>

libcxx/test/std/containers/views/span.iterators/rend.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // constexpr       reverse_iterator  rend() const noexcept;
 // constexpr const_reverse_iterator crend() const noexcept;
 
 #include <span>
 #include <cassert>

libcxx/test/std/containers/views/span.sub/first.pass.cpp

 // -*- C++ -*-
 //===------------------------------ span ---------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // template<size_t Count>
 //  constexpr span<element_type, Count> first() const;
 //
 // constexpr span<element_type, dynamic_extent> first(size_type count) const;
 //

libcxx/test/std/containers/views/span.sub/last.pass.cpp

 // -*- C++ -*-
 //===------------------------------ span ---------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // template<size_t Count>
 //  constexpr span<element_type, Count> last() const;
 //
 // constexpr span<element_type, dynamic_extent> last(size_type count) const;
 //

libcxx/test/std/containers/views/span.sub/subspan.pass.cpp

 // -*- C++ -*-
 //===------------------------------ span ---------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 // UNSUPPORTED: c++03, c++11, c++14, c++17
+// XFAIL: LIBCXX-DEBUG-FIXME
 
 // <span>
 
 // template<size_t Offset, size_t Count = dynamic_extent>
 //   constexpr span<element_type, see below> subspan() const;
 //
 // constexpr span<element_type, dynamic_extent> subspan(
 //   size_type offset, size_type count = dynamic_extent) const;

libcxx/test/std/input.output/filesystems/class.path/path.nonmember/path.factory.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // UNSUPPORTED: c++03
 // ADDITIONAL_COMPILE_FLAGS: -D_LIBCPP_DISABLE_DEPRECATION_WARNINGS
 
 // <filesystem>
 
 // template <class Source>
 //    path u8path(Source const&);
 // template <class InputIter>

libcxx/test/std/input.output/filesystems/fs.op.funcs/fs.op.proximate/proximate.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // UNSUPPORTED: c++03
 
 // <filesystem>
 
 // path proximate(const path& p, error_code &ec)
 // path proximate(const path& p, const path& base = current_path())
 // path proximate(const path& p, const path& base, error_code& ec);
 

libcxx/test/std/iterators/iterator.primitives/iterator.operations/robust_against_adl.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <iterator>
 
 #include <iterator>
 
 #include "test_macros.h"
 
 struct Incomplete;
 template<class T> struct Holder { T t; };

libcxx/test/std/numerics/numeric.ops/exclusive.scan/exclusive_scan.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T>
 //     OutputIterator exclusive_scan(InputIterator first, InputIterator last,

libcxx/test/std/numerics/numeric.ops/exclusive.scan/exclusive_scan_init_op.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T, class BinaryOperation>
 //     OutputIterator

libcxx/test/std/numerics/numeric.ops/inclusive.scan/inclusive_scan.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T>
 //     OutputIterator inclusive_scan(InputIterator first, InputIterator last,

libcxx/test/std/numerics/numeric.ops/inclusive.scan/inclusive_scan_op.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T, class BinaryOperation>
 //     OutputIterator

libcxx/test/std/numerics/numeric.ops/inclusive.scan/inclusive_scan_op_init.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T, class BinaryOperation>
 //     OutputIterator

libcxx/test/std/numerics/numeric.ops/transform.exclusive.scan/transform_exclusive_scan_init_bop_uop.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T,
 //          class BinaryOperation, class UnaryOperation>

libcxx/test/std/numerics/numeric.ops/transform.inclusive.scan/transform_inclusive_scan_bop_uop.pass.cpp

 
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T,
 //          class BinaryOperation, class UnaryOperation>

libcxx/test/std/numerics/numeric.ops/transform.inclusive.scan/transform_inclusive_scan_bop_uop_init.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <numeric>
 // UNSUPPORTED: c++03, c++11, c++14
 // UNSUPPORTED: clang-8
 // UNSUPPORTED: gcc-9
 
 // Became constexpr in C++20
 // template<class InputIterator, class OutputIterator, class T,
 //          class BinaryOperation, class UnaryOperation>

libcxx/test/std/strings/basic.string/string.modifiers/string_assign/iterator.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <string>
 
 // template<class InputIterator>
 //   basic_string& assign(InputIterator first, InputIterator last);
 
 #include <string>
 #include <cassert>
 

libcxx/test/std/strings/char.traits/char.traits.specializations/char.traits.specializations.char/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // UNSUPPORTED: clang-8
 
 // <string>
 
 // template<> struct char_traits<char>
 
 // static char_type* copy(char_type* s1, const char_type* s2, size_t n);
 

libcxx/test/std/strings/char.traits/char.traits.specializations/char.traits.specializations.char16_t/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <string>
 
 // template<> struct char_traits<char16_t>
 
 // static char_type* copy(char_type* s1, const char_type* s2, size_t n);
 
 #include <string>
 #include <cassert>

libcxx/test/std/strings/char.traits/char.traits.specializations/char.traits.specializations.char32_t/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <string>
 
 // template<> struct char_traits<char32_t>
 
 // static char_type* copy(char_type* s1, const char_type* s2, size_t n);
 
 #include <string>
 #include <cassert>

libcxx/test/std/strings/char.traits/char.traits.specializations/char.traits.specializations.char8_t/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-8
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <string>
 
 // template<> struct char_traits<char8_t>
 
 // static char_type* copy(char_type* s1, const char_type* s2, size_t n);
 
 #include <string>
 #include <cassert>

libcxx/test/std/strings/char.traits/char.traits.specializations/char.traits.specializations.wchar.t/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: clang-8
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // <string>
 
 // template<> struct char_traits<wchar_t>
 
 // static char_type* copy(char_type* s1, const char_type* s2, size_t n);
 
 #include <string>
 #include <cassert>

libcxx/test/std/strings/string.view/string.view.ops/copy.pass.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
+// XFAIL: LIBCXX-DEBUG-FIXME
+
 // XFAIL: gcc-10
 //     GCC's __builtin_strlen isn't constexpr yet
 
 // <string_view>
 
 // size_type copy(charT* s, size_type n, size_type pos = 0) const;
 
 // Throws: out_of_range if pos > size().

libcxx/utils/ci/buildkite-pipeline.yml

steps:
 
   #
   # General testing with the default configuration, under all the supported
   # Standard modes, with Clang and GCC and on Apple. This catches most issues
   # upfront.
   #
   - wait
 
-  - label: "C++03"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx03"
-    artifact_paths:
-      - "**/test-results.xml"
-      - "**/*.abilist"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "C++11"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx11"
-    artifact_paths:
-      - "**/test-results.xml"
-      - "**/*.abilist"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "C++14"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx14"
-    artifact_paths:
-      - "**/test-results.xml"
-      - "**/*.abilist"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "C++17"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx17"
-    artifact_paths:
-      - "**/test-results.xml"
-      - "**/*.abilist"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "C++20"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx20"
-    artifact_paths:
-      - "**/test-results.xml"
-      - "**/*.abilist"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
   - label: "C++2b"
     command: "libcxx/utils/ci/run-buildbot generic-cxx2b"
     artifact_paths:
       - "**/test-results.xml"
       - "**/*.abilist"
     agents:
       queue: "libcxx-builders"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
 
-  - label: "MacOS C++20"
-    command: "libcxx/utils/ci/run-buildbot generic-cxx20"
+  - label: "Debug Mode"
+    command: "libcxx/utils/ci/run-buildbot generic-debug"
     artifact_paths:
       - "**/test-results.xml"
       - "**/*.abilist"
     agents:
-      queue: "libcxx-builders-macos"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "GCC/C++20"
-    command: "libcxx/utils/ci/run-buildbot generic-gcc"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  #
-  # All other supported configurations of libc++.
-  #
-  - wait
-
-  - label: "-fno-exceptions"
-    command: "libcxx/utils/ci/run-buildbot generic-noexceptions"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Static libraries"
-    command: "libcxx/utils/ci/run-buildbot generic-static"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "ASAN"
-    command: "libcxx/utils/ci/run-buildbot generic-asan"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "TSAN"
-    command: "libcxx/utils/ci/run-buildbot generic-tsan"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "UBSAN"
-    command: "libcxx/utils/ci/run-buildbot generic-ubsan"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "With LLVM's libunwind"
-    command: "libcxx/utils/ci/run-buildbot generic-with_llvm_unwinder"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Single-threaded"
-    command: "libcxx/utils/ci/run-buildbot generic-singlethreaded"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "No debug mode"
-    command: "libcxx/utils/ci/run-buildbot generic-no-debug"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "No Filesystem"
-    command: "libcxx/utils/ci/run-buildbot generic-no-filesystem"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "No random device"
-    command: "libcxx/utils/ci/run-buildbot generic-no-random_device"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "No locale"
-    command: "libcxx/utils/ci/run-buildbot generic-no-localization"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Benchmarks"
-    command: "libcxx/utils/ci/run-buildbot benchmarks"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Documentation"
-    command: "libcxx/utils/ci/run-buildbot documentation"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Legacy standalone build"
-    command: "libcxx/utils/ci/run-buildbot legacy-standalone"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
       queue: "libcxx-builders"
     retry:
       automatic:
         - exit_status: -1  # Agent was lost
           limit: 2
-
-  - label: "Unified standalone build"
-    command: "libcxx/utils/ci/run-buildbot unified-standalone"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Runtimes build"
-    command: "libcxx/utils/ci/run-buildbot runtimes-build"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Windows"
-    # TODO: The CI runner doesn't have bash in the path currently. Once it
-    # has that, remove the absolute path and just call 'bash' here.
-    command: "\"\\Program Files\\Git\\usr\\bin\\bash\" libcxx/utils/ci/run-buildbot generic-win"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "windows"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  # Build with the configuration we use to generate libc++.dylib on Apple platforms
-  - label: "Apple system"
-    command: "libcxx/utils/ci/run-buildbot x86_64-apple-system"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-macos"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Apple system -fno-exceptions"
-    command: "libcxx/utils/ci/run-buildbot x86_64-apple-system-noexceptions"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-macos"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  # Test back-deployment to older Apple platforms
-  - label: "Apple back-deployment macosx10.9"
-    command: "libcxx/utils/ci/run-buildbot x86_64-apple-system-backdeployment-10.9"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-macos10.15" # TODO: For now, we're running the back-deployment tests for 10.9 on 10.15, because we don't have proper 10.9 machines
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Apple back-deployment macosx10.15"
-    command: "libcxx/utils/ci/run-buildbot x86_64-apple-system-backdeployment-10.15"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-macos10.15"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "AArch64"
-    command: "libcxx/utils/ci/run-buildbot aarch64"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "aarch64"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "AArch64 -fno-exceptions"
-    command: "libcxx/utils/ci/run-buildbot aarch64-noexceptions"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "aarch64"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Armv8"
-    command: "libcxx/utils/ci/run-buildbot armv8"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "armv8l"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Armv8 -fno-exceptions"
-    command: "libcxx/utils/ci/run-buildbot armv8-noexceptions"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "armv8l"
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Armv7"
-    command: "libcxx/utils/ci/run-buildbot armv7"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "armv8l" # Compiling for v7, running on v8 hardware
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2
-
-  - label: "Armv7 -fno-exceptions"
-    command: "libcxx/utils/ci/run-buildbot armv7-noexceptions"
-    artifact_paths:
-      - "**/test-results.xml"
-    agents:
-      queue: "libcxx-builders-linaro-arm"
-      arch: "armv8l" # Compiling for v7, running on v8 hardware
-    retry:
-      automatic:
-        - exit_status: -1  # Agent was lost
-          limit: 2

libcxx/utils/ci/run-buildbot

cmake -S "${MONOREPO_ROOT}/libcxx" \
           -DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=NO \
           -DLIBCXX_ENABLE_FILESYSTEM=YES \
           -DCMAKE_CXX_FLAGS="-D_LIBCPP_HAS_NO_INT128" \
           -DLIBCXX_TEST_COMPILER_FLAGS="-D_LIBCPP_HAS_NO_INT128" \
           -DLIBCXX_TEST_LINKER_FLAGS="-loldnames"
     echo "+++ Running the libc++ tests"
     ${NINJA} -vC "${BUILD_DIR}" check-cxx
 ;;
+generic-debug)
+    clean
+    generate-cmake -C "${MONOREPO_ROOT}/libcxx/cmake/caches/Generic-debug.cmake"
+    check-cxx-cxxabi
+;;
 *)
     echo "${BUILDER} is not a known configuration"
     exit 1
 ;;
 esac

libcxx/utils/libcxx/test/config.py

class Configuration(object):
     def configure_debug_mode(self):
         debug_level = self.get_lit_conf('debug_level', None)
         if not debug_level:
             return
         if debug_level not in ['0', '1']:
             self.lit_config.fatal('Invalid value for debug_level "%s".'
                                   % debug_level)
         self.cxx.compile_flags += ['-D_LIBCPP_DEBUG=%s' % debug_level]
+        self.config.available_features.add('debug_level_%s' % debug_level)
 
     def configure_sanitizer(self):
         san = self.get_lit_conf('use_sanitizer', '').strip()
         if san:
             # Search for llvm-symbolizer along the compiler path first
             # and then along the PATH env variable.
             symbolizer_search_paths = os.environ.get('PATH', '')
             cxx_path = libcxx.util.which(self.cxx.path)