[libc++] Require the use of clang-verify in .fail.cpp tests that don't fail without it

[ldionne]

Summary

Some tests do not fail at all when -verify is not supported, unless some
arbitrary warning flag is added to make them fail. We currently used
-Werror=unused-result to make them fail, but doing so makes the test
suite a lot more inscrutable. It seems better to just disable those
tests when -verify is not supported.

[EricWF]

These tests still require that "arbitrary" warning flag be enabled.

I'm also a bit afraid that the verify-support tag will get accidentally disabled and these tests along with it.
Also, do we care about GCC?

[ldionne]

In D76256#1930359, @EricWF wrote:

These tests still require that "arbitrary" warning flag be enabled.

I don't understand? The only reason these tests were using that flag was to cause a compilation error (instead of a warning) on GCC. If we properly require that we're able to check the diagnostics, which is the goal, we can stop trying to make them fail arbitrarily on GCC.

Basically, it's kind of as if we had said "for these tests (and these tests only), enable -Werror=unused-typedef on GCC cause we know that's going to make them fail. Instead, my point is that we can just disable them cause they're not doing much good.

I'm also a bit afraid that the verify-support tag will get accidentally disabled and these tests along with it.

I think we need to trust our config.py here. I agree it's complicated and I'm working on fixing it, but this change is actually one step exactly in that direction.

Also, do we care about GCC?

These tests already don't do much on GCC, they just confirm that -Werror-unused-result works properly.

[EricWF]

In D76256#1930417, @ldionne wrote:

In D76256#1930359, @EricWF wrote:

These tests still require that "arbitrary" warning flag be enabled.

I don't understand? The only reason these tests were using that flag was to cause a compilation error (instead of a warning) on GCC. If we properly require that we're able to check the diagnostics, which is the goal, we can stop trying to make them fail arbitrarily on GCC.

Previously these tests required -Werror=-Wunused-result. Now they require -Wunused-result. The requirement for an arbitrary flag remains, only the spelling changed.
That said, I'm fine with this.

Basically, it's kind of as if we had said "for these tests (and these tests only), enable -Werror=unused-typedef on GCC cause we know that's going to make them fail. Instead, my point is that we can just disable them cause they're not doing much good.

I'm also a bit afraid that the verify-support tag will get accidentally disabled and these tests along with it.

I think we need to trust our config.py here. I agree it's complicated and I'm working on fixing it, but this change is actually one step exactly in that direction.

Every Clang we test against should support -verify. How about we simply say // UNSUPPORTED: gcc?
That would assuage my concern.

Also, do we care about GCC?

These tests already don't do much on GCC, they just confirm that -Werror-unused-result works properly.

LGTM if we agree on the // UNSUPPORTED: gcc change.

[ldionne]

In D76256#1936968, @EricWF wrote:

These tests already don't do much on GCC, they just confirm that -Werror-unused-result works properly.

LGTM if we agree on the // UNSUPPORTED: gcc change.

So, more generally, would you be OK with dropping the notion of // REQUIRES: verify-support and instead translate that into // UNSUPPORTED: gcc? I believe the best would be to simplify the logic for how we currently determine the verify-support feature by just checking whether we're using GCC, but leave // REQUIRES: verify-support as that provides more information semantically than // UNSUPPORTED: gcc. WDYT? If we're on the same page, I can do that as a follow up change.

[ldionne]

In D76256#1936998, @ldionne wrote:

In D76256#1936968, @EricWF wrote:

These tests already don't do much on GCC, they just confirm that -Werror-unused-result works properly.

LGTM if we agree on the // UNSUPPORTED: gcc change.

So, more generally, would you be OK with dropping the notion of // REQUIRES: verify-support and instead translate that into // UNSUPPORTED: gcc? I believe the best would be to simplify the logic for how we currently determine the verify-support feature by just checking whether we're using GCC, but leave // REQUIRES: verify-support as that provides more information semantically than // UNSUPPORTED: gcc. WDYT? If we're on the same page, I can do that as a follow up change.

I'll check this in as-is and we can talk offline about exactly what you mean by // UNSUPPORTED: gcc. Whatever change we make, we should make it for all tests that have // REQUIRES: verify-support.

libcxx/test/libcxx/depr/depr.default.allocator/allocator.members/allocate.cxx2a.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 // <memory>
 
 // allocator:
 // T* allocate(size_t n, const void* hint);
 
 //  In C++20, parts of std::allocator<T> have been removed.
 //  However, for backwards compatibility, if _LIBCPP_ENABLE_CXX20_REMOVED_ALLOCATOR_MEMBERS

 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::allocator<int> a;
     a.allocate(3, nullptr); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/libcxx/diagnostics/enable_nodiscard.fail.cpp

 // to the appropriate warning-generating attribute when _LIBCPP_ENABLE_NODISCARD
 // is explicitly provided.
 
 // UNSUPPORTED: c++98, c++03
 
 // GCC 7 is the first version to introduce [[nodiscard]]
 // UNSUPPORTED: gcc-5, gcc-6
 
+// REQUIRES: verify-support
+
 // MODULES_DEFINES: _LIBCPP_ENABLE_NODISCARD
 #define _LIBCPP_ENABLE_NODISCARD
 
 #include <__config>
 
 _LIBCPP_NODISCARD_EXT int foo() { return 42; }
 _LIBCPP_NODISCARD_AFTER_CXX17 int bar() { return 42; }
 
 int main(int, char**) {
   foo(); // expected-error-re {{ignoring return value of function declared with {{'nodiscard'|warn_unused_result}} attribute}}
   bar(); // expected-error-re {{ignoring return value of function declared with {{'nodiscard'|warn_unused_result}} attribute}}
   (void)foo(); // OK. void casts disable the diagnostic.
   (void)bar();
 
   return 0;
 }

libcxx/test/libcxx/diagnostics/enable_nodiscard_disable_after_cxx17.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03
+// REQUIRES: verify-support
 
 // GCC 7 is the first version to introduce [[nodiscard]]
 // UNSUPPORTED: gcc-5, gcc-6
 
 // Test that _LIBCPP_DISABLE_NODISCARD_EXT only disables _LIBCPP_NODISCARD_EXT
 // and not _LIBCPP_NODISCARD_AFTER_CXX17.
 
 // MODULES_DEFINES: _LIBCPP_ENABLE_NODISCARD

libcxx/test/libcxx/diagnostics/enable_nodiscard_disable_nodiscard_ext.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03
+// REQUIRES: verify-support
 
 // GCC 7 is the first version to introduce [[nodiscard]]
 // UNSUPPORTED: gcc-5, gcc-6
 
 
 // MODULES_DEFINES: _LIBCPP_ENABLE_NODISCARD
 // MODULES_DEFINES: _LIBCPP_DISABLE_NODISCARD_EXT
 #define _LIBCPP_ENABLE_NODISCARD

libcxx/test/libcxx/diagnostics/nodiscard_aftercxx17.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // Test that _LIBCPP_NODISCARD_AFTER_CXX17 works
 //  #define _LIBCPP_NODISCARD_AFTER_CXX17 [[nodiscard]]
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
+// REQUIRES: verify-support
 
 #include <__config>
 
 _LIBCPP_NODISCARD_AFTER_CXX17 int foo() { return 6; }
 
 int main(int, char**)
 {
   foo();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/libcxx/diagnostics/nodiscard_extensions.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03
-
-// GCC versions prior to 7.0 don't provide the required [[nodiscard]] attribute.
-// UNSUPPORTED: gcc-5, gcc-6
+// REQUIRES: verify-support
 
 // AppleClang9 doesn't yet support C++17's implicitly synthesized deduction
 // guides from existing ctors, needed by default_searcher() below.
 // UNSUPPORTED: apple-clang-9
 
 // Test that entities declared [[nodiscard]] as an extension by libc++, are
 // only actually declared such when _LIBCPP_ENABLE_NODISCARD is specified.
 

libcxx/test/libcxx/thread/thread.lock/thread.lock.guard/nodiscard.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: libcpp-has-no-threads
+// REQUIRES: verify-support
 
 // [[nodiscard]] on constructors isn't supported by all compilers
 // UNSUPPORTED: clang-6, clang-7, clang-8, clang-9
 // UNSUPPORTED: apple-clang-9, apple-clang-10, apple-clang-11
 // UNSUPPORTED: gcc-5
 
 // [[nodiscard]] isn't supported in C++98 and C++03 (not even as an extension)
 // UNSUPPORTED: c++98, c++03

libcxx/test/std/containers/associative/map/map.access/empty.fail.cpp

 // <map>
 
 // class map
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <map>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::map<int, int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/associative/multimap/empty.fail.cpp

 // <map>
 
 // class multimap
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <map>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::multimap<int, int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/associative/multiset/empty.fail.cpp

 // <set>
 
 // class multiset
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <set>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::multiset<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/associative/set/empty.fail.cpp

 // <set>
 
 // class set
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <set>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::set<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/container.adaptors/priority.queue/priqueue.members/empty.fail.cpp

 // <queue>
 
 // class priority_queue
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <queue>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::priority_queue<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/container.adaptors/queue/queue.defn/empty.fail.cpp

 // <queue>
 
 // class queue
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <queue>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::queue<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/container.adaptors/stack/stack.defn/empty.fail.cpp

 // <stack>
 
 // class stack
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <stack>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::stack<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/array/empty.fail.cpp

 // <array>
 
 // class array
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <array>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
 
     std::array<int, 1> c;
     c.empty(); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
     std::array<int, 0> c0;
     c0.empty(); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/deque/deque.capacity/empty.fail.cpp

 // <deque>
 
 // class deque
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <deque>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::deque<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/forwardlist/empty.fail.cpp

 // <forward_list>
 
 // class forward_list
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <forward_list>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::forward_list<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/list/list.capacity/empty.fail.cpp

 // <list>
 
 // class list
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <list>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::list<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/vector.bool/empty.fail.cpp

 // <vector>
 
 // class vector
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <vector>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::vector<bool> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/sequences/vector/vector.capacity/empty.fail.cpp

 // <vector>
 
 // class vector
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <vector>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::vector<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/unord/unord.map/empty.fail.cpp

 // <unordered_map>
 
 // class unordered_map
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <unordered_map>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::unordered_map<int, int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/unord/unord.multimap/empty.fail.cpp

 // <unordered_map>
 
 // class unordered_multimap
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <unordered_map>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::unordered_multimap<int, int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/unord/unord.multiset/empty.fail.cpp

 // <unordered_set>
 
 // class unordered_multiset
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <unordered_set>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::unordered_multiset<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/containers/unord/unord.set/empty.fail.cpp

 // <unordered_set>
 
 // class unordered_set
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <unordered_set>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::unordered_set<int> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/input.output/filesystems/class.path/path.member/path.decompose/empty.fail.cpp

 // <filesystem>
 
 // class path
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include "filesystem_include.h"
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     fs::path c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/iterators/iterator.container/empty.array.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <iterator>
 
 // <iterator>
 // template <class T, size_t N> constexpr bool empty(const T (&array)[N]) noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <vector>
 #include <iterator>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     int c[5];
     std::empty(c);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/iterators/iterator.container/empty.container.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <iterator>
 
 // <iterator>
 // template <class C> constexpr auto empty(const C& c) -> decltype(c.empty());
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <vector>
 #include <iterator>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::vector<int> c;
     std::empty(c);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/iterators/iterator.container/empty.initializer_list.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <iterator>
 
 // <iterator>
 // template <class E> constexpr bool empty(initializer_list<E> il) noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <initializer_list>
 #include <iterator>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::initializer_list<int> c = { 4 };
     std::empty(c);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.placement/new_array_ptr.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <new>
 
 // void* operator new[](std::size_t, void *);
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <new>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     char buffer[100];
     ::operator new[](4, buffer);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.placement/new_ptr.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <new>
 
 // void* operator new(std::size_t, void *);
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <new>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     char buffer[100];
     ::operator new(4, buffer);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new_size.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <new>
 
 // void* operator new(std::size_t);
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <new>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     ::operator new(4);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new_size_nothrow.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <new>
 
 // void* operator new(std::size_t, std::nothrow_t &);
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <new>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     ::operator new(4, std::nothrow);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/language.support/support.dynamic/ptr.launder/launder.nodiscard.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <new>
 
 // template <class T> constexpr T* launder(T* p) noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <new>
 #include <cassert>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     int *p = nullptr;
     std::launder(p);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/re/re.results/re.results.size/empty.fail.cpp

 // -*- C++ -*-
 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // <regex>
 
 // class match_results<BidirectionalIterator, Allocator>
 // bool empty() const;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <regex>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::match_results<const char*> c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/strings/basic.string/string.capacity/empty.fail.cpp

 // <string>
 
 // class deque
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <string>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::string c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/strings/string.view/string.view.capacity/empty.fail.cpp

 // <string_view>
 
 // class string_view
 
 // bool empty() const noexcept;
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <string_view>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::string_view c;
     c.empty();  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/thread/futures/futures.async/async.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // UNSUPPORTED: libcpp-has-no-threads
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 // <future>
 
 // template <class F, class... Args>
 //     future<typename result_of<F(Args...)>::type>
 //     async(F&& f, Args&&... args);
 
 // template <class F, class... Args>

 
 int foo (int x) { return x; }
 
 int main(int, char**)
 {
     std::async(                    foo, 3); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
     std::async(std::launch::async, foo, 3); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/utilities/allocator.adaptor/allocator.adaptor.members/allocate_size.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 // <memory>
 
 // template <class OuterAlloc, class... InnerAllocs>
 //   class scoped_allocator_adaptor
 
 // pointer allocate(size_type n);
 
 #include <scoped_allocator>
 #include <cassert>
 
 #include "allocators.h"
 
 int main(int, char**)
 {
     std::scoped_allocator_adaptor<A1<int>> a;
     a.allocate(10);  // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/utilities/allocator.adaptor/allocator.adaptor.members/allocate_size_hint.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 // <memory>
 
 // template <class OuterAlloc, class... InnerAllocs>
 //   class scoped_allocator_adaptor
 
 // pointer allocate(size_type n, const_void_pointer hint);
 
 #include <scoped_allocator>
 #include <cassert>
 
 #include "allocators.h"
 
 int main(int, char**)
 {
     std::scoped_allocator_adaptor<A1<int>> a;
     a.allocate(10, (const void*)0); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/utilities/memory/allocator.traits/allocator.traits.members/allocate.fail.cpp

 // struct allocator_traits
 // {
 //     static pointer allocate(allocator_type& a, size_type n);
 //     ...
 // };
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 #include <memory>
 #include <cstdint>
 #include <cassert>
 
 #include "test_macros.h"
 
 template <class T>

 };
 
 int main(int, char**)
 {
     A<int> a;
     std::allocator_traits<A<int> >::allocate(a, 10);          // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
     std::allocator_traits<A<int> >::allocate(a, 10, nullptr); // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/test/std/utilities/memory/default.allocator/allocator.members/allocate.fail.cpp

 //===----------------------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 // UNSUPPORTED: c++98, c++03, c++11, c++14, c++17
 // UNSUPPORTED: clang-3.3, clang-3.4, clang-3.5, clang-3.6, clang-3.7, clang-3.8
+// REQUIRES: verify-support
 
 // <memory>
 
 // allocator:
 // T* allocate(size_t n);
 
 #include <memory>
 #include <cassert>
 
 #include "test_macros.h"
 
 int main(int, char**)
 {
     std::allocator<int> a;
     a.allocate(3);          // expected-error {{ignoring return value of function declared with 'nodiscard' attribute}}
 
     return 0;
 }

libcxx/utils/libcxx/test/format.py

def _evaluate_fail_test(self, test, test_cxx, parsers):
         # when using Clang.
         if test_cxx.type != 'gcc':
             test_cxx.flags += ['-fsyntax-only']
         if use_verify:
             test_cxx.useVerify()
             test_cxx.useWarnings()
             if '-Wuser-defined-warnings' in test_cxx.warning_flags:
                 test_cxx.warning_flags += ['-Wno-error=user-defined-warnings']
-        else:
-            # We still need to enable certain warnings on .fail.cpp test when
-            # -verify isn't enabled. Such as -Werror=unused-result. However,
-            # we don't want it enabled too liberally, which might incorrectly
-            # allow unrelated failure tests to 'pass'.
-            #
-            # Therefore, we check if the test was expected to fail because of
-            # nodiscard before enabling it
-            test_str_list = [b'ignoring return value', b'nodiscard',
-                             b'NODISCARD']
-            if any(test_str in contents for test_str in test_str_list):
-                test_cxx.flags += ['-Werror=unused-result']
         cmd, out, err, rc = test_cxx.compile(source_path, out=os.devnull)
         check_rc = lambda rc: rc == 0 if use_verify else rc != 0
         report = libcxx.util.makeReport(cmd, out, err, rc)
         if check_rc(rc):
             return lit.Test.Result(lit.Test.PASS, report)
         else:
             report += ('Expected compilation to fail!\n' if not use_verify else
                        'Expected compilation using verify to pass!\n')
             return lit.Test.Result(lit.Test.FAIL, report)