Page MenuHomePhabricator

[libc++] Use named Lit features to flag back-deployment XFAILs
AcceptedPublic

Authored by ldionne on Sat, Mar 18, 7:42 AM.

Details

Reviewers
philnik
Mordante
Group Reviewers
Restricted Project
Summary

Instead of writing something like XFAIL: use_system_cxx_lib && target=...
to XFAIL back-deployment tests, introduce named Lit features like
availability-shared_mutex-missing to represent those. This makes the
XFAIL annotations leaner, and solves the problem of XFAIL comments
potentially getting out of sync. This would also make it easier for
another vendor to add their own annotations to the test suite by simply
changing how the feature is defined for their OS releases, instead
of having to modify hundreds of tests to add repetitive annotations.

This doesn't touch *all* annotations -- only annotations that were widely
duplicated are given named features (e.g. when filesystem or shared_mutex
were introduced). I still think it probably doesn't make sense to have a
named feature for every single fix we make to the dylib.

This is in essence a revert of 2659663, but since then the test suite
has changed significantly. Back when I did 2659663, the configuration
files we have for the test suite right now were being bootstrapped and
it wasn't clear how to provide these features for back-deployment in
that context. Since then, we have a streamlined way of defining these
features in features.py and that doesn't impact the ability for a
configuration file to stay minimal.

The original motivation for this change was that I am about to propose
a change that would touch essentially all XFAIL annotations for back-deployment
in the test suite, and this greatly reduces the number of lines changed
by that upcoming change, in addition to making the test suite generally
better.

Diff Detail

Unit TestsFailed

TimeTest
1,850 mslibcxx CI Apple back-deployment macosx10.9 > apple-libc++-backdeployment-cfg-in.std/containers::iterator.rel_ops.compile.pass.cpp
Script: -- : 'COMPILED WITH'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/std/containers/iterator.rel_ops.compile.pass.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk --target=x86_64-apple-macosx10.9 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/build/apple-system-backdeployment-10.9/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -Werror=thread-safety -Wuser-defined-warnings -fsyntax-only
260 mslibcxx CI No exceptions > llvm-libc++-shared-cfg-in.std/language_support/support_dynamic/new_delete/new_delete_array::new_align_val_t_nothrow.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-17 /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new_align_val_t_nothrow.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -fno-exceptions -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -Werror=thread-safety -Wuser-defined-warnings -lc++experimental -nostdlib++ -L /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -Wl,-rpath,/home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -lc++ -pthread -o /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/test/std/language.support/support.dynamic/new.delete/new.delete.array/Output/new_align_val_t_nothrow.pass.cpp.dir/t.tmp.exe
270 mslibcxx CI No exceptions > llvm-libc++-shared-cfg-in.std/language_support/support_dynamic/new_delete/new_delete_array::new_array_nothrow.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-17 /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.array/new_array_nothrow.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -fno-exceptions -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -Werror=thread-safety -Wuser-defined-warnings -lc++experimental -nostdlib++ -L /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -Wl,-rpath,/home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -lc++ -pthread -o /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/test/std/language.support/support.dynamic/new.delete/new.delete.array/Output/new_array_nothrow.pass.cpp.dir/t.tmp.exe
270 mslibcxx CI No exceptions > llvm-libc++-shared-cfg-in.std/language_support/support_dynamic/new_delete/new_delete_single::new_align_val_t_nothrow.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-17 /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new_align_val_t_nothrow.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -fno-exceptions -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -Werror=thread-safety -Wuser-defined-warnings -lc++experimental -nostdlib++ -L /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -Wl,-rpath,/home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -lc++ -pthread -o /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/test/std/language.support/support.dynamic/new.delete/new.delete.single/Output/new_align_val_t_nothrow.pass.cpp.dir/t.tmp.exe
280 mslibcxx CI No exceptions > llvm-libc++-shared-cfg-in.std/language_support/support_dynamic/new_delete/new_delete_single::new_nothrow.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-17 /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new_nothrow.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -fno-exceptions -Werror -Wall -Wctad-maybe-unsupported -Wextra -Wshadow -Wundef -Wunused-template -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -Werror=thread-safety -Wuser-defined-warnings -lc++experimental -nostdlib++ -L /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -Wl,-rpath,/home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/lib -lc++ -pthread -o /home/libcxx-builder/.buildkite-agent/builds/0f82a65fa6c4-1/llvm-project/libcxx-ci/build/generic-noexceptions/test/std/language.support/support.dynamic/new.delete/new.delete.single/Output/new_nothrow.pass.cpp.dir/t.tmp.exe

Event Timeline

ldionne created this revision.Sat, Mar 18, 7:42 AM
ldionne requested review of this revision.Sat, Mar 18, 7:42 AM
Herald added a project: Restricted Project. · View Herald TranscriptSat, Mar 18, 7:42 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
philnik accepted this revision.Sat, Mar 18, 8:46 AM
This revision is now accepted and ready to land.Sat, Mar 18, 8:46 AM
ldionne updated this revision to Diff 506311.Sat, Mar 18, 1:10 PM

Fix incorrect markup in libcxx/test/std/strings/basic.string/string.capacity/reserve_size.pass.cpp

ldionne updated this revision to Diff 506332.Sat, Mar 18, 2:15 PM

Fix a couple more incorrect XFAILs

Mordante accepted this revision.Sun, Mar 19, 11:13 AM
Mordante added a subscriber: Mordante.

I spot checked the patch, but trust on the CI to catch issues.

There's one question, other than that LGTM.

libcxx/utils/libcxx/test/features.py
357

Since vendors can disable filesystem, can't you tie into the same feature?