Index: compiler-rt/trunk/docs/TestingGuide.rst =================================================================== --- compiler-rt/trunk/docs/TestingGuide.rst +++ compiler-rt/trunk/docs/TestingGuide.rst @@ -0,0 +1,66 @@ +======================================== +Compiler-rt Testing Infrastructure Guide +======================================== + +.. contents:: + :local: + +Overview +======== + +This document is the reference manual for the compiler-rt modifications to the +testing infrastructure. Documentation for the infrastructure itself can be found at +:ref:`llvm_testing_guide`. + +LLVM testing infrastructure organization +======================================== + +The compiler-rt testing infrastructure contains regression tests which are run +as part of the usual ``make check-all`` and are expected to always pass -- they +should be run before every commit. + +Quick start +=========== + +The regressions tests are in the "compiler-rt" module and are normally checked +out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all`` +to run the regression tests after building compiler-rt. + +REQUIRES, XFAIL, etc. +--------------------- + +Sometimes it is necessary to restrict a test to a specific target or mark it as +an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or +``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the +behaviour of this is somewhat quirky in compiler-rt. There are two main +pitfalls to avoid. + +The first pitfall is that these directives perform a substring match on the +triple and as such ``XFAIL: mips`` affects more triples than expected. For +example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and +``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a +trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but +even that has issues as described below. + +The second pitfall is that the default target triple is often inappropriate for +compiler-rt tests since compiler-rt tests may be compiled for multiple targets. +For example, a typical build on an ``x86_64-linux-gnu`` host will often run the +tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark +both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386`` +will have no effect at all. + +To remedy both pitfalls, compiler-rt tests provide a feature string which can +be used to specify a single target. This string is of the form +``target-is-${arch}`` where ``${arch}}`` is one of the values from the +following lines of the CMake output:: + + -- Compiler-RT supported architectures: x86_64;i386 + -- Builtin supported architectures: i386;x86_64 + +So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail +on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386`` +will mark a test as expected to fail on i386 even if the default target triple +is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string +require exact matches so ``XFAIL: target-is-mips``, +``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and +``XFAIL: target-is-mips64el`` all refer to different MIPS targets. Index: compiler-rt/trunk/test/asan/TestCases/Linux/local_alias.cc =================================================================== --- compiler-rt/trunk/test/asan/TestCases/Linux/local_alias.cc +++ compiler-rt/trunk/test/asan/TestCases/Linux/local_alias.cc @@ -6,7 +6,7 @@ // // FIXME: https://github.com/google/sanitizers/issues/316 // XFAIL: android -// XFAIL: mips64 +// XFAIL: target-is-mips64 // // RUN: %clangxx_asan -DBUILD_INSTRUMENTED_DSO=1 -fPIC -shared -mllvm -asan-use-private-alias %s -o %t-INSTRUMENTED-SO.so // RUN: %clangxx -DBUILD_UNINSTRUMENTED_DSO=1 -fPIC -shared %s -o %t-UNINSTRUMENTED-SO.so Index: compiler-rt/trunk/test/asan/TestCases/Linux/odr-violation.cc =================================================================== --- compiler-rt/trunk/test/asan/TestCases/Linux/odr-violation.cc +++ compiler-rt/trunk/test/asan/TestCases/Linux/odr-violation.cc @@ -1,6 +1,6 @@ // FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316 // XFAIL: android -// XFAIL: mips64 +// XFAIL: target-is-mips64 // // We use fast_unwind_on_malloc=0 to have full unwinding even w/o frame // pointers. This setting is not on by default because it's too expensive. Index: compiler-rt/trunk/test/lit.common.configured.in =================================================================== --- compiler-rt/trunk/test/lit.common.configured.in +++ compiler-rt/trunk/test/lit.common.configured.in @@ -28,6 +28,7 @@ set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@) set_default("has_lld", @COMPILER_RT_HAS_LLD_SOURCES_PYBOOL@) set_default("can_symbolize", @CAN_SYMBOLIZE@) +config.available_features.add('target-is-%s' % config.target_arch) # LLVM tools dir can be passed in lit parameters, so try to # apply substitution. Index: compiler-rt/trunk/test/tsan/ignore_lib4.cc =================================================================== --- compiler-rt/trunk/test/tsan/ignore_lib4.cc +++ compiler-rt/trunk/test/tsan/ignore_lib4.cc @@ -4,7 +4,7 @@ // RUN: %env_tsan_opts=suppressions='%t.supp' %run %t 2>&1 | FileCheck %s // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 // powerpc64 big endian bots failed with "FileCheck error: '-' is empty" due // to a segmentation fault. // UNSUPPORTED: powerpc64-unknown-linux-gnu Index: compiler-rt/trunk/test/tsan/longjmp.cc =================================================================== --- compiler-rt/trunk/test/tsan/longjmp.cc +++ compiler-rt/trunk/test/tsan/longjmp.cc @@ -1,7 +1,7 @@ // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 #include #include Index: compiler-rt/trunk/test/tsan/longjmp2.cc =================================================================== --- compiler-rt/trunk/test/tsan/longjmp2.cc +++ compiler-rt/trunk/test/tsan/longjmp2.cc @@ -1,7 +1,7 @@ // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 #include #include Index: compiler-rt/trunk/test/tsan/longjmp3.cc =================================================================== --- compiler-rt/trunk/test/tsan/longjmp3.cc +++ compiler-rt/trunk/test/tsan/longjmp3.cc @@ -1,7 +1,7 @@ // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 #include #include Index: compiler-rt/trunk/test/tsan/longjmp4.cc =================================================================== --- compiler-rt/trunk/test/tsan/longjmp4.cc +++ compiler-rt/trunk/test/tsan/longjmp4.cc @@ -1,7 +1,7 @@ // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 #include #include Index: compiler-rt/trunk/test/tsan/map32bit.cc =================================================================== --- compiler-rt/trunk/test/tsan/map32bit.cc +++ compiler-rt/trunk/test/tsan/map32bit.cc @@ -8,7 +8,7 @@ // https://github.com/google/sanitizers/issues/412 // MAP_32BIT flag for mmap is supported only for x86_64. -// XFAIL: mips64 +// XFAIL: target-is-mips64 // XFAIL: aarch64 // XFAIL: powerpc64 Index: compiler-rt/trunk/test/tsan/signal_longjmp.cc =================================================================== --- compiler-rt/trunk/test/tsan/signal_longjmp.cc +++ compiler-rt/trunk/test/tsan/signal_longjmp.cc @@ -4,7 +4,7 @@ // https://github.com/google/sanitizers/issues/482 // Longjmp assembly has not been implemented for mips64 yet -// XFAIL: mips64 +// XFAIL: target-is-mips64 // This test fails on powerpc64 BE (VMA=44), a segmentation fault // error happens at the second assignment // "((volatile int *volatile)mem)[1] = 1".