[sanitizers] Make it possible to XFAIL on the effective target, not just the default.

[dsanders]

Summary

The triple is not the right thing to XFAIL on since LIT only sees the default
triple and not the effective triple chosen by any -target option in the RUN
directives. This discrepancy is shown in the table below:

Default Triple   | Options                           | XFAIL  | LIT's expected result | Desired expectation
=================+===================================+========+=======================+====================
mips-linux-gnu   | -target mips-linux-gnu            |        | Pass                  | Pass  
mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
mips-linux-gnu   | -target mips-linux-gnu            | mips   | Fail                  | Fail  
mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass* (debatable**)
mips-linux-gnu   | -target mips-linux-gnu            | mips-  | Fail                  | Fail  
mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips-  | Fail                  | Pass* 
mips-linux-gnu   | -target mips-linux-gnu            | mips64 | Pass                  | Pass  
mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips64 | Pass                  | Fail* 
mips64-linux-gnu | -target mips-linux-gnu            |        | Pass                  | Pass  
mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
mips64-linux-gnu | -target mips-linux-gnu            | mips   | Fail                  | Fail* 
mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass (debatable**)
mips64-linux-gnu | -target mips-linux-gnu            | mips-  | Pass                  | Fail* 
mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips-  | Pass                  | Pass  
mips64-linux-gnu | -target mips-linux-gnu            | mips64 | Fail                  | Pass* 
mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips64 | Fail                  | Fail  
x64_64-linux-gnu | -target i386-linux-gnu            |        | Pass                  | Pass
x64_64-linux-gnu | -target x86_64-linux-gnu          |        | Pass                  | Pass
x64_64-linux-gnu | -target i386-linux-gnu            | i386   | Pass                  | Fail*
x64_64-linux-gnu | -target x86_64-linux-gnu          | i386   | Pass                  | Pass
x64_64-linux-gnu | -target i386-linux-gnu            | x86_64 | Fail                  | Pass
x64_64-linux-gnu | -target x86_64-linux-gnu          | x86_64 | Fail                  | Fail*
* These all differ from LIT's current behaviour.
** People's expectations vary depending on whether they know that LIT does a
 substring match on the default triple or think it's an exact match on an
 architecture.

This patch adds "target-is-${target_arch}" to the available features list and
updates the mips XFAIL's to use them. XFAIL'ing on these features will
correctly account for the target being tested. Making the table:

Options                           | XFAIL            | LIT's expected result
==================================+==================+======================
-target mips-linux-gnu            |                  | Pass
-target mips64-linux-gnu -mabi=64 |                  | Pass
-target mips-linux-gnu            | target-is-mips   | Fail
-target mips64-linux-gnu -mabi=64 | target-is-mips   | Pass
-target mips-linux-gnu            | target-is-mips64 | Pass
-target mips64-linux-gnu -mabi=64 | target-is-mips64 | Fail
-target i386-linux-gnu            |                  | Pass
-target x86_64-linux-gnu          |                  | Pass
-target i386-linux-gnu            | target-is-i386   | Fail
-target x86_64-linux-gnu          | target-is-i386   | Pass
-target i386-linux-gnu            | target-is-x86_64 | Pass
-target x86_64-linux-gnu          | target-is-x86_64 | Fail

[dsanders]

ping

[dsanders]

ping

[probinson]

No documentation of the target-is-xxx feature. This really matters because the difference between 'mips64' and 'target-is-mips64' seems to be very subtle and even somewhat magical.

[dsanders]

Added documentation of target-is-${arch} and the quirks that make it necessary.
There's no place for such documentation yet so I've created a new file to go
along with TestingGuide.rst from the llvm project.

Also, there's broken cross reference to the llvm document at the moment.
Phabricator won't let me add the necessary anchor to llvm/docs/TestingGuide.rst
in this review but I'll add it in the commit.

[probinson]

That is a really clear description, thanks! LGTM.

compiler-rt/trunk/docs/TestingGuide.rst

+========================================
+Compiler-rt Testing Infrastructure Guide
+========================================
+
+.. contents::
+   :local:
+
+Overview
+========
+
+This document is the reference manual for the compiler-rt modifications to the
+testing infrastructure. Documentation for the infrastructure itself can be found at
+:ref:`llvm_testing_guide`.
+
+LLVM testing infrastructure organization
+========================================
+
+The compiler-rt testing infrastructure contains regression tests which are run
+as part of the usual ``make check-all`` and are expected to always pass -- they
+should be run before every commit.
+
+Quick start
+===========
+
+The regressions tests are in the "compiler-rt" module and are normally checked
+out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all``
+to run the regression tests after building compiler-rt.
+
+REQUIRES, XFAIL, etc.
+---------------------
+
+Sometimes it is necessary to restrict a test to a specific target or mark it as
+an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or
+``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the
+behaviour of this is somewhat quirky in compiler-rt. There are two main
+pitfalls to avoid.
+
+The first pitfall is that these directives perform a substring match on the
+triple and as such ``XFAIL: mips`` affects more triples than expected. For
+example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and
+``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a
+trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but
+even that has issues as described below.
+
+The second pitfall is that the default target triple is often inappropriate for
+compiler-rt tests since compiler-rt tests may be compiled for multiple targets.
+For example, a typical build on an ``x86_64-linux-gnu`` host will often run the
+tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark
+both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386``
+will have no effect at all.
+
+To remedy both pitfalls, compiler-rt tests provide a feature string which can
+be used to specify a single target. This string is of the form
+``target-is-${arch}`` where ``${arch}}`` is one of the values from the
+following lines of the CMake output::
+
+  -- Compiler-RT supported architectures: x86_64;i386
+  -- Builtin supported architectures: i386;x86_64
+
+So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail
+on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386``
+will mark a test as expected to fail on i386 even if the default target triple
+is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string
+require exact matches so ``XFAIL: target-is-mips``,
+``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and
+``XFAIL: target-is-mips64el`` all refer to different MIPS targets.

compiler-rt/trunk/test/asan/TestCases/Linux/local_alias.cc

 // Test that mixing instrumented and non-instrumented code doesn't lead to crash.
 // Build two modules (one is instrumented, another is not) that have globals
 // with same names. Check, that ASan doesn't crash with CHECK failure or
 // false positive global-buffer-overflow due to sanitized library poisons
 // globals from non-sanitized one.
 //
 // FIXME: https://github.com/google/sanitizers/issues/316
 // XFAIL: android
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 //
 // RUN: %clangxx_asan -DBUILD_INSTRUMENTED_DSO=1 -fPIC -shared -mllvm -asan-use-private-alias %s -o %t-INSTRUMENTED-SO.so
 // RUN: %clangxx -DBUILD_UNINSTRUMENTED_DSO=1 -fPIC -shared %s -o %t-UNINSTRUMENTED-SO.so
 // RUN: %clangxx %s -c -mllvm -asan-use-private-alias -o %t.o
 // RUN: %clangxx_asan %t.o %t-UNINSTRUMENTED-SO.so %t-INSTRUMENTED-SO.so -o %t-EXE
 // RUN: %env_asan_opts=use_odr_indicator=true %run %t-EXE
 
 #if defined (BUILD_INSTRUMENTED_DSO)

compiler-rt/trunk/test/asan/TestCases/Linux/odr-violation.cc

 // FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316
 // XFAIL: android
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 //
 // We use fast_unwind_on_malloc=0 to have full unwinding even w/o frame
 // pointers. This setting is not on by default because it's too expensive.
 //
 // Different size: detect a bug if detect_odr_violation>=1
 // RUN: %clangxx_asan -DBUILD_SO=1 -fPIC -shared %s -o %t-ODR-SO.so
 // RUN: %clangxx_asan %s %t-ODR-SO.so -Wl,-R. -o %t-ODR-EXE
 // RUN: %env_asan_opts=fast_unwind_on_malloc=0:detect_odr_violation=1 not %run %t-ODR-EXE 2>&1 | FileCheck %s

compiler-rt/trunk/test/lit.common.configured.in

 set_default("compiler_id", "@COMPILER_RT_TEST_COMPILER_ID@")
 set_default("python_executable", "@PYTHON_EXECUTABLE@")
 set_default("compiler_rt_debug", @COMPILER_RT_DEBUG_PYBOOL@)
 set_default("compiler_rt_libdir", "@COMPILER_RT_LIBRARY_OUTPUT_DIR@")
 set_default("emulator", "@COMPILER_RT_EMULATOR@")
 set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
 set_default("has_lld", @COMPILER_RT_HAS_LLD_SOURCES_PYBOOL@)
 set_default("can_symbolize", @CAN_SYMBOLIZE@)
+config.available_features.add('target-is-%s' % config.target_arch)
 
 # LLVM tools dir can be passed in lit parameters, so try to
 # apply substitution.
 try:
   config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
 except KeyError as e:
   key, = e.args
   lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key, key))
 
 # Setup attributes common for all compiler-rt projects.
 lit_config.load_config(config, "@COMPILER_RT_SOURCE_DIR@/test/lit.common.cfg")

compiler-rt/trunk/test/tsan/ignore_lib4.cc

 // RUN: %clangxx_tsan -O1 %s -DLIB -fPIC -shared -o %T/libignore_lib4.so
 // RUN: %clangxx_tsan -O1 %s -o %t
 // RUN: echo "called_from_lib:libignore_lib4.so" > %t.supp
 // RUN: %env_tsan_opts=suppressions='%t.supp' %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // powerpc64 big endian bots failed with "FileCheck error: '-' is empty" due
 // to a segmentation fault.
 // UNSUPPORTED: powerpc64-unknown-linux-gnu
 // aarch64 bots failed with "called_from_lib suppression 'libignore_lib4.so'
 //                           is matched against 2 libraries".
 // UNSUPPORTED: aarch64
 
 // Test longjmp in ignored lib.

compiler-rt/trunk/test/tsan/longjmp.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <setjmp.h>
 
 int foo(jmp_buf env) {
   longjmp(env, 42);
 }

compiler-rt/trunk/test/tsan/longjmp2.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <setjmp.h>
 
 int foo(sigjmp_buf env) {
   printf("env=%p\n", env);
   siglongjmp(env, 42);

compiler-rt/trunk/test/tsan/longjmp3.cc

 // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <pthread.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <setjmp.h>
 
 void bar(jmp_buf env) {
   volatile int x = 42;

compiler-rt/trunk/test/tsan/longjmp4.cc

 // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <pthread.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <setjmp.h>
 #include <string.h>
 
 void bar(jmp_buf env) {

compiler-rt/trunk/test/tsan/map32bit.cc

 // RUN: %clangxx_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
 #include "test.h"
 #include <stdint.h>
 #include <errno.h>
 #include <sys/mman.h>
 
 // Test for issue:
 // https://github.com/google/sanitizers/issues/412
 
 // MAP_32BIT flag for mmap is supported only for x86_64.
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // XFAIL: aarch64
 // XFAIL: powerpc64
 
 // MAP_32BIT doesn't exist on OS X.
 // UNSUPPORTED: darwin
 
 void *Thread(void *ptr) {
   *(int*)ptr = 42;

compiler-rt/trunk/test/tsan/signal_longjmp.cc

 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // Test case for longjumping out of signal handler:
 // https://github.com/google/sanitizers/issues/482
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // This test fails on powerpc64 BE (VMA=44), a segmentation fault
 // error happens at the second assignment
 // "((volatile int *volatile)mem)[1] = 1".
 // XFAIL: powerpc64-unknown-linux-gnu
 
 #include <setjmp.h>
 #include <signal.h>
 #include <stdlib.h>