[compiler-rt] Build custom libcxx with libcxxabi

[Hahnfeld]

Summary

This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:

1. The resulting libFuzzer doesn't expose C++ internals and looks like a plain C library.
2. We don't have to manually link in libstdc++ to provide cxxabi.
3. The sanitizer tests cannot interfere with an installed version of libc++.so in LD_LIBRARY_PATH.

[phosek]

compiler-rt/lib/fuzzer/CMakeLists.txt

138 ↗

(On Diff #186147)

I don't think this is correct either. The problem is that on Linux you might end up mixing different C++ libraries and different C++ ABI implementations. Ideally, libFuzzer would be built against hermetic libc++abi and libc++ so it can be linked into any program independently of which C++ library it uses (or even plain C programs). This wasn't possible before, but it's possible now that libunwind, libc++abi and libc++ supports building hermetic static library.

So the ideal solution here is to build libc++abi in addition to libc++ (as hermetic libraries) and use those the same way we already do today. The resulting libFuzzer library would look like a C library from the consumer perspective. I was planning on doing that, I just haven't had time yet but I hope to get to it soon.

[Hahnfeld]

@phosek Excellent idea. I tried to implement what you described and it seems to work great! The only minor disadvantage is that we now need to disable the tests if COMPILER_RT_LIBCXXABI_PATH is not found...

[phosek]

In D58013#1396761, @Hahnfeld wrote:

@phosek Excellent idea. I tried to implement what you described and it seems to work great! The only minor disadvantage is that we now need to disable the tests if COMPILER_RT_LIBCXXABI_PATH is not found...

This is awesome, exactly what I had in mind, thank you so much for working on this!

compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt

21 ↗

(On Diff #186705)

Should we also set -DLIBCXX_ENABLE_EXCEPTIONS=OFF to avoid the dependency on unwinder (e.g. libunwind or libgcc_s)? We already do this for libFuzzer but would it also make sense for MSan and TSan copies?

[Hahnfeld]

Cleaunup CustomLibcxx/CMakeLists.txt and build custom libcxx without exceptions.

[Hahnfeld]

compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt

21 ↗

(On Diff #186705)

Yes, I guess that makes sense.

compiler-rt/lib/fuzzer/CMakeLists.txt

139 ↗

(On Diff #186803)

I've also tried to remove the ABI namespace (and adjusting LIBFUZZER_CFLAGS) but that resulted in Segfaults during runtime. From a first look it appeared that some relocations were broken in the binary, resulting in calling address 0x0, but I'll need to investigate. I'd say that's out of the scope of this patch anyway.

[vitalybuka]

LGTM

[phosek]

LGTM

[Hahnfeld]

I needed to push rCRT354231 to fix the sanitizer bots. Let me know if this change is appropriate or want me to revert. In that case we need to run another round of clobber builds to fix CMake configuration (needed because the patch changes the source directory for ExternalProject_Add).

[phosek]

This also broke our toolchain builders. The relevant part of the log is:

[357/763] Performing configure step for 'libcxx_fuzzer_aarch64'
-- The C compiler identification is Clang 9.0.0
-- The CXX compiler identification is Clang 9.0.0
-- Check for working C compiler: /b/s/w/ir/kitchen-workdir/recipe_cleanup/clangrCZ026/llvm_build_dir/./bin/clang
-- Check for working C compiler: /b/s/w/ir/kitchen-workdir/recipe_cleanup/clangrCZ026/llvm_build_dir/./bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Detecting CXX compile features
-- Detecting CXX compile features - failed
-- Configuring for standalone build.
-- Linker detection: LLD
-- Performing Test LLVM_LIBSTDCXX_MIN
-- Performing Test LLVM_LIBSTDCXX_MIN - Failed
CMake Error at /b/s/w/ir/kitchen-workdir/llvm-project/llvm/cmake/modules/CheckCompilerVersion.cmake:72 (message):
  libstdc++ version must be at least 4.8.
Call Stack (most recent call first):
  /b/s/w/ir/kitchen-workdir/llvm-project/llvm/cmake/modules/HandleLLVMOptions.cmake:9 (include)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/cmake/Modules/HandleOutOfTreeLLVM.cmake:99 (include)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/cmake/Modules/HandleOutOfTreeLLVM.cmake:142 (configure_out_of_tree_llvm)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/CMakeLists.txt:29 (include)


-- Configuring incomplete, errors occurred!

I don't know yet why the LLVM_LIBSTDCXX_MIN check is failing but I'm going to do a local build to find out.

[phosek]

I found the issue, but I don't know how to easily solve this. The problem is that when being built in standalone mode, libc++abi and libc++ includes LLVM's HandleLLVMOptions which uses checks that try to link C++ library, but that may not be ready yet in case of the runtimes build. In runtimes build I solve this by [adding -nostdlib++ to required flags](https://github.com/llvm/llvm-project/blob/master/llvm/runtimes/CMakeLists.txt#L100). I can do that because in case of runtimes build I know that we'll be always using the just built Clang that supports that flag. I could address this issue by also including -nostdlib++ in required flags in compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt but that's not a general solution because that file in theory also supports other compilers that may not have this flag (older Clang versions, GCC). We could try and use -nostdlib and then include builtins manually but that's a lot of extra code that would need to be duplicated to handle cases like compiler-rt builtins and libgcc.

The general solution is to stop relying on checks that assume working C++ library from runtimes (it's where we're building C++ library after all, we should assume a working C++ library), but that's going to require more significant refactor and I'd like to unbreak our bots ASAP so we'll need some other solution.

[phosek]

In D58013#1400665, @phosek wrote:

I found the issue, but I don't know how to easily solve this. The problem is that when being built in standalone mode, libc++abi and libc++ includes LLVM's HandleLLVMOptions which uses checks that try to link C++ library, but that may not be ready yet in case of the runtimes build. In runtimes build I solve this by [adding -nostdlib++ to required flags](https://github.com/llvm/llvm-project/blob/master/llvm/runtimes/CMakeLists.txt#L100). I can do that because in case of runtimes build I know that we'll be always using the just built Clang that supports that flag. I could address this issue by also including -nostdlib++ in required flags in compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt but that's not a general solution because that file in theory also supports other compilers that may not have this flag (older Clang versions, GCC). We could try and use -nostdlib and then include builtins manually but that's a lot of extra code that would need to be duplicated to handle cases like compiler-rt builtins and libgcc.

The general solution is to stop relying on checks that assume working C++ library from runtimes (it's where we're building C++ library after all, we should assume a working C++ library), but that's going to require more significant refactor and I'd like to unbreak our bots ASAP so we'll need some other solution.

D58331 is what I came up with so far.

[Hahnfeld]

@phosek Sorry for the breakage. I guess there are just too many configurations for non-trivial changes to the build system :-/

CMakeLists.txt

foreach(path IN ITEMS ${LLVM_MAIN_SRC_DIR}/projects/libcxx
                         ${LLVM_MAIN_SRC_DIR}/runtimes/libcxx
                         ${LLVM_MAIN_SRC_DIR}/../libcxx
                         ${LLVM_EXTERNAL_LIBCXX_SOURCE_DIR})
     if(IS_DIRECTORY ${path})
       set(COMPILER_RT_LIBCXX_PATH ${path})
       break()
     endif()
   endforeach()
+  foreach(path IN ITEMS ${LLVM_MAIN_SRC_DIR}/projects/libcxxabi
+                        ${LLVM_MAIN_SRC_DIR}/runtimes/libcxxabi
+                        ${LLVM_MAIN_SRC_DIR}/../libcxxabi
+                        ${LLVM_EXTERNAL_LIBCXXABI_SOURCE_DIR})
+    if(IS_DIRECTORY ${path})
+      set(COMPILER_RT_LIBCXXABI_PATH ${path})
+      break()
+    endif()
+  endforeach()
 endif()
 
 set(COMPILER_RT_LLD_PATH ${LLVM_MAIN_SRC_DIR}/tools/lld)
 if(EXISTS ${COMPILER_RT_LLD_PATH}/ AND LLVM_TOOL_LLD_BUILD)
   set(COMPILER_RT_HAS_LLD TRUE)
 else()
   set(COMPILER_RT_LLD_PATH ${LLVM_MAIN_SRC_DIR}/../lld)
   if(EXISTS ${COMPILER_RT_LLD_PATH}/ AND LLVM_TOOL_LLD_BUILD)

cmake/Modules/AddCompilerRT.cmake

 # add_custom_libcxx(<name> <prefix>
 #                   DEPS <list of build deps>
 #                   CFLAGS <list of compile flags>
 #                   USE_TOOLCHAIN)
 macro(add_custom_libcxx name prefix)
   if(NOT COMPILER_RT_LIBCXX_PATH)
     message(FATAL_ERROR "libcxx not found!")
   endif()
+  if(NOT COMPILER_RT_LIBCXXABI_PATH)
+    message(FATAL_ERROR "libcxxabi not found!")
+  endif()
 
   cmake_parse_arguments(LIBCXX "USE_TOOLCHAIN" "" "DEPS;CFLAGS;CMAKE_ARGS" ${ARGN})
 
   if(LIBCXX_USE_TOOLCHAIN)
     set(compiler_args -DCMAKE_C_COMPILER=${COMPILER_RT_TEST_COMPILER}
                       -DCMAKE_CXX_COMPILER=${COMPILER_RT_TEST_CXX_COMPILER})
     if(NOT COMPILER_RT_STANDALONE_BUILD AND NOT RUNTIMES_BUILD)
       set(toolchain_deps $<TARGET_FILE:clang>)
macro(add_custom_libcxx name prefix)
 
   string(REPLACE ";" " " LIBCXX_CXX_FLAGS "${LIBCXX_CFLAGS}")
   get_property(CXX_FLAGS CACHE CMAKE_CXX_FLAGS PROPERTY VALUE)
   set(LIBCXX_CXX_FLAGS "${LIBCXX_CXX_FLAGS} ${CXX_FLAGS}")
 
   ExternalProject_Add(${name}
     DEPENDS ${name}-clobber ${LIBCXX_DEPS}
     PREFIX ${prefix}
-    SOURCE_DIR ${COMPILER_RT_LIBCXX_PATH}
+    SOURCE_DIR ${COMPILER_RT_SOURCE_DIR}/cmake/Modules/CustomLibcxx
     STAMP_DIR ${STAMP_DIR}
     BINARY_DIR ${BINARY_DIR}
     CMAKE_ARGS ${CMAKE_PASSTHROUGH_VARIABLES}
                ${compiler_args}
                -DCMAKE_C_FLAGS=${LIBCXX_C_FLAGS}
                -DCMAKE_CXX_FLAGS=${LIBCXX_CXX_FLAGS}
                -DCMAKE_BUILD_TYPE=Release
                -DLLVM_PATH=${LLVM_MAIN_SRC_DIR}
                -DLLVM_BINARY_DIR=${prefix}
                -DLLVM_LIBRARY_OUTPUT_INTDIR=${prefix}/lib
-               -DLIBCXX_STANDALONE_BUILD=ON
+               -DCOMPILER_RT_LIBCXX_PATH=${COMPILER_RT_LIBCXX_PATH}
+               -DCOMPILER_RT_LIBCXXABI_PATH=${COMPILER_RT_LIBCXXABI_PATH}
                ${LIBCXX_CMAKE_ARGS}
     INSTALL_COMMAND ""
     STEP_TARGETS configure build
     BUILD_ALWAYS 1
     USES_TERMINAL_CONFIGURE 1
     USES_TERMINAL_BUILD 1
     USES_TERMINAL_INSTALL 1
     EXCLUDE_FROM_ALL TRUE

cmake/Modules/CustomLibcxx/CMakeLists.txt

+cmake_minimum_required(VERSION 3.4.3)
+
+# Build static libcxxabi.
+set(LIBCXXABI_STANDALONE_BUILD 1)
+set(LIBCXXABI_ENABLE_SHARED OFF CACHE BOOL "")
+set(LIBCXXABI_HERMETIC_STATIC_LIBRARY ON CACHE STRING "")
+set(LIBCXXABI_LIBCXX_PATH ${COMPILER_RT_LIBCXX_PATH} CACHE PATH "")
+set(LIBCXXABI_INCLUDE_TESTS OFF CACHE BOOL "")
+add_subdirectory(${COMPILER_RT_LIBCXXABI_PATH} ${CMAKE_CURRENT_BINARY_DIR}/cxxabi)
+
+# Build static libcxx without exceptions.
+set(LIBCXX_STANDALONE_BUILD 1)
+set(LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_SHARED OFF CACHE BOOL "")
+set(LIBCXX_ENABLE_EXCEPTIONS OFF CACHE BOOL "")
+set(LIBCXX_HERMETIC_STATIC_LIBRARY ON CACHE BOOL "")
+
+# Use above libcxxabi.
+set(LIBCXX_CXX_ABI "libcxxabi" CACHE STRING "")
+set(LIBCXX_CXX_ABI_INTREE 1)
+set(LIBCXX_ENABLE_STATIC_ABI_LIBRARY ON CACHE BOOL "")
+set(LIBCXX_CXX_ABI_INCLUDE_PATHS ${COMPILER_RT_LIBCXXABI_PATH}/include CACHE PATH "")
+
+add_subdirectory(${COMPILER_RT_LIBCXX_PATH} ${CMAKE_CURRENT_BINARY_DIR}/cxx)

lib/fuzzer/CMakeLists.txt

CHECK_CXX_SOURCE_COMPILES("
   static thread_local int blah;
   int main() {
   return 0;
   }
   " HAS_THREAD_LOCAL)
 
 set(LIBFUZZER_CFLAGS ${SANITIZER_COMMON_CFLAGS})
 
-if(OS_NAME MATCHES "Linux|Fuchsia" AND COMPILER_RT_LIBCXX_PATH)
+if(OS_NAME MATCHES "Linux|Fuchsia" AND
+   COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH)
   list(APPEND LIBFUZZER_CFLAGS -nostdinc++ -D_LIBCPP_ABI_VERSION=Fuzzer)
   # Remove -stdlib= which is unused when passing -nostdinc++.
   string(REGEX REPLACE "-stdlib=[a-zA-Z+]*" "" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS})
 elseif(TARGET cxx-headers OR HAVE_LIBCXX)
   set(LIBFUZZER_DEPS cxx-headers)
 endif()
 
 append_list_if(COMPILER_RT_HAS_OMIT_FRAME_POINTER_FLAG -fno-omit-frame-pointer LIBFUZZER_CFLAGS)

 add_compiler_rt_runtime(clang_rt.fuzzer_no_main
   STATIC
   OS ${FUZZER_SUPPORTED_OS}
   ARCHS ${FUZZER_SUPPORTED_ARCH}
   OBJECT_LIBS RTfuzzer
   CFLAGS ${LIBFUZZER_CFLAGS}
   PARENT_TARGET fuzzer)
 
-if(OS_NAME MATCHES "Linux|Fuchsia" AND COMPILER_RT_LIBCXX_PATH)
+if(OS_NAME MATCHES "Linux|Fuchsia" AND
+   COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH)
   macro(partially_link_libcxx name dir arch)
     set(cxx_${arch}_merge_dir "${CMAKE_CURRENT_BINARY_DIR}/cxx_${arch}_merge.dir")
     file(MAKE_DIRECTORY ${cxx_${arch}_merge_dir})
     add_custom_command(TARGET clang_rt.${name}-${arch} POST_BUILD
       COMMAND ${CMAKE_LINKER} --whole-archive "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>" --no-whole-archive ${dir}/lib/libc++.a -r -o ${name}.o
       COMMAND ${CMAKE_OBJCOPY} --localize-hidden ${name}.o
       COMMAND ${CMAKE_COMMAND} -E remove "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>"
       COMMAND ${CMAKE_AR} qcs "$<TARGET_LINKER_FILE:clang_rt.${name}-${arch}>" ${name}.o
       WORKING_DIRECTORY ${cxx_${arch}_merge_dir}
     )
   endmacro()
 
   foreach(arch ${FUZZER_SUPPORTED_ARCH})
     get_target_flags_for_arch(${arch} TARGET_CFLAGS)
     set(LIBCXX_${arch}_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/libcxx_fuzzer_${arch})
     add_custom_libcxx(libcxx_fuzzer_${arch} ${LIBCXX_${arch}_PREFIX}
       CFLAGS ${TARGET_CFLAGS}
-             -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS=1
-             -fvisibility=hidden
       CMAKE_ARGS -DCMAKE_CXX_COMPILER_WORKS=ON
-                 -DLIBCXX_ENABLE_EXCEPTIONS=OFF
-                 -DLIBCXX_ENABLE_SHARED=OFF
-                 -DLIBCXX_ABI_NAMESPACE=Fuzzer
-                 -DLIBCXX_CXX_ABI=none)
+                 -DLIBCXX_ABI_NAMESPACE=Fuzzer)
     target_compile_options(RTfuzzer.${arch} PRIVATE -isystem ${LIBCXX_${arch}_PREFIX}/include/c++/v1)
     add_dependencies(RTfuzzer.${arch} libcxx_fuzzer_${arch}-build)
     target_compile_options(RTfuzzer_main.${arch} PRIVATE -isystem ${LIBCXX_${arch}_PREFIX}/include/c++/v1)
     add_dependencies(RTfuzzer_main.${arch} libcxx_fuzzer_${arch}-build)
     partially_link_libcxx(fuzzer_no_main ${LIBCXX_${arch}_PREFIX} ${arch})
     partially_link_libcxx(fuzzer ${LIBCXX_${arch}_PREFIX} ${arch})
   endforeach()
 endif()
 
 if(COMPILER_RT_INCLUDE_TESTS)
   add_subdirectory(tests)
 endif()

lib/fuzzer/tests/CMakeLists.txt

 
 foreach(lib ${SANITIZER_TEST_CXX_LIBRARIES})
   list(APPEND LIBFUZZER_UNITTEST_LINK_FLAGS -l${lib})
 endforeach()
 if(NOT WIN32)
   list(APPEND LIBFUZZER_UNITTEST_LINK_FLAGS -lpthread)
 endif()
 
-if("${CMAKE_SYSTEM_NAME}" STREQUAL "Linux" AND COMPILER_RT_LIBCXX_PATH)
+if(CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
+   COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH)
   list(APPEND LIBFUZZER_UNITTEST_CFLAGS -nostdinc++)
 endif()
 
 if(COMPILER_RT_DEFAULT_TARGET_ARCH IN_LIST FUZZER_SUPPORTED_ARCH)
   # libFuzzer unit tests are only run on the host machine.
   set(arch ${COMPILER_RT_DEFAULT_TARGET_ARCH})
 
   set(LIBFUZZER_TEST_RUNTIME RTFuzzerTest.${arch})
   if(APPLE)
     set(LIBFUZZER_TEST_RUNTIME_OBJECTS
       $<TARGET_OBJECTS:RTfuzzer.osx>)
   else()
     set(LIBFUZZER_TEST_RUNTIME_OBJECTS
       $<TARGET_OBJECTS:RTfuzzer.${arch}>)
   endif()
   add_library(${LIBFUZZER_TEST_RUNTIME} STATIC
     ${LIBFUZZER_TEST_RUNTIME_OBJECTS})
   set_target_properties(${LIBFUZZER_TEST_RUNTIME} PROPERTIES
     ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
     FOLDER "Compiler-RT Runtime tests")
 
-  if("${CMAKE_SYSTEM_NAME}" STREQUAL "Linux" AND COMPILER_RT_LIBCXX_PATH)
+  if(CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
+     COMPILER_RT_LIBCXX_PATH AND
+     COMPILER_RT_LIBCXXABI_PATH)
     file(GLOB libfuzzer_headers ../*.h)
     set(LIBFUZZER_TEST_RUNTIME_DEPS libcxx_fuzzer_${arch}-build ${libfuzzer_headers})
     set(LIBFUZZER_TEST_RUNTIME_CFLAGS -isystem ${LIBCXX_${arch}_PREFIX}/include/c++/v1)
     set(LIBFUZZER_TEST_RUNTIME_LINK_FLAGS ${LIBCXX_${arch}_PREFIX}/lib/libc++.a)
   endif()
 
   set(FuzzerTestObjects)
   generate_compiler_rt_tests(FuzzerTestObjects

lib/msan/tests/CMakeLists.txt

 set(MSAN_UNITTEST_INSTRUMENTED_CFLAGS
   ${MSAN_UNITTEST_COMMON_CFLAGS}
   -fsanitize=memory
   -fsanitize-memory-track-origins
   -mllvm -msan-keep-going=1
 )
 set(MSAN_UNITTEST_LINK_FLAGS
   -fsanitize=memory
-  # Don't need -stdlib=libc++ because we explicitly list libc++.so in the linker
+  # Don't need -stdlib=libc++ because we explicitly list libc++.a in the linker
   # inputs.
-  # FIXME: we build libcxx without cxxabi and need libstdc++ to provide it.
-  -lstdc++
 )
 
 append_list_if(COMPILER_RT_HAS_LIBDL -ldl MSAN_UNITTEST_LINK_FLAGS)
 
 macro(msan_compile obj_list source arch kind cflags)
   sanitizer_test_compile(
     ${obj_list} ${source} ${arch}
     KIND ${kind}
macro(add_msan_tests_for_arch arch kind cflags)
   set(MSAN_TEST_OBJECTS ${MSAN_INST_TEST_OBJECTS} ${MSAN_INST_GTEST})
   set(MSAN_TEST_DEPS ${MSAN_TEST_OBJECTS} libcxx_msan_${arch}-build
                      ${MSAN_LOADABLE_SO})
   if(NOT COMPILER_RT_STANDALONE_BUILD)
     list(APPEND MSAN_TEST_DEPS msan)
   endif()
   get_target_flags_for_arch(${arch} TARGET_LINK_FLAGS)
   add_compiler_rt_test(MsanUnitTests "Msan-${arch}${kind}-Test" ${arch}
-    OBJECTS ${MSAN_TEST_OBJECTS} ${MSAN_LIBCXX_SO}
+    OBJECTS ${MSAN_TEST_OBJECTS} "${MSAN_LIBCXX_DIR}/libc++.a"
     DEPS ${MSAN_TEST_DEPS}
     LINK_FLAGS ${MSAN_UNITTEST_LINK_FLAGS}
-               ${TARGET_LINK_FLAGS}
-               "-Wl,-rpath=${CMAKE_CURRENT_BINARY_DIR}"
-               "-Wl,-rpath=${LIBCXX_PREFIX}/lib")
+               ${TARGET_LINK_FLAGS})
 endmacro()
 
 # We should only build MSan unit tests if we can build instrumented libcxx.
-if(COMPILER_RT_CAN_EXECUTE_TESTS AND COMPILER_RT_LIBCXX_PATH)
+if(COMPILER_RT_CAN_EXECUTE_TESTS AND
+   COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH)
   foreach(arch ${MSAN_SUPPORTED_ARCH})
     get_target_flags_for_arch(${arch} TARGET_CFLAGS)
     set(LIBCXX_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/../libcxx_msan_${arch})
     add_custom_libcxx(libcxx_msan_${arch} ${LIBCXX_PREFIX}
       DEPS ${MSAN_RUNTIME_LIBRARIES}
       CFLAGS ${MSAN_LIBCXX_CFLAGS} ${TARGET_CFLAGS}
       USE_TOOLCHAIN)
-    set(MSAN_LIBCXX_SO ${LIBCXX_PREFIX}/lib/libc++.so)
+    set(MSAN_LIBCXX_DIR ${LIBCXX_PREFIX}/lib/)
 
     add_msan_tests_for_arch(${arch} "" "")
     add_msan_tests_for_arch(${arch} "-with-call"
                             "-mllvm;-msan-instrumentation-with-call-threshold=0")
   endforeach()
 endif()

lib/tsan/CMakeLists.txt

file(GLOB _tsan_platform_sources rtl/tsan*posix* rtl/tsan*mac*
                                    rtl/tsan*linux*)
   list(REMOVE_ITEM _tsan_generic_sources ${_tsan_platform_sources})
   set_source_files_properties(${_tsan_generic_sources}
     PROPERTIES COMPILE_FLAGS "--sysroot=.")
 endif()
 
 # Build libcxx instrumented with TSan.
 if(COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH AND
    COMPILER_RT_TEST_COMPILER_ID STREQUAL "Clang" AND
    NOT ANDROID)
   set(libcxx_tsan_deps)
   foreach(arch ${TSAN_SUPPORTED_ARCH})
     get_target_flags_for_arch(${arch} TARGET_CFLAGS)
     set(LIBCXX_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/libcxx_tsan_${arch})
     add_custom_libcxx(libcxx_tsan_${arch} ${LIBCXX_PREFIX}
       DEPS ${TSAN_RUNTIME_LIBRARIES}

test/fuzzer/lit.cfg

 
 config.substitutions.append(('%build_dir', config.cmake_binary_dir))
 libfuzzer_src_root = os.path.join(config.compiler_rt_src_root, "lib", "fuzzer")
 config.substitutions.append(('%libfuzzer_src', libfuzzer_src_root))
 
 def generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True, msan_enabled=False):
   compiler_cmd = config.clang
   extra_cmd = config.target_flags
-  if config.clang and config.stdlib == 'libc++':
-    link_cmd = '-stdlib=libc++ -Wl,-rpath=%s' % config.runtime_library_dir
-  elif config.clang and config.stdlib == 'static-libc++':
-    link_cmd = '-stdlib=libc++ -lc++abi -static-libstdc++ -Wl,-rpath=%s' % (
-        config.runtime_library_dir)
-  elif any(x in config.target_triple for x in ('darwin', 'freebsd')):
-    link_cmd = '-lc++'
-  elif 'windows-msvc' in config.target_triple:
-    link_cmd = ''
-  else:
-    link_cmd = '-lstdc++'
 
   if is_cpp and 'windows-msvc' in config.target_triple:
     std_cmd = '--driver-mode=cl'
   elif is_cpp:
     std_cmd = '--driver-mode=g++ -std=c++11'
   else:
     std_cmd = ''
 
   if msan_enabled:
     sanitizers = ['memory']
   else:
     sanitizers = ['address']
   if fuzzer_enabled:
     sanitizers.append('fuzzer')
   sanitizers_cmd = ('-fsanitize=%s' % ','.join(sanitizers))
   return " ".join([
     compiler_cmd,
     std_cmd,
-    link_cmd,
     "-O2 -gline-tables-only",
     sanitizers_cmd,
     "-I%s" % libfuzzer_src_root,
     extra_cmd
   ])
 
 config.substitutions.append(('%cpp_compiler',
       generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True)

test/msan/CMakeLists.txt

if(COMPILER_RT_HAS_LLD AND arch STREQUAL "x86_64" AND NOT (APPLE OR WIN32))
     add_msan_testsuite(${arch} True False)
   endif()
 endforeach()
 
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND MSAN_TEST_DEPS msan)
 endif()
 
-if(COMPILER_RT_INCLUDE_TESTS AND COMPILER_RT_LIBCXX_PATH)
+if(COMPILER_RT_INCLUDE_TESTS AND
+   COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH)
   configure_lit_site_cfg(
     ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.in
     ${CMAKE_CURRENT_BINARY_DIR}/Unit/lit.site.cfg)
   list(APPEND MSAN_TEST_DEPS MsanUnitTests)
   list(APPEND MSAN_TESTSUITES ${CMAKE_CURRENT_BINARY_DIR}/Unit)
 endif()
 
 add_lit_testsuite(check-msan "Running the MemorySanitizer tests"
   ${MSAN_TESTSUITES}
   DEPENDS ${MSAN_TEST_DEPS}
   )
 set_target_properties(check-msan PROPERTIES FOLDER "Compiler-RT Misc")

test/tsan/CMakeLists.txt

 set(TSAN_LIT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 
 set(TSAN_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS})
 if(${COMPILER_RT_DEFAULT_TARGET_ARCH} MATCHES "x86_64")
   list(APPEND TSAN_TEST_DEPS GotsanRuntimeCheck)
 endif()
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND TSAN_TEST_DEPS tsan)
 endif()
 if(COMPILER_RT_LIBCXX_PATH AND
+   COMPILER_RT_LIBCXXABI_PATH AND
    COMPILER_RT_TEST_COMPILER_ID STREQUAL "Clang"
    AND NOT APPLE AND NOT ANDROID)
   list(APPEND TSAN_TEST_DEPS libcxx_tsan)
   set(TSAN_HAS_LIBCXX True)
 else()
   set(TSAN_HAS_LIBCXX False)
 endif()
 

test/tsan/Linux/check_memcpy.c

+// Test that verifies TSan runtime doesn't contain compiler-emitted
+// memcpy/memmove calls. It builds the binary with TSan and check's
+// its objdump.
+
+// RUN: %clang_tsan -O1 %s -o %t
+// RUN: llvm-objdump -d %t | FileCheck %s
+
+// REQUIRES: compiler-rt-optimized
+
+int main() {
+  return 0;
+}
+
+// CHECK-NOT: callq {{.*<(__interceptor_)?mem(cpy|set)>}}
+// tail calls:
+// CHECK-NOT: jmpq {{.*<(__interceptor_)?mem(cpy|set)>}}
+

test/tsan/Linux/check_memcpy.cc

-// Test that verifies TSan runtime doesn't contain compiler-emitted
-// memcpy/memmove calls. It builds the binary with TSan and passes it to
-// check_memcpy.sh script.
-
-// RUN: %clangxx_tsan -O1 %s -o %t
-// RUN: llvm-objdump -d %t | FileCheck %s
-
-// REQUIRES: compiler-rt-optimized
-
-int main() {
-  return 0;
-}
-
-// CHECK-NOT: callq {{.*<(__interceptor_)?mem(cpy|set)>}}
-// tail calls:
-// CHECK-NOT: jmpq {{.*<(__interceptor_)?mem(cpy|set)>}}
-

test/tsan/lit.cfg

 if default_tsan_opts:
   config.environment['TSAN_OPTIONS'] = default_tsan_opts
   default_tsan_opts += ':'
 config.substitutions.append(('%env_tsan_opts=',
                              'env TSAN_OPTIONS=' + default_tsan_opts))
 
 # GCC driver doesn't add necessary compile/link flags with -fsanitize=thread.
 if config.compiler_id == 'GNU':
-  extra_cflags = ["-fPIE", "-pthread", "-ldl", "-lstdc++", "-lrt", "-pie"]
+  extra_cflags = ["-fPIE", "-pthread", "-ldl", "-lrt", "-pie"]
 else:
   extra_cflags = []
 
 tsan_incdir = config.test_source_root + "/../"
 # Setup default compiler flags used with -fsanitize=thread option.
 clang_tsan_cflags = (["-fsanitize=thread",
                       "-Wall"] +
                       [config.target_cflags] +
                       config.debug_info_flags +
                       extra_cflags +
                       ["-I%s" % tsan_incdir])
 clang_tsan_cxxflags = config.cxx_mode_flags + clang_tsan_cflags + ["-std=c++11"] + ["-I%s" % tsan_incdir]
 # Add additional flags if we're using instrumented libc++.
 # Instrumented libcxx currently not supported on Darwin.
 if config.has_libcxx and config.host_os != 'Darwin':
   # FIXME: Dehardcode this path somehow.
   libcxx_path = os.path.join(config.compiler_rt_obj_root, "lib",
                              "tsan", "libcxx_tsan_%s" % config.target_arch)
   libcxx_incdir = os.path.join(libcxx_path, "include", "c++", "v1")
   libcxx_libdir = os.path.join(libcxx_path, "lib")
-  libcxx_so = os.path.join(libcxx_libdir, "libc++.so")
+  libcxx_a = os.path.join(libcxx_libdir, "libc++.a")
   clang_tsan_cxxflags += ["-nostdinc++",
                           "-I%s" % libcxx_incdir,
-                          libcxx_so,
-                          "-Wl,-rpath=%s" % libcxx_libdir]
+                          libcxx_a]
 
 def build_invocation(compile_flags):
   return " " + " ".join([config.clang] + compile_flags) + " "
 
 config.substitutions.append( ("%clang_tsan ", build_invocation(clang_tsan_cflags)) )
 config.substitutions.append( ("%clangxx_tsan ", build_invocation(clang_tsan_cxxflags)) )
 
 # Define CHECK-%os to check for OS-dependent output.

test/tsan/static_init6.cc

-// RUN: %clangxx_tsan %linux_static_libstdcplusplus -O1 %s -o %t && %run %t 2>&1 \
-// RUN: | FileCheck %s
+// RUN: %clangxx_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 #include <pthread.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <sched.h>
 
 struct Cache {
   int x;
   explicit Cache(int x)