Support for multiarch runtimes layout

[phosek]

Summary

This change adds a support for multiarch style runtimes layout, so in
addition to the existing layout where runtimes get installed to:

lib/clang/$version/lib/$os

Clang now allows runtimes to be installed to:

lib/clang/$version/$target/lib

This also includes libc++, libc++abi and libunwind; today those are
assumed to be in Clang library directory built for host, with the
new layout it is possible to install libc++, libc++abi and libunwind
into the runtime directory built for different targets.

The use of new layout is enabled by setting the
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR CMake variable and is supported by both
projects and runtimes layouts. The runtimes CMake build has been further
modified to use the new layout when building runtimes for multiple
targets.

[phosek]

Currently everything is a single patch but I'd be happy to split into separate patches and submit those separately against each subproject if you're fine with the overall design.

[rnk]

The overall goal makes sense to me, but I feel like this patch adds many CMake variables to accomplish it, which are only set from the Fuchsia CMake cache. I feel like it would be simpler to have one CMake flag that clang and the runtime libraries look at to decide if they want to use the old libclang_rt.$arch.$lib paths, or the new clang/$version/$triple/lib/$lib paths.

In other words, we should aim to make the CMake cache scripts as simple as possible. Most developers (so far as I know) don't use the cache scripts to configure their local builds. They directly invoke LLVM's CMake, usually with a shell script with their favorite options. Ideally, changing the runtime library layout this way would be adding a simple flag to that script, rather than requiring developers to switch to these cache scripts or duplicating the logic of the cache scripts.

[phosek]

In D45604#1068915, @rnk wrote:

The overall goal makes sense to me, but I feel like this patch adds many CMake variables to accomplish it, which are only set from the Fuchsia CMake cache. I feel like it would be simpler to have one CMake flag that clang and the runtime libraries look at to decide if they want to use the old libclang_rt.$arch.$lib paths, or the new clang/$version/$triple/lib/$lib paths.

In other words, we should aim to make the CMake cache scripts as simple as possible. Most developers (so far as I know) don't use the cache scripts to configure their local builds. They directly invoke LLVM's CMake, usually with a shell script with their favorite options. Ideally, changing the runtime library layout this way would be adding a simple flag to that script, rather than requiring developers to switch to these cache scripts or duplicating the logic of the cache scripts.

It's not necessarily Fuchsia-specific, but it relies on the runtimes build which is currently only used by Fuchsia and Android AFAIK, that build model makes some things easier, e.g. compiler-rt is always built for a single target only (list of targets is configured via cache files). I'm fine changing this to guard the layout used behind a single CMake variable (e.g. ENABLE_RUNTIME_TARGET_DIR), but it'll require more changes to compiler-rt build, is that fine with you?

[phosek]

I've modified the patch following @rnk suggestion, the new layout is enabled by a single CMake option CLANG_ENABLE_PER_TARGET_RUNTIME_DIR. I'm still working on fixing tests, but I've tested the build and it's working for both runtimes/ and projects/ layout. Let me know what you think.

[phosek]

In D45604#1084815, @phosek wrote:

I've modified the patch following @rnk suggestion, the new layout is enabled by a single CMake option CLANG_ENABLE_PER_TARGET_RUNTIME_DIR. I'm still working on fixing tests, but I've tested the build and it's working for both runtimes/ and projects/ layout. Let me know what you think.

One issue I ran into while working on getting tests to work is that in the current patch, enabling CLANG_ENABLE_PER_TARGET_RUNTIME_DIR enables the new layout in the driver and disables the old one (e.g. when constructing compiler-rt paths). However, there are plenty of tests that hardcode the libclang_rt.builtins-<arch>.a name and those would all have to be disabled when the new layout is enabled. Maybe it'd be better to only enable the new layout but support fallback back onto the old one (i.e. search both the new and the old path for compiler-rt libraries)?

[rnk]

I went on vacation and haven't had more time to look at this since I got back. Maybe @hans and @vitalybuka might be able to help review this.

[hans]

I only looked a little at the Clang parts (the compiler-rt and libcxx cmake stuff I'm not familiar with). It seems reasonable, and would make it easier to ship a cross-compiler I suppose, but I think folks with a higher-level Clang view need to take a look too.

clang/lib/Driver/ToolChain.cpp

383 ↗

(On Diff #144819)

Why is it using the default target triple here and not the current one? (same for the ToolChain ctor).

And the #ifdefs seems unfortunate.. could Clang just look in the target-specific directory if it exists and fall back otherwise or something?

[filcab]

clang/lib/Driver/ToolChain.cpp

90 ↗

(On Diff #144819)

Maybe use cmakedefine01 and change this to a simple if?

392 ↗

(On Diff #144819)

These last two lines can probably get hoisted out of the if and make it easier to notice the difference between the two paths. Same comment as above about using a "normal if" instead of #if.

clang/lib/Driver/ToolChains/Fuchsia.cpp

152 ↗

(On Diff #144819)

Why "compute triple + set triple + return Triple.str()" instead of a plain "compute triple + return arch + '-' + os"?

clang/lib/Driver/ToolChains/Fuchsia.h

69 ↗

(On Diff #144819)

Why the extra default argument? The patch doesn't seem to show any call to this function that requires a single arg.

[beanz]

Looping in @bruno in case he has feedback.

[phosek]

clang/lib/Driver/ToolChain.cpp

383 ↗

(On Diff #144819)

getDefaultTargetTriple() returns either default triple or the current one if overriden via --target= option, I just reused the existing value. I think this should be called simply getTargetTriple(), that should be a simple refactoring in Driver (famous last words).

I could do the fallback, I had that implemented in the first version of the patch but then Reid suggested guarding this feature behind a flag. I can still keep the flag, but only use it for CMake and make the driver support both variants.

[phosek]

All tests now seem to be passing when ENABLE_PER_TARGET_RUNTIME_DIR is set and all prerequisites have landed.

[rnk]

compiler-rt/cmake/base-config-ix.cmake

72 ↗

(On Diff #149839)

Does this usage still work in a standalone compiler-rt build? I'd expect it might warn about using an undefined variable. You might need to duplicate the option into each top-level cmake and do some ugly if (NOT DEFINED ) hacks, or use some of the existing "if standalone" checks.

[phosek]

compiler-rt/cmake/base-config-ix.cmake

72 ↗

(On Diff #149839)

Yes, this expression is evaluated as "True if given a variable that is defined to a value that is not a false constant. False otherwise." I checked the standalone build and it's working as before.

[phosek]

Ping? Does anyone have anymore comments?

[vitalybuka]

Does it have to be a single patch?

[rnk]

In D45604#1128585, @vitalybuka wrote:

Does it have to be a single patch?

I think it makes sense for it to be one. It's technically possible to split out the clang changes, but there would be no motivation without changing the installation layout of compiler-rt, libc++, and the rest.

In D45604#1128571, @phosek wrote:

Ping? Does anyone have anymore comments?

Let's land it. If we have anymore feedback, we can fix it in post.

[Hahnfeld]

Should ENABLE_PER_TARGET_RUNTIME_DIR be declared at LLVM's top level CMakeLists.txt? If I read the patch correctly, it's in clang so llvm/runtimes/ relies on the default value being OFF (hard to tell because the patch doesn't follow the current layout of the SVN repository but rather a git monorepo?)

[phosek]

In D45604#1141891, @Hahnfeld wrote:

Should ENABLE_PER_TARGET_RUNTIME_DIR be declared at LLVM's top level CMakeLists.txt? If I read the patch correctly, it's in clang so llvm/runtimes/ relies on the default value being OFF (hard to tell because the patch doesn't follow the current layout of the SVN repository but rather a git monorepo?)

I'm fine moving that option to LLVM's CMakeLists.txt, it's more specific to Clang which is why I defined it in Clang's CMakeLists.txt, but it's also needed by all runtimes so LLVM's CMakeLists.txt is probably more appropriate.

CMakeLists.txt

 if ("${COMPILER_RT_DEFAULT_TARGET_ABI}" MATCHES "^android")
   set(ANDROID 1)
 endif()
 pythonize_bool(ANDROID)
 
 set(COMPILER_RT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 set(COMPILER_RT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
 
+pythonize_bool(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR)
+
 # We support running instrumented tests when we're not cross compiling
 # and target a UNIX-like system or Windows.
 # We can run tests on Android even when we are cross-compiling.
 if(("${CMAKE_HOST_SYSTEM}" STREQUAL "${CMAKE_SYSTEM}" AND (UNIX OR WIN32)) OR ANDROID
    OR COMPILER_RT_EMULATOR)
   option(COMPILER_RT_CAN_EXECUTE_TESTS "Can we execute instrumented tests" ON)
 else()
   option(COMPILER_RT_CAN_EXECUTE_TESTS "Can we execute instrumented tests" OFF)

cmake/Modules/AddCompilerRT.cmake

if (${CMAKE_GENERATOR} STREQUAL "Xcode")
     enable_language(ASM)
   else()
     # Pass ASM file directly to the C++ compiler.
     set_source_files_properties(${ARGN} PROPERTIES LANGUAGE C)
   endif()
 endfunction()
 
 macro(set_output_name output name arch)
+  if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR)
+    set(${output} ${name})
+  else()
     if(ANDROID AND ${arch} STREQUAL "i386")
       set(${output} "${name}-i686${COMPILER_RT_OS_SUFFIX}")
     else()
       set(${output} "${name}-${arch}${COMPILER_RT_OS_SUFFIX}")
     endif()
+  endif()
 endmacro()
 
 # Adds static or shared runtime for a list of architectures and operating
 # systems and puts it in the proper directory in the build and install trees.
 # add_compiler_rt_runtime(<name>
 #                         {STATIC|SHARED}
 #                         ARCHS <architectures>
 #                         OS <os list>
foreach(os ${LIB_OS})
       endif()
       list_intersect(LIB_ARCHS_${libname} DARWIN_${os}_ARCHS LIB_ARCHS)
       if(LIB_ARCHS_${libname})
         list(APPEND libnames ${libname})
         set(extra_cflags_${libname} ${DARWIN_${os}_CFLAGS} ${NO_LTO_FLAGS} ${LIB_CFLAGS})
         set(output_name_${libname} ${libname}${COMPILER_RT_OS_SUFFIX})
         set(sources_${libname} ${LIB_SOURCES})
         format_object_libs(sources_${libname} ${os} ${LIB_OBJECT_LIBS})
+        get_compiler_rt_output_dir(${COMPILER_RT_DEFAULT_TARGET_ARCH} output_dir_${libname})
+        get_compiler_rt_install_dir(${COMPILER_RT_DEFAULT_TARGET_ARCH} install_dir_${libname})
       endif()
     endforeach()
   else()
     foreach(arch ${LIB_ARCHS})
       if(NOT CAN_TARGET_${arch})
         message(FATAL_ERROR "Architecture ${arch} can't be targeted")
         return()
       endif()
foreach(arch ${LIB_ARCHS})
         else()
           set_output_name(output_name_${libname} ${name} ${arch})
         endif()
       endif()
       set(sources_${libname} ${LIB_SOURCES})
       format_object_libs(sources_${libname} ${arch} ${LIB_OBJECT_LIBS})
       set(libnames ${libnames} ${libname})
       set(extra_cflags_${libname} ${TARGET_${arch}_CFLAGS} ${NO_LTO_FLAGS} ${LIB_CFLAGS})
+      get_compiler_rt_output_dir(${arch} output_dir_${libname})
+      get_compiler_rt_install_dir(${arch} install_dir_${libname})
     endforeach()
   endif()
 
   if(NOT libnames)
     return()
   endif()
 
   if(LIB_PARENT_TARGET)
else()
       set(COMPONENT_OPTION COMPONENT ${libname})
     endif()
 
     add_library(${libname} ${type} ${sources_${libname}})
     set_target_compile_flags(${libname} ${extra_cflags_${libname}})
     set_target_link_flags(${libname} ${extra_link_flags_${libname}})
     set_property(TARGET ${libname} APPEND PROPERTY
                 COMPILE_DEFINITIONS ${LIB_DEFS})
-    set_target_output_directories(${libname} ${COMPILER_RT_LIBRARY_OUTPUT_DIR})
+    set_target_output_directories(${libname} ${output_dir_${libname}})
     set_target_properties(${libname} PROPERTIES
         OUTPUT_NAME ${output_name_${libname}})
     set_target_properties(${libname} PROPERTIES FOLDER "Compiler-RT Runtime")
     if(LIB_LINK_LIBS)
       target_link_libraries(${libname} ${LIB_LINK_LIBS})
     endif()
     if(${type} STREQUAL "SHARED")
       if(COMMAND llvm_setup_rpath)
         llvm_setup_rpath(${libname})
       endif()
       if(WIN32 AND NOT CYGWIN AND NOT MINGW)
         set_target_properties(${libname} PROPERTIES IMPORT_PREFIX "")
         set_target_properties(${libname} PROPERTIES IMPORT_SUFFIX ".lib")
       endif()
       if(APPLE)
         # Ad-hoc sign the dylibs
         add_custom_command(TARGET ${libname}
           POST_BUILD  
           COMMAND codesign --sign - $<TARGET_FILE:${libname}>
           WORKING_DIRECTORY ${COMPILER_RT_LIBRARY_OUTPUT_DIR}
         )
       endif()
     endif()
     install(TARGETS ${libname}
-      ARCHIVE DESTINATION ${COMPILER_RT_LIBRARY_INSTALL_DIR}
+      ARCHIVE DESTINATION ${install_dir_${libname}}
               ${COMPONENT_OPTION}
-      LIBRARY DESTINATION ${COMPILER_RT_LIBRARY_INSTALL_DIR}
+      LIBRARY DESTINATION ${install_dir_${libname}}
               ${COMPONENT_OPTION}
-      RUNTIME DESTINATION ${COMPILER_RT_LIBRARY_INSTALL_DIR}
+      RUNTIME DESTINATION ${install_dir_${libname}}
               ${COMPONENT_OPTION})
 
     # We only want to generate per-library install targets if you aren't using
     # an IDE because the extra targets get cluttered in IDEs.
     if(NOT CMAKE_CONFIGURATION_TYPES)
       add_custom_target(install-${libname}
                         DEPENDS ${libname}
                         COMMAND "${CMAKE_COMMAND}"
function(rt_externalize_debuginfo name)
   endif()
 endfunction()
 
 
 # Configure lit configuration files, including compiler-rt specific variables.
 function(configure_compiler_rt_lit_site_cfg input output)
   set_llvm_build_mode()
 
+  get_compiler_rt_output_dir(${COMPILER_RT_DEFAULT_TARGET_ARCH} output_dir)
+
   string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} COMPILER_RT_RESOLVED_TEST_COMPILER ${COMPILER_RT_TEST_COMPILER})
-  string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR ${COMPILER_RT_LIBRARY_OUTPUT_DIR})
+  string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR ${output_dir})
 
   configure_lit_site_cfg(${input} ${output})
 endfunction()

cmake/Modules/CompilerRTUtils.cmake

foreach (_file ${intermediate})
     elseif(${_file} MATCHES ".*/.*\\.S" OR ${_file} MATCHES ".*/.*\\.c")
       get_filename_component(_name ${_file} NAME)
       string(REPLACE ".S" ".c" _cname "${_name}")
       list(REMOVE_ITEM intermediate ${_cname})
     endif ()
   endforeach ()
   set(${output_var} ${intermediate} PARENT_SCOPE)
 endfunction()
+
+function(get_compiler_rt_target arch variable)
+  if(ANDROID AND ${arch} STREQUAL "i386")
+    set(target "i686${COMPILER_RT_OS_SUFFIX}-${COMPILER_RT_DEFAULT_TARGET_OS}")
+  else()
+    set(target "${arch}-${COMPILER_RT_DEFAULT_TARGET_OS}")
+  endif()
+  if(COMPILER_RT_DEFAULT_TARGET_ABI)
+    set(target "${target}-${COMPILER_RT_DEFAULT_TARGET_ABI}")
+  endif()
+  set(${variable} ${target} PARENT_SCOPE)
+endfunction()
+
+function(get_compiler_rt_install_dir arch install_dir)
+  if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE)
+    get_compiler_rt_target(${arch} target)
+    set(${install_dir} ${COMPILER_RT_INSTALL_PATH}/${target}/lib PARENT_SCOPE)
+  else()
+    set(${install_dir} ${COMPILER_RT_LIBRARY_INSTALL_DIR} PARENT_SCOPE)
+  endif()
+endfunction()
+
+function(get_compiler_rt_output_dir arch output_dir)
+  if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE)
+    get_compiler_rt_target(${arch} target)
+    set(${output_dir} ${COMPILER_RT_OUTPUT_DIR}/${target}/lib PARENT_SCOPE)
+  else()
+    set(${output_dir} ${COMPILER_RT_LIBRARY_OUTPUT_DIR} PARENT_SCOPE)
+  endif()
+endfunction()

cmake/Modules/SanitizerUtils.cmake

+include(CompilerRTUtils)
+
 set(SANITIZER_GEN_DYNAMIC_LIST
   ${COMPILER_RT_SOURCE_DIR}/lib/sanitizer_common/scripts/gen_dynamic_list.py)
 
 set(SANITIZER_LINT_SCRIPT
   ${COMPILER_RT_SOURCE_DIR}/lib/sanitizer_common/scripts/check_lint.sh)
 
 # Create a target "<name>-<arch>-symbols" that would generate the list of
 # symbols that need to be exported from sanitizer runtime "<name>". Function
add_custom_command(OUTPUT ${stamp}
       COMMAND ${CMAKE_COMMAND} -E touch ${stamp}
       DEPENDS ${target_name} ${SANITIZER_GEN_DYNAMIC_LIST} ${ARG_EXTRA}
       WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
       COMMENT "Generating exported symbols for ${target_name}"
       VERBATIM)
     add_custom_target(${target_name}-symbols ALL
       DEPENDS ${stamp}
       SOURCES ${SANITIZER_GEN_DYNAMIC_LIST} ${ARG_EXTRA})
-
+    get_compiler_rt_install_dir(${arch} install_dir)
     install(FILES $<TARGET_FILE:${target_name}>.syms
-            DESTINATION ${COMPILER_RT_LIBRARY_INSTALL_DIR})
+            DESTINATION ${install_dir})
     if(ARG_PARENT_TARGET)
       add_dependencies(${ARG_PARENT_TARGET} ${target_name}-symbols)
     endif()
   endforeach()
 endmacro()
 
 # This function is only used on Darwin, where undefined symbols must be specified
 # in the linker invocation.

cmake/base-config-ix.cmake

elseif("${COMPILER_RT_TEST_COMPILER}" MATCHES "clang.*.exe$")
   set(COMPILER_RT_TEST_COMPILER_ID Clang)
 else()
   set(COMPILER_RT_TEST_COMPILER_ID GNU)
 endif()
 
 if(NOT DEFINED COMPILER_RT_OS_DIR)
   string(TOLOWER ${CMAKE_SYSTEM_NAME} COMPILER_RT_OS_DIR)
 endif()
+if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE)
+  set(COMPILER_RT_LIBRARY_OUTPUT_DIR
+    ${COMPILER_RT_OUTPUT_DIR})
+  set(COMPILER_RT_LIBRARY_INSTALL_DIR
+    ${COMPILER_RT_INSTALL_PATH})
+else(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR)
   set(COMPILER_RT_LIBRARY_OUTPUT_DIR
     ${COMPILER_RT_OUTPUT_DIR}/lib/${COMPILER_RT_OS_DIR})
   set(COMPILER_RT_LIBRARY_INSTALL_DIR
     ${COMPILER_RT_INSTALL_PATH}/lib/${COMPILER_RT_OS_DIR})
+endif()
 
 if(APPLE)
   # On Darwin if /usr/include doesn't exist, the user probably has Xcode but not
   # the command line tools. If this is the case, we need to find the OS X
   # sysroot to pass to clang.
   if(NOT EXISTS /usr/include)
     execute_process(COMMAND xcodebuild -version -sdk macosx Path
        OUTPUT_VARIABLE OSX_SYSROOT

cmake/config-ix.cmake

foreach(platform ${DARWIN_EMBEDDED_PLATFORMS})
         endforeach()
       endif()
     endforeach()
   endif()
 
   # for list_intersect
   include(CompilerRTUtils)
 
-
   list_intersect(SANITIZER_COMMON_SUPPORTED_ARCH
     ALL_SANITIZER_COMMON_SUPPORTED_ARCH
     COMPILER_RT_SUPPORTED_ARCH
     )
   set(LSAN_COMMON_SUPPORTED_ARCH ${SANITIZER_COMMON_SUPPORTED_ARCH})
   set(UBSAN_COMMON_SUPPORTED_ARCH ${SANITIZER_COMMON_SUPPORTED_ARCH})
   list_intersect(ASAN_SUPPORTED_ARCH
     ALL_ASAN_SUPPORTED_ARCH

test/asan/lit.cfg

 
 def build_invocation(compile_flags):
   return " " + " ".join([config.clang] + compile_flags) + " "
 
 config.substitutions.append( ("%clang ", build_invocation(target_cflags)) )
 config.substitutions.append( ("%clangxx ", build_invocation(target_cxxflags)) )
 config.substitutions.append( ("%clang_asan ", build_invocation(clang_asan_cflags)) )
 config.substitutions.append( ("%clangxx_asan ", build_invocation(clang_asan_cxxflags)) )
-config.substitutions.append( ("%shared_libasan", "libclang_rt.asan-%s.so" % config.target_arch))
+config.substitutions.append( ("%shared_libasan", "libclang_rt.asan%s.so" % config.target_suffix))
 if config.asan_dynamic:
   config.substitutions.append( ("%clang_asan_static ", build_invocation(clang_asan_static_cflags)) )
   config.substitutions.append( ("%clangxx_asan_static ", build_invocation(clang_asan_static_cxxflags)) )
 
 # Windows-specific tests might also use the clang-cl.exe driver.
 if platform.system() == 'Windows':
   clang_cl_cxxflags = ["-Wno-deprecated-declarations",
                        "-WX",
                        "-D_HAS_EXCEPTIONS=0",
                        "-Zi"] + target_cflags
   clang_cl_asan_cxxflags = ["-fsanitize=address"] + clang_cl_cxxflags
   if config.asan_dynamic:
     clang_cl_asan_cxxflags.append("-MD")
 
   clang_cl_invocation = build_invocation(clang_cl_cxxflags)
   clang_cl_invocation = clang_cl_invocation.replace("clang.exe","clang-cl.exe")
   config.substitutions.append( ("%clang_cl ", clang_cl_invocation) )
 
   clang_cl_asan_invocation = build_invocation(clang_cl_asan_cxxflags)
   clang_cl_asan_invocation = clang_cl_asan_invocation.replace("clang.exe","clang-cl.exe")
   config.substitutions.append( ("%clang_cl_asan ", clang_cl_asan_invocation) )
 
-  base_lib = os.path.join(config.compiler_rt_libdir, "clang_rt.asan%%s-%s.lib" % config.target_arch)
+  base_lib = os.path.join(config.compiler_rt_libdir, "clang_rt.asan%%s%s.lib" % config.target_suffix)
   config.substitutions.append( ("%asan_lib", base_lib % "") )
   config.substitutions.append( ("%asan_cxx_lib", base_lib % "_cxx") )
   config.substitutions.append( ("%asan_dll_thunk", base_lib % "_dll_thunk") )
 
 if platform.system() == 'Windows':
   # Don't use -std=c++11 on Windows, as the driver will detect the appropriate
   # default needed to use with the STL.
   config.substitutions.append(("%stdcxx11 ", ""))

test/builtins/Unit/lit.cfg

 default_builtins_opts = ''
 
 # Setup source root.
 config.test_source_root = os.path.dirname(__file__)
 
 # Path to the static library
 is_msvc = get_required_attr(config, "builtins_is_msvc")
 if is_msvc:
-  base_lib = os.path.join(config.compiler_rt_libdir, "clang_rt.builtins-%s.lib "
-                          % config.target_arch)
+  base_lib = os.path.join(config.compiler_rt_libdir, "clang_rt.builtins%s.lib "
+                          % config.target_suffix)
   config.substitutions.append( ("%librt ", base_lib) )
 else:
-  base_lib = os.path.join(config.compiler_rt_libdir, "libclang_rt.builtins-%s.a"
-                          % config.target_arch)
+  base_lib = os.path.join(config.compiler_rt_libdir, "libclang_rt.builtins%s.a"
+                          % config.target_suffix)
   config.substitutions.append( ("%librt ", base_lib + ' -lc -lm ') )
 
 builtins_source_dir = os.path.join(
   get_required_attr(config, "compiler_rt_src_root"), "lib", "builtins")
 builtins_lit_source_dir = get_required_attr(config, "builtins_lit_source_dir")
 
 extra_link_flags = ["-nodefaultlibs"]
 

test/fuzzer/lit.cfg

 config.substitutions.append(('%build_dir', config.cmake_binary_dir))
 libfuzzer_src_root = os.path.join(config.compiler_rt_src_root, "lib", "fuzzer")
 config.substitutions.append(('%libfuzzer_src', libfuzzer_src_root))
 
 def generate_compiler_cmd(is_cpp=True, fuzzer_enabled=True):
   compiler_cmd = config.clang
   extra_cmd = config.target_flags
   if config.clang and config.stdlib == 'libc++':
-    link_cmd = '-stdlib=libc++ -Wl,-rpath=%s' % config.llvm_library_dir
+    link_cmd = '-stdlib=libc++ -Wl,-rpath=%s' % config.runtime_library_dir
   elif config.clang and config.stdlib == 'static-libc++':
     link_cmd = '-stdlib=libc++ -lc++abi -static-libstdc++ -Wl,-rpath=%s' % (
-        config.llvm_library_dir)
+        config.runtime_library_dir)
   elif any(x in config.target_triple for x in ('darwin', 'freebsd')):
     link_cmd = '-lc++'
   else:
     link_cmd = '-lstdc++'
 
   std_cmd = '--driver-mode=g++ -std=c++11' if is_cpp else ''
   sanitizers = ['address']
   if fuzzer_enabled:

test/fuzzer/lit.site.cfg.in

 config.cmake_binary_dir = "@CMAKE_BINARY_DIR@"
 config.llvm_library_dir = "@LLVM_LIBRARY_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 
 # Load common config for all compiler-rt lit tests.
 lit_config.load_config(config,
     "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
 
+if config.enable_per_target_runtime_dir:
+  config.runtime_library_dir = config.compiler_rt_libdir
+else:
+  config.runtime_library_dir = "@LLVM_LIBRARY_DIR@"
+
 lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg")

test/lit.common.configured.in

 set_default("host_arch", "@HOST_ARCH@")
 set_default("target_arch", "@COMPILER_RT_DEFAULT_TARGET_ARCH@")
 set_default("host_os", "@HOST_OS@")
 set_default("llvm_build_mode", "@LLVM_BUILD_MODE@")
 set_default("llvm_src_root", "@LLVM_MAIN_SRC_DIR@")
 set_default("llvm_obj_root", "@LLVM_BINARY_DIR@")
 set_default("compiler_rt_src_root", "@COMPILER_RT_SOURCE_DIR@")
 set_default("compiler_rt_obj_root", "@COMPILER_RT_BINARY_DIR@")
+set_default("enable_per_target_runtime_dir", @LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_PYBOOL@)
 set_default("llvm_tools_dir", "@LLVM_TOOLS_DIR@")
 set_default("llvm_shlib_dir", "@LLVM_LIBRARY_OUTPUT_INTDIR@")
 set_default("gold_executable", "@GOLD_EXECUTABLE@")
 set_default("clang", "@COMPILER_RT_RESOLVED_TEST_COMPILER@")
 set_default("compiler_id", "@COMPILER_RT_TEST_COMPILER_ID@")
 set_default("python_executable", "@PYTHON_EXECUTABLE@")
 set_default("compiler_rt_debug", @COMPILER_RT_DEBUG_PYBOOL@)
 set_default("compiler_rt_libdir", "@COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR@")
 set_default("emulator", "@COMPILER_RT_EMULATOR@")
 set_default("asan_shadow_scale", "@COMPILER_RT_ASAN_SHADOW_SCALE@")
 set_default("apple_platform", "osx")
 set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
 set_default("has_lld", @COMPILER_RT_HAS_LLD_PYBOOL@)
 set_default("can_symbolize", @CAN_SYMBOLIZE@)
 set_default("use_lld", False)
 set_default("use_thinlto", False)
 set_default("use_lto", config.use_thinlto)
 set_default("android", @ANDROID_PYBOOL@)
 config.available_features.add('target-is-%s' % config.target_arch)
 
+if config.enable_per_target_runtime_dir:
+  set_default("target_suffix", "")
+else:
+  set_default("target_suffix", "-%s" % config.target_arch)
+
 # LLVM tools dir can be passed in lit parameters, so try to
 # apply substitution.
 try:
   config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
   config.clang = config.clang % lit_config.params
   config.compiler_rt_libdir = config.compiler_rt_libdir % lit_config.params
 except KeyError as e:
   key, = e.args
   lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key, key))
 
 if not os.path.exists(config.clang):
   lit_config.fatal("Can't find compiler on path %r" % config.clang)
 
 # Setup attributes common for all compiler-rt projects.
 lit_config.load_config(config, "@COMPILER_RT_SOURCE_DIR@/test/lit.common.cfg")

test/scudo/lit.cfg

 # -*- Python -*-
 
 import os
 
 # Setup config name.
 config.name = 'Scudo' + config.name_suffix
 
 # Setup source root.
 config.test_source_root = os.path.dirname(__file__)
 
 # Path to the shared library
-shared_libscudo = os.path.join(config.compiler_rt_libdir, "libclang_rt.scudo-%s.so" % config.target_arch)
+shared_libscudo = os.path.join(config.compiler_rt_libdir, "libclang_rt.scudo%s.so" % config.target_suffix)
 
 # Test suffixes.
 config.suffixes = ['.c', '.cc', '.cpp']
 
 # C & CXX flags.
 c_flags = ([config.target_cflags] +
            ["-pthread",
            "-fPIE",

test/xray/lit.cfg

 clang_xray_cxxflags = config.cxx_mode_flags + clang_xray_cflags
 
 
 def build_invocation(compile_flags):
   return ' ' + ' '.join([config.clang] + compile_flags) + ' '
 
 # Assume that llvm-xray is in the config.llvm_tools_dir.
 llvm_xray = os.path.join(config.llvm_tools_dir, 'llvm-xray')
-host_arch = config.host_arch
-if host_arch == 'amd64':
-  host_arch = 'x86_64'
 
 # Setup substitutions.
 if config.host_os == "Linux":
   libdl_flag = "-ldl"
 else:
   libdl_flag = ""
 
 config.substitutions.append(
     ('%clang ', build_invocation([config.target_cflags])))
 config.substitutions.append(
     ('%clangxx ',
      build_invocation(config.cxx_mode_flags + [config.target_cflags])))
 config.substitutions.append(
     ('%clang_xray ', build_invocation(clang_xray_cflags)))
 config.substitutions.append(
     ('%clangxx_xray', build_invocation(clang_xray_cxxflags)))
 config.substitutions.append(
     ('%llvm_xray', llvm_xray))
 config.substitutions.append(
     ('%xraylib',
         ('-lm -lpthread %s -lrt -L%s '
-         '-Wl,-whole-archive -lclang_rt.xray-%s -Wl,-no-whole-archive')
-        % (libdl_flag, config.compiler_rt_libdir, host_arch)))
+         '-Wl,-whole-archive -lclang_rt.xray%s -Wl,-no-whole-archive')
+        % (libdl_flag, config.compiler_rt_libdir, config.target_suffix)))
 
 # Default test suffixes.
 config.suffixes = ['.c', '.cc', '.cpp']
 
 if config.host_os not in ['FreeBSD', 'Linux', 'NetBSD']:
   config.unsupported = True
 elif '64' not in config.host_arch:
   if 'arm' in config.host_arch:
     if '-mthumb' in config.target_cflags:
       config.unsupported = True
   else:
     config.unsupported = True