[CMake] Detect the compiler runtime and standard library

[phosek]

Summary

Rather then requiring the user to specify runtime the compiler
runtime and C++ standard library, or trying to guess them which is
error-prone, use auto-detection by parsing the compiler link output.

[dim]

Do we still leave the defaults for particular OSes here? I'm asking because for example in FreeBSD, we use libcxxrt, but you *won't* find it in linker output, since our libc++.so is a linker script:

$ clang++ /usr/local/share/cmake/Modules/DummyCXXFile.cxx -###
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
Target: x86_64-unknown-freebsd11.2
Thread model: posix
InstalledDir: /usr/bin
 "/usr/bin/clang++" "-cc1" "-triple" "x86_64-unknown-freebsd11.2" "-emit-obj" "-mrelax-all" "-disable-free" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "DummyCXXFile.cxx" "-mrelocation-model" "static" "-mthread-model" "posix" "-mdisable-fp-elim" "-masm-verbose" "-mconstructor-aliases" "-munwind-tables" "-target-cpu" "x86-64" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/usr/lib/clang/6.0.0" "-internal-isystem" "/usr/include/c++/v1" "-fdeprecated-macro" "-fdebug-compilation-dir" "/home/dim/src/llvm/trunk" "-ferror-limit" "19" "-fmessage-length" "160" "-fobjc-runtime=gnustep" "-fcxx-exceptions" "-fexceptions" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-o" "/tmp/DummyCXXFile-5ffab3.o" "-x" "c++" "/usr/local/share/cmake/Modules/DummyCXXFile.cxx"
 "/usr/bin/ld" "--eh-frame-hdr" "-dynamic-linker" "/libexec/ld-elf.so.1" "--hash-style=both" "--enable-new-dtags" "-o" "a.out" "/usr/lib/crt1.o" "/usr/lib/crti.o" "/usr/lib/crtbegin.o" "-L/usr/lib" "/tmp/DummyCXXFile-5ffab3.o" "-lc++" "-lm" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lc" "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "/usr/lib/crtend.o" "/usr/lib/crtn.o"

E.g., you see -lc++, and our /usr/lib/libc++.so contains:

$ cat /usr/lib/libc++.so
/* $FreeBSD: stable/11/lib/libc++/libc++.ldscript 253917 2013-08-03 16:23:43Z dim $ */
GROUP ( /usr/lib/libc++.so.1 /usr/lib/libcxxrt.so )

So what would check_link_libraries find in this case?

[phosek]

check_link_libraries would find c++ and set -lc++ as SANITIZER_CXX_ABI_LIBRARY which is presumably what it does today since I don't see libcxxrt in the CXXABIS list, so there should be no functional difference. The autodetection logic is only used when SANITIZER_CXX_ABI is set to "default", so you can still override it through CMake same as today.

[phosek]

One of the positive side-effects of this change is that it also avoids passing unnecessary --rtlib= and --stdlib= flags which eliminates spurious warnings in Fuchsia build that forced us to disable -Werror, this would affect any platform that uses non-default --rtlib= or --stdlib= values.

[vitalybuka]

compiler-rt/CMakeLists.txt

157 ↗

(On Diff #146721)

What does happen with the unwind?

compiler-rt/cmake/Modules/CompilerRTUtils.cmake

155 ↗

(On Diff #146721)

Should this message be in some if() statement?

329 ↗

(On Diff #146721)

why not to use try_compile?

354 ↗

(On Diff #146721)

Should this variable names be upper case?

[phosek]

compiler-rt/CMakeLists.txt

157 ↗

(On Diff #146721)

That's a great question, this is relying on libc++ being a linker script that pulls in all the additional dependencies such as libunwind which is how libc++ is normally installed on platforms that support linker scripts, except for macOS where they use different mechanism but this should still work there. It'd only break on platforms where libunwind has to be linked explicitly, but those are probably broken in current setup as well and I don't know how to easily handle those.

compiler-rt/cmake/Modules/CompilerRTUtils.cmake

155 ↗

(On Diff #146721)

This is a leftover debugging print, removed.

329 ↗

(On Diff #146721)

In this case I'm only using --print-libgcc-file-name to find out what the runtime is so we don't need to compile anything. I could possibly completely drop this and just use check_link_libraries below and then see if output contains either libclang_rt.builtins or libgcc, the downside is that I'd need to iterate over all libraries in that list and try to MATCH each one of them against the regex which might be slower than actually invoking the compiler again. WDYT?

[phosek]

Ping?

[beanz]

This all looks reasonable to me. I had a few small comments below. There are a few isolated bits of code cleanup that you included in this patch, and I'd prefer to see those land separately.

compiler-rt/cmake/Modules/CompilerRTUtils.cmake

329 ↗

(On Diff #146721)

--print-libgcc-file-name is broken on Darwin (it returns an incorrect result). I suspect that this is actually alright because you're not actually using the file, but we should probably look into fixing that.

compiler-rt/cmake/Modules/HandleCompilerRT.cmake

1 ↗

(On Diff #146977)

This just looks like some generally good code cleanup. Can you put it in a separate patch? No review necessary.

compiler-rt/cmake/config-ix.cmake

129 ↗

(On Diff #146977)

This is also just a generally good fix. Can you also commit this separately?

[vitalybuka]

please move unrelated change into separate patches

[phosek]

compiler-rt/cmake/config-ix.cmake

129 ↗

(On Diff #146977)

https://reviews.llvm.org/D47031

[rnk]

This is causing warnings on Windows:

-- Targeting WebAssembly
clang-cl.exe: warning: unknown argument ignored in clang-cl: '--print-libgcc-file-name' [-Wunknown-argument]
clang-cl.exe: error: no input files
-- Compiler-RT supported architectures: x86_64

The fact that you are blindly passing --print-libgcc-file-name suggests that this code doesn't work with MSVC or any other compiler that has a non-GCC command line interface. Things might work, but I'd prefer it if it was relanded after proactively considering those cases, rather than patching a new pile of conditionals.

Looks like MSVC also warns:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/28806/steps/cmake/logs/stdio

-- Targeting XCore
DummyCXXFile.cxx
Microsoft (R) Incremental Linker Version 14.00.24215.1
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:DummyCXXFile.exe 
DummyCXXFile.obj 
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '--print-libgcc-file-name'
cl : Command line error D8003 : missing source filename

CMakeLists.txt

 endif()
 
 option(COMPILER_RT_DEBUG "Build runtimes with full debug info" OFF)
 option(COMPILER_RT_EXTERNALIZE_DEBUGINFO
   "Generate dSYM files and strip executables and libraries (Darwin Only)" OFF)
 # COMPILER_RT_DEBUG_PYBOOL is used by lit.common.configured.in.
 pythonize_bool(COMPILER_RT_DEBUG)
 
-include(HandleCompilerRT)
-include(config-ix)
-
 if(APPLE AND SANITIZER_MIN_OSX_VERSION AND SANITIZER_MIN_OSX_VERSION VERSION_LESS "10.9")
   # Mac OS X prior to 10.9 had problems with exporting symbols from
   # libc++/libc++abi.
   set(cxxabi_supported OFF)
 else()
   set(cxxabi_supported ON)
 endif()
 
-option(SANITIZER_ALLOW_CXXABI "Allow use of C++ ABI details in ubsan" ON)
+option(SANITIZER_ALLOW_CXXABI "Allow use of C++ ABI details in sanitizers" ON)
 
 set(SANITIZE_CAN_USE_CXXABI OFF)
 if (cxxabi_supported AND SANITIZER_ALLOW_CXXABI)
   set(SANITIZER_CAN_USE_CXXABI ON)
 endif()
 pythonize_bool(SANITIZER_CAN_USE_CXXABI)
 
+check_link_libraries(cxx_link_libraries)
+
 set(SANITIZER_CXX_ABI "default" CACHE STRING
     "Specify C++ ABI library to use.")
-set(CXXABIS none default libcxxabi libstdc++ libc++)
+set(CXXABIS none default libc++ libstdc++)
 set_property(CACHE SANITIZER_CXX_ABI PROPERTY STRINGS ;${CXXABIS})
 
 if (SANITIZER_CXX_ABI STREQUAL "default")
-  if (HAVE_LIBCXXABI AND COMPILER_RT_DEFAULT_TARGET_ONLY)
-    set(SANITIZER_CXX_ABI_LIBNAME "libcxxabi")
-    set(SANITIZER_CXX_ABI_INTREE 1)
-  elseif (APPLE)
-    set(SANITIZER_CXX_ABI_LIBNAME "libcxxabi")
+  if(APPLE)
+    set(SANITIZER_CXX_ABI_LIBNAME "libc++")
     set(SANITIZER_CXX_ABI_SYSTEM 1)
-  else()
+  elseif("stdc++" IN_LIST cxx_link_libraries)
     set(SANITIZER_CXX_ABI_LIBNAME "libstdc++")
     set(SANITIZER_CXX_ABI_SYSTEM 1)
+  elseif("c++" IN_LIST cxx_link_libraries)
+    set(SANITIZER_CXX_ABI_LIBNAME "libc++")
   endif()
 else()
   set(SANITIZER_CXX_ABI_LIBNAME "${SANITIZER_CXX_ABI}")
 endif()
 
-if (SANITIZER_CXX_ABI_LIBNAME STREQUAL "libcxxabi")
-  if (SANITIZER_CXX_ABI_INTREE)
-    if (NOT LIBCXXABI_ENABLE_STATIC_UNWINDER AND (TARGET unwind_shared OR HAVE_LIBUNWIND))
-      list(APPEND SANITIZER_CXX_ABI_LIBRARY unwind_shared)
-    elseif (LIBCXXABI_ENABLE_STATIC_UNWINDER AND (TARGET unwind_static OR HAVE_LIBUNWIND))
-      list(APPEND SANITIZER_CXX_ABI_LIBRARY unwind_static)
-    endif()
-    if (NOT LIBCXX_ENABLE_STATIC_ABI_LIBRARY AND (TARGET cxxabi_shared OR HAVE_LIBCXXABI))
+if (SANITIZER_CXX_ABI_LIBNAME STREQUAL "libc++")
+  if(SANITIZER_CXX_ABI_SYSTEM)
+    list(APPEND SANITIZER_CXX_ABI_LIBRARY "c++")
+  else()
+    if(TARGET cxxabi_shared OR (HAVE_LIBCXXABI AND NOT LIBCXX_ENABLE_STATIC_ABI_LIBRARY))
       list(APPEND SANITIZER_CXX_ABI_LIBRARY cxxabi_shared)
-    elseif (LIBCXX_ENABLE_STATIC_ABI_LIBRARY AND (TARGET cxxabi_static OR HAVE_LIBCXXABI))
+    elseif(TARGET cxxabi_shared OR (HAVE_LIBCXXABI AND LIBCXX_ENABLE_STATIC_ABI_LIBRARY))
       list(APPEND SANITIZER_CXX_ABI_LIBRARY cxxabi_static)
-    endif()
     else()
-    list(APPEND SANITIZER_CXX_ABI_LIBRARY "c++abi")
-  endif()
-elseif (SANITIZER_CXX_ABI_LIBNAME STREQUAL "libc++")
       list(APPEND SANITIZER_CXX_ABI_LIBRARY "c++")
+    endif()
+  endif()
 elseif (SANITIZER_CXX_ABI_LIBNAME STREQUAL "libstdc++")
-  append_list_if(COMPILER_RT_HAS_LIBSTDCXX stdc++ SANITIZER_CXX_ABI_LIBRARY)
+  list(APPEND SANITIZER_CXX_ABI_LIBRARY "stdc++")
 endif()
 
-option(SANITIZER_USE_COMPILER_RT "Use compiler-rt builtins instead of libgcc" OFF)
+check_runtime_library(runtime_library)
+if(runtime_library MATCHES ".*libgcc.*")
+  set(COMPILER_RT_RUNTIME_LIBRARY "libgcc")
+elseif(runtime_library MATCHES ".*libclang_rt.builtins.*")
+  set(COMPILER_RT_RUNTIME_LIBRARY "compiler-rt")
+endif()
+
+include(config-ix)
 
 #================================
 # Setup Compiler Flags
 #================================
 
 if(MSVC)
   # Override any existing /W flags with /W4. This is what LLVM does.  Failing to
   # remove other /W[0-4] flags will result in a warning about overriding a

 append_list_if(COMPILER_RT_HAS_WD4291_FLAG /wd4291 SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_WD4391_FLAG /wd4391 SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_WD4722_FLAG /wd4722 SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_WD4800_FLAG /wd4800 SANITIZER_COMMON_CFLAGS)
 
 # Set common link flags.
 append_list_if(COMPILER_RT_HAS_NODEFAULTLIBS_FLAG -nodefaultlibs SANITIZER_COMMON_LINK_FLAGS)
 
-if (SANITIZER_USE_COMPILER_RT)
-  list(APPEND SANITIZER_COMMON_LINK_FLAGS -rtlib=compiler-rt)
-  find_compiler_rt_library(builtins COMPILER_RT_BUILTINS_LIBRARY)
+if (COMPILER_RT_RUNTIME_LIBRARY STREQUAL "compiler-rt")
   list(APPEND SANITIZER_COMMON_LINK_LIBS ${COMPILER_RT_BUILTINS_LIBRARY})
 else()
   if (ANDROID)
     append_list_if(COMPILER_RT_HAS_GCC_LIB gcc SANITIZER_COMMON_LINK_LIBS)
   else()
     append_list_if(COMPILER_RT_HAS_GCC_S_LIB gcc_s SANITIZER_COMMON_LINK_LIBS)
   endif()
 endif()

cmake/Modules/CompilerRTUtils.cmake

foreach (_file ${intermediate})
     elseif(${_file} MATCHES ".*/.*\\.S" OR ${_file} MATCHES ".*/.*\\.c")
       get_filename_component(_name ${_file} NAME)
       string(REPLACE ".S" ".c" _cname "${_name}")
       list(REMOVE_ITEM intermediate ${_cname})
     endif ()
   endforeach ()
   set(${output_var} ${intermediate} PARENT_SCOPE)
 endfunction()
+
+function(check_runtime_library variable)
+  set(COMPILER_COMMAND ${CMAKE_CXX_COMPILER} ${SANITIZER_COMMON_CFLAGS}
+      "--print-libgcc-file-name")
+  if (CMAKE_CXX_COMPILER_ID MATCHES Clang AND CMAKE_CXX_COMPILER_TARGET)
+    list(APPEND COMPILER_COMMAND "--target=${CMAKE_CXX_COMPILER_TARGET}")
+  endif()
+  execute_process(
+      COMMAND ${COMPILER_COMMAND}
+      RESULT_VARIABLE COMMAND_RESULT
+      OUTPUT_VARIABLE LIBRARY_FILE
+  )
+  string(STRIP "${LIBRARY_FILE}" LIBRARY_FILE)
+  if(COMMAND_RESULT EQUAL 0)
+    set(${variable} "${LIBRARY_FILE}" PARENT_SCOPE)
+  endif()
+endfunction()
+
+function(check_link_libraries variable)
+  set(COMPILER_COMMAND "${CMAKE_CXX_COMPILER}" "${SANITIZER_COMMON_CFLAGS}"
+    "${CMAKE_ROOT}/Modules/DummyCXXFile.cxx" "-###")
+  if (CMAKE_CXX_COMPILER_ID MATCHES Clang AND CMAKE_CXX_COMPILER_TARGET)
+    list(APPEND COMPILER_COMMAND "--target=${CMAKE_CXX_COMPILER_TARGET}")
+  endif()
+  execute_process(
+      COMMAND ${COMPILER_COMMAND}
+      RESULT_VARIABLE COMMAND_RESULT
+      ERROR_VARIABLE COMMAND_OUTPUT
+  )
+  if(COMMAND_RESULT EQUAL 0)
+    get_filename_component(linker ${CMAKE_LINKER} NAME)
+    string(REGEX REPLACE "([][+.*?()^$])" "\\\\\\1" linker "${linker}")
+    set(LINKER_REGEX "^( *|.*[/\\])(${linker}|([^/\\]+-)?ld|collect2)[^/\\]*( |$)")
+    set(EXCLUDE_REGEX "collect2 version |^[A-Za-z0-9_]+=")
+    string(REGEX REPLACE "\r?\n" ";" output_lines "${COMMAND_OUTPUT}")
+    foreach(line IN LISTS output_lines)
+      if(line MATCHES "${LINKER_REGEX}" AND NOT line MATCHES "${EXCLUDE_REGEX}")
+        separate_arguments(args NATIVE_COMMAND "${line}")
+        list(GET args 0 cmd)
+      endif()
+      if("${cmd}" MATCHES "${LINKER_REGEX}")
+        foreach(arg IN LISTS args)
+          if("${arg}" MATCHES "^-l([^:].*)$")
+            set(lib "${CMAKE_MATCH_1}")
+            list(APPEND libs ${lib})
+          elseif("${arg}" MATCHES "^(.:)?[/\\].*\\.a$")
+            list(APPEND libs ${arg})
+          endif()
+        endforeach()
+        break()
+      endif()
+    endforeach()
+    set(${variable} "${libs}" PARENT_SCOPE)
+  endif()
+endfunction()

cmake/config-ix.cmake

 include(CMakePushCheckState)
 include(CheckCCompilerFlag)
 include(CheckCXXCompilerFlag)
 include(CheckLibraryExists)
 include(CheckSymbolExists)
 include(TestBigEndian)
 
 function(check_linker_flag flag out_var)
   cmake_push_check_state()
   set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} ${flag}")
   check_cxx_compiler_flag("" ${out_var})
   cmake_pop_check_state()
 endfunction()
 
 check_library_exists(c fopen "" COMPILER_RT_HAS_LIBC)
-if (NOT SANITIZER_USE_COMPILER_RT)
+if (COMPILER_RT_RUNTIME_LIBRARY STREQUAL "compiler-rt")
+  include(HandleCompilerRT)
+  find_compiler_rt_library(builtins COMPILER_RT_BUILTINS_LIBRARY)
+else()
   if (ANDROID)
     check_library_exists(gcc __gcc_personality_v0 "" COMPILER_RT_HAS_GCC_LIB)
   else()
     check_library_exists(gcc_s __gcc_personality_v0 "" COMPILER_RT_HAS_GCC_S_LIB)
   endif()
 endif()
 
 check_c_compiler_flag(-nodefaultlibs COMPILER_RT_HAS_NODEFAULTLIBS_FLAG)
 if (COMPILER_RT_HAS_NODEFAULTLIBS_FLAG)
   set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -nodefaultlibs")
   if (COMPILER_RT_HAS_LIBC)
     list(APPEND CMAKE_REQUIRED_LIBRARIES c)
   endif ()
-  if (SANITIZER_USE_COMPILER_RT)
-    list(APPEND CMAKE_REQUIRED_FLAGS -rtlib=compiler-rt)
-    find_compiler_rt_library(builtins COMPILER_RT_BUILTINS_LIBRARY)
+  if (COMPILER_RT_RUNTIME_LIBRARY STREQUAL "compiler-rt")
     list(APPEND CMAKE_REQUIRED_LIBRARIES "${COMPILER_RT_BUILTINS_LIBRARY}")
   elseif (COMPILER_RT_HAS_GCC_S_LIB)
     list(APPEND CMAKE_REQUIRED_LIBRARIES gcc_s)
   elseif (COMPILER_RT_HAS_GCC_LIB)
     list(APPEND CMAKE_REQUIRED_LIBRARIES gcc)
   endif ()
 endif ()