[CMake] Support runtimes targets without specifying triple

[phosek]

Summary

Currently, for BUILTIN_TARGETS and RUNTIME_TARGETS you can either use
the special "default" value, or a target triple.

For the "default" value, we don't set any target triple and passthrough
a subset of CMake variables into the subbuild. This is typically used
on Darwin where we build universal binaries and a single target triple
therefore isn't sufficient.

For the target triple value, you can set arbitrary CMake variables
through RUNTIMES_<target>_<variable>, but we always set target triple
to <target>. This gives more flexibility, because we can precisely
control what variables are set in the subbuild, but is unsuitable for
platforms like Darwin.

To address this, we would like to introduce a third option which is
similar to the second option, except we won't set target triple in
the subbuild (you can always do so yourself by setting the appropriate
CMake variable, e.g. RUNTIMES_<name>_CMAKE_C_COMPILER_TARGET=<triple>).

This change is a first step in that direction, by eliminating the support
of target triples from builtin_register_target and runtime_register_target
helper functions.

[smeenai]

I don't understand the motivation for this change.

The Darwin compiler-rt support works by overriding the OSX_ARCHITECTURES target property (which makes CMake pass -arch flags), and emptying CMAKE_OSX_DEPLOYMENT_TARGET and passing its own -m*-version-min flags. The Clang driver seems to have some special support where if your triple is x86_64-apple-darwin (I assume other architectures would work too, but I haven't checked), it'll override the architecture and the deployment target based on the -arch and -m*-version-min flags, which means the -target is effectively a no-op in this case (besides enabling the overwriting based on those other flags). The result is that you can add x86_64-apple-darwin to your LLVM_BUILTIN_TARGETS and LLVM_RUNTIME_TARGETS, and then control the actual platforms and architectures via the appropriate CMake variables for that target (COMPILER_RT_ENABLE_<PLATFORM>, DARWIN_<PLATFORM>_BUILTIN_ARCHS, etc.), which is what I set up in D86313, and what we use internally. Does that not work for your use case?

[phosek]

In D117263#3242414, @smeenai wrote:

I don't understand the motivation for this change.

The Darwin compiler-rt support works by overriding the OSX_ARCHITECTURES target property (which makes CMake pass -arch flags), and emptying CMAKE_OSX_DEPLOYMENT_TARGET and passing its own -m*-version-min flags. The Clang driver seems to have some special support where if your triple is x86_64-apple-darwin (I assume other architectures would work too, but I haven't checked), it'll override the architecture and the deployment target based on the -arch and -m*-version-min flags, which means the -target is effectively a no-op in this case (besides enabling the overwriting based on those other flags). The result is that you can add x86_64-apple-darwin to your LLVM_BUILTIN_TARGETS and LLVM_RUNTIME_TARGETS, and then control the actual platforms and architectures via the appropriate CMake variables for that target (COMPILER_RT_ENABLE_<PLATFORM>, DARWIN_<PLATFORM>_BUILTIN_ARCHS, etc.), which is what I set up in D86313, and what we use internally. Does that not work for your use case?

I'm mostly concerned about the potential confusion. Specifically, I'd like to start building libunwind, libc++abi, libc++ as a universal library for arm64 and x86_64 in our toolchain and I'd like to use CMAKE_OSX_ARCHITECTURES for that purpose. I could do what you've suggested, it's just that seeing something like:

RUNTIMES_x86_64-apple-darwin_CMAKE_OSX_ARCHITECTURES=arm64;x64

is in my opinion more confusing than something like:

RUNTIMES_darwin_CMAKE_OSX_ARCHITECTURES=arm64;x64

which is what I'm trying to achieve. I'm interested in your experience and thoughts on this, especially if there are other alternatives you think I should consider.

[phosek]

Another point I'd also add is that using CMAKE_{ASM,C,CXX}_COMPILER_TARGET as we do today isn't always sufficient. We ran into this recently on Windows. The problem is that CMake only processes CMAKE_{ASM,C,CXX}_COMPILER_TARGET after it does compiler detection, but in the case of Clang, --target= option changes the compiler behavior which can throw CMake compiler detection off. Specifically on Windows, CMake would detect Clang as "clang-cl in GNU compatibility mode" and to workaround it, we had to start passing --target= manually via CMAKE_{ASM,C,CXX}_FLAGS so the target triple handling runtimes build does right now looses some of its value. I'd more inclined to just deprecate it and require users to set those flags manually, but I'm not sure if that's feasible at this point since it'd be a breaking change.

[smeenai]

Sorry, this fell off my radar! Your reasoning makes sense to me, and in particular, I'd forgotten that only compiler-t has the Darwin multi-arch setup (and you'd still need to handle libc++/libc++abi/libunwind differently).

llvm/runtimes/CMakeLists.txt

256

This part of the multilib setup has always confused me. Is there only ever going to be one extra name (as is the case in this diff)? If so, maybe something like BASE_NAME or a different name would convey the intent better?

[smeenai]

This was breaking our internal builds. rG03610cdeb979 should fix it.

llvm/runtimes/CMakeLists.txt

llvm_ExternalProject_Add(builtins
                            PASSTHROUGH_PREFIXES COMPILER_RT
                                                 DARWIN
                                                 SANITIZER
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${LLVM_TARGET_TRIPLE}
                            ${EXTRA_ARGS})
 endfunction()
 
-function(builtin_register_target compiler_rt_path target)
-  cmake_parse_arguments(ARG "" "" "DEPENDS" ${ARGN})
-
-  check_apple_target(${target} builtin)
+function(builtin_register_target compiler_rt_path name)
+  cmake_parse_arguments(ARG "" "" "DEPENDS;CMAKE_ARGS;EXTRA_ARGS" ${ARGN})
 
-  get_cmake_property(variableNames VARIABLES)
-  foreach(variableName ${variableNames})
-    string(FIND "${variableName}" "BUILTINS_${target}" out)
+  set(${name}_extra_args ${ARG_CMAKE_ARGS})
+  get_cmake_property(variable_names VARIABLES)
+  foreach(variable_name ${variable_names})
+    string(FIND "${variable_name}" "BUILTINS_${name}" out)
     if("${out}" EQUAL 0)
-      string(REPLACE "BUILTINS_${target}_" "" new_name ${variableName})
-      string(REPLACE ";" "|" new_value "${${variableName}}")
-      list(APPEND ${target}_extra_args "-D${new_name}=${new_value}")
+      string(REPLACE "BUILTINS_${name}_" "" new_name ${variable_name})
+      string(REPLACE ";" "|" new_value "${${variable_name}}")
+      list(APPEND ${name}_extra_args "-D${new_name}=${new_value}")
     endif()
   endforeach()
 
-  llvm_ExternalProject_Add(builtins-${target}
+  llvm_ExternalProject_Add(builtins-${name}
                            ${compiler_rt_path}/lib/builtins
                            DEPENDS ${ARG_DEPENDS}
                            CMAKE_ARGS -DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_DIR}
                                       -DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_TOOLS_BINARY_DIR}
-                                      -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
                                       ${COMMON_CMAKE_ARGS}
-                                      ${${target}_extra_args}
+                                      ${${name}_extra_args}
                            USE_TOOLCHAIN
-                           TARGET_TRIPLE ${target}
-                           ${EXTRA_ARGS})
+                           ${EXTRA_ARGS} ${ARG_EXTRA_ARGS})
 endfunction()
 
 # If compiler-rt is present we need to build the builtin libraries first. This
 # is required because the other runtimes need the builtin libraries present
 # before the just-built compiler can pass the configuration tests.
 get_compiler_rt_path(compiler_rt_path)
 if(compiler_rt_path)
   if(NOT LLVM_BUILTIN_TARGETS)
     builtin_default_target(${compiler_rt_path}
       DEPENDS clang-resource-headers)
   else()
     if("default" IN_LIST LLVM_BUILTIN_TARGETS)
       builtin_default_target(${compiler_rt_path}
         DEPENDS clang-resource-headers)
       list(REMOVE_ITEM LLVM_BUILTIN_TARGETS "default")
     else()
       add_custom_target(builtins)
       add_custom_target(install-builtins)
       add_custom_target(install-builtins-stripped)
     endif()
 
     foreach(target ${LLVM_BUILTIN_TARGETS})
+      check_apple_target(${target} builtin)
+
       builtin_register_target(${compiler_rt_path} ${target}
-        DEPENDS clang-resource-headers)
+        DEPENDS clang-resource-headers
+        CMAKE_ARGS -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
+        EXTRA_ARGS TARGET_TRIPLE ${target})
 
       add_dependencies(builtins builtins-${target})
       add_dependencies(install-builtins install-builtins-${target})
       add_dependencies(install-builtins-stripped install-builtins-${target}-stripped)
     endforeach()
   endif()
   set(builtins_dep builtins)
   # We don't need to depend on the builtins if we're building instrumented
llvm_ExternalProject_Add(runtimes
                                          ${SUB_COMPONENTS}
                                          ${SUB_CHECK_TARGETS}
                                          ${SUB_INSTALL_TARGETS}
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${LLVM_TARGET_TRIPLE}
                            ${EXTRA_ARGS})
 endfunction()
 
-# runtime_register_target(target)
+# runtime_register_target(name)
 #   Utility function to register external runtime target.
-function(runtime_register_target name target)
-  cmake_parse_arguments(ARG "" "" "DEPENDS;CMAKE_ARGS" ${ARGN})
+function(runtime_register_target name)
+  cmake_parse_arguments(ARG "" "" "DEPENDS;CMAKE_ARGS;BASE_NAME;EXTRA_ARGS" ${ARGN})
   include(${LLVM_BINARY_DIR}/runtimes/${name}/Components.cmake OPTIONAL)
   set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${LLVM_BINARY_DIR}/runtimes/${name}/Components.cmake)
 
-  check_apple_target(${target} runtime)
-
-  set(${name}_deps ${ARG_DEPENDS})
-  if(NOT name STREQUAL target)
-    list(APPEND ${name}_deps runtimes-${target})
-  endif()
-
   foreach(runtime_name ${runtime_names})
     set(${runtime_name}-${name} ${runtime_name})
     set(install-${runtime_name}-${name} install-${runtime_name})
     set(install-${runtime_name}-${name}-stripped install-${runtime_name}-stripped)
     list(APPEND ${name}_extra_targets ${runtime_name}-${name} install-${runtime_name}-${name} install-${runtime_name}-${name}-stripped)
     if(LLVM_INCLUDE_TESTS)
       set(check-${runtime_name}-${name} check-${runtime_name} )
       list(APPEND ${name}_test_targets check-${runtime_name}-${name})
     endif()
   endforeach()
 
-  foreach(target_name IN LISTS SUB_COMPONENTS)
-    set(${target_name}-${name} ${target_name})
-    list(APPEND ${name}_extra_targets ${target_name}-${name})
+  foreach(component IN LISTS SUB_COMPONENTS)
+    set(${component}-${name} ${component})
+    list(APPEND ${name}_extra_targets ${component}-${name})
   endforeach()
 
-  foreach(target_name IN LISTS SUB_INSTALL_TARGETS)
-    set(${target_name}-${name} ${target_name})
-    set(${target_name}-${name}-stripped ${target_name}-stripped)
-    list(APPEND ${name}_extra_targets ${target_name}-${name} ${target_name}-${name}-stripped)
+  foreach(target IN LISTS SUB_INSTALL_TARGETS)
+    set(${target}-${name} ${target})
+    set(${target}-${name}-stripped ${target}-stripped)
+    list(APPEND ${name}_extra_targets ${target}-${name} ${target}-${name}-stripped)
   endforeach()
 
   foreach(component ${LLVM_RUNTIME_DISTRIBUTION_COMPONENTS})
     if(NOT component IN_LIST SUB_COMPONENTS)
       set(${component}-${name} ${component})
       set(install-${component}-${name} install-${component})
       set(install-${component}-${name}-stripped install-${component}-stripped)
       list(APPEND ${name}_extra_targets ${component}-${name} install-${component}-${name} install-${component}-${name}-stripped)
if(LLVM_INCLUDE_TESTS)
 
     set(component_check_targets)
     foreach(component IN LISTS LLVM_RUNTIME_DISTRIBUTION_COMPONENTS)
       if(NOT "check-${component}" IN_LIST SUB_CHECK_TARGETS)
         list(APPEND component_check_targets "check-${component}")
       endif()
     endforeach()
 
-    foreach(target_name IN LISTS SUB_CHECK_TARGETS component_check_targets)
-      set(${target_name}-${name} ${target_name})
-      list(APPEND ${name}_test_targets ${target_name}-${name})
-      list(APPEND test_targets ${target_name}-${name})
+    foreach(target IN LISTS SUB_CHECK_TARGETS component_check_targets)
+      set(${target}-${name} ${target})
+      list(APPEND ${name}_test_targets ${target}-${name})
+      list(APPEND test_targets ${target}-${name})
     endforeach()
     set(test_targets "${test_targets}" PARENT_SCOPE)
   endif()
 
   set(${name}_extra_args ${ARG_CMAKE_ARGS})
-  get_cmake_property(variableNames VARIABLES)
-  foreach(variableName ${variableNames})
-    string(FIND "${variableName}" "RUNTIMES_${target}_" out)
+  get_cmake_property(variable_names VARIABLES)
+  foreach(extra_name IN ITEMS ${name} ${ARG_BASE_NAME})
+    foreach(variable_name ${variable_names})
+      string(FIND "${variable_name}" "RUNTIMES_${extra_name}_" out)
       if("${out}" EQUAL 0)
-      string(REPLACE "RUNTIMES_${target}_" "" new_name ${variableName})
-      string(REPLACE ";" "|" new_value "${${variableName}}")
+        string(REPLACE "RUNTIMES_${extra_name}_" "" new_name ${variable_name})
+        string(REPLACE ";" "|" new_value "${${variable_name}}")
         list(APPEND ${name}_extra_args "-D${new_name}=${new_value}")
       endif()
     endforeach()
-  if(NOT "${name}" STREQUAL "${target}")
-    foreach(variableName ${variableNames})
-      string(FIND "${variableName}" "RUNTIMES_${name}_" out)
-      if("${out}" EQUAL 0)
-        string(REPLACE "RUNTIMES_${name}_" "" new_name ${variableName})
-        string(REPLACE ";" "|" new_value "${${variableName}}")
-        list(APPEND ${name}_extra_args "-D${new_name}=${new_value}")
-      endif()
-    endforeach()
-  endif()
 
-  if(NOT RUNTIMES_${name}_LLVM_ENABLE_RUNTIMES AND NOT RUNTIMES_${target}_LLVM_ENABLE_RUNTIMES)
+  if(NOT RUNTIMES_${name}_LLVM_ENABLE_RUNTIMES)
     string(REPLACE ";" "|" LLVM_ENABLE_RUNTIMES_PASSTHROUGH "${LLVM_ENABLE_RUNTIMES}")
     list(APPEND ${name}_extra_args -DLLVM_ENABLE_RUNTIMES=${LLVM_ENABLE_RUNTIMES_PASSTHROUGH})
   endif()
 
   if(NOT RUNTIMES_${name}_LLVM_USE_LINKER AND NOT RUNTIMES_${target}_LLVM_USE_LINKER)
     list(APPEND ${name}_extra_args -DLLVM_USE_LINKER=${LLVM_USE_LINKER})
   endif()
 
   set_enable_per_target_runtime_dir()
 
   llvm_ExternalProject_Add(runtimes-${name}
                            ${CMAKE_CURRENT_SOURCE_DIR}/../../runtimes
-                           DEPENDS ${${name}_deps}
+                           DEPENDS ${ARG_DEPENDS}
                            # Builtins were built separately above
-                           CMAKE_ARGS -DCOMPILER_RT_BUILD_BUILTINS=Off
+                           CMAKE_ARGS -DCOMPILER_RT_BUILD_BUILTINS=OFF
                                       -DLLVM_INCLUDE_TESTS=${LLVM_INCLUDE_TESTS}
-                                      -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
                                       -DLLVM_ENABLE_PROJECTS_USED=${LLVM_ENABLE_PROJECTS_USED}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR}
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_CXX_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
                                       -DLLVM_RUNTIMES_TARGET=${name}
                                       ${COMMON_CMAKE_ARGS}
                                       ${${name}_extra_args}
                            EXTRA_TARGETS ${${name}_extra_targets}
                                          ${${name}_test_targets}
                            USE_TOOLCHAIN
-                           TARGET_TRIPLE ${target}
-                           ${EXTRA_ARGS})
+                           ${EXTRA_ARGS} ${ARG_EXTRA_ARGS})
 endfunction()
 
 if(runtimes)
   # Create a runtimes target that uses this file as its top-level CMake file.
   # The runtimes target is a configuration of all the runtime libraries
   # together in a single CMake invocation.
   set(extra_deps "")
   if("openmp" IN_LIST LLVM_ENABLE_RUNTIMES)
else()
     foreach(name ${LLVM_RUNTIME_TARGETS})
       if(builtins_dep)
         if (LLVM_BUILTIN_TARGETS)
           set(builtins_dep_name "${builtins_dep}-${name}")
         else()
           set(builtins_dep_name ${builtins_dep})
         endif()
       endif()
-      runtime_register_target(${name} ${name}
-        DEPENDS ${builtins_dep_name})
+
+      check_apple_target(${name} runtime)
+
+      runtime_register_target(${name}
+        DEPENDS ${builtins_dep_name}
+        CMAKE_ARGS -DLLVM_DEFAULT_TARGET_TRIPLE=${name}
+        EXTRA_ARGS TARGET_TRIPLE ${name})
 
       add_dependencies(runtimes runtimes-${name})
       add_dependencies(runtimes-configure runtimes-${name}-configure)
       add_dependencies(install-runtimes install-runtimes-${name})
       add_dependencies(install-runtimes-stripped install-runtimes-${name}-stripped)
       if(LLVM_INCLUDE_TESTS)
         add_dependencies(check-runtimes check-runtimes-${name})
         add_dependencies(runtimes-test-depends runtimes-test-depends-${name})
       endif()
       foreach(runtime_name ${runtime_names})
         add_dependencies(${runtime_name} ${runtime_name}-${name})
         add_dependencies(install-${runtime_name} install-${runtime_name}-${name})
         add_dependencies(install-${runtime_name}-stripped install-${runtime_name}-${name}-stripped)
       endforeach()
       foreach(component ${LLVM_RUNTIME_DISTRIBUTION_COMPONENTS})
         add_dependencies(${component} ${component}-${name})
         add_dependencies(install-${component} install-${component}-${name})
         add_dependencies(install-${component}-stripped install-${component}-${name}-stripped)
       endforeach()
     endforeach()
 
     foreach(multilib ${LLVM_RUNTIME_MULTILIBS})
       foreach(name ${LLVM_RUNTIME_MULTILIB_${multilib}_TARGETS})
-        runtime_register_target(${name}+${multilib} ${name}
+        runtime_register_target(${name}+${multilib}
           DEPENDS runtimes-${name}
-          CMAKE_ARGS -DLLVM_RUNTIMES_PREFIX=${name}/
-                     -DLLVM_RUNTIMES_LIBDIR_SUBDIR=${multilib})
+          CMAKE_ARGS -DLLVM_DEFAULT_TARGET_TRIPLE=${name}
+                     -DLLVM_RUNTIMES_PREFIX=${name}/
+                     -DLLVM_RUNTIMES_LIBDIR_SUBDIR=${multilib}
+          BASE_NAME ${name}
+          EXTRA_ARGS TARGET_TRIPLE ${name})
+
         add_dependencies(runtimes runtimes-${name}+${multilib})
         add_dependencies(runtimes-configure runtimes-${name}+${multilib}-configure)
         add_dependencies(install-runtimes install-runtimes-${name}+${multilib})
         add_dependencies(install-runtimes-stripped install-runtimes-${name}+${multilib}-stripped)
         foreach(runtime_name ${runtime_names})
           add_dependencies(${runtime_name} ${runtime_name}-${name}+${multilib})
           add_dependencies(install-${runtime_name} install-${runtime_name}-${name}+${multilib})
           add_dependencies(install-${runtime_name}-stripped install-${runtime_name}-${name}+${multilib}-stripped)