Passthrough LLVM_USE_LLD LLVM_USE_LINKER into runtimes

[vitalybuka]

Summary

Fixes https://github.com/llvm/llvm-project/issues/42339

[phosek]

llvm/runtimes/CMakeLists.txt

334

We should also pass it through here it here.

[vitalybuka]

D115812 is not needed, -fuse-ld should be in CMAKE_EXE_LINKER_FLAGS CMAKE_MODULE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS

[vitalybuka]

runtimes-${name}

[sylvestre.ledru]

Excellent, thanks

[phosek]

LGTM

[mehdi_amini]

This breaks our bot right now, I'm not sure how/why, maybe our CMake invocation isn't correct? It's not obvious to me though...

Here is what I see in the log:

cmake -G Ninja ${src_dir}/llvm -DLLVM_TARGETS_TO_BUILD="X86;NVPTX;AMDGPU" -DLLVM_ENABLE_PROJECTS="lld;clang;mlir" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=Off -DLIBCXX_CXX_ABI=default -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi"  -DLLVM_ENABLE_LLD=ON  -DLLVM_DISTRIBUTION_COMPONENTS="clang;runtimes" -DLLVM_CCACHE_BUILD=ON
ninja distribution

But then later in the process:

[3938/4157] Performing configure step for 'runtimes'
-- The C compiler identification is Clang 14.0.0
-- The CXX compiler identification is Clang 14.0.0
-- The ASM compiler identification is Clang
-- Found assembler: /tmp/ci-jSvVzRPAaN/./bin/clang
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /tmp/ci-jSvVzRPAaN/./bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /tmp/ci-jSvVzRPAaN/./bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test Terminfo_LINKABLE
-- Performing Test Terminfo_LINKABLE - Success
-- Found Terminfo: /usr/lib/x86_64-linux-gnu/libtinfo.so
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- Performing Test LLVM_RUNTIMES_LINKING_WORKS
-- Performing Test LLVM_RUNTIMES_LINKING_WORKS - Success
-- Performing Test LLVM_RUNTIMES_SUPPORT_NOSTDLIBXX_FLAG
-- Performing Test LLVM_RUNTIMES_SUPPORT_NOSTDLIBXX_FLAG - Success
-- Performing Test LLVM_RUNTIMES_SUPPORT_NOSTDINCXX_FLAG
-- Performing Test LLVM_RUNTIMES_SUPPORT_NOSTDINCXX_FLAG - Success
-- Linker detection: LLD
CMake Error at /var/lib/buildkite-agent/builds/buildkite-588bd64db9-m924d-1/mlir/mlir-core/llvm/cmake/modules/HandleLLVMOptions.cmake:280 (message):
  LLVM_ENABLE_LLD and LLVM_USE_LINKER can't be set at the same time
Call Stack (most recent call first):
  CMakeLists.txt:125 (include)
 
-- Configuring incomplete, errors occurred!
See also "/tmp/ci-jSvVzRPAaN/runtimes/runtimes-bins/CMakeFiles/CMakeOutput.log".
See also "/tmp/ci-jSvVzRPAaN/runtimes/runtimes-bins/CMakeFiles/CMakeError.log".
[3951/4157] Linking CXX static library lib/libMLIRTestReducer.a
FAILED: runtimes/runtimes-stamps/runtimes-configure /tmp/ci-jSvVzRPAaN/runtimes/runtimes-stamps/runtimes-configure
cd /tmp/ci-jSvVzRPAaN/runtimes/runtimes-bins && /usr/bin/cmake -DCMAKE_C_COMPILER=/tmp/ci-jSvVzRPAaN/./bin/clang -DCMAKE_CXX_COMPILER=/tmp/ci-jSvVzRPAaN/./bin/clang++ -DCMAKE_ASM_COMPILER=/tmp/ci-jSvVzRPAaN/./bin/clang -DCMAKE_LINKER=/tmp/ci-jSvVzRPAaN/./bin/ld.lld -DCMAKE_AR=/tmp/ci-jSvVzRPAaN/./bin/llvm-ar -DCMAKE_RANLIB=/tmp/ci-jSvVzRPAaN/./bin/llvm-ranlib -DCMAKE_NM=/tmp/ci-jSvVzRPAaN/./bin/llvm-nm -DCMAKE_OBJDUMP=/tmp/ci-jSvVzRPAaN/./bin/llvm-objdump -DCMAKE_OBJCOPY=/tmp/ci-jSvVzRPAaN/./bin/llvm-objcopy -DCMAKE_STRIP=/tmp/ci-jSvVzRPAaN/./bin/llvm-strip -DCMAKE_READELF=/tmp/ci-jSvVzRPAaN/./bin/llvm-readelf -DCMAKE_C_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_CXX_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_ASM_COMPILER_TARGET=x86_64-unknown-linux-gnu -DCMAKE_INSTALL_PREFIX=/usr/local -DLLVM_BINARY_DIR=/tmp/ci-jSvVzRPAaN -DLLVM_CONFIG_PATH=/tmp/ci-jSvVzRPAaN/bin/llvm-config -DLLVM_ENABLE_WERROR=OFF -DLLVM_HOST_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_HAVE_LINK_VERSION_SCRIPT=1 -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=OFF -DLLVM_USE_RELATIVE_PATHS_IN_FILES=OFF -DLLVM_LIT_ARGS=-sv -DLLVM_SOURCE_PREFIX= -DPACKAGE_VERSION=14.0.0git -DCMAKE_BUILD_TYPE=Release -DCMAKE_MAKE_PROGRAM=/usr/bin/ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCOMPILER_RT_BUILD_BUILTINS=Off -DLLVM_INCLUDE_TESTS=ON -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_ENABLE_PROJECTS_USED=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON -DLLVM_BUILD_TOOLS=ON -DCMAKE_C_COMPILER_WORKS=ON -DCMAKE_CXX_COMPILER_WORKS=ON -DCMAKE_ASM_COMPILER_WORKS=ON -DHAVE_LLVM_LIT=ON "-DLLVM_ENABLE_RUNTIMES=libcxx;libcxxabi" -DLLVM_ENABLE_LLD=ON -DLLVM_USE_LINKER=lld -DLIBCXX_CXX_ABI=default -GNinja /var/lib/buildkite-agent/builds/buildkite-588bd64db9-m924d-1/mlir/mlir-core/llvm/runtimes/../../runtimes && /usr/bin/cmake -E touch /tmp/ci-jSvVzRPAaN/runtimes/runtimes-stamps//runtimes-configure

[mehdi_amini]

Oh I see https://reviews.llvm.org/rG96aa1e73b3da2252fda883d07cc0600167fc7c79 ; that makes fix our bot as well

[smeenai]

I don't think this is correct when cross-compiling. For example, if you're cross-compiling for Windows on a Linux host and using gold as your Linux host linker, it doesn't make sense to also use gold for your Windows links. PASSTHROUGH_PREFIXES is also annoying because it's added last in the external project invocation (https://github.com/apple/llvm-project/blob/7ca7a2547f00e34f5ec91be776a1d0bbca74b7a9/llvm/cmake/modules/LLVMExternalProjectUtils.cmake#L308-L309), so there's no way to override it. Is there an alternative solution that would work here while not breaking the cross-compilation case?

[smeenai]

llvm/runtimes/CMakeLists.txt

341

Would it work to add -DLLVM_USE_LINKER=${LLVM_USE_LINKER} instead of PASSTHROUGH_PREFIXES? That'd still be incorrect for cross-compilation by default, but then we could at least override it by using the target-specific runtime arguments.

[phosek]

In D115852#3226050, @smeenai wrote:

I don't think this is correct when cross-compiling. For example, if you're cross-compiling for Windows on a Linux host and using gold as your Linux host linker, it doesn't make sense to also use gold for your Windows links. PASSTHROUGH_PREFIXES is also annoying because it's added last in the external project invocation (https://github.com/apple/llvm-project/blob/7ca7a2547f00e34f5ec91be776a1d0bbca74b7a9/llvm/cmake/modules/LLVMExternalProjectUtils.cmake#L308-L309), so there's no way to override it. Is there an alternative solution that would work here while not breaking the cross-compilation case?

I think for the cross-compilation cases, we should use the same strategy we use for LLVM_ENABLE_RUNTIMES where we don't do passthrough if the variable name prefixed with the target is set. So if RUNTIMES_x86_64-pc-windows-msvc_LLVM_USE_LLD is set, we would pass through that one and not LLVM_USE_LLD. Do you think that would be sufficient?

[smeenai]

In D115852#3226064, @phosek wrote:

In D115852#3226050, @smeenai wrote:

I don't think this is correct when cross-compiling. For example, if you're cross-compiling for Windows on a Linux host and using gold as your Linux host linker, it doesn't make sense to also use gold for your Windows links. PASSTHROUGH_PREFIXES is also annoying because it's added last in the external project invocation (https://github.com/apple/llvm-project/blob/7ca7a2547f00e34f5ec91be776a1d0bbca74b7a9/llvm/cmake/modules/LLVMExternalProjectUtils.cmake#L308-L309), so there's no way to override it. Is there an alternative solution that would work here while not breaking the cross-compilation case?

I think for the cross-compilation cases, we should use the same strategy we use for LLVM_ENABLE_RUNTIMES where we don't do passthrough if the variable name prefixed with the target is set. So if RUNTIMES_x86_64-pc-windows-msvc_LLVM_USE_LLD is set, we would pass through that one and not LLVM_USE_LLD. Do you think that would be sufficient?

Yup, that sounds good to me.

llvm/runtimes/CMakeLists.txt

llvm_ExternalProject_Add(runtimes
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default}
                                       -DLLVM_BUILD_TOOLS=${LLVM_BUILD_TOOLS}
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_CXX_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       ${COMMON_CMAKE_ARGS}
                                       ${RUNTIMES_CMAKE_ARGS}
                            PASSTHROUGH_PREFIXES LLVM_ENABLE_RUNTIMES
+                                                LLVM_USE_LLD
+                                                LLVM_USE_LINKER
                                                 ${ARG_PREFIXES}
                            EXTRA_TARGETS ${extra_targets}
                                          ${test_targets}
                                          ${SUB_COMPONENTS}
                                          ${SUB_CHECK_TARGETS}
                                          ${SUB_INSTALL_TARGETS}
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${TARGET_TRIPLE}
if(NOT "${name}" STREQUAL "${target}")
     endforeach()
   endif()
 
   if(NOT RUNTIMES_${name}_LLVM_ENABLE_RUNTIMES AND NOT RUNTIMES_${target}_LLVM_ENABLE_RUNTIMES)
     string(REPLACE ";" "|" LLVM_ENABLE_RUNTIMES_PASSTHROUGH "${LLVM_ENABLE_RUNTIMES}")
     list(APPEND ${name}_extra_args -DLLVM_ENABLE_RUNTIMES=${LLVM_ENABLE_RUNTIMES_PASSTHROUGH})
   endif()
 
   llvm_ExternalProject_Add(runtimes-${name}
                            ${CMAKE_CURRENT_SOURCE_DIR}/../../runtimes
                            DEPENDS ${${name}_deps}
                            # Builtins were built separately above
                            CMAKE_ARGS -DCOMPILER_RT_BUILD_BUILTINS=Off
                                       -DLLVM_INCLUDE_TESTS=${LLVM_INCLUDE_TESTS}
                                       -DLLVM_DEFAULT_TARGET_TRIPLE=${target}
                                       -DLLVM_ENABLE_PROJECTS_USED=${LLVM_ENABLE_PROJECTS_USED}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON
                                       -DCMAKE_C_COMPILER_WORKS=ON
                                       -DCMAKE_CXX_COMPILER_WORKS=ON
                                       -DCMAKE_ASM_COMPILER_WORKS=ON
                                       -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON
                                       -DLLVM_RUNTIMES_TARGET=${name}
                                       ${COMMON_CMAKE_ARGS}
                                       ${${name}_extra_args}
+                           PASSTHROUGH_PREFIXES LLVM_USE_LLD
+                                                LLVM_USE_LINKER
                            EXTRA_TARGETS ${${name}_extra_targets}
                                          ${${name}_test_targets}
                            USE_TOOLCHAIN
                            TARGET_TRIPLE ${target}
                            ${EXTRA_ARGS})
 endfunction()
 
 if(runtimes)