Page MenuHomePhabricator

Fix RUNPATH not accounting for LLVM_ENABLE_PER_TARGET_RUNTIME_DIR
Needs ReviewPublic

Authored by arcivanov on May 21 2022, 11:39 AM.

Details

Reviewers
EricWF
Group Reviewers
Restricted Project
Summary

I'm incorporating by reference the data from this issue: https://github.com/llvm/llvm-project/issues/54955
This commit fixes this issue.

Furthermore the issue always occurs when building runtimes as part of the two-stage distro build as LLVM_ENABLE_PER_TARGET_RUNTIME_DIR is always ON for all architectures except AIX for any and all selected runtimes.

Diff Detail

Unit TestsFailed

TimeTest
4,470 mslibcxx CI Apple back-deployment macosx10.15 > apple-libc++-backdeployment-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 50'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/libcxx/modules_include.sh.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk --target=x86_64-apple-macosx10.15 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/build/apple-system-backdeployment-10.15/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fmodules -fcxx-modules -fsyntax-only -DTEST_0
4,290 mslibcxx CI Apple back-deployment macosx10.9 > apple-libc++-backdeployment-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 50'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/libcxx/modules_include.sh.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk --target=x86_64-apple-macosx10.9 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/build/apple-system-backdeployment-10.9/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fmodules -fcxx-modules -fsyntax-only -DTEST_0
1,130 mslibcxx CI Apple back-deployment macosx11.0 arm64 > apple-libc++-backdeployment-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 50'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/static-162-255-21-37-macminivault-com/llvm-project/libcxx-ci/libcxx/test/libcxx/modules_include.sh.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk --target=arm64-apple-macosx11.0 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/static-162-255-21-37-macminivault-com/llvm-project/libcxx-ci/build/apple-system-backdeployment-11.0/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/static-162-255-21-37-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fmodules -fcxx-modules -fsyntax-only -DTEST_0
1,380 mslibcxx CI MacOS arm64 > llvm-libc++-shared-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 50'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/f1-1-macminivault-com/llvm-project/libcxx-ci/libcxx/test/libcxx/modules_include.sh.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk --target=arm64-apple-darwin21.4.0 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/f1-1-macminivault-com/llvm-project/libcxx-ci/build/generic-cxx20/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/f1-1-macminivault-com/llvm-project/libcxx-ci/build/generic-cxx20/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/f1-1-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++20 -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fmodules -fcxx-modules -fsyntax-only -DTEST_0
4,920 mslibcxx CI MacOS x86_64 > llvm-libc++-shared-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 50'; /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/libcxx/modules_include.sh.cpp -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk --target=x86_64-apple-darwin21.3.0 -nostdinc++ -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/build/generic-cxx20/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/build/generic-cxx20/include/c++/v1 -I /Users/libcxx-buildkite-agent/libcxx.buildkite-agent/builds/y10-8-macminivault-com/llvm-project/libcxx-ci/libcxx/test/support -std=c++20 -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fmodules -fcxx-modules -fsyntax-only -DTEST_0

Event Timeline

arcivanov created this revision.May 21 2022, 11:39 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 21 2022, 11:39 AM
Herald added a subscriber: mgorny. · View Herald Transcript
arcivanov requested review of this revision.May 21 2022, 11:39 AM
EricWF requested changes to this revision.May 21 2022, 1:56 PM
EricWF added a subscriber: EricWF.

I think this changes necessary to fix libc++, assuming it actually uses this cmake infrastructure, which I'm not sure it does.

How is this been tested?

Also, I apologize I didn't mean to request changes but I can't unselect it on mobile

This revision now requires changes to proceed.May 21 2022, 1:56 PM

Well, this affects *at least* runtimes libunwind, libcxx and libcxxabi and at least on Linux.
The situation occurs when project are built depending on any of those runtimes being directly colocated, i.e. referenced by runpath.

The issue was discovered as part of a single-shot two-stage build with said runtimes.
More details: https://discourse.llvm.org/t/bootstrap-build-with-llvm-runtimes-clang-shared-libc-build-failure-stage2-bins-due-to-libc-so-not-found/60141/7?u=arcivanov

The saga continues. The libcxx/abi do not appear to have any rpath reference to the libunwind:

$ ldd bin/clang-14
        linux-vdso.so.1 (0x00007ffcfd7cc000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7970ad0000)
        libclang-cpp.so.14 => /home/user/Documents/src/project/llvm.twostage.build.manylinux2014/tools/clang/stage2-bins/bin/../lib/libclang-cpp.so.14 (0x00007f796ce70000)
        libLLVM-14.so => /home/user/Documents/src/project/llvm.twostage.build.manylinux2014/tools/clang/stage2-bins/bin/../lib/libLLVM-14.so (0x00007f796971b000)
        libc++.so.1 => /home/user/Documents/src/project/llvm.twostage.build.manylinux2014/tools/clang/stage2-bins/bin/../lib/x86_64-unknown-linux-gnu/libc++.so.1 (0x00007f7969627000)
        libc++abi.so.1 => /home/user/Documents/src/project/llvm.twostage.build.manylinux2014/tools/clang/stage2-bins/bin/../lib/x86_64-unknown-linux-gnu/libc++abi.so.1 (0x00007f79695ec000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f7969510000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f79694f5000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f79692ec000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f7970b02000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f79692e7000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f79692e0000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f79692c6000)
        libunwind.so.1 => not found
        libunwind.so.1 => not found

It looks like 2-stage builds with embedded shared library runtimes are fundamentally broken with respect to rpath/runpath.
Additional revision is incoming.

arcivanov updated this revision to Diff 431848.May 24 2022, 5:28 PM

When libcxx and libcxxabi depend on shared libunwind that is installed with them in the target-triple directory, both libraries also require an '$ORIGIN' runpath.

arcivanov added a comment.EditedMay 24 2022, 6:03 PM
$ ldd clang-14 
        linux-vdso.so.1 (0x00007ffeb49f7000)
        libclang-cpp.so.14 => /home/user/Documents/src/project/llvm.twostage.bin/bin/./../lib/libclang-cpp.so.14 (0x00007fc547ad7000)
        libLLVM-14.so => /home/user/Documents/src/project/llvm.twostage.bin/bin/./../lib/libLLVM-14.so (0x00007fc54437f000)
        libc++.so.1 => /home/user/Documents/src/project/llvm.twostage.bin/bin/./../lib/x86_64-unknown-linux-gnu/libc++.so.1 (0x00007fc54428b000)
        libc++abi.so.1 => /home/user/Documents/src/project/llvm.twostage.bin/bin/./../lib/x86_64-unknown-linux-gnu/libc++abi.so.1 (0x00007fc544252000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fc54414b000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc54412e000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fc543f25000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc54b739000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fc543f0b000)
        libxml2.so.2 => /lib64/libxml2.so.2 (0x00007fc543d81000)
        libatomic.so.1 => /lib64/libatomic.so.1 (0x00007fc543d77000)
        libunwind.so.1 => /home/user/Documents/src/project/llvm.twostage.bin/bin/./../lib/x86_64-unknown-linux-gnu/libunwind.so.1 (0x00007fc543d69000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fc543d3d000)
$ ldd libc++.so.1.0 
        linux-vdso.so.1 (0x00007ffe0af16000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f823dd6b000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f823dc8f000)
        libatomic.so.1 => /lib64/libatomic.so.1 (0x00007f823dc85000)
        libc++abi.so.1 => /home/user/Documents/src/project/llvm.twostage.bin/lib/x86_64-unknown-linux-gnu/./libc++abi.so.1 (0x00007f823dc4c000)
        libunwind.so.1 => /home/user/Documents/src/project/llvm.twostage.bin/lib/x86_64-unknown-linux-gnu/./libunwind.so.1 (0x00007f823dc40000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f823e095000)

Windows doesn't need the rpath.

I'm not sure what to do here to validate this as builds keep randomly failing due to timeouts.

for what it's worth: As part of upgrading the Salesforce-internal clang toolchain to clang 15.0.0, I also ran into the RUNPATH issues. This patch here solved those issues for us

DoDoENT added a subscriber: DoDoENT.Sep 7 2022, 4:00 AM