Page MenuHomePhabricator

[clang] [driver] Enable static linking to libc++
AbandonedPublic

Authored by tbaeder on Feb 4 2021, 11:45 AM.

Details

Summary

Hi,

currently the following fails:

clang++ -stdlib=libc++ -static ./test.cpp

because libc++ needs libc++abi and pthreads.

This change has already been proposed in a different version, using -static-libc++ in https://reviews.llvm.org/D63329

And https://reviews.llvm.org/D60794 tried to do it in libc++ instead of clang.

This version makes both linking with -static as above work, as well as -static-libstdc++ -stdlib=libc++.

Diff Detail

Unit TestsFailed

TimeTest
40 msx64 debian > Clang.Driver::libcxx-link.cpp
Script: -- : 'RUN: at line 4'; /mnt/disks/ssd0/agent/llvm-project/build/bin/clang++ /mnt/disks/ssd0/agent/llvm-project/clang/test/Driver/libcxx-link.cpp -stdlib=libc++
370 msx64 debian > libarcher.races::task-dependency.c
Script: -- : 'RUN: at line 13'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang -fopenmp -pthread -fno-experimental-isel -g -O1 -fsanitize=thread -I /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests -I /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/runtime/src -L /mnt/disks/ssd0/agent/llvm-project/build/lib -Wl,-rpath,/mnt/disks/ssd0/agent/llvm-project/build/lib /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-dependency.c -o /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-dependency.c.tmp -latomic && env TSAN_OPTIONS='ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1' /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/deflake.bash /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-dependency.c.tmp 2>&1 | tee /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-dependency.c.tmp.log | /mnt/disks/ssd0/agent/llvm-project/build/./bin/FileCheck /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-dependency.c
360 msx64 debian > libarcher.races::task-taskgroup-unrelated.c
Script: -- : 'RUN: at line 13'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang -fopenmp -pthread -fno-experimental-isel -g -O1 -fsanitize=thread -I /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests -I /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/runtime/src -L /mnt/disks/ssd0/agent/llvm-project/build/lib -Wl,-rpath,/mnt/disks/ssd0/agent/llvm-project/build/lib /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-taskgroup-unrelated.c -o /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskgroup-unrelated.c.tmp -latomic && env TSAN_OPTIONS='ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1' /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/deflake.bash /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskgroup-unrelated.c.tmp 2>&1 | tee /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskgroup-unrelated.c.tmp.log | /mnt/disks/ssd0/agent/llvm-project/build/./bin/FileCheck /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-taskgroup-unrelated.c
330 msx64 debian > libarcher.races::task-taskwait-nested.c
Script: -- : 'RUN: at line 13'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang -fopenmp -pthread -fno-experimental-isel -g -O1 -fsanitize=thread -I /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests -I /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/runtime/src -L /mnt/disks/ssd0/agent/llvm-project/build/lib -Wl,-rpath,/mnt/disks/ssd0/agent/llvm-project/build/lib /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-taskwait-nested.c -o /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskwait-nested.c.tmp -latomic && env TSAN_OPTIONS='ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1' /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/deflake.bash /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskwait-nested.c.tmp 2>&1 | tee /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-taskwait-nested.c.tmp.log | /mnt/disks/ssd0/agent/llvm-project/build/./bin/FileCheck /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-taskwait-nested.c
330 msx64 debian > libarcher.races::task-two.c
Script: -- : 'RUN: at line 13'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang -fopenmp -pthread -fno-experimental-isel -g -O1 -fsanitize=thread -I /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests -I /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/runtime/src -L /mnt/disks/ssd0/agent/llvm-project/build/lib -Wl,-rpath,/mnt/disks/ssd0/agent/llvm-project/build/lib /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-two.c -o /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-two.c.tmp -latomic && env TSAN_OPTIONS='ignore_noninstrumented_modules=0:ignore_noninstrumented_modules=1' /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/deflake.bash /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-two.c.tmp 2>&1 | tee /mnt/disks/ssd0/agent/llvm-project/build/projects/openmp/tools/archer/tests/races/Output/task-two.c.tmp.log | /mnt/disks/ssd0/agent/llvm-project/build/./bin/FileCheck /mnt/disks/ssd0/agent/llvm-project/openmp/tools/archer/tests/races/task-two.c
View Full Test Results (15 Failed)

Event Timeline

tbaeder requested review of this revision.Feb 4 2021, 11:45 AM
tbaeder created this revision.
Herald added a project: Restricted Project. · View Herald TranscriptFeb 4 2021, 11:45 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript

I'm not convinced that this is the right way of handling it. Libc++ can be used on top of a number of C++ ABI libraries (both libsupc++/libstdc++ and libcxxabi), and how they are linked varies with how a toolchain is assembled. In particular, if libcxx is built with LIBCXX_ENABLE_STATIC_ABI_LIBRARY enabled, the libcxxabi static library is merged into libc++.a, making static linking work just fine.

joerg added a subscriber: joerg.Feb 4 2021, 1:52 PM

The NetBSD part is most definitely not acceptable.

MaskRay requested changes to this revision.EditedFeb 4 2021, 2:07 PM
MaskRay added a subscriber: MaskRay.

It is difficult for clang driver to make the decision because libc++ has multiple C++ ABI implementations as @mstorsjo said.

Persoanlly (https://maskray.me/blog/2020-12-12-c++-exception-handling-abi#use-libc-and-libcabi) I use:

clang++ -stdlib=libc++ -static-libstdc++ -nostdlib++ a.cc -Wl,--push-state,-Bstatic -lc++ -lc++abi -Wl,--pop-state -pthread

This is still a bit inferior because there is a duplicate -lc++ passed by the driver.

Dynamically link against libc++.so (which depends on libc++abi.so) (additionally specify -pthread if threads are used):
clang++ -stdlib=libc++ -nostdlib++ a.cc -lc++ -lc++abi (clang -stdlib=libc++ a.cc -lc++ -lc++abi does not pass -lm to the linker.)
Omitting -lc++abi most works, but can fail if your program reference some definitions in <exception> <stdexcept> directly and you are using gold or LLD.

This revision now requires changes to proceed.Feb 4 2021, 2:07 PM

The goal here is for distributions to be able to build libc++ one way and then have it work with clang without requiring that users add additional linker flags besides -static or -stdlib=libc++. Is there a combination of CMake arguments we can use when building libc++ to make all static and shared linking with libc++ work out of the box?

In particular, if libcxx is built with LIBCXX_ENABLE_STATIC_ABI_LIBRARY enabled, the libcxxabi static library is merged into libc++.a, making static linking work just fine.

I can't test this right now, but it is manually specifying pthread not needed in that case?

Persoanlly (https://maskray.me/blog/2020-12-12-c++-exception-handling-abi#use-libc-and-libcabi) I use:

clang++ -stdlib=libc++ -static-libstdc++ -nostdlib++ a.cc -Wl,--push-state,-Bstatic -lc++ -lc++abi -Wl,--pop-state -pthread

This is still a bit inferior because there is a duplicate -lc++ passed by the driver.

Dynamically link against libc++.so (which depends on libc++abi.so) (additionally specify -pthread if threads are used):
clang++ -stdlib=libc++ -nostdlib++ a.cc -lc++ -lc++abi (clang -stdlib=libc++ a.cc -lc++ -lc++abi does not pass -lm to the linker.)
Omitting -lc++abi most works, but can fail if your program reference some definitions in <exception> <stdexcept> directly and you are using gold or LLD.

Are you suggesting clang should do the same?

Or should clang just not do anything here and people should be aware they need to pass additional flags when using libc++ (on Linux)?

In particular, if libcxx is built with LIBCXX_ENABLE_STATIC_ABI_LIBRARY enabled, the libcxxabi static library is merged into libc++.a, making static linking work just fine.

I can't test this right now, but it is manually specifying pthread not needed in that case?

That would only bundle libcxxabi, not pthreads, so that'd still be required.

joerg added a comment.Feb 5 2021, 1:49 AM

I can't speak for other systems, but forcing libpthread to be linked is in general not desirable as it creates a non-trivial runtime cost, especially when <thread> is not used. That's still a pretty common use case of C++.

The problem is that we have -lc++ (optional -lc++abi/-lcxxrt/others) (optional -lpthread)) dependency. While we can control -lpthread with -pthread (which does extra things on its own), we cannot control (optional -lc++abi/-lcxxrt/others), which is depended by the installation configuration (e.g. LIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY, or libc++.a as a linker script (https://bugs.llvm.org/show_bug.cgi?id=46321), or .deplibs) and whether you are specifying --sysroot. Perhaps I had the opportunity to redesign the matter, I would let -stdlib=libc++ not affect linker options at all. Users just should specify -lc++ and related options by themselves.

we cannot control (optional -lc++abi/-lcxxrt/others), which is depended by the installation configuration (e.g. LIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY, or libc++.a as a linker script (https://bugs.llvm.org/show_bug.cgi?id=46321), or .deplibs) and whether you are specifying --sysroot. Perhaps I had the opportunity to redesign the matter, I would let -stdlib=libc++ not affect linker options at all. Users just should specify -lc++ and related options by themselves.

Right, but at least some of these sound like the can or should be solved in libc++ then. Reading https://reviews.llvm.org/D60794, I thought this was not going to happen and the solution should instead be in clang.

What is the ideal solution here? If -static -stdlib=libc++ is not meant to work, is the working solution documented anywhere that we could link people to? If your blog post isn't official of course :)

I think the root cause of the push back you're getting is that this is normally handled by vendors when they decide how they're going to ship libc++ on their platform or in their toolchain.

For example, on Apple platforms, we ship libc++.dylib, and we make sure to re-export libc++abi.dylib's symbols from libc++.dylib. The result is that you only need to specify -lc++ (which the compiler does by default), and everything "works". Since we don't ship libc++.a, there's no out-of-the-box solution to make it work - it's not officially supported. Some folks do diverge from that, however we assume they are willing to get their hands dirty and use the flags required to make it work (i.e. -lc++ -lc++abi). My understanding is that the same goes for other platforms, where each has their preferred way of shipping libc++. If we were to officially ship libc++.a in the SDKs for Apple platforms, I could imagine that we'd want to merge libc++abi.a into libc++.a, and there again you'd only need to specify -static -lc++ and everything would work (unless I'm missing something).

Making that work out-of-the-box would be something we'd want to achieve in how we're building and shipping libc++, not by modifying the behaviour of the Clang driver on multiple platforms.

[...]

Or should clang just not do anything here and people should be aware they need to pass additional flags when using libc++ (on Linux)?

The TL;DR of what I wrote above is that I believe this is the correct answer. My main area is Apple platforms and not Linux, though, so this is just my .02 as someone who ships libc++.

Making -stdlib=libc++ not affect linker option for -static and -static-pie looks good to me, for the GNU toolchain. They are expected to pass -lc++ -lc++abi if they use separate libc++abi.a.
Just specifying -lc++abi does not work with GNU ld and gold anyway.
It may make some users unhappy if we do that for dynamic linking.

tbaeder abandoned this revision.Feb 18 2021, 4:36 AM

Thanks for the input everyone.