[runtimes] Support ELF dependent libraries feature

[phosek]

Summary

As of r360984, LLD supports dependent libraries feature for ELF.
libunwind and libc++ have library dependencies: libdl and libpthread
respectively, which means that when libunwind and libc++ are being
statically linked (using -static-libstdc++ flag), user has to manually
specify -ldl -lpthread which is onerous. This change includes the
lib pragma to specify the library dependencies directly in the source
that uses those libraries. This doesn't make any difference when using
linkers that don't support dependent libraries. However, when using
LLD that has dependent libraries feature, users no longer have to
manually specifying library dependencies when using static linking,
linker will pick them up automatically. The use of this feature is
conditionalized on CMake build option and can be enabled by vendors.

[phosek]

Would it be possible to take a look? I believe this is a better alternative to both D54724 and D54726 which doesn't require any changes to the driver or special linker scripts.

[ldionne]

I guess I don't have any strong objection, but I think I had a preference for handling it in the build system per https://reviews.llvm.org/D54726. This would conceivably work on other platforms that support linker scripts and it's less intrusive to the source code. But if you feel strongly that this is the right way to go, I'm OK with it.

[bd1976llvm]

Great to see someone wanting to use this. During the RFC we decided not to set SHF_EXCLUDE on the .deplibs ELF section that is used to store the strings from these pragmas. This means that other linkers that don't support dependent libraries will not remove the sections and they will end up in the output. This shouldn't cause any functional issues; however, it might be noticed by developers and commented on. Perhaps we should change this decision? Also, have you considered the other option of having the build system invoke the clang with the --dependent-library=<specifier> switch. This is an alternative approach where modifying the source code is not appropriate, an example use is: https://reviews.llvm.org/D61742.

[phosek]

In D62090#1510537, @ldionne wrote:

I guess I don't have any strong objection, but I think I had a preference for handling it in the build system per https://reviews.llvm.org/D54726. This would conceivably work on other platforms that support linker scripts and it's less intrusive to the source code. But if you feel strongly that this is the right way to go, I'm OK with it.

I have a slight preference for this approach for two reasons: (1) with linker scripts we have to rename the static library which may lead to some confusion, e.g. simply running readelf -sW libc++.a will fail because libc++.a is now a linker script; (2) it localizes the dependency information to where it's needed, e.g. the pthread library dependency is only included if _LIBCPP_HAS_THREAD_API_PTHREAD is defined.

In D62090#1510816, @bd1976llvm wrote:

Great to see someone wanting to use this. During the RFC we decided not to set SHF_EXCLUDE on the .deplibs ELF section that is used to store the strings from these pragmas. This means that other linkers that don't support dependent libraries will not remove the sections and they will end up in the output. This shouldn't cause any functional issues; however, it might be noticed by developers and commented on. Perhaps we should change this decision?

I think we should consider that.

In D62090#1510816, @bd1976llvm wrote:

Also, have you considered the other option of having the build system invoke the clang with the --dependent-library=<specifier> switch. This is an alternative approach where modifying the source code is not appropriate, an example use is: https://reviews.llvm.org/D61742.

That might be an option as well and is something that can be done from CMake invocation via CFLAGS. The downside is AFAICT that the information will no longer be localized to where it's needed and instead applied to all translation units inside the archive e.g. even if you use part of libc++ that doesn't need pthread, you'll still get the dependency. An alternative would be to set that flag on a per translation unit basis but I'm not sure whether that's better than having the pragma directly in the source.

[ldionne]

I don't have any objections to this (but it doesn't impact my platform).

[phosek]

I've conditionalized the use of this feature on CMake build option which defaults to OFF and can be turned on by vendors as needed. Hopefully that's cleaner than the previous version where it was enabled unconditionally.

[ldionne]

It's not entirely clear to me that adding an option to control this is better than just doing it unconditionally. Have you run into a scenario where you didn't want to enable it (and if so, why?).

If that's "just in case", I'd suggest doing it unconditionally and adding an option later if a vendor wants to disable it.

libcxx/CMakeLists.txt

547 ↗

(On Diff #201812)

Please use the per-target form. We've done a lot of work to remove global definitions and flags, let's keep working in that direction.

[phosek]

In D62090#1521187, @ldionne wrote:

It's not entirely clear to me that adding an option to control this is better than just doing it unconditionally. Have you run into a scenario where you didn't want to enable it (and if so, why?).

If that's "just in case", I'd suggest doing it unconditionally and adding an option later if a vendor wants to disable it.

That makes sense, I've removed the option but I've moved the pragma from headers into source files to avoid it being injected into users code through includes.

src/algorithm.cpp

 //===----------------------- algorithm.cpp --------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "algorithm"
 #include "random"
+#ifndef _LIBCPP_HAS_NO_THREADS
 #include "mutex"
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
+#endif
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 template void __sort<__less<char>&, char*>(char*, char*, __less<char>&);
 template void __sort<__less<wchar_t>&, wchar_t*>(wchar_t*, wchar_t*, __less<wchar_t>&);
 template void __sort<__less<signed char>&, signed char*>(signed char*, signed char*, __less<signed char>&);
 template void __sort<__less<unsigned char>&, unsigned char*>(unsigned char*, unsigned char*, __less<unsigned char>&);
 template void __sort<__less<short>&, short*>(short*, short*, __less<short>&);

src/chrono.cpp

 #if !defined(_LIBCPP_HAS_NO_MONOTONIC_CLOCK)
 #if __APPLE__
 #include <mach/mach_time.h>  // mach_absolute_time, mach_timebase_info_data_t
 #elif !defined(_LIBCPP_WIN32API) && !defined(CLOCK_MONOTONIC)
 #error "Monotonic clock not implemented"
 #endif
 #endif
 
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "rt")
+#endif
+
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 namespace chrono
 {
 
 // system_clock
 
 const bool system_clock::is_steady;

src/condition_variable.cpp

 
 #ifndef _LIBCPP_HAS_NO_THREADS
 
 #include "condition_variable"
 #include "thread"
 #include "system_error"
 #include "__undef_macros"
 
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
+
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 condition_variable::~condition_variable()
 {
     __libcpp_condvar_destroy(&__cv_);
 }
 
 void

src/debug.cpp

 //===-------------------------- debug.cpp ---------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "__config"
 #include "__debug"
 #include "functional"
 #include "algorithm"
 #include "string"
 #include "cstdio"
 #include "__hash_table"
+#ifndef _LIBCPP_HAS_NO_THREADS
 #include "mutex"
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
+#endif
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 std::string __libcpp_debug_info::what() const {
   string msg = __file_;
   msg += ":" + to_string(__line_) + ": _LIBCPP_ASSERT '";
   msg += __pred_;
   msg += "' failed. ";

src/experimental/memory_resource.cpp

 //===------------------------ memory_resource.cpp -------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "experimental/memory_resource"
 
 #ifndef _LIBCPP_HAS_NO_ATOMIC_HEADER
 #include "atomic"
 #elif !defined(_LIBCPP_HAS_NO_THREADS)
 #include "mutex"
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
 #endif
 
 _LIBCPP_BEGIN_NAMESPACE_LFTS_PMR
 
 // memory_resource
 
 //memory_resource::~memory_resource() {}
 

src/filesystem/operations.cpp

 #if !defined(__APPLE__)
 #define _LIBCPP_USE_CLOCK_GETTIME
 #endif
 
 #if !defined(CLOCK_REALTIME) || !defined(_LIBCPP_USE_CLOCK_GETTIME)
 #include <sys/time.h> // for gettimeofday and timeval
 #endif                // !defined(CLOCK_REALTIME)
 
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "rt")
+#endif
+
 #if defined(_LIBCPP_COMPILER_GCC)
 #if _GNUC_VER < 500
 #pragma GCC diagnostic ignored "-Wmissing-field-initializers"
 #endif
 #endif
 
 _LIBCPP_BEGIN_NAMESPACE_FILESYSTEM
 

src/memory.cpp

 //===------------------------ memory.cpp ----------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "memory"
 #ifndef _LIBCPP_HAS_NO_THREADS
 #include "mutex"
 #include "thread"
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
 #endif
 #include "include/atomic_support.h"
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 const allocator_arg_t allocator_arg = allocator_arg_t();
 
 bad_weak_ptr::~bad_weak_ptr() _NOEXCEPT {}

src/mutex.cpp

 //===------------------------- mutex.cpp ----------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "mutex"
 #include "limits"
 #include "system_error"
 #include "include/atomic_support.h"
 #include "__undef_macros"
 
+#ifndef _LIBCPP_HAS_NO_THREADS
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
+#endif
+
 _LIBCPP_BEGIN_NAMESPACE_STD
 #ifndef _LIBCPP_HAS_NO_THREADS
 
 const defer_lock_t  defer_lock = {};
 const try_to_lock_t try_to_lock = {};
 const adopt_lock_t  adopt_lock = {};
 
 mutex::~mutex() _NOEXCEPT

src/shared_mutex.cpp

 //===---------------------- shared_mutex.cpp ------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "__config"
 #ifndef _LIBCPP_HAS_NO_THREADS
 
 #include "shared_mutex"
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 // Shared Mutex Base
 __shared_mutex_base::__shared_mutex_base()
     : __state_(0)
 {
 }

src/thread.cpp

 #if defined(__NetBSD__)
 #pragma weak pthread_create // Do not create libpthread dependency
 #endif
 
 #if defined(_LIBCPP_WIN32API)
 #include <windows.h>
 #endif
 
+#if defined(__unix__) &&  defined(__ELF__) && defined(__clang__)
+#pragma comment(lib, "pthread")
+#endif
+
 _LIBCPP_BEGIN_NAMESPACE_STD
 
 thread::~thread()
 {
     if (!__libcpp_thread_isnull(&__t_))
         terminate();
 }