[cmake] Split linked libraries into private & public, for linker script

[mgorny]

Summary

Introduce LIBCXX_LIBRARIES_PUBLIC in addition to LIBCXX_LIBRARIES that
holds 'public' interface libraries -- that is, libraries that both
libc++ links to and programs linked against it need to link to.

Currently this includes the ABI library and optionally -lunwind (when
LIBCXXABI_USE_LLVM_UNWINDER is on). The libraries are included in the
linker script, in order to make it possible to link C++ programs using
clang with compiler-rt runtime out-of-the-box.

[rmaprath]

The patch generally makes sense to me.

I wonder if the test configuration can also benefit from this; test/libcxx/test/target_info.py manually updates the link line to use -lgcc_s or -lunwind depending on how the build is configured. Presumably, this workaround will no longer be needed if the linker script does that for us?

(Note:- -lgcc_s is linked twice there to satisfy some weird dependency, it's not a mistake)

[mgorny]

In D25008#555041, @rmaprath wrote:

The patch generally makes sense to me.

I wonder if the test configuration can also benefit from this; test/libcxx/test/target_info.py manually updates the link line to use -lgcc_s or -lunwind depending on how the build is configured. Presumably, this workaround will no longer be needed if the linker script does that for us?

I was originally thinking about that but we don't use the linker script on APPLE and I have no clue how that system is handled or whether it suffers from the same issue at all.

(Note:- -lgcc_s is linked twice there to satisfy some weird dependency, it's not a mistake)

Thanks for letting me know. I almost prepared a patch for that ;-D.

[mgorny]

Side note: I see that gcc is solving similar problem via .spec files for its libraries, e.g.:

$ cat /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libgomp.spec 
# This spec file is read by gcc when linking.  It is used to specify the
# standard libraries we need in order to link with libgomp.
*link_gomp: -lgomp %{static: -ldl }

If we wanted to be more portable, I guess we could establish something similar for clang, and install such a file along with libc++. However, I don't know if that's a problem.

[mgorny]

I will be reworking this to use CMake's PRIVATE and PUBLIC linked libs consistently across the build, linker scripts and possibly tests.

@EricWF, would you prefer if I killed the add_library_flags* macros or extended them to support specifying PRIVATE and PUBLIC libs separately?

[mgorny]

Here's my patch v2. I have to admit it's not very different from the original one but the goal is to provide a more generic interface for future improvement.

[EricWF]

lib/CMakeLists.txt

43

Please handle the special case for Apple here, where it manually re-exports the ABI lib.

82

I don't think libgcc_s should be considered a public library like libunwind because both Clang and GCC link it as a default system library.

168

Should the libraries be added using target_link_libraries(foo PRIVATE ...) and target_link_libraries(foo PUBLIC ...) now?

[mgorny]

lib/CMakeLists.txt

82

However, clang doesn't add it if you use compiler-rt. In which case you are left without an unwinder.

[mgorny]

Apple re-export and PRIVATE/PUBLIC covered.

[mgorny]

As discussed on IRC, limited the unwinder deps to -lunwind, and I'll work on making clang include -lunwind by default.

[EricWF]

One concern I have is being able to specify libraries in the correct order when we have two different lists. However I'm not concerned enough not to try it.

LGTM.

lib/CMakeLists.txt

45

Urg. I think this whole setup is borked on Apple. This was an existing issue though, so you don't have to fix it. this LGTM for now.

[Hahnfeld]

Hi Michal,

this currently breaks for me: -lunwind will find the system default libunwind.so.8 instead of LLVMs libunwind.so.1 which breaks some tests.
This has worked magically before because libc++ and libc++abi where explicitely linked against LLVMs libunwind.so.1. Could we force that in the linker script as well?

Thanks,
Jonas

[mgorny]

I think it'd be technically possible to force a specific SONAME in the linker script but I don't think that's a good idea. Clang should be able to use any libunwind implementation, and libc++ shouldn't really be encoding internal implementation details to the point of specific SONAME.

On the other hand, if we are doing a complete in-tree build of LLVM with libunwind, I guess it'd be reasonable to avoid combining the system library with the just-built library. Assuming we want to force 'our' library, and considering the fact that the tests are specifying all dependencies explicitly anyway, I think the most correct solution for that would be to skip the linker script for tests.

I see two possible solutions for that: either we pass full SONAME of the library when linking tests, or we provide an additional symlink to skip linker script. Gentoo already does the latter, and I wanted to add support for that into libcxx anyway, so that may be the solution. The idea is that besides libc++.so (the linker script), additional libc++_shared.so symlink is installed that references the library directly.

[Hahnfeld]

LIBCXXABI_USE_LLVM_UNWINDER implies to me: "Use LLVM's libunwind whenever you use libc++abi". This has worked until now and I would vote for this to be the right thing to do.

[joerg]

So the historical reason for the libgcc_s dance for glibc was the use of unwinding for thread cancellation. With all the glibc versions I can find, going back to SLES11 SP1, this is no longer the case. To check for yourself, look for _Unwind* symbols in libc.so and libc.a. Only findfde seems to be exposed now.

As such, I think we can avoid all this complexity.

[Hahnfeld]

What are the possibilities to proceed here?

[mgorny]

In D25008#581889, @Hahnfeld wrote:

What are the possibilities to proceed here?

To be honest, I don't have any good idea besides renaming the library. Keeping two non-interchangeable libraries under the same name is bound to fail.

[Hahnfeld]

Please disregard the noise, I think the error lays in the testing of compiler-rt which clears LIBRARY_PATH and therefore doesn't find the right libunwind!

lib/CMakeLists.txt

 
 add_library_flags_if(LIBCXX_ENABLE_STATIC_ABI_LIBRARY "-Wl,--whole-archive" "-Wl,-Bstatic")
 add_library_flags("${LIBCXX_CXX_ABI_LIBRARY}")
 add_library_flags_if(LIBCXX_ENABLE_STATIC_ABI_LIBRARY "-Wl,-Bdynamic" "-Wl,--no-whole-archive")
 
 if (APPLE AND LLVM_USE_SANITIZER)
   if (("${LLVM_USE_SANITIZER}" STREQUAL "Address") OR
       ("${LLVM_USE_SANITIZER}" STREQUAL "Address;Undefined") OR
       ("${LLVM_USE_SANITIZER}" STREQUAL "Undefined;Address"))
     set(LIBFILE "libclang_rt.asan_osx_dynamic.dylib")
   elseif("${LLVM_USE_SANITIZER}" STREQUAL "Undefined")
     set(LIBFILE "libclang_rt.ubsan_osx_dynamic.dylib")
   elseif("${LLVM_USE_SANITIZER}" STREQUAL "Thread")
     set(LIBFILE "libclang_rt.tsan_osx_dynamic.dylib")
   else()
     message(WARNING "LLVM_USE_SANITIZER=${LLVM_USE_SANITIZER} is not supported on OS X")
   endif()
   if (LIBFILE)
     execute_process(COMMAND ${CMAKE_CXX_COMPILER} -print-file-name=lib OUTPUT_VARIABLE LIBDIR RESULT_VARIABLE Result)

 add_library_flags_if(LIBCXX_HAS_RT_LIB rt)
 add_library_flags_if(LIBCXX_HAS_GCC_S_LIB gcc_s)
 add_library_flags_if(LIBCXX_HAVE_CXX_ATOMICS_WITH_LIB atomic)
 
 # Setup flags.
 if (NOT WIN32)
   add_flags_if_supported(-fPIC)
 endif()
 add_link_flags_if_supported(-nodefaultlibs)
 
 if ( APPLE AND (LIBCXX_CXX_ABI_LIBNAME STREQUAL "libcxxabi" OR
                 LIBCXX_CXX_ABI_LIBNAME STREQUAL "none"))
   if (NOT DEFINED LIBCXX_LIBCPPABI_VERSION)
     set(LIBCXX_LIBCPPABI_VERSION "2") # Default value
     execute_process(
       COMMAND xcrun --show-sdk-version
       OUTPUT_VARIABLE sdk_ver
PROPERTIES
       SOVERSION     "${LIBCXX_ABI_VERSION}"
   )
   list(APPEND LIBCXX_TARGETS "cxx_shared")
 endif()
 
 # Build the static library.
 if (LIBCXX_ENABLE_STATIC)
   add_library(cxx_static STATIC $<TARGET_OBJECTS:cxx_objects>)
   target_link_libraries(cxx_static ${LIBCXX_LIBRARIES})
   set_target_properties(cxx_static
     PROPERTIES
       LINK_FLAGS    "${LIBCXX_LINK_FLAGS}"
       OUTPUT_NAME   "c++"
   )
   list(APPEND LIBCXX_TARGETS "cxx_static")
 endif()
 

 # Generate a linker script inplace of a libc++.so symlink. Rerun this command
 # after cxx builds.
 if (LIBCXX_ENABLE_SHARED AND LIBCXX_ENABLE_ABI_LINKER_SCRIPT)
   # Get the name of the ABI library and handle the case where CXXABI_LIBNAME
   # is a target name and not a library. Ex cxxabi_shared.
   set(SCRIPT_ABI_LIBNAME "${LIBCXX_CXX_ABI_LIBRARY}")
   if (SCRIPT_ABI_LIBNAME STREQUAL "cxxabi_shared")
     set(SCRIPT_ABI_LIBNAME "c++abi")
+  elseif(SCRIPT_ABI_LIBNAME STREQUAL "")
+    message(FATAL_ERROR "LIBCXX_ENABLE_ABI_LINKER_SCRIPT can not find an ABI library")
   endif()
+
+  # Prepare a complete list of dependant libraries for linking.
+  set(SCRIPT_LINKED_LIBS "-l${SCRIPT_ABI_LIBNAME}")
+  if (LIBCXXABI_USE_LLVM_UNWINDER)
+    set(SCRIPT_LINKED_LIBS "${SCRIPT_LINKED_LIBS} -lunwind")
+  elseif (LIBCXX_HAS_GCC_S_LIB)
+    set(SCRIPT_LINKED_LIBS "${SCRIPT_LINKED_LIBS} -lgcc_s")
+  endif()
+
   # Generate a linker script inplace of a libc++.so symlink. Rerun this command
   # after cxx builds.
   add_custom_command(TARGET cxx_shared POST_BUILD
     COMMAND
       ${PYTHON_EXECUTABLE} ${LIBCXX_SOURCE_DIR}/utils/gen_link_script/gen_link_script.py
     ARGS
       "$<TARGET_LINKER_FILE:cxx_shared>"
-      "${SCRIPT_ABI_LIBNAME}"
+      "${SCRIPT_LINKED_LIBS}"
     WORKING_DIRECTORY ${LIBCXX_BUILD_DIR}
   )
 endif()
 
 if (LIBCXX_INSTALL_LIBRARY)
   if (LIBCXX_INSTALL_EXPERIMENTAL_LIBRARY)
     set(experimental_lib cxx_experimental)
   endif()

utils/gen_link_script/gen_link_script.py

 import os
 import sys
 
 def print_and_exit(msg):
     sys.stderr.write(msg + '\n')
     sys.exit(1)
 
 def usage_and_exit():
-    print_and_exit("Usage: ./gen_link_script.py [--help] [--dryrun] <path/to/libcxx.so> <abi_libname>")
+    print_and_exit("Usage: ./gen_link_script.py [--help] [--dryrun] <path/to/libcxx.so> <libs>")
 
 def help_and_exit():
     help_msg = \
 """Usage
 
-  gen_link_script.py [--help] [--dryrun] <path/to/libcxx.so> <abi_libname>
+  gen_link_script.py [--help] [--dryrun] <path/to/libcxx.so> <libs>
 
-  Generate a linker script that links libc++ to the proper ABI library.
+  Generate a linker script that links libc++ to the proper ABI library
+  (and other dependant libraries).
   The script replaces the specified libc++ symlink.
   An example script for c++abi would look like "INPUT(libc++.so.1 -lc++abi)".
 
 Arguments
   <path/to/libcxx.so> - The top level symlink to the versioned libc++ shared
                         library. This file is replaced with a linker script.
-  <abi_libname>       - The name of the ABI library to use in the linker script.
-                        The name must be one of [c++abi, stdc++, supc++, cxxrt].
+  <libs>              - The list of libraries (-l flags) to use in the linker
+                        script.
 
 Exit Status:
   0 if OK,
   1 if the action failed.
 """
     print_and_exit(help_msg)
 
 def parse_args():
     args = list(sys.argv)
     del args[0]
     if len(args) == 0:
         usage_and_exit()
     if args[0] == '--help':
         help_and_exit()
     dryrun = '--dryrun' == args[0]
     if dryrun:
         del args[0]
     if len(args) != 2:
         usage_and_exit()
     symlink_file = args[0]
-    abi_libname = args[1]
-    return dryrun, symlink_file, abi_libname
+    abi_libs = args[1]
+    return dryrun, symlink_file, abi_libs
 
 def main():
-    dryrun, symlink_file, abi_libname = parse_args()
+    dryrun, symlink_file, abi_libs = parse_args()
 
     # Check that the given libc++.so file is a valid symlink.
     if not os.path.islink(symlink_file):
         print_and_exit("symlink file %s is not a symlink" % symlink_file)
 
     # Read the symlink so we know what libc++ to link to in the linker script.
     linked_libcxx = os.readlink(symlink_file)
 
-    # Check that the abi_libname is one of the supported values.
-    supported_abi_list = ['c++abi', 'stdc++', 'supc++', 'cxxrt']
-    if abi_libname not in supported_abi_list:
-        print_and_exit("abi name '%s' is not supported: Use one of %r" %
-                        (abi_libname, supported_abi_list))
-
     # Generate the linker script contents and print the script and destination
     # information.
-    contents = "INPUT(%s -l%s)" % (linked_libcxx, abi_libname)
+    contents = "INPUT(%s %s)" % (linked_libcxx, abi_libs)
     print("GENERATING SCRIPT: '%s' as file %s" % (contents, symlink_file))
 
     # Remove the existing libc++ symlink and replace it with the script.
     if not dryrun:
         os.unlink(symlink_file)
         with open(symlink_file, 'w') as f:
             f.write(contents + "\n")
 
 
 if __name__ == '__main__':
     main()