[CMake] Do not use libtool on Apple platforms when building LLVM with (full) LTO.

[fhahn]

Summary

libtool does not support bitcode files generated by upstream LLVM in the latest
toolchains. When building LLVM with full LTO, we should use llvm-ranlib/llvm-ar.

I am not sure if this is the best way to solve the underlying problem:
being able to do a stage2 bootstrap build of LLVM master with LTO on
Darwin. Unfortunately, CMAKE_AR/CMAKE_RANLIB are automatically populated
if they are not provided. Otherwise we could limit avoiding using
libtool only when the user specified CMAKE_AR/CMAKE_RANLIB manually with LTO.

Any thoughts very appreciated!

[dexonsmith]

Shouldn't this just send DYLD_LIBRARY_PATH through to the second stage, so that ar and ranlib use the just-built libLTO.dylib?

[mehdi_amini]

llvm/CMakeLists.txt

55

Why is this specific to full LTO? Isn't the same mechanism used for ThinLTO to build the symbol table for the archive?

[steven_wu]

If you use cmake variable CLANG_ENABLE_BOOTSTRAP, it should take care of DYLD_LIBRARY_PATH so libtool is using the just built libLTO to create static library. The setup of the DYLD_LIBRARY_PATH is just below.

llvm/CMakeLists.txt

97

This sets up DYLD_LIBRARY_PATH

[fhahn]

Thanks for taking a look so quickly! I'll look into how to best set-up DYLD_LIBRARY_PATH in that case. I am after a more lightweight solution than CLANG_ENABLE_BOOTSTRAP, i.e. creating/configuring the separate build stages manually.

[steven_wu]

When you are manually configuring the bootstrap, I still don't think trying to bypass libtool is the correct fix because you still need to specify the correct llvm-ar to use. The default CMAKE_AR inferred by cmake will not work. Maybe it is an option to add the control to force llvm-ar if wished, then you have two options when you manually configuring bootstrap:

• DYLD_LIBRARY_PATH points to libLTO.dylib directory
• CMAKE_AR points to just built llvm-ar and set the variable to force llvm-ar

I don't think the second one is easier than the first one.

[fhahn]

Thanks for the feedback, setting DYLD_LIBRARY_PATH as suggested does what I want :)

llvm/CMakeLists.txt

 
 if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
   message(STATUS "No build type selected, default to Debug")
   set(CMAKE_BUILD_TYPE "Debug" CACHE STRING "Build type (default Debug)" FORCE)
 endif()
 
 # This should only apply if you are both on an Apple host, and targeting Apple.
 if(CMAKE_HOST_APPLE AND APPLE)
+
+  # Only use libtool when not doing full LTO. When bootstrapping LLVM master
+  # with LTO, we have to use llvm-ranlib/llvm-ar, as libtool may not support
+  # the newest LLVM bitcode version.
+  string(TOUPPER "${LLVM_ENABLE_LTO}" uppercase_LLVM_ENABLE_LTO)
+  if(NOT uppercase_LLVM_ENABLE_LTO STREQUAL "FULL" OR CMAKE_AR_val)
       # if CMAKE_LIBTOOL is not set, try and find it with xcrun or find_program
       if(NOT CMAKE_LIBTOOL)
         if(NOT CMAKE_XCRUN)
           find_program(CMAKE_XCRUN NAMES xcrun)
         endif()
         if(CMAKE_XCRUN)
           execute_process(COMMAND ${CMAKE_XCRUN} -find libtool
             OUTPUT_VARIABLE CMAKE_LIBTOOL
             OUTPUT_STRIP_TRAILING_WHITESPACE)
         endif()
 
         if(NOT CMAKE_LIBTOOL OR NOT EXISTS CMAKE_LIBTOOL)
           find_program(CMAKE_LIBTOOL NAMES libtool)
         endif()
       endif()
 
       get_property(languages GLOBAL PROPERTY ENABLED_LANGUAGES)
       if(CMAKE_LIBTOOL)
         set(CMAKE_LIBTOOL ${CMAKE_LIBTOOL} CACHE PATH "libtool executable")
         message(STATUS "Found libtool - ${CMAKE_LIBTOOL}")
 
         execute_process(COMMAND ${CMAKE_LIBTOOL} -V
           OUTPUT_VARIABLE LIBTOOL_V_OUTPUT
           OUTPUT_STRIP_TRAILING_WHITESPACE)
         if("${LIBTOOL_V_OUTPUT}" MATCHES ".*cctools-([0-9.]+).*")
           string(REGEX REPLACE ".*cctools-([0-9.]+).*" "\\1" LIBTOOL_VERSION
             ${LIBTOOL_V_OUTPUT})
           if(NOT LIBTOOL_VERSION VERSION_LESS "862")
             set(LIBTOOL_NO_WARNING_FLAG "-no_warning_for_no_symbols")
           endif()
         endif()
 
         foreach(lang ${languages})
           set(CMAKE_${lang}_CREATE_STATIC_LIBRARY
             "\"${CMAKE_LIBTOOL}\" -static ${LIBTOOL_NO_WARNING_FLAG} -o <TARGET> \
             <LINK_FLAGS> <OBJECTS> ")
         endforeach()
       endif()
+  endif()
 
   # If DYLD_LIBRARY_PATH is set we need to set it on archiver commands
   if(DYLD_LIBRARY_PATH)
     set(dyld_envar "DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}")
     foreach(lang ${languages})
       foreach(cmd ${CMAKE_${lang}_CREATE_STATIC_LIBRARY})
         list(APPEND CMAKE_${lang}_CREATE_STATIC_LIBRARY_NEW
              "${dyld_envar} ${cmd}")
       endforeach()
       set(CMAKE_${lang}_CREATE_STATIC_LIBRARY
         ${CMAKE_${lang}_CREATE_STATIC_LIBRARY_NEW})