Adding llvm-shlib to CMake build system with a few new bells and whistles

[beanz]

Summary

This patch adds a new CMake build setting LLVM_BUILD_LLVM_DYLIB, which defaults to OFF. When set to ON, this will generate a shared library containing most of LLVM. The contents of the shared library can be overriden by specifying LLVM_DYLIB_COMPONENTS. LLVM_DYLIB_COMPONENTS can be set to a semi-colon delimited list of any LLVM components that you llvm-config can resolve.

On Windows, unless you are using Cygwin, you must specify an explicit symbol export file using LLVM_EXPORTED_SYMBOL_FILE. On Cygwin and all unix-like platforms if you do not specify LLVM_EXPORTED_SYMBOL_FILE, an export file containing only the LLVM C API will be auto-generated from the list of LLVM components specified in LLVM_DYLIB_COMPONENTS.

[rnk]

At a really high level, I wonder why we don't annotate the entire LLVM C API as attribute((visibility("default"))) / dllexport / whatever is needed on MachO.

tools/llvm-shlib/CMakeLists.txt

54 ↗

(On Diff #15202)

I'm not sure if this is always correct with funky generators like VS and XCode.

59 ↗

(On Diff #15202)

Linux (and other OSs) won't add this leading underscore to functions with extern "C" linkage, so nm's output will look like 'T LLVM*'.

59 ↗

(On Diff #15202)

It looks like you can skip the removal of leading underscores, add_llvm_symbol_exports appears to do this for you on Mac.

[beanz]

I'm not really sure tagging the C API with default visibility would solve the problem I'm trying to solve. Marking the C API as exported, doesn't mark the C++ API to not be exported - which is what I'm trying to do here.

New patches coming soon, and comments below.

tools/llvm-shlib/CMakeLists.txt

54 ↗

(On Diff #15202)

I'm not too concerned about VS because generally Windows support will need to be done by someone else. I'm a bit weak WRT Windows development. I'll make sure this does the right thing for Xcode though. I mostly work with SublimeText & Ninja.

59 ↗

(On Diff #15202)

Good catch. on both of these. I'll update the patches.

[beanz]

tools/llvm-shlib/CMakeLists.txt

59 ↗

(On Diff #15202)

Looking at this further. I do actually need to strip the _. The sed line in add_llvm_symbol_exports actually adds a leading _ so that darwin can be compatible with Linux & Windows export lists that don't have leading underscores.

[beanz]

Fixed generation for Xcode. This should work for VS too if someone wants to take the time to implement the export list generation. I also caught an error I overlooked. I was inadvertently generating an empty static library along side the dylib.

[rnk]

lgtm

I'm sure the awk is broken on some platforms, but this is a good start and we can let people try it and fix it as needed.

tools/llvm-shlib/CMakeLists.txt

1 ↗

(On Diff #15295)

Can you add a short comment explaining what this library is? It's not as readily apparent as the other tools.

67 ↗

(On Diff #15295)

Our CMake generally doesn't repeat the contents of the 'if' and 'foreach' clauses. This one doesn't match, so I'd nuke it and the others.

Closed by commit rL220490 (authored by cbieneman).

[chapuni]

IMO, I wonder the term "DYLIB" here might be odd. I mis-imagined this were darwin-specific stuff at first.

That said, I don't have better idea for now.

• Configure provides --enable-shared
• CMakefiles provides BUILD_SHARED_LIBS.
• FYI, in past, I tried to introduce LLVM_ENABLE_SHARED. (See D2942).

[beanz]

I'm not particularly attached to any specific naming. The problem however is that all three major OS types have different nomenclature for the same construct. On Linux they are shared objects (so), on Windows they are dynamically linked libraries (dll), and on Darwin things are even more confusing because they are referred to in documentation as shared libraries, but the extension is dylib which refers to the dynamic linking aspect.

That said, I do see your point that dylib does sound awfully "Darwin". An alternative naming could be LLVM_BUILD_LLVM_SHLIB, which would more clearly map to the llvm-shlib "tool" which generates the library.

[rnk]

I try to use "DSO" as an OS-neutral term, but it stands for "dynamic shared
object", so it still shows Unix roots.

[chapuni]

FYI, it should work also on mingw.

• binutils nm.exe is shipped by most mingw distros. You may assume nm.exe available also on mingw.
• It works if awk is available. I have gawk.exe in my msys tree.

I guess it would also work on msvc with a few tweaks.

I suggest to avoid awk|sort|sed. Could we use python here? Just only python is required to build the tree.
I will work for that.

llvm/trunk/CMakeLists.txt

 option (LLVM_BUILD_DOCS "Build the llvm documentation." OFF)
 option (LLVM_INCLUDE_DOCS "Generate build targets for llvm documentation." ON)
 option (LLVM_ENABLE_DOXYGEN "Use doxygen to generate llvm API documentation." OFF)
 option (LLVM_ENABLE_SPHINX "Use Sphinx to generate llvm documentation." OFF)
 
 option (LLVM_BUILD_EXTERNAL_COMPILER_RT
   "Build compiler-rt as an external project." OFF)
 
+option(LLVM_BUILD_LLVM_DYLIB "Build libllvm dynamic library" OFF)
+
 # All options referred to from HandleLLVMOptions have to be specified
 # BEFORE this include, otherwise options will not be correctly set on
 # first cmake run
 include(config-ix)
 
 # By default, we target the host, but this can be overridden at CMake
 # invocation time.
 set(LLVM_DEFAULT_TARGET_TRIPLE "${LLVM_HOST_TRIPLE}" CACHE STRING

llvm/trunk/tools/CMakeLists.txt

if( (CMAKE_SYSTEM_NAME STREQUAL "Linux")
     add_llvm_tool_subdirectory(gold)
   else()
     ignore_llvm_tool_subdirectory(gold)
   endif()
 else()
   ignore_llvm_tool_subdirectory(gold)
 endif()
 
+if( LLVM_BUILD_LLVM_DYLIB )
+  add_llvm_tool_subdirectory(llvm-shlib)
+else()
+  ignore_llvm_tool_subdirectory(llvm-shlib)
+endif()
+
 add_llvm_external_project(clang)
 
 if( NOT LLVM_INCLUDE_TOOLS STREQUAL "bootstrap-only" )
   add_llvm_external_project(lld)
   add_llvm_external_project(lldb)
 
   # Automatically add remaining sub-directories containing a 'CMakeLists.txt'
   # file as external projects.
   add_llvm_implicit_external_projects()
 endif()
 
 set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} PARENT_SCOPE)

llvm/trunk/tools/llvm-shlib/CMakeLists.txt

+# This tool creates a shared library from the LLVM libraries. Generating this
+# library is enabled by setting LLVM_BUILD_LLVM_DYLIB=yes on the CMake
+# commandline. By default the shared library only exports the LLVM C API.
+
+
+# You can configure which libraries from LLVM you want to include in the shared
+# library by setting LLVM_DYLIB_COMPONENTS to a semi-colon delimited list of
+# LLVM components. All compoenent names handled by llvm-config are valid.
+
+if(NOT DEFINED LLVM_DYLIB_COMPONENTS)
+  set(LLVM_LINK_COMPONENTS
+    ${LLVM_TARGETS_TO_BUILD}
+    Analysis
+    AsmPrinter
+    BitWriter
+    CodeGen
+    Core
+    ExecutionEngine
+    IPA
+    IPO
+    IRReader
+    InstCombine
+    Instrumentation
+    Interpreter
+    Linker
+    MC
+    MCJIT
+    ObjCARCOpts
+    Object
+    ScalarOpts
+    SelectionDAG
+    Support
+    Target
+    TransformUtils
+    Vectorize
+    native
+    )
+else()
+  set(LLVM_LINK_COMPONENTS ${LLVM_DYLIB_COMPONENTS})
+endif()
+
+add_definitions( -DLLVM_VERSION_INFO=\"${PACKAGE_VERSION}\" )
+
+set(SOURCES
+  libllvm.cpp
+  )
+
+if(NOT DEFINED LLVM_EXPORTED_SYMBOL_FILE)
+
+  if( WIN32 AND NOT CYGWIN )
+    message(FATAL_ERROR "Auto-generation not implemented for Win32 without GNU utils. Please specify LLVM_EXPORTED_SYMBOL_FILE.")
+  endif()
+
+  # To get the export list for a single llvm library:
+  # nm ${LIB_PATH} | awk "/T _LLVM/ { print $3 }" | sort -u | sed -e "s/^_//g" > ${LIB_PATH}.exports
+
+  set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_BINARY_DIR}/libllvm.exports)
+
+  llvm_map_components_to_libnames(LIB_NAMES ${LLVM_LINK_COMPONENTS})
+
+  foreach (lib ${LIB_NAMES})
+    
+    set(LIB_DIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib)
+    set(LIB_NAME ${LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${lib})
+    set(LIB_PATH ${LIB_NAME}${CMAKE_STATIC_LIBRARY_SUFFIX})
+    set(LIB_EXPORTS_PATH ${LIB_NAME}.exports)
+
+    list(APPEND LLVM_DYLIB_REQUIRED_EXPORTS ${LIB_EXPORTS_PATH})
+
+    add_custom_command(OUTPUT ${LIB_EXPORTS_PATH}
+      COMMAND nm ${LIB_PATH} | awk "/T _LLVM/ || /T LLVM/ { print $3 }" | sort -u | sed -e "s/^_//g" > ${LIB_EXPORTS_PATH}
+      WORKING_DIRECTORY ${LIB_DIR}
+      DEPENDS ${lib}
+      COMMENT "Generating Export list for ${lib}..."
+      VERBATIM )
+  endforeach ()
+
+  add_custom_command(OUTPUT ${LLVM_EXPORTED_SYMBOL_FILE}
+    COMMAND cat ${LLVM_DYLIB_REQUIRED_EXPORTS} > ${LLVM_EXPORTED_SYMBOL_FILE}
+    WORKING_DIRECTORY ${LIB_DIR}
+    DEPENDS ${LLVM_DYLIB_REQUIRED_EXPORTS}
+    COMMENT "Generating combined export list...")
+
+endif()
+
+add_llvm_library(LLVM SHARED ${SOURCES})
+
+add_dependencies(LLVM ${LLVM_EXPORTED_SYMBOL_FILE})
+
+if (APPLE)
+  set_property(TARGET LLVM APPEND_STRING PROPERTY
+              LINK_FLAGS
+              " -compatibility_version ${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR} -current_version ${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}")
+endif()
+

llvm/trunk/tools/llvm-shlib/libllvm.cpp

+//===-libllvm.cpp - LLVM Shared Library -----------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is empty and serves only the purpose of making CMake happy because
+// you can't define a target with no sources.
+//
+//===----------------------------------------------------------------------===//