[cmake] Always build the libLLVM shared library

[tstellar]

Summary

We discourage non-developers from using the BUILD_SHARED_LIBS option
and recommend using LLVM_BUILD_LLVM_DYLIB=ON for building LLVM as
a shared library. We should make it as easy as possible for
users to build with our recommended configuration, so to help with that
this patch makes building libLLVM.so the default and removes the
LLVM_BUILD_LLVM_DYLIB option altogether.

Removing this option will help simplify the build system and also
help downstream users of LLVM, because they won't need to add
special logic to their own build systems to detect whether or not libLLVM
was built and installed.

[lebedev.ri]

llvm/docs/CMake.rst

568

What happens to this limitation?
Does -DBUILD_SHARED_LIBS=ON begins to just work?

[compnerd]

I'm kinda torn on this. Although this simplifies the build, linking the DSO on Linux is painful unless you externalise the debug info (i.e. build with fission). On the other hand, people should be encouraged to setup the distributions and use that to build the full set that they need. I think that marking libLLVM as EXCLUDE_FROM_ALL is a decent compromise.

[tstellar]

In D70026#1740962, @compnerd wrote:

I'm kinda torn on this. Although this simplifies the build, linking the DSO on Linux is painful unless you externalise the debug info (i.e. build with fission). On the other hand, people should be encouraged to setup the distributions and use that to build the full set that they need. I think that marking libLLVM as EXCLUDE_FROM_ALL is a decent compromise.

Is it more painful than linking any other tool in the tools/ dir?

[beanz]

I very much dislike using EXCLUDE_FROM_ALL. I really believe all should be all, not most things I think are important. In general iteration cycles the check-* targets won't build or link libLLVM unless you specify LLVM_LINK_LLVM_DYLIB, and if you actually want to build everything as a means of testing your changes the all target should do that.

On the specifics of this change I think there are some problems because of CMake's lack of cache invalidation. Changing this variable won't actually change anything until people delete their build directories.

Instead if you remove all uses of the variable, and you could add:

if (MSVC or BUILD_SHARED_LIBS)
  if (LLVM_LINK_LLVM_DYLIB)
    message(FATAL_ERROR "LLVM_LINK_LLVM_DYLIB is not supported with BUILD_SHARED_LIBS or using MSVC")
  endif()
  return()
endif()

to llvm-shlib/CMakelists.txt to handle Win32, and it should have roughly the same impact without cache invalidation problems.

[tstellar]

llvm/CMakeLists.txt

573–577

@beanz Does setting the variables here override the cache variables?

[smeenai]

llvm/CMakeLists.txt

573–577

Yup, setting the variable here will override the cached value. Any reason to prefer this to just removing all uses of the variable though?

[tstellar]

llvm/CMakeLists.txt

573–577

I thought it would be cleaner to keep the variable than to replace it with if (MSVC) everywhere.

[tstellar]

llvm/docs/CMake.rst

568

D70179 will fix this.

[beanz]

llvm/CMakeLists.txt

573–577

There is really only one place the if (MSVC) check would be needed, in the llvm-shlib/CMakeLists.txt file. All the other uses of the variable could disappear.

[tstellar]

Remove all uses of the LLVM_BUILD_LLVM_DYLIB variable.

[tstellar]

llvm/CMakeLists.txt

651–652

This change I'm not so sure about. Can we drop the WIN32 part of the condition? Does CYGWIN support building shared libs?

[tstellar]

Ping. All the prerequisite patches have been committed now.

[compnerd]

llvm/CMakeLists.txt

651–652

Why the desire to drop WIN32? Both Windows and cygwin support shared libraries.

llvm/tools/llvm-config/CMakeLists.txt

60

I think that this should really be NOT CMAKE_SYSTEM_NAME STREQUAL Windows. I can build for windows using clang (not clang-cl) in which case, we get the value wrong.

llvm/tools/llvm-shlib/CMakeLists.txt

12

Similar

[tstellar]

Use correct checks for Windows.

[merge_guards_bot]

Build result: FAILURE - Could not check out parent git hash "6f585e1ccfdce0c807fc2844e5c9cc3e6694101d". It was not found in the repository. Did you configure the "Parent Revision" in Phabricator properly? Trying to apply the patch to the master branch instead...

ERROR: arc patch failed with error code 1. Check build log for details.
Log files: console-log.txt, CMakeCache.txt

[tstellar]

llvm/CMakeLists.txt

651–652

I just wasn't sure what WIN32 meant. Does it mean all of windows or just 32-bit windows? I'm fine with keeping it.

[compnerd]

llvm/CMakeLists.txt

651–652

Yes, it means all windows builds.

[smeenai]

Anything holding this up?

Resigning as reviewer to remove this from my queue.

llvm/CMakeLists.txt

set(LLVM_DYLIB_COMPONENTS "all" CACHE STRING
     "Semicolon-separated list of components to include in libLLVM, or \"all\".")
 endif()
 
 if(MSVC)
   option(LLVM_BUILD_LLVM_C_DYLIB "Build LLVM-C.dll (Windows only)" ON)
   # Set this variable to OFF here so it can't be set with a command-line
   # argument.
   set (LLVM_LINK_LLVM_DYLIB OFF)
   if (BUILD_SHARED_LIBS)
     message(FATAL_ERROR "BUILD_SHARED_LIBS options is not supported on Windows.")
   endif()
 else()
   option(LLVM_LINK_LLVM_DYLIB "Link tools against the libllvm dynamic library" OFF)
   option(LLVM_BUILD_LLVM_C_DYLIB "Build libllvm-c re-export library (Darwin only)" OFF)
-  set(LLVM_BUILD_LLVM_DYLIB_default OFF)
-  if(LLVM_LINK_LLVM_DYLIB OR LLVM_BUILD_LLVM_C_DYLIB)
-    set(LLVM_BUILD_LLVM_DYLIB_default ON)
-  endif()
-  option(LLVM_BUILD_LLVM_DYLIB "Build libllvm dynamic library" ${LLVM_BUILD_LLVM_DYLIB_default})
 endif()
 
 if (LLVM_LINK_LLVM_DYLIB AND BUILD_SHARED_LIBS)
   message(FATAL_ERROR "Cannot enable BUILD_SHARED_LIBS with LLVM_LINK_LLVM_DYLIB.  We recommend disabling BUILD_SHARED_LIBS.")
 endif()
 
 option(LLVM_OPTIMIZED_TABLEGEN "Force TableGen to be built with optimization" OFF)
 if(CMAKE_CROSSCOMPILING OR (LLVM_OPTIMIZED_TABLEGEN AND (LLVM_ENABLE_ASSERTIONS OR CMAKE_CONFIGURATION_TYPES)))

 # By default, we target the host, but this can be overridden at CMake
 # invocation time.
 set(LLVM_DEFAULT_TARGET_TRIPLE "${LLVM_HOST_TRIPLE}" CACHE STRING
   "Default target for which LLVM will generate code." )
 set(TARGET_TRIPLE "${LLVM_DEFAULT_TARGET_TRIPLE}")
 message(STATUS "LLVM host triple: ${LLVM_HOST_TRIPLE}")
 message(STATUS "LLVM default target triple: ${LLVM_DEFAULT_TARGET_TRIPLE}")
 
 if(WIN32 OR CYGWIN)
-  if(BUILD_SHARED_LIBS OR LLVM_BUILD_LLVM_DYLIB)
+  if(BUILD_SHARED_LIBS)
     set(LLVM_ENABLE_PLUGINS_default ON)
   else()
     set(LLVM_ENABLE_PLUGINS_default OFF)
   endif()
 else()
   set(LLVM_ENABLE_PLUGINS_default ${LLVM_ENABLE_PIC})
 endif()
 option(LLVM_ENABLE_PLUGINS "Enable plugin support" ${LLVM_ENABLE_PLUGINS_default})

llvm/docs/BuildingADistribution.rst

 distributed pieces without needing to build all configured targets.
 
 Special Notes for Library-only Distributions
 --------------------------------------------
 
 One of the most powerful features of LLVM is its library-first design mentality
 and the way you can compose a wide variety of tools using different portions of
 LLVM. Even in this situation using *BUILD_SHARED_LIBS* is not supported. If you
-want to distribute LLVM as a shared library for use in a tool, the recommended
-method is using *LLVM_BUILD_LLVM_DYLIB*, and you can use *LLVM_DYLIB_COMPONENTS*
-to configure which LLVM components are part of libLLVM.
-Note: *LLVM_BUILD_LLVM_DYLIB* is not available on Windows.
+want to distribute LLVM as a shared library then distribute the libLLVM.so library.
 
 Options for Optimizing LLVM
 ===========================
 
 There are four main build optimizations that our CMake build system supports.
 When performing a bootstrap build it is not beneficial to do anything other than
 setting *CMAKE_BUILD_TYPE* to ``Release`` for the stage-1 compiler. This is
 because the more intensive optimizations are expensive to perform and the

 
 The simplest and least significant way to reduce binary size is to set the
 *CMAKE_BUILD_TYPE* variable to ``MinSizeRel``, which will set the compiler
 optimization level to ``-Os`` which optimizes for binary size. This will have
 both the least benefit to size and the least impact on performance.
 
 The most impactful way to reduce binary size is to dynamically link LLVM into
 all the tools. This reduces code size by decreasing duplication of common code
-between the LLVM-based tools. This can be done by setting the following two
-CMake options to ``On``: *LLVM_BUILD_LLVM_DYLIB* and *LLVM_LINK_LLVM_DYLIB*.
+between the LLVM-based tools. This can be done by setting the following
+CMake option to ``On``: *LLVM_LINK_LLVM_DYLIB*.
 
 .. warning::
   Distributions should never be built using the *BUILD_SHARED_LIBS* CMake
   option. (:ref:`See the warning above for more explanation <shared_libs>`.).
 
 Relevant CMake Options
 ======================
 
 This section provides documentation of the CMake options that are intended to
 help construct distributions. This is not an exhaustive list, and many
 additional options are documented in the :doc:`CMake` page. Some key options
 that are already documented include: *LLVM_TARGETS_TO_BUILD*,
-*LLVM_ENABLE_PROJECTS*, *LLVM_BUILD_LLVM_DYLIB*, and *LLVM_LINK_LLVM_DYLIB*.
+*LLVM_ENABLE_PROJECTS*, and *LLVM_LINK_LLVM_DYLIB*.
 
 **LLVM_ENABLE_RUNTIMES**:STRING
   When building a distribution that includes LLVM runtime projects (i.e. libcxx,
   compiler-rt, libcxxabi, libunwind...), it is important to build those projects
   with the just-built compiler.
 
 **LLVM_DISTRIBUTION_COMPONENTS**:STRING
   This variable can be set to a semi-colon separated list of LLVM build system

llvm/docs/CMake.rst

**LLVM_INSTALL_OCAMLDOC_HTML_DIR**:STRING
   `share/doc/llvm/ocaml-html`.
 
 **LLVM_CREATE_XCODE_TOOLCHAIN**:BOOL
   macOS Only: If enabled CMake will generate a target named
   'install-xcode-toolchain'. This target will create a directory at
   $CMAKE_INSTALL_PREFIX/Toolchains containing an xctoolchain directory which can
   be used to override the default system tools.
 
-**LLVM_BUILD_LLVM_DYLIB**:BOOL
-  If enabled, the target for building the libLLVM shared library is added.
-  This library contains all of LLVM's components in a single shared library.
-  Defaults to OFF. This cannot be used in conjunction with BUILD_SHARED_LIBS.
-  Tools will only be linked to the libLLVM shared library if LLVM_LINK_LLVM_DYLIB
-  is also ON.
-  The components in the library can be customised by setting LLVM_DYLIB_COMPONENTS
-  to a list of the desired components.
-  This option is not available on Windows.
-
 **LLVM_LINK_LLVM_DYLIB**:BOOL
   If enabled, tools will be linked with the libLLVM shared library. Defaults
-  to OFF. Setting LLVM_LINK_LLVM_DYLIB to ON also sets LLVM_BUILD_LLVM_DYLIB
-  to ON.
+  to OFF.
   This option is not available on Windows.
 
 **BUILD_SHARED_LIBS**:BOOL
   Flag indicating if each LLVM component (e.g. Support) is built as a shared
   library (ON) or as a static library (OFF). Its default value is OFF. On
   Windows, shared libraries may be used when building with MinGW, including
   mingw-w64, but not when building with the Microsoft toolchain.
 
   .. note:: BUILD_SHARED_LIBS is only recommended for use by LLVM developers.
-            If you want to build LLVM as a shared library, you should use the
-            ``LLVM_BUILD_LLVM_DYLIB`` option.
+            If you want to link against shared libraries, use libLLVM.so, which
+            is always available.
 
 **LLVM_OPTIMIZED_TABLEGEN**:BOOL
   If enabled and building a debug or asserts build the CMake build system will
   generate a Release build tree to build a fully optimized tablegen for use
   during the build. Enabling this option can significantly speed up build times
   especially when building LLVM in Debug configurations.
 
 **LLVM_REVERSE_ITERATION**:BOOL

llvm/docs/GettingStarted.rst

 |                         | include: clang, libcxx, libcxxabi, libunwind, lldb,|
 |                         | compiler-rt, lld, polly, or debuginfo-tests.       |
 +-------------------------+----------------------------------------------------+
 | LLVM_ENABLE_SPHINX      | Build sphinx-based documentation from the source   |
 |                         | code. This is disabled by default because it is    |
 |                         | slow and generates a lot of output. Sphinx version |
 |                         | 1.5 or later recommended.                          |
 +-------------------------+----------------------------------------------------+
-| LLVM_BUILD_LLVM_DYLIB   | Generate libLLVM.so. This library contains a       |
-|                         | default set of LLVM components that can be         |
-|                         | overridden with ``LLVM_DYLIB_COMPONENTS``. The     |
-|                         | default contains most of LLVM and is defined in    |
-|                         | ``tools/llvm-shlib/CMakelists.txt``. This option is|
-|                         | not avialable on Windows.                          |
-+-------------------------+----------------------------------------------------+
 | LLVM_OPTIMIZED_TABLEGEN | Builds a release tablegen that gets used during    |
 |                         | the LLVM build. This can dramatically speed up     |
 |                         | debug builds.                                      |
 +-------------------------+----------------------------------------------------+
 
 To configure LLVM, follow these steps:
 
 #. Change directory into the object root directory:

llvm/docs/ReleaseNotes.rst

* Enabled MemorySSA as a loop dependency. Since
   with a small number of overall memory instructions. This choice was motivated
   by experimental results showing compile and run time benefits or replacing the
   AliasSetTracker usage with MemorySSA without any performance penalties.
   The fact that MemorySSA is now preserved by and available in a series of loop
   passes, also opens up opportunities for its use in those respective passes.
 * The BasicBlockPass, BBPassManager and all their uses were deleted in
   `this revision <https://reviews.llvm.org/rG9f0ff0b2634bab6a5be8dace005c9eb24d386dd1>`_.
 
-* The LLVM_BUILD_LLVM_DYLIB and LLVM_LINK_LLVM_DYLIB CMake options are no longer
-  available on Windows.
+* The LLVM_LINK_LLVM_DYLIB CMake option is no longer available on Windows.
+
+* The LLVM_BUILD_LLVM_DYLIB option has been removed.  The libLLVM shared object
+  is now always built on all platforms except Windows, so this option is no longer
+  necessary.
 
 .. NOTE
    If you would like to document a larger change, then you can add a
    subsection about it right here. You can copy the following boilerplate
    and un-indent it (the indentation causes it to be inside this comment).
 
    Special New Feature
    -------------------

llvm/tools/CMakeLists.txt

 # Build polly before the tools: the tools link against polly when
 # LINK_POLLY_INTO_TOOLS is set.
 if(WITH_POLLY)
   add_llvm_external_project(polly)
 else()
   set(LLVM_TOOL_POLLY_BUILD Off)
 endif()
 
-if(NOT LLVM_BUILD_LLVM_DYLIB AND NOT LLVM_BUILD_LLVM_C_DYLIB)
-  set(LLVM_TOOL_LLVM_SHLIB_BUILD Off)
-endif()
-
 if(NOT LLVM_USE_INTEL_JITEVENTS )
   set(LLVM_TOOL_LLVM_JITLISTENER_BUILD Off)
 endif()
 
 if(CYGWIN OR NOT LLVM_ENABLE_PIC)
   set(LLVM_TOOL_LTO_BUILD Off)
 endif()
 

llvm/tools/llvm-config/BuildVariables.inc.in

 #define LLVM_LDFLAGS "@LLVM_LDFLAGS@"
 #define LLVM_CXXFLAGS "@LLVM_CXXFLAGS@"
 #define LLVM_BUILDMODE "@LLVM_BUILDMODE@"
 #define LLVM_LIBDIR_SUFFIX "@LLVM_LIBDIR_SUFFIX@"
 #define LLVM_TARGETS_BUILT "@LLVM_TARGETS_BUILT@"
 #define LLVM_SYSTEM_LIBS "@LLVM_SYSTEM_LIBS@"
 #define LLVM_BUILD_SYSTEM "@LLVM_BUILD_SYSTEM@"
 #define LLVM_HAS_RTTI @LLVM_HAS_RTTI@
-#define LLVM_ENABLE_DYLIB @LLVM_BUILD_LLVM_DYLIB@
+#define LLVM_ENABLE_DYLIB @LLVM_ENABLE_DYLIB@
 #define LLVM_LINK_DYLIB @LLVM_LINK_LLVM_DYLIB@
 #define LLVM_ENABLE_SHARED @BUILD_SHARED_LIBS@
 #define LLVM_DYLIB_COMPONENTS "@LLVM_DYLIB_COMPONENTS_expanded@"
 #define LLVM_DYLIB_VERSION "@LLVM_DYLIB_VERSION@"
 #define LLVM_HAS_GLOBAL_ISEL @LLVM_HAS_GLOBAL_ISEL@
 #define LLVM_TOOLS_INSTALL_DIR "@LLVM_TOOLS_INSTALL_DIR@"

llvm/tools/llvm-config/CMakeLists.txt

 set(LLVM_DYLIB_VERSION "${LLVM_VERSION_MAJOR}${LLVM_VERSION_SUFFIX}")
 set(LLVM_HAS_GLOBAL_ISEL "ON")
 
 # Use the C++ link flags, since they should be a superset of C link flags.
 set(LLVM_LDFLAGS "${CMAKE_CXX_LINK_FLAGS}")
 set(LLVM_BUILDMODE ${CMAKE_BUILD_TYPE})
 set(LLVM_SYSTEM_LIBS ${SYSTEM_LIBS})
 string(REPLACE ";" " " LLVM_TARGETS_BUILT "${LLVM_TARGETS_TO_BUILD}")
+set(LLVM_ENABLE_DYLIB NOT CMAKE_SYSTEM_NAME STREQUAL Windows)
 llvm_canonicalize_cmake_booleans(
-  LLVM_BUILD_LLVM_DYLIB
+  LLVM_ENABLE_DYLIB
   LLVM_LINK_LLVM_DYLIB
   LLVM_HAS_RTTI
   LLVM_HAS_GLOBAL_ISEL
   BUILD_SHARED_LIBS)
 llvm_expand_pseudo_components(LLVM_DYLIB_COMPONENTS_expanded "${LLVM_DYLIB_COMPONENTS}")
 configure_file(${BUILDVARIABLES_SRCPATH} ${BUILDVARIABLES_OBJPATH} @ONLY)
 
 # Set build-time environment(s).

llvm/tools/llvm-shlib/CMakeLists.txt

-# This tool creates a shared library from the LLVM libraries. Generating this
-# library is enabled by setting LLVM_BUILD_LLVM_DYLIB=yes on the CMake
-# commandline. By default the shared library only exports the LLVM C API.
+# This tool creates a shared library from the LLVM libraries.
+# By default the shared library only exports the LLVM C API.
 
 set(SOURCES
   libllvm.cpp
   )
 
 if(LLVM_LINK_LLVM_DYLIB AND LLVM_DYLIB_EXPORTED_SYMBOL_FILE)
   message(WARNING "Using LLVM_LINK_LLVM_DYLIB with LLVM_DYLIB_EXPORTED_SYMBOL_FILE may not work. Use at your own risk.")
 endif()
 
-if(LLVM_BUILD_LLVM_DYLIB)
-  if(MSVC)
-    message(FATAL_ERROR "Generating libLLVM is not supported on MSVC")
-  endif()
-
+if(NOT CMAKE_SYSTEM_NAME STREQUAL Windows)
   llvm_map_components_to_libnames(LIB_NAMES ${LLVM_DYLIB_COMPONENTS})
 
   # Exclude libLLVMTableGen for the following reasons:
   #  - it is only used by internal *-tblgen utilities;
   #  - it pollutes the global options space.
   list(REMOVE_ITEM LIB_NAMES "LLVMTableGen")
 
   if(LLVM_DYLIB_EXPORTED_SYMBOL_FILE)
if(NOT CMAKE_SYSTEM_NAME STREQUAL Windows)
   endif()
 endif()
 
 if(LLVM_BUILD_LLVM_C_DYLIB AND NOT MSVC)
   if(NOT APPLE)
     message(FATAL_ERROR "Generating libLLVM-c is only supported on Darwin")
   endif()
 
-  if(NOT LLVM_BUILD_LLVM_DYLIB)
-    message(FATAL_ERROR "Generating libLLVM-c requires LLVM_BUILD_LLVM_C_DYLIB on Darwin")
-  endif()
-
   # To get the export list for a single llvm library:
   # nm ${LIB_PATH} | awk "/T _LLVM/ { print $3 }" | sort -u | sed -e "s/^_//g" > ${LIB_PATH}.exports
 
   set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_BINARY_DIR}/libllvm-c.exports)
 
   set(LIB_DIR ${CMAKE_BINARY_DIR}/${CMAKE_CFG_INTDIR}/lib${LLVM_LIBDIR_SUFFIX})
   set(LIB_NAME ${LIB_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}LLVM)
   set(LIB_PATH ${LIB_NAME}${CMAKE_SHARED_LIBRARY_SUFFIX})