[lld] Add CMake options to disable individual linkers

[arichardson]

Summary

This is useful in downstream toolchain builds that only support a single
(or maybe two) output formats. For CHERI LLVM, most users compile
the toolchain from source, so disabling the non-ELF linkers reduces
build time (about 100 C++ source files fewer to compile) and reduces
the lld binary size by about 4MB (106MB -> 102MB) for a statically
linked build.

Another example where this can be useful is the FreeBSD lld build,
which has a local patch to only include ELF support in lld.cpp.
This change will allow removing the downstream patch and
instead use the relevant LLD_ENABLE_*_LINKER defines.

This patch should not result in any long-term maintainance issues,
as it only adds a few if() lines to the CMake build and skips tests
for some subdirectories.

[thakis]

Is the motivation disk size? How much of a difference does this make?

[arichardson]

In D108120#2949178, @thakis wrote:

Is the motivation disk size? How much of a difference does this make?

For a release+asserts build it's about 3.8MB difference. Not huge but noticeable:

$ ls -l bin/lld.*.s*
-rwxrwxr-x 1 alr48 alr48 106799760 Aug 17 13:57 bin/lld.all.stripped
-rwxrwxr-x 1 alr48 alr48 102797984 Aug 17 13:53 bin/lld.elf.stripped
$ size bin/lld.*.s*
   text	   data	    bss	    dec	    hex	filename
99852875	6934040	 500314	107287229	66512bd	bin/lld.all.stripped
95985096	6805040	 474962	103265098	627b34a	bin/lld.elf.stripped

My main motivation was actually build time and looking for files that can easily be omitted.
Building all LLD flavours results in 156 ninja commands (after rm -rf tools/lld) whereas only building the ELF linker results in 54 build commands.
FreeBSD builds LLVM using a separate Makefile based system, so they only have a small downstream #if 0 patch in lld.cpp. Therefore it's not a huge local diff that would be removed by this patch.

My real motivation is only building the ELF linker for our CHERI toolchain (and I think Arm might do the same for their Morello toolchain once our changes have been merged).
Right now we don't ship binaries, so we expect all our users to compile LLVM from source (possibly on laptops with only a few CPU cores), so not building approximately 100 source files could be a noticeable speedup.

[arichardson]

@thakis If you want I can also have a look at updating the GN build files (assuming this patch is considered acceptable).

[arichardson]

Fix unused function warning

[MaskRay]

Looks fine to me.

lld/include/lld/Common/config.h.cmake

1

The header is not needed. Many config.h.cmake files don't use the header.

[dim]

Indeed in the FreeBSD base system we only build the ELF part and patch lldMain to stub out the rest, see https://github.com/freebsd/freebsd-src/commit/59948e95d8deadafca3acd659b6dfc78b708f117

But this is nicer to have, and can save you some time when using the CMake build.

[arichardson]

lld/include/lld/Common/config.h.cmake

1

Will drop it, I believe I just copied it from the LLVM config header.

[thakis]

If this doesn't help you much upstream, and since it's not a big size change, is it really worth to add the complexity here upstream?

(Don't worry about the gn build though, that's easy to update if this lands.)

[smeenai]

In D108120#2952966, @thakis wrote:

If this doesn't help you much upstream, and since it's not a big size change, is it really worth to add the complexity here upstream?

(Don't worry about the gn build though, that's easy to update if this lands.)

We're interested in this as well, and it'd be nice to have it upstream :) (I haven't run the size numbers on our end yet, to be fair.)

[sbc100]

In D108120#2952982, @smeenai wrote:

In D108120#2952966, @thakis wrote:

If this doesn't help you much upstream, and since it's not a big size change, is it really worth to add the complexity here upstream?

(Don't worry about the gn build though, that's easy to update if this lands.)

We're interested in this as well, and it'd be nice to have it upstream :) (I haven't run the size numbers on our end yet, to be fair.)

+1 to avoiding extra complexity unless the size wins really are significant (I'm not sure how we go about assessing that).

[arichardson]

In D108120#2952966, @thakis wrote:

If this doesn't help you much upstream, and since it's not a big size change, is it really worth to add the complexity here upstream?

(Don't worry about the gn build though, that's easy to update if this lands.)

While it doesn't help very much for FreeBSD (one of the downstream forks I contribute to) since FreeBSD doesn't use the CMake build system, it does help for the CHERI downstream (the downstream fork I work on for my job).
I think this doesn't add very much complexity: just a few ifdefs in a file that rarely changes and a few if() statements in CMake.

To me a 3.8MB size reduction seems worth this complexity but if you disagree I can commit this downstream instead.

[dim]

It's mostly about shaving off quite a bit of build time, if you can skip all the formats that you never use. The size reduction is also nice, but is not really the essential part.

(I would compare this more to e.g. the LLVM_TARGETS_TO_BUILD list, which allows you to select only the architectures you're interested in, saving you lots of build time in many cases.)

[arichardson]

Drop unnecessary copyright from config.h header.

@sbc100 are you still strongly opposed to this patch?

[emaste]

IMO this is worth it for the build time reduction, the size reduction is a nice bonus but not the main benefit.

[sbc100]

In D108120#3020506, @arichardson wrote:

Drop unnecessary copyright from config.h header.

@sbc100 are you still strongly opposed to this patch?

Nope, not a strong objection. Just not sure the incremental wins are worth it. If folks downstream see real value in shaving a few cycles of the build time this seems like a reasonable approach.

[MaskRay]

lld/tools/lld/lld.cpp

154

then and else should be swapped.

[arichardson]

lld/tools/lld/lld.cpp

154

Like this?

#if LLD_ENABLE_COFF_LINKER
      return !mingw::link(args, exitEarly, stdoutOS, stderrOS);
#else
      missingLinkerSupport("COFF");
#endif

I originally used ! to catch cases where the macro is not defined (since we don't build with -Wundef), but I could use != 0 to get the same error with the branches inverted.

[MaskRay]

lld/tools/lld/lld.cpp

154

!= 0 looks good

[smeenai]

We're supportive of this, as stated before.

lld/CMakeLists.txt

option(LLD_BUILD_TOOLS
   "Build the lld tools. If OFF, just generate build targets." ON)
 
 option(LLD_DEFAULT_LD_LLD_IS_MINGW
     "Use MinGW as the default backend for ld.lld. If OFF, ELF will be used." OFF)
 if (LLD_DEFAULT_LD_LLD_IS_MINGW)
   add_definitions("-DLLD_DEFAULT_LD_LLD_IS_MINGW=1")
 endif()
 
+option(LLD_ENABLE_ELF_LINKER "Build the ELF linker" ON)
+option(LLD_ENABLE_COFF_LINKER "Build the COFF linker" ON)
+option(LLD_ENABLE_MACHO_LINKER "Build the Mach-O linker" ON)
+option(LLD_ENABLE_WASM_LINKER "Build the WASM linker" ON)
+configure_file(${CMAKE_CURRENT_SOURCE_DIR}/include/lld/Common/config.h.cmake
+               ${CMAKE_CURRENT_BINARY_DIR}/include/lld/Common/config.h)
+
 if (MSVC)
   add_definitions(-wd4530) # Suppress 'warning C4530: C++ exception handler used, but unwind semantics are not enabled.'
   add_definitions(-wd4062) # Suppress 'warning C4062: enumerator X in switch of enum Y is not handled' from system header.
 endif()
 
 include_directories(BEFORE
   ${CMAKE_CURRENT_BINARY_DIR}/include
   ${CMAKE_CURRENT_SOURCE_DIR}/include
   )
 
 if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
   install(DIRECTORY include/
     DESTINATION include
     FILES_MATCHING
     PATTERN "*.h"
     )
 endif()
 
 add_subdirectory(Common)
+if (LLD_ENABLE_MACHO_LINKER)
   add_subdirectory(lib)
+endif()
 add_subdirectory(tools/lld)
 
 if (LLVM_INCLUDE_TESTS)
   add_subdirectory(test)
   add_subdirectory(unittests)
 endif()
 
 add_subdirectory(docs)
+if (LLD_ENABLE_COFF_LINKER)
   add_subdirectory(COFF)
+  add_subdirectory(MinGW)
+endif()
+if (LLD_ENABLE_ELF_LINKER)
   add_subdirectory(ELF)
+endif()
+if (LLD_ENABLE_MACHO_LINKER)
   add_subdirectory(MachO)
-add_subdirectory(MinGW)
+endif()
+if (LLD_ENABLE_WASM_LINKER)
   add_subdirectory(wasm)
+endif()
 
 add_subdirectory(cmake/modules)

lld/include/lld/Common/config.h.cmake

+/*===------- lld/Common/config.h - lld configuration --------------*- C -*-===*/
+/*                                                                            */
+/* Part of the LLVM Project, under the Apache License v2.0 with LLVM          */
+/* Exceptions.                                                                */
+/* See https://llvm.org/LICENSE.txt for license information.                  */
+/* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception                    */
+/*                                                                            */
+/*===----------------------------------------------------------------------===*/
+
+#cmakedefine01 LLD_ENABLE_COFF_LINKER
+#cmakedefine01 LLD_ENABLE_ELF_LINKER
+#cmakedefine01 LLD_ENABLE_MACHO_LINKER
+#cmakedefine01 LLD_ENABLE_WASM_LINKER

lld/test/COFF/lit.local.cfg

+if 'lld-coff' not in config.available_features:
+  config.unsupported = True
+
 config.environment['LLD_IN_TEST'] = '2'

lld/test/ELF/lit.local.cfg

+if 'lld-elf' not in config.available_features:
+  config.unsupported = True
+
 config.suffixes = ['.test', '.s', '.ll', '.bat']

lld/test/MachO/lit.local.cfg

 # -*- Python -*-
 
 import os
 
+if 'lld-macho' not in config.available_features:
+    config.unsupported = True
+
 # We specify the most commonly-used archs and platform versions in our tests
 # here. Tests which need different settings can just append to this, as only
 # the last value will be used.
 #
 # Note however that this does not apply to `-syslibroot`: each instance of that
 # flag will append to the set of library roots. As such, we define a separate
 # alias for each platform.
 

lld/test/MinGW/lit.local.cfg

+if 'lld-coff' not in config.available_features:
+  config.unsupported = True

lld/test/darwin/lit.local.cfg

+if 'lld-macho' not in config.available_features:
+  config.unsupported = True

lld/test/lit.cfg.py

if config.have_libxar:
     config.available_features.add('xar')
 
 if config.have_libxml2:
     config.available_features.add('libxml2')
 
 if config.have_dia_sdk:
     config.available_features.add("diasdk")
 
+if config.have_lld_coff_linker:
+    config.available_features.add('lld-coff')
+if config.have_lld_elf_linker:
+    config.available_features.add('lld-elf')
+if config.have_lld_macho_linker:
+    config.available_features.add('lld-macho')
+if config.have_lld_wasm_linker:
+    config.available_features.add('lld-wasm')
+
 if config.sizeof_void_p == 8:
     config.available_features.add("llvm-64-bits")
 
 tar_executable = lit.util.which('tar', config.environment['PATH'])
 if tar_executable:
     env = os.environ
     env['LANG'] = 'C'
     tar_version = subprocess.Popen(

lld/test/lit.site.cfg.py.in

 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@Python3_EXECUTABLE@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
 config.have_libxar = @LLVM_HAVE_LIBXAR@
 config.have_libxml2 = @LLVM_ENABLE_LIBXML2@
 config.sizeof_void_p = @CMAKE_SIZEOF_VOID_P@
 config.ld_lld_default_mingw = @LLD_DEFAULT_LD_LLD_IS_MINGW@
 
+config.have_lld_coff_linker = lit.util.pythonize_bool("@LLD_ENABLE_COFF_LINKER@")
+config.have_lld_elf_linker = lit.util.pythonize_bool("@LLD_ENABLE_ELF_LINKER@")
+config.have_lld_macho_linker = lit.util.pythonize_bool("@LLD_ENABLE_MACHO_LINKER@")
+config.have_lld_wasm_linker = lit.util.pythonize_bool("@LLD_ENABLE_WASM_LINKER@")
+
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
     config.lld_libs_dir = config.lld_libs_dir % lit_config.params
     config.lld_tools_dir = config.lld_tools_dir % lit_config.params
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLD_SOURCE_DIR@/test/lit.cfg.py")

lld/test/mach-o/lit.local.cfg

+if 'lld-macho' not in config.available_features:
+  config.unsupported = True
 
 # mach-o test cases encode input files in yaml and use .yaml extension
 config.suffixes = ['.yaml']
 config.excludes = ['Inputs']

lld/test/wasm/lit.local.cfg

-if 'wasm' not in config.available_features:
+if 'wasm' not in config.available_features or 'lld-wasm' not in config.available_features:
   config.unsupported = True
 
 config.suffixes = ['.test', '.yaml', '.ll', '.s']

lld/tools/lld/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Support
   )
 
 add_lld_tool(lld
   lld.cpp
 
   SUPPORT_PLUGINS
   )
 export_executable_symbols_for_plugins(lld)
 
 target_link_libraries(lld
   PRIVATE
   lldCommon
-  lldCOFF
-  lldDriver
-  lldELF
-  lldMachO
-  lldMinGW
-  lldWasm
   )
+set(LLD_DEFAULT_SYMLINKS)
+if (LLD_ENABLE_COFF_LINKER)
+  target_link_libraries(lld PRIVATE lldCOFF lldMinGW)
+  list(APPEND LLD_DEFAULT_SYMLINKS ld.lld lld-link)
+endif()
+if (LLD_ENABLE_ELF_LINKER)
+  target_link_libraries(lld PRIVATE lldELF)
+  list(APPEND LLD_DEFAULT_SYMLINKS ld.lld)
+endif()
+if (LLD_ENABLE_MACHO_LINKER)
+  target_link_libraries(lld PRIVATE lldDriver lldMachO)
+  list(APPEND LLD_DEFAULT_SYMLINKS ld64.lld ld64.lld.darwinnew ld64.lld.darwinold)
+endif()
+if (LLD_ENABLE_WASM_LINKER)
+  target_link_libraries(lld PRIVATE lldWasm)
+  list(APPEND LLD_DEFAULT_SYMLINKS wasm-ld)
+endif()
+list(REMOVE_DUPLICATES LLD_DEFAULT_SYMLINKS)
 
 install(TARGETS lld
   RUNTIME DESTINATION bin)
 
 if(NOT LLD_SYMLINKS_TO_CREATE)
-  set(LLD_SYMLINKS_TO_CREATE
-      lld-link ld.lld ld64.lld ld64.lld.darwinnew ld64.lld.darwinold wasm-ld)
+  set(LLD_SYMLINKS_TO_CREATE ${LLD_DEFAULT_SYMLINKS})
 endif()
 
 foreach(link ${LLD_SYMLINKS_TO_CREATE})
   add_lld_symlink(${link} lld)
 endforeach()

lld/tools/lld/lld.cpp

 //  - lld-link:  COFF (Windows)
 //  - ld-wasm:   WebAssembly
 //
 // lld can be invoked as "lld" along with "-flavor" option. This is for
 // backward compatibility and not recommended.
 //
 //===----------------------------------------------------------------------===//
 
+#include "lld/Common/config.h"
+
 #include "lld/Common/Driver.h"
 #include "lld/Common/ErrorHandler.h"
 #include "lld/Common/Memory.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/ADT/Twine.h"
enum Flavor {
   Wasm,      // -flavor wasm
 };
 
 [[noreturn]] static void die(const Twine &s) {
   llvm::errs() << s << "\n";
   exit(1);
 }
 
+[[noreturn]] LLVM_ATTRIBUTE_UNUSED static void
+missingLinkerSupport(StringRef format) {
+  die(format + " support in lld was disabled. Please use a different linker");
+}
+
 static Flavor getFlavor(StringRef s) {
   return StringSwitch<Flavor>(s)
       .CasesLower("ld", "ld.lld", "gnu", Gnu)
       .CasesLower("wasm", "ld-wasm", Wasm)
       .CaseLower("link", WinLink)
       .CasesLower("ld64", "ld64.lld", "darwin", "darwinnew",
                   "ld64.lld.darwinnew", Darwin)
       .CasesLower("darwinold", "ld64.lld.darwinold", DarwinOld)

 
 /// Universal linker main(). This linker emulates the gnu, darwin, or
 /// windows linker based on the argv[0] or -flavor option.
 static int lldMain(int argc, const char **argv, llvm::raw_ostream &stdoutOS,
                    llvm::raw_ostream &stderrOS, bool exitEarly = true) {
   std::vector<const char *> args(argv, argv + argc);
   switch (parseFlavor(args)) {
   case Gnu:
-    if (isPETarget(args))
+    if (isPETarget(args)) {
+#if !LLD_ENABLE_COFF_LINKER
+      missingLinkerSupport("COFF");
+#else
       return !mingw::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
+    }
+#if !LLD_ENABLE_ELF_LINKER
+    missingLinkerSupport("ELF");
+#else
     return !elf::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
   case WinLink:
+#if !LLD_ENABLE_COFF_LINKER
+    missingLinkerSupport("COFF");
+#else
     return !coff::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
   case Darwin:
+#if !LLD_ENABLE_MACHO_LINKER
+    missingLinkerSupport("Mach-O");
+#else
     return !macho::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
   case DarwinOld:
+#if !LLD_ENABLE_MACHO_LINKER
+    missingLinkerSupport("Mach-O");
+#else
     return !mach_o::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
   case Wasm:
+#if !LLD_ENABLE_WASM_LINKER
+    missingLinkerSupport("WASM");
+#else
     return !lld::wasm::link(args, exitEarly, stdoutOS, stderrOS);
+#endif
   default:
     die("lld is a generic driver.\n"
         "Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld"
         " (WebAssembly) instead");
   }
 }
 
 // Similar to lldMain except that exceptions are caught.

lld/unittests/CMakeLists.txt

 add_custom_target(LLDUnitTests)
 set_target_properties(LLDUnitTests PROPERTIES FOLDER "lld tests")
 
 set(CMAKE_BUILD_WITH_INSTALL_RPATH OFF)
 
 # add_lld_unittest(test_dirname file1.cpp file2.cpp)
 #
 # Will compile the list of files together and link against lld
 # Produces a binary named 'basename(test_dirname)'.
 function(add_lld_unittest test_dirname)
   add_unittest(LLDUnitTests ${test_dirname} ${ARGN})
   target_link_libraries(${test_dirname} ${LLVM_COMMON_LIBS})
 endfunction()
 
+# Currently, all unit tests depend on the old Mach-O linker
+if (LLD_ENABLE_MACHO_LINKER)
   add_subdirectory(DriverTests)
   add_subdirectory(MachOTests)
+endif()