Make lld::wasm::link() or lld::elf::link() more thread-safe

[seanyoung]

Summary

lld::wasm::link() or lld::elf::link() call InitializeAllTargets() which is not thread-safe if another thread is attempting to use the llvm libraries.

This is an issue for Solang:

https://github.com/hyperledger-labs/solang/blob/master/src/linker/linker.cpp

It uses ldd by calling lld::wasm::link() but another thread might be trying to build an llvm module, which fails if InitializeAllTargets() gets called.

Ideally I'd like to make lld entirely thread-safe and more amiable for using as a a library, that is a much larger change.

[int3]

Mach-O changes lgtm

[sbc100]

wasm changes lgtm assuming other backends agree.

[MaskRay]

ELF looks good.

Ideally I'd like to make lld entirely thread-safe and more amiable for using as a a library, that is a much larger change.

I still want to give a caution:

1. if a port calls fatal or error more than --error-limit times, the process will exit.
2. some ports may reset cl::opt and cl::list options so you cannot rely on these option values after calling lld.

Sometimes it is better to spawn an external lld process to do the link.

[thakis]

Ideally I'd like to make lld entirely thread-safe and more amiable for using as a a library, that is a much larger change.

This is an explicit non-goal.

[dblaikie]

In D93184#2453408, @thakis wrote:

Ideally I'd like to make lld entirely thread-safe and more amiable for using as a a library, that is a much larger change.

This is an explicit non-goal.

Part of the original motivation for that decision was to unblock progress on making a fast production quality linker. Now that that seems to have been achieved - I think it'd be worth revisiting the discussion/accepting patches to enhance the library-compatibility of lld and assess them based on their performance impact, etc.

[seanyoung]

This patch improves the library-compatibility by make it more thread-safe, and it has no performance impact since it simply moves the initialization to an earlier stage.

[MaskRay]

I favor the idea. initLLVM should probably be renamed, because InitLLVM exists as a class which may make users confused.

[seanyoung]

• Rename initLLVM() to initializeLLVM()
• Rebase against latest main

lld/COFF/Driver.cpp

if (auto *path = getenv("LLD_REPRODUCE"))
     return std::string(path);
 
   return None;
 }
 
 void LinkerDriver::link(ArrayRef<const char *> argsArr) {
   ScopedTimer rootTimer(Timer::root());
 
-  // Needed for LTO.
-  InitializeAllTargetInfos();
-  InitializeAllTargets();
-  InitializeAllTargetMCs();
-  InitializeAllAsmParsers();
-  InitializeAllAsmPrinters();
-
   // If the first command line argument is "/lib", link.exe acts like lib.exe.
   // We call our own implementation of lib.exe that understands bitcode files.
   if (argsArr.size() > 1 && StringRef(argsArr[1]).equals_lower("/lib")) {
     if (llvm::libDriverMain(argsArr.slice(1)) != 0)
       fatal("lib failed");
     return;
   }
 

lld/ELF/Driver.cpp

 // Add a given library by searching it from input search paths.
 void LinkerDriver::addLibrary(StringRef name) {
   if (Optional<std::string> path = searchLibrary(name))
     addFile(*path, /*withLOption=*/true);
   else
     error("unable to find library -l" + name, ErrorTag::LibNotFound, {name});
 }
 
-// This function is called on startup. We need this for LTO since
-// LTO calls LLVM functions to compile bitcode files to native code.
-// Technically this can be delayed until we read bitcode files, but
-// we don't bother to do lazily because the initialization is fast.
-static void initLLVM() {
-  InitializeAllTargets();
-  InitializeAllTargetMCs();
-  InitializeAllAsmPrinters();
-  InitializeAllAsmParsers();
-}
-
 // Some command line options or some combinations of them are not allowed.
 // This function checks for such errors.
 static void checkOptions() {
   // The MIPS ABI as of 2016 does not support the GNU-style symbol lookup
   // table which is a relatively new feature.
   if (config->emachine == EM_MIPS && config->gnuHash)
     error("the .gnu.hash section is not compatible with the MIPS target");
 
void LinkerDriver::main(ArrayRef<const char *> argsArr) {
 
   // Initialize time trace profiler.
   if (config->timeTraceEnabled)
     timeTraceProfilerInitialize(config->timeTraceGranularity, config->progName);
 
   {
     llvm::TimeTraceScope timeScope("ExecuteLinker");
 
-    initLLVM();
     createFiles(args);
     if (errorCount())
       return;
 
     inferMachineType();
     setConfigs(args);
     checkOptions();
     if (errorCount())

lld/MachO/Driver.cpp

if (auto *dylibFile = dyn_cast<DylibFile>(file)) {
         dylibFile->reexport = true;
         return true;
       }
     }
   }
   return false;
 }
 
-// This function is called on startup. We need this for LTO since
-// LTO calls LLVM functions to compile bitcode files to native code.
-// Technically this can be delayed until we read bitcode files, but
-// we don't bother to do lazily because the initialization is fast.
-static void initLLVM() {
-  InitializeAllTargets();
-  InitializeAllTargetMCs();
-  InitializeAllAsmPrinters();
-  InitializeAllAsmParsers();
-}
-
 static void compileBitcodeFiles() {
   auto lto = make<BitcodeCompiler>();
   for (InputFile *file : inputFiles)
     if (auto *bitcodeFile = dyn_cast<BitcodeFile>(file))
       lto->add(*bitcodeFile);
 
   for (ObjFile *file : lto->compile())
     inputFiles.push_back(file);
if (args.hasArg(OPT_v)) {
     message(StringRef("Framework search paths:") +
             (config->frameworkSearchPaths.size()
                  ? "\n\t" + llvm::join(config->frameworkSearchPaths, "\n\t")
                  : ""));
     freeArena();
     return !errorCount();
   }
 
-  initLLVM(); // must be run before any call to addFile()
-
   for (const auto &arg : args) {
     const auto &opt = arg->getOption();
     warnIfDeprecatedOption(opt);
     warnIfUnimplementedOption(opt);
     // TODO: are any of these better handled via filtered() or getLastArg()?
     switch (opt.getID()) {
     case OPT_INPUT:
       addFile(arg->getValue(), false);

lld/tools/lld/lld.cpp

 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/CrashRecoveryContext.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/PluginLoader.h"
 #include "llvm/Support/Signals.h"
+#include "llvm/Support/TargetSelect.h"
 #include <cstdlib>
 
 #if !defined(_MSC_VER) && !defined(__MINGW32__)
 #include <signal.h> // for raise
 #include <unistd.h> // for _exit
 #endif
 
 using namespace lld;
static Flavor parseFlavor(std::vector<const char *> &v) {
 
   // Deduct the flavor from argv[0].
   StringRef arg0 = path::filename(v[0]);
   if (arg0.endswith_lower(".exe"))
     arg0 = arg0.drop_back(4);
   return parseProgname(arg0);
 }
 
+// This function is called on startup. We need this for LTO since
+// LTO calls LLVM functions to compile bitcode files to native code.
+// Technically this can be delayed until we read bitcode files, but
+// we don't bother to do lazily because the initialization is fast.
+static void initLLVM() {
+  InitializeAllTargets();
+  InitializeAllTargetMCs();
+  InitializeAllAsmPrinters();
+  InitializeAllAsmParsers();
+}
+
 /// Universal linker main(). This linker emulates the gnu, darwin, or
 /// windows linker based on the argv[0] or -flavor option.
 static int lldMain(int argc, const char **argv, llvm::raw_ostream &stdoutOS,
                    llvm::raw_ostream &stderrOS, bool exitEarly = true) {
   std::vector<const char *> args(argv, argv + argc);
+
+  initLLVM();
+
   switch (parseFlavor(args)) {
   case Gnu:
     if (isPETarget(args))
       return !mingw::link(args, exitEarly, stdoutOS, stderrOS);
     return !elf::link(args, exitEarly, stdoutOS, stderrOS);
   case WinLink:
     return !coff::link(args, exitEarly, stdoutOS, stderrOS);
   case Darwin:

lld/wasm/Driver.cpp

 // Create enum with OPT_xxx values for each option in Options.td
 enum {
   OPT_INVALID = 0,
 #define OPTION(_1, _2, ID, _4, _5, _6, _7, _8, _9, _10, _11, _12) OPT_##ID,
 #include "Options.inc"
 #undef OPTION
 };
 
-// This function is called on startup. We need this for LTO since
-// LTO calls LLVM functions to compile bitcode files to native code.
-// Technically this can be delayed until we read bitcode files, but
-// we don't bother to do lazily because the initialization is fast.
-static void initLLVM() {
-  InitializeAllTargets();
-  InitializeAllTargetMCs();
-  InitializeAllAsmPrinters();
-  InitializeAllAsmParsers();
-}
-
 class LinkerDriver {
 public:
   void link(ArrayRef<const char *> argsArr);
 
 private:
   void createFiles(opt::InputArgList &args);
   void addFile(StringRef path);
   void addLibrary(StringRef name);
bool link(ArrayRef<const char *> args, bool canExitEarly, raw_ostream &stdoutOS,
   errorHandler().errorLimitExceededMsg =
       "too many errors emitted, stopping now (use "
       "-error-limit=0 to see all errors)";
   stderrOS.enable_colors(stderrOS.has_colors());
 
   config = make<Configuration>();
   symtab = make<SymbolTable>();
 
-  initLLVM();
   LinkerDriver().link(args);
 
   // Exit immediately if we don't need to return to the caller.
   // This saves time because the overhead of calling destructors
   // for all globally-allocated objects is not negligible.
   if (canExitEarly)
     exitLld(errorCount() ? 1 : 0);