[MLIR] Temporary workaround for calling the LLD ELF driver as-a-lib

[aganea]

Summary

This fixes the situation described in https://github.com/llvm/llvm-project/issues/53475 with a repro exposed by https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

This is purposely just a workaround to unblock users. This could be transplanted to the release/14.x branch if need be. A proper fix will later be provided in D119049

[krzysz00]

Other than a minor typo, fixes the problem and looks good

mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp

446

Missing namespace causing compilation failure

[aganea]

namespace fix.

[aganea]

mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp

446

Fixed, thanks for testing! I don't have the ROCm libs installed. I only tested with the repro you provided in https://github.com/ROCmSoftwarePlatform/D108850-lld-bug-reproduction

[krzysz00]

LGTM, we can poke at long-term solutions later

This at least gets everything testable again and lets us update downstream to LLVM tip.

[MaskRay]

lld/ELF/Driver.cpp

139

If this is not ELF-specific, seems that it needs to live in lld::cleanup ? If the API (perhaps not intended as an API), we can give it an long discouraging name.

[aganea]

Use more generic lld::cleanup() instead of lld::elf::cleanup()

lld/ELF/Driver.cpp

bool elf::link(ArrayRef<const char *> args, llvm::raw_ostream &stdoutOS,
 
   config->progName = args[0];
 
   driver->linkerMain(args);
 
   return errorCount() == 0;
 }
 
+void elf::cleanup() {
+  // Delete the global context and clear the global context pointer, so that it
+  // cannot be accessed anymore.
+  CommonLinkerContext::destroy();
+}
+
 // Parses a linker -m option.
 static std::tuple<ELFKind, uint16_t, uint8_t> parseEmulation(StringRef emul) {
   uint8_t osabi = 0;
   StringRef s = emul;
   if (s.endswith("_fbsd")) {
     s = s.drop_back(5);
     osabi = ELFOSABI_FREEBSD;
   }

lld/include/lld/Common/Driver.h

 namespace mingw {
 bool link(llvm::ArrayRef<const char *> args, llvm::raw_ostream &stdoutOS,
           llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput);
 }
 
 namespace elf {
 bool link(llvm::ArrayRef<const char *> args, llvm::raw_ostream &stdoutOS,
           llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput);
-}
+
+// Temporary API that forces global state cleanup between explicit calls to the
+// ELF driver above. DO NOT USE - this will be replaced by safeLldMain(). See
+// discussion in https://reviews.llvm.org/D119049.
+void cleanup();
+} // namespace elf
 
 namespace macho {
 bool link(llvm::ArrayRef<const char *> args, llvm::raw_ostream &stdoutOS,
           llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput);
 }
 
 namespace wasm {
 bool link(llvm::ArrayRef<const char *> args, llvm::raw_ostream &stdoutOS,
           llvm::raw_ostream &stderrOS, bool exitEarly, bool disableOutput);
 }
 }
 
 #endif

mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp

if (llvm::sys::fs::createTemporaryFile("kernel", "hsaco", tempHsacoFD,
     return {};
   }
   llvm::FileRemover cleanupHsaco(tempHsacoFilename);
 
   {
     static std::mutex mutex;
     const std::lock_guard<std::mutex> lock(mutex);
     // Invoke lld. Expect a true return value from lld.
-    if (!lld::elf::link({"ld.lld", "-shared", tempIsaBinaryFilename.c_str(),
+    bool r = lld::elf::link({"ld.lld", "-shared", tempIsaBinaryFilename.c_str(),
                              "-o", tempHsacoFilename.c_str()},
-                        llvm::outs(), llvm::errs(), /*exitEarly=*/true,
-                        /*disableOutput=*/false)) {
+                            llvm::outs(), llvm::errs(), /*exitEarly=*/false,
+                            /*disableOutput=*/false);
+    // Allow for calling the driver again in the same process.
+    lld::cleanup();
+    if (!r) {
       emitError(loc, "lld invocation error");
       return {};
     }
   }
 
   // Load the HSA code object.
   auto hsacoFile = openInputFile(tempHsacoFilename);
   if (!hsacoFile) {