[lto] add getLinkerOpts()

[inglorion]

Summary

Some compilers, including MSVC and Clang, allow linker options to be specified in source files. In the legacy LTO API, there is a getLinkerOpts() method that returns linker options for the bitcode module being processed. This change adds that method to the new API, so that the COFF linker can get the right linker options when using the new LTO API.

[pcc]

lib/LTO/LTO.cpp

257

We deliberately enable lazy loading of metadata here for performance reasons (debug metadata can be expensive to load).

Eventually I think we will want to add the linker options to the bitcode symbol table (see pr27551), so that we will not need to load metadata at all in the COFF linker at symbol resolution time.

In the meantime, I think it would be best to produce the linker options string on demand in getLinkerOpts after calling materializeMetadata. That way, linkers for other object formats don't pay the cost of loading metadata. Can you please also add a FIXME to add this to the symbol table?

277

This loop (and the one below) can be a range for loop.

286

It's probably best to emit the /export flags from here as well, see TargetLoweringObjectFileCOFF::emitLinkerFlagsForGlobal.

[mehdi_amini]

lib/LTO/LTO.cpp

257

I agree.

I'll add that technically if it is really important for you to get cheaply this you could hook it up to the MD lazy-loading scheme to load this without loading all the MDs.

[inglorion]

Thanks for the comments! I've changed it so that metadata is only materialized if someone actually calls getLinkerOpts, and converted the loops to use ranges.

[pcc]

lib/LTO/LTO.cpp

311

This will lead to a use after free I think.

[pcc]

lib/LTO/LTO.cpp

311

Sorry, no, I didn't notice that this was a field.

I would just remove the field and make this return a std::string.

[inglorion]

lib/LTO/LTO.cpp

311

Hmm, I don't think so. The StringRef here is a reference to LinkerOpts, which is a std::string stored on the InputFile. So that should still be valid after this function returns. LOS.str() just calls flush() on the ostream and returns the StringRef. I think this is ok.

[inglorion]

remove LinkerOpts field and use std::string in the return value

[inglorion]

Made getLinkerOpts() also sythesize /export flags for symbols with dllexport linkage, as @pcc suggested

[inglorion]

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

922

I moved this into a separate function so that we don't have to do the whole TargetTriple, TargetRegistry, SubtargetFeatures, etc. etc. dance to create a TargetLOweringObjectFileImpl, just to call one method that doesn't need all that information.

[pcc]

lib/LTO/LTO.cpp

305–309

Remove braces

314

The Mods list should never be empty, nor should the module's target triple, so you can initialize TT with just Mods[0].Mod->getTargetTriple().

319

You could enumerate SymTab.symbols() instead to avoid needing to befriend InputFile.

[inglorion]

Removed braces, friend declaration, and defensive programming.
Iterate ModuleSymbolTable instead of using symbol_iterator.

[pcc]

LGTM, thanks

include/llvm/CodeGen/TargetLoweringObjectFileImpl.h

MCSection *getStaticCtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
   MCSection *getStaticDtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
 
   void emitLinkerFlagsForGlobal(raw_ostream &OS,
                                 const GlobalValue *GV) const override;
 };
 
+void emitLinkerFlagsForGlobalCOFF(raw_ostream &OS, const GlobalValue *GV,
+                                  const Triple &TT, Mangler &Mangler);
+
 } // end namespace llvm
 
 #endif

include/llvm/LTO/LTO.h

 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/CodeGen/Analysis.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/LTO/Config.h"
 #include "llvm/Linker/IRMover.h"
 #include "llvm/Object/IRObjectFile.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/thread.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Transforms/IPO/FunctionImport.h"
 
 namespace llvm {
 
 class BitcodeModule;
 class Error;
public:
   /// exposes only the information that an LTO client should need in order to do
   /// symbol resolution.
   ///
   /// This object is ephemeral; it is only valid as long as an iterator obtained
   /// from symbols() refers to it.
   class Symbol {
     friend symbol_iterator;
     friend LTO;
+    friend InputFile;
 
     ArrayRef<ModuleSymbolTable::Symbol>::iterator I;
     const ModuleSymbolTable &SymTab;
     const InputFile *File;
     uint32_t Flags;
     SmallString<64> Name;
 
     bool shouldSkip() {
public:
 
   /// A range over the symbols in this InputFile.
   iterator_range<symbol_iterator> symbols() {
     return llvm::make_range(
         symbol_iterator(SymTab.symbols().begin(), SymTab, this),
         symbol_iterator(SymTab.symbols().end(), SymTab, this));
   }
 
+  /// Returns linker options specified in the input file.
+  Expected<std::string> getLinkerOpts();
+
   /// Returns the path to the InputFile.
   StringRef getName() const;
 
   /// Returns the source file path specified at compile time.
   StringRef getSourceFileName() const;
 
   // Returns a table with all the comdats used by this file.
   ArrayRef<StringRef> getComdatTable() const { return Comdats; }

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

if (ComdatKey == GV) {
       }
     } else {
       return COFF::IMAGE_COMDAT_SELECT_ASSOCIATIVE;
     }
   }
   return 0;
 }
 
+void llvm::emitLinkerFlagsForGlobalCOFF(raw_ostream &OS, const GlobalValue *GV,
+                                        const Triple &TT, Mangler &Mangler) {
+  if (!GV->hasDLLExportStorageClass() || GV->isDeclaration())
+    return;
+
+  if (TT.isKnownWindowsMSVCEnvironment())
+    OS << " /EXPORT:";
+  else
+    OS << " -export:";
+
+  if (TT.isWindowsGNUEnvironment() || TT.isWindowsCygwinEnvironment()) {
+    std::string Flag;
+    raw_string_ostream FlagOS(Flag);
+    Mangler.getNameWithPrefix(FlagOS, GV, false);
+    FlagOS.flush();
+    if (Flag[0] == GV->getParent()->getDataLayout().getGlobalPrefix())
+      OS << Flag.substr(1);
+    else
+      OS << Flag;
+  } else {
+    Mangler.getNameWithPrefix(OS, GV, false);
+  }
+
+  if (!GV->getValueType()->isFunctionTy()) {
+    if (TT.isKnownWindowsMSVCEnvironment())
+      OS << ",DATA";
+    else
+      OS << ",data";
+  }
+}
+
 MCSection *TargetLoweringObjectFileCOFF::getExplicitSectionGlobal(
     const GlobalObject *GO, SectionKind Kind, const TargetMachine &TM) const {
   int Selection = 0;
   unsigned Characteristics = getCOFFSectionFlags(Kind, TM);
   StringRef Name = GO->getSection();
   StringRef COMDATSymName = "";
   if (GO->hasComdat()) {
     Selection = getSelectionForCOFF(GO);

 MCSection *TargetLoweringObjectFileCOFF::getStaticDtorSection(
     unsigned Priority, const MCSymbol *KeySym) const {
   return getContext().getAssociativeCOFFSection(
       cast<MCSectionCOFF>(StaticDtorSection), KeySym, 0);
 }
 
 void TargetLoweringObjectFileCOFF::emitLinkerFlagsForGlobal(
     raw_ostream &OS, const GlobalValue *GV) const {
-  if (!GV->hasDLLExportStorageClass() || GV->isDeclaration())
-    return;
-
-  const Triple &TT = getTargetTriple();
-
-  if (TT.isKnownWindowsMSVCEnvironment())
-    OS << " /EXPORT:";
-  else
-    OS << " -export:";
-
-  if (TT.isWindowsGNUEnvironment() || TT.isWindowsCygwinEnvironment()) {
-    std::string Flag;
-    raw_string_ostream FlagOS(Flag);
-    getMangler().getNameWithPrefix(FlagOS, GV, false);
-    FlagOS.flush();
-    if (Flag[0] == GV->getParent()->getDataLayout().getGlobalPrefix())
-      OS << Flag.substr(1);
-    else
-      OS << Flag;
-  } else {
-    getMangler().getNameWithPrefix(OS, GV, false);
-  }
-
-  if (!GV->getValueType()->isFunctionTy()) {
-    if (TT.isKnownWindowsMSVCEnvironment())
-      OS << ",DATA";
-    else
-      OS << ",data";
-  }
+  emitLinkerFlagsForGlobalCOFF(OS, GV, getTargetTriple(), getMangler());
 }

lib/LTO/LTO.cpp

 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTO.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/BitcodeReader.h"
 #include "llvm/Bitcode/BitcodeWriter.h"
 #include "llvm/CodeGen/Analysis.h"
+#include "llvm/CodeGen/TargetLoweringObjectFileImpl.h"
 #include "llvm/IR/AutoUpgrade.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/Mangler.h"
+#include "llvm/IR/Metadata.h"
 #include "llvm/LTO/LTOBackend.h"
 #include "llvm/Linker/IRMover.h"
 #include "llvm/Object/ModuleSummaryIndexObjectFile.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SHA1.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/Threading.h"
Expected<std::unique_ptr<InputFile>> InputFile::create(MemoryBufferRef Object) {
   if (BMsOrErr->empty())
     return make_error<StringError>("Bitcode file does not contain any modules",
                                    inconvertibleErrorCode());
 
   // Create an InputModule for each module in the InputFile, and add it to the
   // ModuleSymbolTable.
   for (auto BM : *BMsOrErr) {
     Expected<std::unique_ptr<Module>> MOrErr =
         BM.getLazyModule(File->Ctx, /*ShouldLazyLoadMetadata*/ true,
                          /*IsImporting*/ false);
     if (!MOrErr)
       return MOrErr.takeError();
 
     size_t SymBegin = File->SymTab.symbols().size();
     File->SymTab.addModule(MOrErr->get());
     size_t SymEnd = File->SymTab.symbols().size();
 
     for (const auto &C : (*MOrErr)->getComdatSymbolTable()) {
       auto P = File->ComdatMap.insert(
           std::make_pair(&C.second, File->Comdats.size()));
       assert(P.second);
       (void)P;
       File->Comdats.push_back(C.first());
     }
 
     File->Mods.push_back({BM, std::move(*MOrErr), SymBegin, SymEnd});
   }
 
   return std::move(File);
 }
 
 Expected<int> InputFile::Symbol::getComdatIndex() const {
   if (!isGV())
     return -1;
   const GlobalObject *GO = getGV()->getBaseObject();
   if (!GO)
     return make_error<StringError>("Unable to determine comdat of alias!",
                                    inconvertibleErrorCode());
   if (const Comdat *C = GO->getComdat()) {
     auto I = File->ComdatMap.find(C);
     assert(I != File->ComdatMap.end());
     return I->second;
   }
   return -1;
 }
 
+Expected<std::string> InputFile::getLinkerOpts() {
+  std::string LinkerOpts;
+  raw_string_ostream LOS(LinkerOpts);
+  // Extract linker options from module metadata.
+  for (InputModule &Mod : Mods) {
+    std::unique_ptr<Module> &M = Mod.Mod;
+    if (auto E = M->materializeMetadata())
+      return std::move(E);
+    if (Metadata *Val = M->getModuleFlag("Linker Options")) {
+      MDNode *LinkerOptions = cast<MDNode>(Val);
+      for (const MDOperand &MDOptions : LinkerOptions->operands()) {
+        for (const MDOperand &MDOption : cast<MDNode>(MDOptions)->operands()) {
+          LOS << " " << cast<MDString>(MDOption)->getString();
+        }
+      }
+    }
+  }
+
+  // Synthesize export flags for symbols with dllexport storage.
+  std::string TTStr = Mods.empty() ? "" : Mods[0].Mod->getTargetTriple();
+  if (TTStr.empty())
+    TTStr = sys::getDefaultTargetTriple();
+  const Triple TT(TTStr);
+  Mangler M;
+  for (auto &Sym : symbols())
+    if (Sym.isGV())
+      emitLinkerFlagsForGlobalCOFF(LOS, Sym.getGV(), TT, M);
+  LOS.flush();
+  return LinkerOpts;
+}
+
 StringRef InputFile::getName() const {
   return Mods[0].BM.getModuleIdentifier();
 }
 
 StringRef InputFile::getSourceFileName() const {
   return Mods[0].Mod->getSourceFileName();
 }