Add an LTO interface to parse metadata and extract linker options

[ygao]

Summary

Hi,
This patch adds the following LTO interface to parse metadata nodes and extract linker options and dependent
libraries from a bitcode module. The new API functions are:

unsigned int lto_module_get_num_deplibs(lto_module_t mod) : Returns the number of dependent libraries in the object module.
const char* lto_module_get_deplib(lto_module_t mod, unsigned int index) : Returns the ith dependent library in the module.
unsigned int lto_module_get_num_linkeropts(lto_module_t mod) : Returns the number of linker options in the object module.
const char* lto_module_get_linkeropt(lto_module_t mod, unsigned int index) : Returns the ith linker option in the module.

These new APIs are very similar to the existing APIs for symbols.

Could someone take a look whether this is good to go in? Does anyone have strong opinion on this proposed interface?

Many thanks,

• Gao.

[ygao]

ping?

[ygao]

On Sun, Dec 15, 2013 at 11:43 AM, "Duncan P. N. Exon Smith" <[e-mail]> wrote:

Hi Gao,

The implementation looks correct, but given that it adds to the C API (which needs to be stable),
can someone else chime in?

Also, would you please explain the use case for embedding linker options in the object file metadata?
Why wouldn’t they be passed directly to the linker? Sorry if I missed an earlier thread on this, or if
I’m missing something obvious.

Duncan

Hi Duncan,

Many thanks for reviewing my patch!

Embedding linker options in the object file metadata has been the way to support
MSVC-style auto linking since r181426. I guess your real question is why we need
to add new APIs instead of compiling to native object files and then letting the
linker extract the linker options from the object files. Please clarify if that
is not the case.

I have to discuss with our linker team for any specific use cases, but I think
there are a few linker options which affect symbol references used by LTO,
and they may be needed before the symbol resolution phase. For example,
/include:<symbol> tells the linker to pretend that a symbol is referenced. Also,
if /dll and /def:<module-definition file> are used, then the linker need to read
the input .def file and preserve the exported symbols specified in that file from
being deleted by LTO passes.

Reference: http://llvm.org/docs/LinkTimeOptimization.html

• Gao.

[ygao]

Rebased the patch to trunk r197851.

After discussing with the linker team, I believe we have two important use
cases that need to be solved by extending the LTO plugin APIs.

First, we have game codes that use bitcode archives as argument to pragma
comment libs. In order to have knowledge of all symbol references, the linker
needs to add bitcode files included in the dependent libraries to its input
queue. And these bitcode files could in turn refer to more dependent libraries.
So there is an iterative process of scanning bitcode files for dependencies.

To be able to support bitcode archives simplifies the process of adding LTO to
existing build system. For example, say, there is a Makefile that generates an
intermediate library libfoo.a that is compiled out of source files liba.cpp and
libb.cpp and is referenced in a source file a.cpp with

#pragma comment(lib, "foo").

Without LTO, running "make all" will put native object files liba.o and libb.o
into libfoo.a. With LTO, running "make all CFLAGS=-flto" will put bitcode files
liba.o and libb.o into libfoo.a; there would be no need to change the Makefile.

Secondly, the libraries referenced in pragma comment lib need to be added to
the input queue to be scanned immediately after the current file. So, for
example, the link order would be different if the library dependencies are
extracted from a merged, post-LTO, object.

[rnk]

Interesting. This is essentially exposing the info that we would have put into the object file if we had generated one.

include/llvm/Target/TargetLoweringObjectFile.h

60 ↗

(On Diff #6220)

This should return a StringRef.

lib/LTO/LTOModule.cpp

817 ↗

(On Diff #6220)

Now _deplibs and _linkeropts hold pointers into global metadata. Is that OK? Is that metadata guaranteed to outlive LTOModule? Might be better to copy here.

[ygao]

On Sun, Jan 5, 2014 at 4:08 PM, "Duncan P. N. Exon Smith" <[e-mail]> wrote:

Thanks for the explanation; I understand the use case now.
LGTM, once you’ve satisfied Reid’s comments about string lifetime.
Duncan
P.S. Sorry for the long delay!

Hi Duncan, thanks again for the review! I only just came back from my vacation yesterday.

On Thu, Dec 26, 2013 at 11:48 AM, "Reid Kleckner" <[e-mail]> wrote:

Interesting. This is essentially exposing the info that we would have put into the
object file if we had generated one.
================
Comment at: include/llvm/Target/TargetLoweringObjectFile.h:60
@@ +59,3 @@
+ /// option string. Returns NULL if the option does not specify a library.
+ virtual const char *getDepLibFromLinkerOpt(StringRef LinkerOption) const {

+ return NULL;

This should return a StringRef.

Comment at: lib/LTO/LTOModule.cpp:817
@@ +816,3 @@
+ getObjFileLowering().getDepLibFromLinkerOpt(Op))
+ _deplibs.push_back(DepLibName);

+ else if (!Op.empty())

Now _deplibs and _linkeropts hold pointers into global metadata. Is that OK? Is that
metadata guaranteed to outlive LTOModule? Might be better to copy here.

I updated the patch to address both of these points.

I would like to make a somewhat related observation since you mentioned string
lifetime. If you take a look at the _symbols vector inside struct LTOModule, each
NameAndAttributes entry includes both a pointer to the symbol name and a pointer
to the symbol itself (as a GlobalValue). The symbol name referenced from
NameAndAttributes is a copy with a StringSet. The symbol referenced from
the same NameAndAttributes struct does not appear to be a copy; it is just part
of the input llvm::Module. If the symbol is known to have a long lifetime, it
would indicate that the copy of the symbol name is redundant. If the symbol's
lifetime is not guaranteed, then maybe the reference to it could become invalidated
at some point.

[rnk]

Nick, can you look at this, since it touches the LTO interface? IIRC you had previous objections to having flags in the LTO interface.

The basic issue is that we support embedding linker options in MachO and COFF, but without this change, they aren't exposed in LTO.

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

8 ↗

(On Diff #6379)

Is that dash "one" or "ell"? I would assume you want -l.

[ygao]

It should be dash-ell. Fixed. Thanks!

[nlewycky]

The libLTO API here is fine.

I'm surprised to see the return of dependent libraries in modules. For history,
see:

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20121126/157053.html

Secondly, I don't think this needs to be metadata. It doesn't need to hold weak
references to IR values. I *think* I'd rather they were first-class properties
of llvm::Module? This would require some general agreement on llvm-dev.

A Module more or less corresponds to a single .o file, which makes storing
properties like linker flags a bit unusual. (This is true regardless of whether
you make it a first class property or use named metadata.) Could you explain
why it belongs here?

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

407 ↗

(On Diff #6390)

Hmm. You want to stick a linker option parser inside codegen? Am I correct in understanding that COFF object files have linker flags in them?

Who fills in these linker options and dependent libraries? Is there any reason we couldn't hoist the linker option parsing into there?

[kledzik]

include/llvm-c/lto.h

201 ↗

(On Diff #6390)

When you add new APIs to <llvm-c/lto.h> be sure to bump up LTO_API_VERSION, so that clients to do compiler configuration checks if the API exists.

[ygao]

include/llvm-c/lto.h

201 ↗

(On Diff #6390)

Hi Nick,

I saw your discussion with Sean on the mailing list on this topic. I will bump up LTO_API_VERSION as you requested.

• Gao.

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

407 ↗

(On Diff #6390)

Hi Nick,

Am I correct in understanding that COFF object files have linker flags in them?

I assume you mean the COFF .drectve section.

The translation from IR's "Linker Options" named metadata to .drectve entries happens in
TargetLoweringObjectFileCOFF::emitModuleFlags() for COFF; and in
TargetLoweringObjectFileMachO::emitModuleFlags() for MachO. That is why I thought
TargetLoweringObjectFile{COFF,MachO} are good places to add these new functions. These
emitModuleFlags() routines do not appear to parse the linker options; they just output the
metadata strings as-is into the relevant directive sections.

The generation of these "Linker Options" named metadata happens in the clang
front end. For example, in cfe/trunk/lib/CodeGen/TargetInfo.cpp, look in
WinX86_32TargetCodeGenInfo::getDependentLibraryOption() and
TargetCodeGenInfo::getDependentLibraryOption(). The metadata strings received by the
backend are already "packaged" and I was trying to extract the library names back from there.

I would argue that by adding these routines to extract the library names, the results returned
by the LTO APIs could stay independent from how the linker options are packaged in the IR
metadata, although I should admit that the "/DEFAULTLIB:<library_name>" format used by
COFF has been around for quite a while and probably will not change.

[ygao]

Rebased to trunk r199503 and added the documentation similar to r199429.

[ygao]

Closed by commit rL199759 (authored by @ygao).

llvm/trunk/include/llvm-c/lto.h

 
 /**
  * @defgroup LLVMCLTO LTO
  * @ingroup LLVMC
  *
  * @{
  */
 
-#define LTO_API_VERSION 7
+#define LTO_API_VERSION 8
 
 /**
  * \since prior to LTO_API_VERSION=3
  */
 typedef enum {
     LTO_SYMBOL_ALIGNMENT_MASK              = 0x0000001F, /* log2 of alignment */
     LTO_SYMBOL_PERMISSIONS_MASK            = 0x000000E0,
     LTO_SYMBOL_PERMISSIONS_CODE            = 0x000000A0,

 /**
  * Returns the attributes of the ith symbol in the object module.
  *
  * \since prior to LTO_API_VERSION=3
  */
 extern lto_symbol_attributes
 lto_module_get_symbol_attribute(lto_module_t mod, unsigned int index);
 
+
+/**
+ * Returns the number of dependent libraries in the object module.
+ *
+ * \since LTO_API_VERSION=8
+ */
+extern unsigned int
+lto_module_get_num_deplibs(lto_module_t mod);
+
+
+/**
+ * Returns the ith dependent library in the module.
+ *
+ * \since LTO_API_VERSION=8
+ */
+extern const char*
+lto_module_get_deplib(lto_module_t mod, unsigned int index);
+
+
+/**
+ * Returns the number of linker options in the object module.
+ *
+ * \since LTO_API_VERSION=8
+ */
+extern unsigned int
+lto_module_get_num_linkeropts(lto_module_t mod);
+
+
+/**
+ * Returns the ith linker option in the module.
+ *
+ * \since LTO_API_VERSION=8
+ */
+extern const char*
+lto_module_get_linkeropt(lto_module_t mod, unsigned int index);
+
+
 /**
  * Diagnostic severity.
  *
  * \since LTO_API_VERSION=7
  */
 typedef enum {
   LTO_DS_ERROR,
   LTO_DS_WARNING,

llvm/trunk/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h

 };
 
 
 
 class TargetLoweringObjectFileMachO : public TargetLoweringObjectFile {
 public:
   virtual ~TargetLoweringObjectFileMachO() {}
 
+  /// getDepLibFromLinkerOpt - Extract the dependent library name from a linker
+  /// option string. Returns StringRef() if the option does not specify a library.
+  virtual StringRef getDepLibFromLinkerOpt(StringRef LinkerOption) const;
+
   /// emitModuleFlags - Emit the module flags that specify the garbage
   /// collection information.
   virtual void emitModuleFlags(MCStreamer &Streamer,
                                ArrayRef<Module::ModuleFlagEntry> ModuleFlags,
                                Mangler *Mang, const TargetMachine &TM) const;
 
   virtual const MCSection *
   SelectSectionForGlobal(const GlobalValue *GV, SectionKind Kind,
public:
   virtual const MCSection *
   getExplicitSectionGlobal(const GlobalValue *GV, SectionKind Kind,
                            Mangler *Mang, const TargetMachine &TM) const;
 
   virtual const MCSection *
   SelectSectionForGlobal(const GlobalValue *GV, SectionKind Kind,
                          Mangler *Mang, const TargetMachine &TM) const;
 
+  /// getDepLibFromLinkerOpt - Extract the dependent library name from a linker
+  /// option string. Returns StringRef() if the option does not specify a library.
+  virtual StringRef getDepLibFromLinkerOpt(StringRef LinkerOption) const;
+
   /// emitModuleFlags - Emit Obj-C garbage collection and linker options.  Only
   /// linker option emission is implemented for COFF.
   virtual void emitModuleFlags(MCStreamer &Streamer,
                                ArrayRef<Module::ModuleFlagEntry> ModuleFlags,
                                Mangler *Mang, const TargetMachine &TM) const;
 };
 
 } // end namespace llvm
 
 #endif

llvm/trunk/include/llvm/LTO/LTOModule.h

struct NameAndAttributes {
     uint32_t           attributes;
     bool               isFunction;
     const llvm::GlobalValue *symbol;
   };
 
   llvm::OwningPtr<llvm::Module>           _module;
   llvm::OwningPtr<llvm::TargetMachine>    _target;
   llvm::MCObjectFileInfo ObjFileInfo;
+  StringSet                               _linkeropt_strings;
+  std::vector<const char *>               _deplibs;
+  std::vector<const char *>               _linkeropts;
   std::vector<NameAndAttributes>          _symbols;
 
   // _defines and _undefines only needed to disambiguate tentative definitions
   StringSet                               _defines;
   llvm::StringMap<NameAndAttributes>      _undefines;
   std::vector<const char*>                _asm_undefines;
   llvm::MCContext                         _context;
 
public:
 
   /// getSymbolName - Get the name of the symbol at the specified index.
   const char *getSymbolName(uint32_t index) {
     if (index < _symbols.size())
       return _symbols[index].name;
     return NULL;
   }
 
+  /// getDependentLibraryCount - Get the number of dependent libraries
+  uint32_t getDependentLibraryCount() {
+    return _deplibs.size();
+  }
+
+  /// getDependentLibrary - Get the dependent library at the specified index.
+  const char *getDependentLibrary(uint32_t index) {
+    if (index < _deplibs.size())
+      return _deplibs[index];
+    return NULL;
+  }
+
+  /// getLinkerOptCount - Get the number of linker options
+  uint32_t getLinkerOptCount() {
+    return _linkeropts.size();
+  }
+
+  /// getLinkerOpt - Get the linker option at the specified index.
+  const char *getLinkerOpt(uint32_t index) {
+    if (index < _linkeropts.size())
+      return _linkeropts[index];
+    return NULL;
+  }
+
   /// getLLVVMModule - Return the Module.
   llvm::Module *getLLVVMModule() { return _module.get(); }
 
   /// getAsmUndefinedRefs -
   const std::vector<const char*> &getAsmUndefinedRefs() {
     return _asm_undefines;
   }
 
 private:
+  /// parseMetadata - Parse metadata from the module
+  // FIXME: it only parses "Linker Options" metadata at the moment
+  void parseMetadata();
+
   /// parseSymbols - Parse the symbols from the module and model-level ASM and
   /// add them to either the defined or undefined lists.
   bool parseSymbols(std::string &errMsg);
 
   /// addPotentialUndefinedSymbol - Add a symbol which isn't defined just yet
   /// to a list to be resolved later.
   void addPotentialUndefinedSymbol(const llvm::GlobalValue *dcl, bool isFunc);
 

llvm/trunk/include/llvm/Target/TargetLoweringObjectFile.h

public:
   /// done.  This specifies the current context for codegen, and gives the
   /// lowering implementations a chance to set up their default sections.
   virtual void Initialize(MCContext &ctx, const TargetMachine &TM);
   
   virtual void emitPersonalityValue(MCStreamer &Streamer,
                                     const TargetMachine &TM,
                                     const MCSymbol *Sym) const;
 
+  /// getDepLibFromLinkerOpt - Extract the dependent library name from a linker
+  /// option string. Returns StringRef() if the option does not specify a library.
+  virtual StringRef getDepLibFromLinkerOpt(StringRef LinkerOption) const {
+    return StringRef();
+  }
+
   /// emitModuleFlags - Emit the module flags that the platform cares about.
   virtual void emitModuleFlags(MCStreamer &,
                                ArrayRef<Module::ModuleFlagEntry>,
                                Mangler *, const TargetMachine &) const {
   }
 
   /// shouldEmitUsedDirectiveFor - This hook allows targets to selectively
   /// decide not to emit the UsedDirective for some symbols in llvm.used.

llvm/trunk/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

getContext().getELFSection(".fini_array", ELF::SHT_FINI_ARRAY,
                                ELF::SHF_ALLOC,
                                SectionKind::getDataRel());
 }
 
 //===----------------------------------------------------------------------===//
 //                                 MachO
 //===----------------------------------------------------------------------===//
 
+/// getDepLibFromLinkerOpt - Extract the dependent library name from a linker
+/// option string. Returns StringRef() if the option does not specify a library.
+StringRef TargetLoweringObjectFileMachO::
+getDepLibFromLinkerOpt(StringRef LinkerOption) const {
+  const char *LibCmd = "-l";
+  if (LinkerOption.startswith(LibCmd))
+    return LinkerOption.substr(strlen(LibCmd));
+  return StringRef();
+}
+
 /// emitModuleFlags - Perform code emission for module flags.
 void TargetLoweringObjectFileMachO::
 emitModuleFlags(MCStreamer &Streamer,
                 ArrayRef<Module::ModuleFlagEntry> ModuleFlags,
                 Mangler *Mang, const TargetMachine &TM) const {
   unsigned VersionVal = 0;
   unsigned ImageInfoFlags = 0;
   MDNode *LinkerOptions = 0;
if (Kind.isReadOnly())
     return ReadOnlySection;
 
   if (Kind.isBSS())
     return BSSSection;
 
   return DataSection;
 }
 
+StringRef TargetLoweringObjectFileCOFF::
+getDepLibFromLinkerOpt(StringRef LinkerOption) const {
+  const char *LibCmd = "/DEFAULTLIB:";
+  if (LinkerOption.startswith(LibCmd))
+    return LinkerOption.substr(strlen(LibCmd));
+  return StringRef();
+}
+
 void TargetLoweringObjectFileCOFF::
 emitModuleFlags(MCStreamer &Streamer,
                 ArrayRef<Module::ModuleFlagEntry> ModuleFlags,
                 Mangler *Mang, const TargetMachine &TM) const {
   MDNode *LinkerOptions = 0;
 
   // Look for the "Linker Options" flag, since it's the only one we support.
   for (ArrayRef<Module::ModuleFlagEntry>::iterator

llvm/trunk/lib/LTO/LTOModule.cpp

 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTOModule.h"
 #include "llvm/ADT/OwningPtr.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCParser/MCAsmParser.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/MCTargetAsmParser.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/system_error.h"
+#include "llvm/Target/TargetLowering.h"
+#include "llvm/Target/TargetLoweringObjectFile.h"
 #include "llvm/Target/TargetRegisterInfo.h"
 #include "llvm/Transforms/Utils/GlobalStatus.h"
 using namespace llvm;
 
 LTOModule::LTOModule(llvm::Module *m, llvm::TargetMachine *t)
   : _module(m), _target(t),
     _context(_target->getMCAsmInfo(), _target->getRegisterInfo(), &ObjFileInfo),
     _mangler(t->getDataLayout()) {
LTOModule *LTOModule::makeLTOModule(MemoryBuffer *buffer,
   m->materializeAllPermanently();
 
   LTOModule *Ret = new LTOModule(m.take(), target);
   if (Ret->parseSymbols(errMsg)) {
     delete Ret;
     return NULL;
   }
 
+  Ret->parseMetadata();
+
   return Ret;
 }
 
 /// makeBuffer - Create a MemoryBuffer from a memory range.
 MemoryBuffer *LTOModule::makeBuffer(const void *mem, size_t length) {
   const char *startPtr = (const char*)mem;
   return MemoryBuffer::getMemBuffer(StringRef(startPtr, length), "", false);
 }
for (StringMap<NameAndAttributes>::iterator u =_undefines.begin(),
     // it is a tentative definition.
     if (_defines.count(u->getKey())) continue;
     NameAndAttributes info = u->getValue();
     _symbols.push_back(info);
   }
 
   return false;
 }
+
+/// parseMetadata - Parse metadata from the module
+void LTOModule::parseMetadata() {
+  // Linker Options
+  if (Value *Val = _module->getModuleFlag("Linker Options")) {
+    MDNode *LinkerOptions = cast<MDNode>(Val);
+    for (unsigned i = 0, e = LinkerOptions->getNumOperands(); i != e; ++i) {
+      MDNode *MDOptions = cast<MDNode>(LinkerOptions->getOperand(i));
+      for (unsigned ii = 0, ie = MDOptions->getNumOperands(); ii != ie; ++ii) {
+        MDString *MDOption = cast<MDString>(MDOptions->getOperand(ii));
+        StringRef Op = _linkeropt_strings.
+            GetOrCreateValue(MDOption->getString()).getKey();
+        StringRef DepLibName = _target->getTargetLowering()->
+            getObjFileLowering().getDepLibFromLinkerOpt(Op);
+        if (!DepLibName.empty())
+          _deplibs.push_back(DepLibName.data());
+        else if (!Op.empty())
+          _linkeropts.push_back(Op.data());
+      }
+    }
+  }
+
+  // Add other interesting metadata here.
+}

llvm/trunk/tools/lto/lto.cpp

 
 /// lto_module_get_symbol_attribute - Returns the attributes of the ith symbol
 /// in the object module.
 lto_symbol_attributes lto_module_get_symbol_attribute(lto_module_t mod,
                                                       unsigned int index) {
   return mod->getSymbolAttributes(index);
 }
 
+/// lto_module_get_num_deplibs - Returns the number of dependent libraries in
+/// the object module.
+unsigned int lto_module_get_num_deplibs(lto_module_t mod) {
+  return mod->getDependentLibraryCount();
+}
+
+/// lto_module_get_deplib - Returns the ith dependent library in the module.
+const char* lto_module_get_deplib(lto_module_t mod, unsigned int index) {
+  return mod->getDependentLibrary(index);
+}
+
+/// lto_module_get_num_linkeropts - Returns the number of linker options in the
+/// object module.
+unsigned int lto_module_get_num_linkeropts(lto_module_t mod) {
+  return mod->getLinkerOptCount();
+}
+
+/// lto_module_get_linkeropt - Returns the ith linker option in the module.
+const char* lto_module_get_linkeropt(lto_module_t mod, unsigned int index) {
+  return mod->getLinkerOpt(index);
+}
+
 /// Set a diagnostic handler.
 void lto_codegen_set_diagnostic_handler(lto_code_gen_t cg,
                                         lto_diagnostic_handler_t diag_handler,
                                         void *ctxt) {
   cg->setDiagnosticHandler(diag_handler, ctxt);
 }
 
 /// lto_codegen_create - Instantiates a code generator. Returns NULL if there

llvm/trunk/tools/lto/lto.exports

 lto_get_error_message
 lto_get_version
 lto_initialize_disassembler
 lto_module_create
 lto_module_create_from_fd
 lto_module_create_from_fd_at_offset
 lto_module_create_from_memory
+lto_module_get_deplib
+lto_module_get_linkeropt
+lto_module_get_num_deplibs
+lto_module_get_num_linkeropts
 lto_module_get_num_symbols
 lto_module_get_symbol_attribute
 lto_module_get_symbol_name
 lto_module_get_target_triple
 lto_module_set_target_triple
 lto_module_is_object_file
 lto_module_is_object_file_for_target
 lto_module_is_object_file_in_memory