Introduce libLTO C APIs to target the "resolution-based" new LTO API

[mehdi_amini]

Summary

The goal is to deprecate the existing C API exposed by libLTO and
remove it 2 or 3 releases from now.
This commit introduces the basic replacement APIs, and it not intended
to be a complete replacement at this point.

A new tool is introduced: llvm-liblto. This tool links statically
only to libSupport, and link dynamically to libLTO. It offers a way
to test the new C API in tree without having a linker using this API
available. This was a major painpoint of the existing API.

This is a work-in-progress, I'm sharing it now to show the general
direction I took until now.

[pcc]

A few comments, mostly on the interface.

llvm/include/llvm-c/lto2.h

52

What is the minimum level of C we support in our C interfaces? If it is C89 we should use /* */ in this file.

55

Please be more consistent about type naming (i.e. lto_foo vs LTOFoo).

123

How are you planning to expose other symbol attributes? I'd use an array-of-tagged-unions approach for that as well.

133

Is this data structure needed? I think it would be sufficient to just have a way of querying for the number of symbols and symbol attributes for the nth symbol given an lto_input_t.

161

I would imagine that a client would use this function to destroy the input file after retrieving the list of symbols without adding it to an LTO object, for example if the client is ar or wants to support gold's --start-lib/--end-lib flags.

184

Maybe there should be a tag field in lto_symbol_resolutions_t so that we can introduce new versions of the lto_symbol_resolution_t data structure by incrementing the tag.

189

Document that this function takes ownership of the lto_input_t.

llvm/include/llvm/LTO/LTO.h

125–128

Is this change necessary?

llvm/tools/lto/lto2.cpp

110

This isn't guaranteed to be null terminated (especially after the string table changes), so this probably needs to return a pointer and a length.

[tobiasvk]

llvm/include/llvm/LTO/LTO.h

127

InputFile::symbols() only contains global and non-format specific symbols, so InputFile::Symbol::isGlobal will always be true (and ::isFormatSpecific always false).

If you do need local and/or private symbols too, you could add a symbolsWithLocals() or similar to InputFile. I would have a use for that with LTO + Linker Scripts :)

[mehdi_amini]

Thanks, I'll address all these.

llvm/include/llvm-c/lto2.h

52

Right, I planned to adopt the C like comments.

55

That's another one that I noticed in the middle of the development: the existing LTO C API was likely based on ld64 convention, but I rather use the LLVM one. I'll likely update all the APIs as well.

123

I was planning to add a new API for each accessor. An array isn't great: it requires to allocate memory (which mean deallocating as well) and search in the array for the name when it is the only thing you want. I expect the client side to be more annoying to implement with an array.
(I may have misunderstood what you meant here).

133

I could do it now, I wrote this when the symbols were not backed by an array but some kind of list that we had to iterate through. So having an API with get_ith_symbol(int n) was requiring to iterate n times every call.
(we discuss it on IRC last month).

Now that you added the IR symtab, this does not add much value anymore.

161

Good point, I'll document this!

llvm/include/llvm/LTO/LTO.h

125–128

No in this patch, at some point I was populating a special LTOSymbol structure containing all the information here.

[pcc]

llvm/include/llvm-c/lto2.h

123

Yes, the allocation could be a problem if it were the responsibility of the API. But another possibility would be for the client to allocate the data structure and have the API fill it in.

The client side would look like this:

lto_symbol_attr attrs[2];
attrs[0].tag = tag_name;
attrs[1].tag = tag_visibility;

for (int i = 0; i != n; ++i) {
  lto_get_symbol_attrs(input_file, i, attrs, sizeof(attrs));
  char *name = attrs[0].value.u_ptr;
  lto_visibility vis = attrs[1].value.u_int;
  // process the symbol
}

What do you think?

133

Right, I expect that we will continue to provide random access via the irsymtab (or whatever we replace it with).

llvm/lib/LTO/LTO.cpp

527

I think you can move this check into lto2.cpp now that the symbol list is random access.

[Wallbraker]

llvm/include/llvm-c/lto2.h

61

These will need to be prefixed with LTO or something like that. See the LLVM-C enums.

77

Add to the comment a note about threading. If you use multiple threads does the callback happen on different threads?

Is there a way to avoid using a callback?

109

This API feels very un LLVM-Cy, maybe make the lto_config_t object opaque and have functions to add params to it?

207

Why not just destroy it automatically?

[mehdi_amini]

llvm/include/llvm-c/lto2.h

123

That looks nicer this way.

That said I forgot to mention that I planned to also have a c++ wrapper for the C API, so that the client could have a C++-like API that would target the C API exposed by LTO.
Something like:

namespace {
class LTOSymbol {
  lto_symbol_t Sym;
public:
  LTOStringRef getName() {
    return lto_symbol_name(Sym);
  }
};
}

But also something that would allow to write:

LTOInput Input = ...;
for (LTOSymbol Sym : Input) {
  auto Name = Sym.getName();
  ...
}

So I'd like to have an API that allows to write a nice wrapper for the client.

207

Not auto-destroying leaves the possibility to introduce new APIs that can be called after lto_run without breaking existing client.

llvm/lib/LTO/LTO.cpp

527

Correct!

[steven_wu]

Some high level comments before I look into details. Are we expecting those two APIs to co-exist? Maybe just create a new libLTO and replacing the existing one in few releases? Maybe libLTO.1.dylib and libLTO.2.dylib, with a cmake configuration for which one to symlink to?

Really like the idea of llvm-liblto. It would be even better if we have an easy way to run llvm-lto/llvm-lto2 and llvm-liblto on the same tests and make sure it pass on both side.

llvm/include/llvm-c/lto2.h

+/*==-- llvm-c/lto2.h - LTO Public C Interface ---------------------*- C -*-===*\
+|*                                                                            *|
+|*                     The LLVM Compiler Infrastructure                       *|
+|*                                                                            *|
+|* This file is distributed under the University of Illinois Open Source      *|
+|* License. See LICENSE.TXT for details.                                      *|
+|*                                                                            *|
+|*===----------------------------------------------------------------------===*|
+|*                                                                            *|
+|* This header provides public interface to an abstract link time optimization*|
+|* library.  LLVM provides an implementation of this interface for use with   *|
+|* llvm bitcode files.                                                        *|
+|*                                                                            *|
+\*===----------------------------------------------------------------------===*/
+
+#ifndef LLVM_C_LTO2_H
+#define LLVM_C_LTO2_H
+
+#include <stdint.h>
+
+#ifdef __cplusplus
+#include <cstddef>
+#else
+#include <stddef.h>
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifndef __cplusplus
+#if !defined(_MSC_VER)
+#include <stdbool.h>
+typedef bool lto_bool_t;
+#else
+/* MSVC in particular does not have anything like _Bool or bool in C, but we can
+ at least make sure the type is the same size.  The implementation side will
+ use C++ bool. */
+typedef unsigned char lto_bool_t;
+#endif
+#else
+typedef bool lto_bool_t;
+#endif
+
+/**
+ * @defgroup LLVMCLTO2 LTO Resolution API
+ * @ingroup LLVMC
+ *
+ * @{
+ */
+
+/// Error handling for these API: they all returned a string in case or error,
+/// or a nullptr on success. The string is mallocated and has to be free'd by
+/// the client.
+typedef const char *LTOErrorStr;
+
+/// An enumeration for the kinds of visibility of global values.
+enum VisibilityTypes {
+  DefaultVisibility = 0, ///< The GV is visible
+  HiddenVisibility,      ///< The GV is hidden
+  ProtectedVisibility    ///< The GV is protected
+};
+
+/// The ThinLTO backend instance, it can be a in-memory process, or specialized
+/// to interact with a distributed build system for example. The API only
+/// expose the in-memory threaded instance at this point.
+typedef struct LLVMOpaqueThinLTOBackend *thinlto_backend_t;
+
+/** opaque reference to a code generator */
+typedef struct LLVMOpaqueLTOInstance *lto_instance_t;
+
+/** Single LTO Output file. */
+typedef struct LLVMOpaqueLTOOutput *lto_output_t;
+
+/** Callback passed by the client to the API to receive objects as they are
+ * generated */
+typedef void (*ObjectReadyCallback)(lto_output_t, void *context);
+
+/// Configuration parameters accepted by the LTO API.
+enum lto_config_params_tag {
+  /// Boolean enabling/disabling -save-temps (default to false).
+  LTOCFG_SAVETEMPS,
+  /// Integer [0-3] controlling to the optimization level.
+  LTOCFG_OPTLEVEL
+};
+
+/// Configuration parameter for the LTO API, composed of a tag indicating the
+/// which parameter is set here, and a value that can be an integer or a string.
+typedef struct {
+  /// Tag indicating the type of parameter.
+  lto_config_params_tag Tag;
+  /// Value for the parameter.
+  union {
+    const char *Str;
+    long long int Num;
+  } Value;
+} lto_config_param_t;
+
+/// List of parameters to initializing an LTO instance.
+typedef struct {
+  lto_config_param_t *Params;
+  size_t Length;
+} lto_config_t;
+
+/// Helper function to create an integer parameter.
+lto_config_param_t lto_make_int_param(lto_config_params_tag Tag, int Value);
+
+/// Helper function to create a string parameter.
+lto_config_param_t lto_make_str_param(lto_config_params_tag Tag, const char *);
+
+/**
+ * @defgroup LLVMCLTO2Symbol LTO Symbol
+ * @ingroup LLVMCLTO2
+ *
+ * @{
+ */
+
+/// This represent the opaque type used for representing and iterating over
+/// symbols.
+typedef struct LLVMOpaqueSymbol *lto_symbol_t;
+
+/// Retrieve the name for a symbol.
+LTOErrorStr lto_symbol_name(lto_symbol_t);
+
+/**
+ * @}  // endgroup LLVMCLTO2Symbol
+ */
+
+/// List of symbols in an object file.
+typedef struct {
+  void *Symbols;
+  size_t NumSyms;
+} LTOSymbols;
+
+/// Retrieve the symbol list in an object file.
+LTOErrorStr lto_get_symbol(LTOSymbols, int, lto_symbol_t *);
+
+/** Create an instance of a thinlto_backend_t operating with multiple threads
+* in-memory. If numthread is zero, LLVM will adapt the number of threads to the
+* machine
+*/
+LTOErrorStr thinlto_create_threaded_backend(size_t numthread,
+                                            thinlto_backend_t *);
+
+/** Create an LTO instance with a given thinlto_backend_t instance and a
+ configuration. */
+LTOErrorStr lto_create_instance(lto_config_t, thinlto_backend_t,
+                                lto_instance_t *);
+
+/** Destroy an LTO instance */
+void lto_destroy_instance(lto_instance_t);
+
+/** Opaque reference to a loaded object module */
+typedef struct LLVMOpaqueLTOInput *lto_input_t;
+
+/** Create an LTO input from a bitcode buffer */
+LTOErrorStr lto_create_input(const void *mem, size_t length,
+                             const char *BufferId, lto_input_t *);
+
+/** FIXME: when? */
+void lto_destroy_input(lto_input_t input);
+
+/// Get all the symbols for a given input.
+LTOErrorStr lto_get_symbols(lto_input_t, LTOSymbols *);
+
+/// The resolution for a symbol. The linker must provide a SymbolResolution for
+/// each global symbol based on its internal resolution of that symbol.
+typedef struct {
+  /// The linker has chosen this definition of the symbol.
+  unsigned Prevailing : 1;
+
+  /// The definition of this symbol is unpreemptable at runtime and is known to
+  /// be in this linkage unit.
+  unsigned FinalDefinitionInLinkageUnit : 1;
+
+  /// The definition of this symbol is visible outside of the LTO unit.
+  unsigned VisibleToRegularObj : 1;
+} lto_symbol_resolution_t;
+
+/** List of resolutions, to be passed in when adding an input file to the LTO
+ * instance.
+ */
+typedef struct {
+  lto_symbol_resolution_t *resolutions;
+  size_t num;
+} lto_symbol_resolutions_t;
+
+/** Add an input to the LTO instance */
+LTOErrorStr lto_add_input(lto_instance_t, lto_input_t,
+                          lto_symbol_resolutions_t);
+
+/** Buffer corresponding to an output.  */
+typedef struct {
+  size_t Task;
+  const char *mem;
+  size_t length;
+} LTOBuffer;
+
+/** Get a buffer from an output. */
+LTOBuffer lto_get_output_buffer(lto_output_t);
+
+/** Destroy an LTO output. This will invalidate buffer retrieve from it. */
+void lto_destroy_output(lto_output_t);
+
+/** Run the LTO instance over the set of input that have been added. This is
+ * supposed to be called once per instance. The instance can't be reused and
+ * should be destroyed after this call returns.
+ */
+LTOErrorStr lto_run(lto_instance_t, ObjectReadyCallback, void *CallbackCtx,
+                    const char *cache_dir, const char *OutputFilePrefix);
+
+/**
+ * @} // endgroup LLVMCLTO2
+ */
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* LLVM_C_LTO2_H */

llvm/include/llvm/LTO/LTO.h

public:
     using irsymtab::Symbol::isUndefined;
     using irsymtab::Symbol::isCommon;
     using irsymtab::Symbol::isWeak;
     using irsymtab::Symbol::isIndirect;
     using irsymtab::Symbol::getName;
     using irsymtab::Symbol::getVisibility;
     using irsymtab::Symbol::canBeOmittedFromSymbolTable;
     using irsymtab::Symbol::isTLS;
+    using irsymtab::Symbol::isUsed;
+    using irsymtab::Symbol::isGlobal;
+    using irsymtab::Symbol::isFormatSpecific;
+    using irsymtab::Symbol::isUnnamedAddr;
     using irsymtab::Symbol::getComdatIndex;
     using irsymtab::Symbol::getCommonSize;
     using irsymtab::Symbol::getCommonAlignment;
     using irsymtab::Symbol::getCOFFWeakExternalFallback;
   };
 
   /// A range over the symbols in this InputFile.
   ArrayRef<Symbol> symbols() const { return Symbols; }

llvm/lib/LTO/LTO.cpp

while (MsymI != MsymE) {
           !(Flags & object::BasicSymbolRef::SF_FormatSpecific))
         return;
       ++MsymI;
     }
   };
   Skip();
 
   for (const InputFile::Symbol &Sym : Syms) {
-    assert(ResI != ResE);
+    if (ResI == ResE)
+      return make_error<StringError>("List of resolution is too short",
+                                     std::error_code());
+
     SymbolResolution Res = *ResI++;
     addSymbolToGlobalRes(Sym, Res, 0);
 
     assert(MsymI != MsymE);
     ModuleSymbolTable::Symbol Msym = *MsymI++;
     Skip();
 
     if (GlobalValue *GV = Msym.dyn_cast<GlobalValue *>()) {

llvm/lib/Support/ThreadPool.cpp

 
 #if LLVM_ENABLE_THREADS
 
 // Default to std::thread::hardware_concurrency
 ThreadPool::ThreadPool() : ThreadPool(std::thread::hardware_concurrency()) {}
 
 ThreadPool::ThreadPool(unsigned ThreadCount)
     : ActiveThreads(0), EnableFlag(true) {
+  if (ThreadCount == 0)
+    ThreadCount = std::thread::hardware_concurrency();
+
   // Create ThreadCount threads that will loop forever, wait on QueueCondition
   // for tasks to be queued or the Pool to be destroyed.
   Threads.reserve(ThreadCount);
   for (unsigned ThreadID = 0; ThreadID < ThreadCount; ++ThreadID) {
     Threads.emplace_back([&] {
       while (true) {
         PackagedTaskTy Task;
         {

llvm/test/lit.cfg

for pattern in [r"\bbugpoint\b(?!-)",
                 r"\bllvm-cxxdump\b",
                 r"\bllvm-diff\b",
                 r"\bllvm-dis\b",
                 r"\bllvm-dsymutil\b",
                 r"\bllvm-dwarfdump\b",
                 r"\bllvm-extract\b",
                 r"\bllvm-lib\b",
                 r"\bllvm-link\b",
+                r"\bllvm-liblto\b",
                 r"\bllvm-lto\b",
                 r"\bllvm-lto2\b",
                 r"\bllvm-mc\b",
                 r"\bllvm-mcmarkup\b",
                 r"\bllvm-modextract\b",
                 r"\bllvm-nm\b",
                 r"\bllvm-objdump\b",
                 r"\bllvm-pdbdump\b",

llvm/test/tools/llvm-liblto/X86/Inputs/funcimport2.ll

+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-macosx10.11.0"
+
+
+define i32 @main() #0 {
+entry:
+  call void (...) @foo()
+  ret i32 0
+}
+
+declare void @foo(...) #1

llvm/test/tools/llvm-liblto/X86/Inputs/mixed_lto.ll

+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+declare i32 @g()
+define i32 @main() {
+  call i32 @g()
+  ret i32 0
+}

llvm/test/tools/llvm-liblto/X86/funcimport2.ll

+; Do setup work for all below tests: generate bitcode and combined index
+; RUN: opt -module-summary %s -o %t1.bc
+; RUN: opt -module-summary %p/Inputs/funcimport2.ll -o %t2.bc
+
+; RUN: llvm-liblto %t1.bc %t2.bc -o %t.o -save-temps \
+; RUN:     -r=%t1.bc,_foo,plx \
+; RUN:     -r=%t2.bc,_main,plx \
+; RUN:     -r=%t2.bc,_foo,l
+; RUN: llvm-dis %t.o.1.3.import.bc -o - | FileCheck %s
+; CHECK: define available_externally void @foo()
+
+; We shouldn't do any importing at -O0
+; rm -f %t.o.1.3.import.bc
+; RUN: llvm-liblto %t1.bc %t2.bc -o %t.o -save-temps \
+; RUN:     -O0 \
+; RUN:     -r=%t1.bc,_foo,plx \
+; RUN:     -r=%t2.bc,_main,plx \
+; RUN:     -r=%t2.bc,_foo,l
+; RUN: llvm-dis %t.o.1.3.import.bc -o - | FileCheck %s --check-prefix=CHECKO0
+; CHECKO0: declare void @foo(...)
+
+target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-macosx10.11.0"
+
+define void @foo() #0 {
+entry:
+  ret void
+}

llvm/test/tools/llvm-liblto/X86/lit.local.cfg

+if not 'X86' in config.root.targets:
+  config.unsupported = True
+

llvm/test/tools/llvm-liblto/X86/mixed_lto.ll

+; Test mixed-mode LTO (mix of regular and thin LTO objects)
+; RUN: opt %s -o %t1.o
+; RUN: opt -module-summary %p/Inputs/mixed_lto.ll -o %t2.o
+
+; RUN: llvm-liblto -o %t3.o %t2.o %t1.o -r %t2.o,main,px -r %t2.o,g, -r %t1.o,g,px
+
+; Task 0 is the regular LTO file (this file)
+; RUN: llvm-nm %t3.o.0 | FileCheck %s --check-prefix=NM0
+; NM0: T g
+
+; Task 1 is the (first) ThinLTO file (Inputs/mixed_lto.ll)
+; RUN: llvm-nm %t3.o.1 | FileCheck %s --check-prefix=NM1
+; NM1-DAG: T main
+; NM1-DAG: U g
+
+; Do the same test again, but with the regular and thin LTO modules in the same file.
+; RUN: llvm-cat -b -o %t4.o %t2.o %t1.o
+; RUN: llvm-liblto -o %t5.o %t4.o -r %t4.o,main,px -r %t4.o,g, -r %t4.o,g,px
+; RUN: llvm-nm %t5.o.0 | FileCheck %s --check-prefix=NM0
+; RUN: llvm-nm %t5.o.1 | FileCheck %s --check-prefix=NM1
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+define i32 @g() {
+  ret i32 0
+}

llvm/test/tools/llvm-liblto/X86/nodatalayout.ll

+; RUN: llvm-as < %s > %t1.bc
+
+; Reject input modules without a datalayout.
+; RUN: not llvm-liblto %t1.bc -o %t.o \
+; RUN:  -r %t1.bc,patatino,px 2>&1 | FileCheck %s
+
+; CHECK: input module has no datalayout
+
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @patatino() {
+  ret void
+}

llvm/test/tools/llvm-liblto/errors.ll

+; RUN: llvm-as %s -o %t.bc
+; RUN: not llvm-liblto -o %t2.o %t.bc 2>&1 | FileCheck --check-prefix=ERR1 %s
+; RUN: not llvm-liblto -o %t2.o -r %t.bc,foo,p -r %t.bc,bar,p %t.bc 2>&1 | FileCheck --check-prefix=ERR2 %s
+; RUN: not llvm-liblto -o %t2.o -r %t.bc,foo,q %t.bc 2>&1 | FileCheck --check-prefix=ERR3 %s
+; RUN: not llvm-liblto -o %t2.o -r foo %t.bc 2>&1 | FileCheck --check-prefix=ERR4 %s
+
+; ERR1: missing symbol resolution for {{.*}}.bc,foo
+; ERR2: unused symbol resolution for {{.*}}.bc,bar
+; ERR3: invalid character q in resolution: {{.*}}.bc,foo
+; ERR4: invalid resolution: foo
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@foo = global i32 0

llvm/tools/lto/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   ${LLVM_TARGETS_TO_BUILD}
   BitReader
   Core
   LTO
   MC
   MCDisassembler
   Support
   Target
   )
 
 set(SOURCES
   LTODisassembler.cpp
   lto.cpp
+  lto2.cpp
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/lto.exports)
 
 add_llvm_library(LTO SHARED ${SOURCES} DEPENDS intrinsics_gen)
 
 install(FILES ${LLVM_MAIN_INCLUDE_DIR}/llvm-c/lto.h
   DESTINATION include/llvm-c
   COMPONENT LTO)
 
 if (APPLE)
   set(LTO_VERSION ${LLVM_VERSION_MAJOR})
   if(LLVM_LTO_VERSION_OFFSET)
     math(EXPR LTO_VERSION "${LLVM_VERSION_MAJOR} + ${LLVM_LTO_VERSION_OFFSET}")
   endif()
   set_property(TARGET LTO APPEND_STRING PROPERTY
               LINK_FLAGS
               " -compatibility_version 1 -current_version ${LTO_VERSION}.${LLVM_VERSION_MINOR}.${LLVM_VERSION_PATCH}")
 endif()
+
+set(LLVM_LINK_COMPONENTS "")
+set(LLVM_EXPORTED_SYMBOL_FILE "")
+add_llvm_tool_subdirectory(llvm-liblto)
 No newline at end of file

llvm/tools/lto/llvm-liblto/CMakeLists.txt

+add_llvm_executable(llvm-liblto
+  llvm-liblto.cpp
+)
+target_link_libraries(llvm-liblto LTO LLVMSupport)
 No newline at end of file

llvm/tools/lto/llvm-liblto/llvm-liblto.cpp

+//===-llvm-liblto - LLVM Link Time Optimizer ------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements a tool that links dynamically to the libLTO shared
+// library and interacts with it only through the C API. It is intended to
+// provide a in-tree testing for the linkers that are using libLTO.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm-c/lto2.h"
+
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <list>
+#include <map>
+
+using namespace llvm;
+
+static cl::opt<char>
+    OptLevel("O", cl::desc("Optimization level. [-O0, -O1, -O2, or -O3] "
+                           "(default = '-O2')"),
+             cl::Prefix, cl::ZeroOrMore, cl::init('2'));
+
+static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
+                                            cl::desc("<input bitcode files>"));
+
+static cl::opt<std::string> OutputFilename("o", cl::Required,
+                                           cl::desc("Output filename"),
+                                           cl::value_desc("filename"));
+
+static cl::opt<std::string> CacheDir("cache-dir", cl::desc("Cache Directory"),
+                                     cl::value_desc("directory"));
+
+static cl::opt<bool> SaveTemps("save-temps", cl::desc("Save temporary files"));
+
+static cl::opt<int> Threads("thinlto-threads", cl::init(0));
+
+static cl::list<std::string> SymbolResolutions(
+    "r",
+    cl::desc("Specify a symbol resolution: filename,symbolname,resolution\n"
+             "where \"resolution\" is a sequence (which may be empty) of the\n"
+             "following characters:\n"
+             " p - prevailing: the linker has chosen this definition of the\n"
+             "     symbol\n"
+             " l - local: the definition of this symbol is unpreemptable at\n"
+             "     runtime and is known to be in this linkage unit\n"
+             " x - externally visible: the definition of this symbol is\n"
+             "     visible outside of the LTO unit\n"
+             "A resolution for each symbol must be specified."),
+    cl::ZeroOrMore);
+
+void check(const char *ErrorStr) {
+  if (!ErrorStr)
+    return;
+
+  errs() << "llvm-liblto error: " << ErrorStr << "\n";
+  exit(1);
+}
+void check(const char *ErrorStr, StringRef Msg) {
+  if (!ErrorStr)
+    return;
+
+  errs() << "llvm-liblto error: " << Msg << " " << ErrorStr << "\n";
+  exit(1);
+}
+static void check(Error E, std::string Msg) {
+  if (!E)
+    return;
+  handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
+    errs() << "llvm-lto2: " << Msg << ": " << EIB.message().c_str() << '\n';
+  });
+  exit(1);
+}
+
+template <typename T> static T check(Expected<T> E, std::string Msg) {
+  if (E)
+    return std::move(*E);
+  check(E.takeError(), Msg);
+  return T();
+}
+
+static void check(std::error_code EC, std::string Msg) {
+  check(errorCodeToError(EC), Msg);
+}
+
+template <typename T> static T check(ErrorOr<T> E, std::string Msg) {
+  if (E)
+    return std::move(*E);
+  check(E.getError(), Msg);
+  return T();
+}
+
+// Called by libLTO after completing CodeGen for every output file.
+void Callback(lto_output_t Output, void *Context) {
+  auto Buffer = lto_get_output_buffer(Output);
+  std::string Path = OutputFilename + "." + utostr(Buffer.Task);
+  std::error_code EC;
+  auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
+  check(EC, Path);
+  *S << StringRef(Buffer.mem, Buffer.length);
+}
+
+int main(int argc, char **argv) {
+  cl::ParseCommandLineOptions(argc, argv, "libLTO API test harness");
+
+  // FIXME: Workaround PR30396 which means that a symbol can appear
+  // more than once if it is defined in module-level assembly and
+  // has a GV declaration. We allow (file, symbol) pairs to have multiple
+  // resolutions and apply them in the order observed.
+  std::map<std::pair<std::string, std::string>,
+           std::list<lto_symbol_resolution_t>>
+      CommandLineResolutions;
+  for (std::string R : SymbolResolutions) {
+    StringRef Rest = R;
+    StringRef FileName, SymbolName;
+    std::tie(FileName, Rest) = Rest.split(',');
+    if (Rest.empty()) {
+      llvm::errs() << "invalid resolution: " << R << '\n';
+      return 1;
+    }
+    std::tie(SymbolName, Rest) = Rest.split(',');
+    lto_symbol_resolution_t Res;
+    for (char C : Rest) {
+      if (C == 'p')
+        Res.Prevailing = true;
+      else if (C == 'l')
+        Res.FinalDefinitionInLinkageUnit = true;
+      else if (C == 'x')
+        Res.VisibleToRegularObj = true;
+      else {
+        llvm::errs() << "invalid character " << C << " in resolution: " << R
+                     << '\n';
+        return 1;
+      }
+    }
+    CommandLineResolutions[{FileName, SymbolName}].push_back(Res);
+  }
+
+  std::vector<std::unique_ptr<MemoryBuffer>> MBs;
+  std::vector<lto_config_param_t> Params;
+  std::string OutputTempsFilename = OutputFilename + ".";
+  if (SaveTemps)
+    Params.push_back(
+        lto_make_str_param(LTOCFG_SAVETEMPS, OutputTempsFilename.c_str()));
+  Params.push_back(lto_make_int_param(LTOCFG_OPTLEVEL, OptLevel - '0'));
+
+  lto_config_t Config = {&Params[0], Params.size()};
+  thinlto_backend_t ThinBackend;
+  // FIXME: handle createWriteIndexesThinBackend
+  check(thinlto_create_threaded_backend(Threads, &ThinBackend));
+  lto_instance_t LTOInstance;
+  check(lto_create_instance(Config, ThinBackend, &LTOInstance));
+
+  bool HasErrors = false;
+  for (const std::string &F : InputFilenames) {
+    std::unique_ptr<MemoryBuffer> MB = check(MemoryBuffer::getFile(F), F);
+    lto_input_t LTOInputFile;
+    check(lto_create_input(MB->getBufferStart(), MB->getBufferSize(), F.c_str(),
+                           &LTOInputFile));
+
+    LTOSymbols Symbols;
+    check(lto_get_symbols(LTOInputFile, &Symbols));
+    std::vector<lto_symbol_resolution_t> Res;
+    for (unsigned CurrSymId = 0; CurrSymId < Symbols.NumSyms; ++CurrSymId) {
+      lto_symbol_t Sym;
+      lto_get_symbol(Symbols, CurrSymId, &Sym);
+      auto *SymName = lto_symbol_name(Sym);
+      auto I = CommandLineResolutions.find({F, SymName});
+      if (I == CommandLineResolutions.end()) {
+        llvm::errs() << argv[0] << ": missing symbol resolution for " << F
+                     << ',' << SymName << '\n';
+        HasErrors = true;
+      } else {
+        Res.push_back(I->second.front());
+        I->second.pop_front();
+        if (I->second.empty())
+          CommandLineResolutions.erase(I);
+      }
+    }
+
+    if (HasErrors)
+      continue;
+
+    MBs.push_back(std::move(MB));
+
+    lto_symbol_resolutions_t Resolutions;
+    Resolutions.resolutions = Res.data();
+    Resolutions.num = Res.size();
+
+    check(lto_add_input(LTOInstance, LTOInputFile, Resolutions), F);
+  }
+
+  if (!CommandLineResolutions.empty()) {
+    HasErrors = true;
+    for (auto UnusedRes : CommandLineResolutions)
+      llvm::errs() << argv[0] << ": unused symbol resolution for "
+                   << UnusedRes.first.first << ',' << UnusedRes.first.second
+                   << '\n';
+  }
+  if (HasErrors)
+    return 1;
+
+  check(lto_run(LTOInstance, Callback, nullptr, CacheDir.c_str(),
+                OutputFilename.c_str()));
+}

llvm/tools/lto/lto.exports

 lto_module_is_thinlto
 thinlto_codegen_add_must_preserve_symbol
 thinlto_codegen_add_cross_referenced_symbol
 thinlto_codegen_set_final_cache_size_relative_to_available_space
 thinlto_codegen_set_codegen_only
 thinlto_codegen_disable_codegen
 thinlto_module_get_num_object_files
 thinlto_module_get_object_file
-thinlto_set_generated_objects_dir
+thinlto_set_generated_objects_dir
+thinlto_create_threaded_backend
+lto_create_instance
+lto_destroy_instance
+lto_create_input
+lto_destroy_input
+lto_add_input
+lto_get_output_buffer
+lto_run
+lto_destroy_output
+lto_get_symbols
+lto_make_int_param
+lto_make_str_param
+lto_symbol_name
+lto_get_symbol
 No newline at end of file

llvm/tools/lto/lto2.cpp

+//===-lto2.cpp - LLVM Link Time Optimizer ---------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the resolution-based Link Time Optimization library.
+// This library is intended to be used by linker to optimize code at link time.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm-c/lto2.h"
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Bitcode/BitcodeReader.h"
+#include "llvm/ExecutionEngine/ObjectMemoryBuffer.h"
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/IR/DiagnosticPrinter.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/LTO/Caching.h"
+#include "llvm/LTO/LTO.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <iterator>
+
+using namespace llvm;
+using namespace llvm::lto;
+
+struct LTOOutput {
+  size_t Task;
+  std::unique_ptr<MemoryBuffer> MB;
+};
+
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(LTOOutput, lto_output_t)
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(InputFile, lto_input_t)
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(ThinBackend, thinlto_backend_t)
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(LTO, lto_instance_t)
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(InputFile::Symbol, lto_symbol_t)
+
+// Global initialization for the LLVM internals. Can be called many times but
+// will do something only the first time.
+static void init_once() {
+  llvm::once_flag once;
+  llvm::call_once(once, []() {
+    InitializeAllTargets();
+    InitializeAllTargetMCs();
+    InitializeAllAsmPrinters();
+    InitializeAllAsmParsers();
+  });
+}
+
+extern "C" lto_config_param_t lto_make_int_param(lto_config_params_tag Tag,
+                                                 int Value) {
+  lto_config_param_t Param;
+  Param.Tag = Tag;
+  Param.Value.Num = Value;
+  return Param;
+}
+
+extern "C" lto_config_param_t lto_make_str_param(lto_config_params_tag Tag,
+                                                 const char *Str) {
+  lto_config_param_t Param;
+  Param.Tag = Tag;
+  Param.Value.Str = Str;
+  return Param;
+}
+
+// Create an LTO input from a bitcode buffer
+extern "C" LTOErrorStr lto_create_input(const void *Data, size_t Length,
+                                        const char *BufferId,
+                                        lto_input_t *NewInput) {
+  *NewInput = nullptr;
+  MemoryBufferRef BufferRef(StringRef((const char *)Data, Length), BufferId);
+  Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
+  if (!ObjOrErr) {
+    auto Msg =
+        "Could not read bitcode from file : " + toString(ObjOrErr.takeError());
+    return strdup(Msg.c_str());
+  }
+
+  std::unique_ptr<InputFile> Obj = std::move(*ObjOrErr);
+  *NewInput = wrap(Obj.release());
+  return nullptr;
+}
+
+void lto_destroy_input(lto_input_t input) {
+  InputFile *File = unwrap(input);
+  delete File;
+}
+
+extern "C" LTOErrorStr lto_get_symbol(LTOSymbols Symbols, int Id,
+                                      lto_symbol_t *Sym) {
+  if (Id < 0 || (size_t)Id > Symbols.NumSyms)
+    return strdup("lto_get_symbol invalid ID requested");
+  if (!Sym)
+    return strdup("lto_get_symbol nullptr passed for result");
+
+  *Sym = wrap(&((InputFile::Symbol *)Symbols.Symbols)[Id]);
+  return nullptr;
+}
+
+extern "C" const char *lto_symbol_name(lto_symbol_t Sym) {
+  return unwrap(Sym)->getName().data();
+}
+LTOErrorStr lto_get_symbols(lto_input_t Input, LTOSymbols *Symbols) {
+  std::string Errors;
+  auto SymbolsList = unwrap(Input)->symbols();
+  Symbols->NumSyms = SymbolsList.size();
+  Symbols->Symbols = wrap(SymbolsList.data());
+  return nullptr;
+}
+
+/** If numthread is zero, LLVM will adapt the number of threads to the machine
+ */
+extern "C" LTOErrorStr
+thinlto_create_threaded_backend(size_t numthread,
+                                thinlto_backend_t *NewBackend) {
+  *NewBackend = wrap(new ThinBackend(createInProcessThinBackend(numthread)));
+  return nullptr;
+}
+
+extern "C" LTOErrorStr lto_add_input(lto_instance_t LTO, lto_input_t Input,
+                                     lto_symbol_resolutions_t Resolutions) {
+
+  Error AddErr =
+      unwrap(LTO)->add(std::unique_ptr<InputFile>(unwrap(Input)),
+                       {(llvm::lto::SymbolResolution *)Resolutions.resolutions,
+                        Resolutions.num});
+  if (AddErr)
+    return strdup(toString(std::move(AddErr)).c_str());
+  return nullptr;
+}
+
+class LTOObjectStream : public NativeObjectStream {
+  ObjectReadyCallback AddBuffer;
+  void *CallbackContext;
+  unsigned Task;
+  std::string FileName;
+  SmallVector<char, 0> OutputBuffer;
+  std::function<void(const std::string &)> AddError;
+
+public:
+  LTOObjectStream(ObjectReadyCallback AddBuffer, void *CallbackContext,
+                  unsigned Task, std::string FileName,
+                  std::function<void(const std::string &)> AddError)
+      : NativeObjectStream(make_unique<raw_svector_ostream>(OutputBuffer)),
+        AddBuffer(AddBuffer), CallbackContext(CallbackContext), Task(Task),
+        FileName(std::move(FileName)), AddError(AddError) {}
+
+  LTOObjectStream(ObjectReadyCallback AddBuffer, void *CallbackContext,
+                  unsigned Task,
+                  std::function<void(const std::string &)> AddError)
+      : NativeObjectStream(make_unique<raw_svector_ostream>(OutputBuffer)),
+        AddBuffer(AddBuffer), CallbackContext(CallbackContext), Task(Task),
+        AddError(AddError) {}
+
+  static std::unique_ptr<LTOObjectStream>
+  create(size_t Task, ObjectReadyCallback AddBuffer, void *CallbackContext,
+         const char *CacheDir, const char *OutputFilePrefix,
+         std::function<void(const std::string &)> AddError) {
+    std::string Filename = (Twine(OutputFilePrefix) + "." + Twine(Task)).str();
+    int FD;
+    std::error_code EC =
+        sys::fs::openFileForWrite(Filename, FD, sys::fs::F_None);
+    if (EC) {
+      AddError((Twine("Could not open file ") + Filename.c_str() + ": " +
+                EC.message())
+                   .str());
+      return llvm::make_unique<LTOObjectStream>(
+          AddBuffer, CallbackContext, Task, std::move(Filename), AddError);
+    }
+    return llvm::make_unique<LTOObjectStream>(AddBuffer, CallbackContext, Task,
+                                              AddError);
+  }
+
+  ~LTOObjectStream() {
+    // Make sure the stream is flushed (and file is closed) before sending it.
+    OS.reset();
+    if (FileName.empty()) {
+      auto MemBuf = MemoryBuffer::getMemBuffer(
+          StringRef(OutputBuffer.data(), OutputBuffer.size()), "",
+          /* RequiresNullTerminator */ false);
+      AddBuffer(wrap(new LTOOutput{Task, std::move(MemBuf)}), CallbackContext);
+    } else {
+      // Close the stream
+      OS.reset(nullptr);
+      // Re-open the file
+      auto ErrOrBuf = MemoryBuffer::getFile(FileName);
+      if (ErrOrBuf) {
+        AddError("Can't open file '" + FileName + "': " +
+                 ErrOrBuf.getError().message());
+        return;
+      }
+      AddBuffer(wrap(new LTOOutput{Task, std::move(*ErrOrBuf)}),
+                CallbackContext);
+    }
+  }
+};
+
+extern "C" LTOErrorStr lto_create_instance(lto_config_t ConfParams,
+                                           thinlto_backend_t Backend,
+                                           lto_instance_t *Instance) {
+  lto::Config Conf;
+
+  // Initialize the configuration
+  for (int ParamId = 0; ParamId < (int)ConfParams.Length; ++ParamId) {
+    const auto &Param = ConfParams.Params[ParamId];
+    switch ((int)Param.Tag) {
+    default:
+      return strdup(
+          (Twine("Unknown LTO Config Parameter '") + Twine(Param.Tag) + "'")
+              .str()
+              .c_str());
+    case LTOCFG_SAVETEMPS: {
+      auto Err = Conf.addSaveTemps(Param.Value.Str);
+      if (Err)
+        return strdup(toString(std::move(Err)).c_str());
+      break;
+    }
+    case LTOCFG_OPTLEVEL:
+      Conf.OptLevel = Param.Value.Num;
+      switch (Conf.OptLevel) {
+      case 0:
+        Conf.CGOptLevel = CodeGenOpt::None;
+        break;
+      case 1:
+        Conf.CGOptLevel = CodeGenOpt::Less;
+        break;
+      case 2:
+        Conf.CGOptLevel = CodeGenOpt::Default;
+        break;
+      case 3:
+        Conf.CGOptLevel = CodeGenOpt::Aggressive;
+        break;
+      default:
+        return strdup(("Invalid optimization level: " + Twine(Conf.OptLevel))
+                          .str()
+                          .c_str());
+      }
+      break;
+    }
+  }
+
+  // Create the instance
+  *Instance = wrap(new LTO(std::move(Conf), *unwrap(Backend)));
+  return nullptr;
+}
+
+extern "C" void lto_destroy_instance(lto_instance_t LTO) { delete unwrap(LTO); }
+
+extern "C" void lto_destroy_output(lto_output_t Output) {
+  delete unwrap(Output);
+}
+
+extern "C" LTOBuffer lto_get_output_buffer(lto_output_t Output) {
+  return {unwrap(Output)->Task, unwrap(Output)->MB->getBufferStart(),
+          unwrap(Output)->MB->getBufferSize()};
+}
+
+extern "C" LTOErrorStr lto_run(lto_instance_t Instance,
+                               ObjectReadyCallback ObjCallback, void *Context,
+                               const char *CacheDir,
+                               const char *OutputFilePrefix) {
+  init_once();
+  LTO *LTOInstance = unwrap(Instance);
+  std::string ErrorResult;
+  auto AddStream =
+      [&](size_t Task) -> std::unique_ptr<lto::NativeObjectStream> {
+    return LTOObjectStream::create(Task, ObjCallback, Context, CacheDir,
+                                   OutputFilePrefix,
+                                   [&](const std::string &Error) {
+                                     // FIXME LOCK
+                                     ErrorResult += Error;
+                                   });
+  };
+
+  auto AddBuffer = [&](size_t Task, std::unique_ptr<MemoryBuffer> MB) {
+    // The cache is supposed to call back here with a file-backed buffer.
+    ObjCallback(wrap(new LTOOutput{Task, std::move(MB)}), Context);
+  };
+
+  NativeObjectCache Cache;
+  if (CacheDir && *CacheDir) {
+    auto CacheOrErr = localCache(CacheDir, AddBuffer);
+    if (!CacheOrErr)
+      return strdup(toString(CacheOrErr.takeError()).c_str());
+    Cache = *CacheOrErr;
+  }
+  auto RunErr = LTOInstance->run(AddStream, Cache);
+  if (RunErr)
+    return strdup(toString(std::move(RunErr)).c_str());
+  return nullptr;
+}