Resolution-based LTO API.

[tejohnson]

Summary

This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

[lhames]

lib/CodeGen/LTOBackend.cpp

142–149 ↗

(On Diff #60111)

Apologies if this is a naive question - I'm still wrapping my head around this patch, but could opt/codegen/splitCodeGen be std::functions on Config rather than being plain functions that introspect the Config object to build the pass pipelines? This might allow you to decouple some of the pipeline setup logic from the LTO interface. It may also allow you to remove some of the hooks: E.g. If the user is supplying 'codegen', they can just add any module inspection code to their custom implementation, rather than having to set up 'PreCodeGenModuleHook'.

[mehdi_amini]

I like Lang approach to decouple a bit more the configuration.

include/llvm/CodeGen/LTOBackend.h

59 ↗

(On Diff #60111)

"to add its own resolved symbols": so the linker will modify the module? (I notice the Module is not const in the hook prototype)
What is the use-case?

62 ↗

(On Diff #60111)

What level of "thread-safety" are we talking about? Is it just about the linker internal structure? Can the linker safely assume that only one thread will call one of these callbacks for a given Module? For a given LLVMContext?

67 ↗

(On Diff #60111)

What is the Task "ID" at this point? This would be a task that will not codegen in the case of a parallel backend?

94 ↗

(On Diff #60111)

If the intent is just -save-temps, I'd rather have the Index passed as a const ref.

107 ↗

(On Diff #60111)

Why the Triple string and Target here?

120 ↗

(On Diff #60111)

This is a strange API at this level.
And the naming is very generic for what the description says it does.

136 ↗

(On Diff #60111)

Why this LTOLLVMContext instead of a factory method in the Config class?

include/llvm/LTO/LTO.h

90 ↗

(On Diff #60111)

I think some doc may be nice for the enum values (I have no idea what is comdat_alias right now).

93 ↗

(On Diff #60111)

doc (public API in public header)

292 ↗

(On Diff #60111)

Not clear why ParallelCodeGenParallelismLevel is not part of Config.
Also it does not apply to ThinLTO, which is not clear.

323 ↗

(On Diff #60111)

There is too much state here.
I could see getting rid of almost *all* of these.
Until you call run() on this class, I don't expect that we need any but the Config and ModuleMap.

385 ↗

(On Diff #60111)

Why do we need this here?

[pcc]

include/llvm/CodeGen/LTOBackend.h

59 ↗

(On Diff #60111)

Yes, it would modify the module.

The use case is to work around API deficiencies; the specific issue I had in mind was the special handling for common symbols required in the gold plugin (search for addCommons).

I wouldn't be surprised if we had to implement a similar workaround in libLTO as well, but I would not expect lld for example to need to modify the module, as we control both sides of the interface.

That said, the gold plugin only requires a mutable hook at one point (which happens to be post-internalize at the moment, but I think also pre-opt would work); maybe we should only allow a mutable hook at one of those points.

62 ↗

(On Diff #60111)

Yes, this is with regard to the linker's internal structure.

On the LLVM side I think the right guarantee is a per-module guarantee (so the linker would be able to freely inspect and modify the provided module without having to worry about thread safety). Although of course at the moment this would be a per-context guarantee unless/until we made LLVMContext thread-safe.

I will change this comment to mention this.

67 ↗

(On Diff #60111)

What is the Task "ID" at this point?

I'm not sure what you are asking here. The task ID generally stays consistent across the whole pipeline.

This would be a task that will not codegen in the case of a parallel backend?

Do you mean a distributed backend? In that case, none of the hooks will be called for ThinLTO tasks other than the combined module hook. I will document that.

94 ↗

(On Diff #60111)

Makes sense, will do.

107 ↗

(On Diff #60111)

You're right, we don't need these. I'll change the implementation to use Module::getTargetTriple().

120 ↗

(On Diff #60111)

Yes, agree. After I finish the reorganization I mentioned earlier, I want to move this API to a library-internal header.

136 ↗

(On Diff #60111)

Because the LTOLLVMContext needs to own the DiagHandler.

include/llvm/LTO/LTO.h

90 ↗

(On Diff #60111)

Will do.

93 ↗

(On Diff #60111)

Will do

292 ↗

(On Diff #60111)

The idea was that Config controls everything except how code generation is "orchestrated" (that's why I'm passing in a ThinBackend here for example).

Also it does not apply to ThinLTO, which is not clear.

I will document that more clearly.

323 ↗

(On Diff #60111)

I think the code is made more straightforward and easier to understand by incrementally building the combined module (for regular LTO) or the combined index (for ThinLTO) from add(), as you can see from the code exactly what happens for each module. That requires us to keep this state here.

Anyway, this state is an implementation detail, so it is not as important as the public API.

385 ↗

(On Diff #60111)

Okay, since it looks like we're only using memory buffers here, we can probably just have an array of MemoryBuffers here (or maintain ModuleMap as a MapVector, or something).

lib/CodeGen/LTOBackend.cpp

142–149 ↗

(On Diff #60111)

That's an interesting suggestion. My initial concern with that was that it bakes the required state for each pipeline phase into the API. For example, codegen requires a Triple and a Target, and many of the ThinLTO phases would require a combined module index, and those would need to be passed into the pipeline function via the "hook" in Config. If we ever change the required state, that would mean a lot of churn for each user of the API.

Although it occurred to me that we could avoid baking in the state like this:

// In LTOBackend.cpp
struct PromoteState {
  ModuleSummaryIndex &CombinedIndex;
};
struct CodeGenState {
  Config &C;
  StringRef TheTriple;
  const Target *TheTarget;
  AddStreamFn AddStream;
};

// In LTOBackend.h
struct PromoteState;
bool promote(size_t Task, Module &M, PromoteState &State);

struct CodeGenState;
bool codegen(size_t Task, Module &M, CodeGenState &State);

struct Config {
  // ...
  std::function<bool(size_t Task, Module &M, PromoteState &State)> Promote;
  std::function<bool(size_t Task, Module &M, CodeGenState &State)> CodeGen;
  // ...
};

that would still make the signatures of the Config "hooks" non-uniform, which would also make it more complicated to implement save-temps, which is about 90% of the point of these hooks. (Suppose I want to add save-temps to each pipeline phase, including "pre-optimization". I would not only need to write an individual wrapper function for each phase, but I would also need to know which phase comes "first" in order to run my pre-opt save-temps processing before it.) I would prefer to keep the hooks uniform unless there is a good reason not to do so.

[pcc]

• Address review comments
• Remove TheTarget and TheTriple arguments and unneeded error_category (now that we have StringError)
• Pass index as const ref to CombinedIndexHook
• Remove ThinObjs, and change ModuleMap's type to MapVector
• Add Config::addSaveTemps; add initial llvm-lto2 test
• Wire the gold plugin up to addSaveTemps, and fix several tests that were failing before (didn't notice due to stale outputs in the test directory)
• Add a .resolution.txt to save-temps, and port comdat.ll
• Port emit-llvm.ll test to resolution.txt
• Remove api file feature
• Move LTO.cpp to Resolution subdirectory
• Move LTOBackend to LTO/Resolution; inline upgradeLinkage
• Remove unnecessary #includes from gold-plugin.cpp
• Add comments
• Remove unnecessary dep

[pcc]

In this version of the patch I started exercising the test harness so you can see what that looks like. As previously mentioned I also moved the code to a separate library (lib/LTO/Resolution) so we can depend on it from clang.

I started working on the clang side backend, but there's a hairy output stream yak to shave there, so it'll take a little more time.

include/llvm/CodeGen/LTOBackend.h

120 ↗

(On Diff #60111)

I ended up folding this into the caller on the regular LTO side, as ThinLTO linkage upgrades now go via the module summary index.

include/llvm/LTO/LTO.h

90 ↗

(On Diff #60111)

I ended up removing this error category and just using StringError for errors produced by this module.

93 ↗

(On Diff #60111)

See above

[mehdi_amini]

include/llvm/CodeGen/LTOBackend.h

67 ↗

(On Diff #60111)

I meant the LTO parallel codegen: after merging the IR you will call this hook with a tuple <task-ID, module> that will never go through the codegen.

include/llvm/LTO/Resolution/LTOBackend.h

71 ↗

(On Diff #61155)

The thread safety in LLVM is at the LLVMContext level, not the module level.

[pcc]

include/llvm/CodeGen/LTOBackend.h

67 ↗

(On Diff #60111)

In this case the task id would be 0, and it would be split into tasks 0..N-1.

Although task 0 will end up using a different module for codegen (i.e. whichever partition happened to be assigned task id 0), the module provided to the hook at this point is still significant, as any changes made to the module (e.g. adding common symbols for gold) will be reflected in the partitions.

include/llvm/LTO/Resolution/LTOBackend.h

71 ↗

(On Diff #61155)

Yes, but I wanted to describe the intention here should we ever relax the thread safety in LLVM. (For example, if we ever made it safe for multiple code generation threads to operate on a single module, the guarantee here would be stronger than the default guarantee.)

[tejohnson]

Looks like it is getting very close. A few comments/questions below.

Also, it is a huge patch - suggest committing in a few different pieces:

1. Move existing LTO.{cpp,h} to Resolution subdirectory
2. Add LTOBackend.cpp, changes to LTO.{cpp,h}, and llvm-lto2 + tests
3. Changes to gold-plugin.cpp to use new interfaces

include/llvm/LTO/Resolution/LTO.h

321 ↗

(On Diff #61155)

Is the ThinLTO support still TODO? It looks like this can be removed as I see it here now.

include/llvm/LTO/Resolution/LTOBackend.h

106 ↗

(On Diff #61155)

Although it is not called for the backend process (which also has a Config object per D21545). I do know what you are trying to say here - I think the distinction is that this is always invoked via LTO::run regardless of whether the backend is in-process.

Whereas the earlier hooks are only invoked from the backends themselves (regarless of whether invoked in process or in a separate backend process)

111 ↗

(On Diff #61155)

Add doxygen comment

lib/LTO/Resolution/LTO.cpp

167 ↗

(On Diff #61155)

Can we avoid creating this if we are not going to perform ThinLTO? E.g. Create lazily in runThinLto if it is still null?

370 ↗

(On Diff #61155)

Could move this to parent class since it is in both derived classes

561 ↗

(On Diff #61155)

Can you document why we are initializing Task to ParallelCodeGenParallelismLevel? I thought earlier it said somewhere that regular LTO (parallel or not) was TaskId 0 and ThinLTO started at TaskId 1.

lib/LTO/Resolution/LTOBackend.cpp

32 ↗

(On Diff #61155)

I'm confused by what is going on here with OldHook.

test/LTO/Resolution/X86/comdat.ll

14 ↗

(On Diff #61155)

Does this need to be specified here? It looks like a suitable default (with none of the flags) will be constructed by default if not.

73 ↗

(On Diff #61155)

What causes the difference w.r.t. gold here (protected vs not)?

test/tools/gold/X86/coff.ll

19 ↗

(On Diff #61155)

Why the loss of local_unnamed_addr?

test/tools/gold/X86/thinlto_alias.ll

24 ↗

(On Diff #61155)

Since thinBackend() now invokes thinLTOInternalizeModule, why is this failing?

tools/gold/gold-plugin.cpp

730 ↗

(On Diff #61155)

Can we skip this if options::thinlto_index_only? Otherwise Backend is simply overwritten just below.

tools/llvm-lto2/llvm-lto2.cpp

2 ↗

(On Diff #61155)

Should probably add a comment at the top of the file about why this exists then.

20 ↗

(On Diff #61155)

Need to describe format of "resolution" (e.g. 'p','l','x' and meanings). Also, it would be good to mention the default resolution for anything not specified here (which from the code it appears to be non-p, non-l, and non-x).

[pcc]

• Refresh
• Address review comments

[pcc]

Also, it is a huge patch - suggest committing in a few different pieces:

Makes sense. Could also split it up into differential revisions if it makes it easier to review.

include/llvm/LTO/Resolution/LTOBackend.h

106 ↗

(On Diff #61155)

Clarified.

lib/LTO/Resolution/LTO.cpp

167 ↗

(On Diff #61155)

Maybe, but that sounds like a premature optimization to me.

lib/LTO/Resolution/LTOBackend.cpp

32 ↗

(On Diff #61155)

This is calling the hook provided by the linker, which still needs to run (e.g. in order to add common symbols). If the linker's hook returned false, we need to pass that result through.

test/LTO/Resolution/X86/comdat.ll

14 ↗

(On Diff #61155)

After thinking about it some more, I reckon that in order to reduce the possibility of mistakes, we probably don't want to implement a default resolution in llvm-lto2, nor should we accept resolutions for non-existent symbols. I have made that change to the test harness.

73 ↗

(On Diff #61155)

Unlike the existing gold plugin, we do not resolve the visibility of each symbol and apply it to the combined module. It is unnecessary to do so because gold has already resolved the symbol's visibility using information provided by the plugin (i.e. LDPV_*) and will apply it to the final output file. The corresponding gold test (test/LTO/Resolution/X86/comdat.ll) shows that the symbol receives the correct visibility.

test/tools/gold/X86/coff.ll

19 ↗

(On Diff #61155)

LTO only tracks whether the symbol is global unnamed_addr. We do not track local_unnamed_addr for similar reasons as for visibility: it is the linker's job to keep track of whether the presence of that attribute can permit internalization (or, for Mach-O, auto-hide), and its presence in the IR shouldn't affect the generated code.

test/tools/gold/X86/thinlto_alias.ll

24 ↗

(On Diff #61155)

The internalization done in thinLTOInternalizeModule for symbols that are not externally visible is separate from the internalization done for preempted alias targets (search for InternalLinkage in IRMover.cpp), which is not yet implemented for ThinLTO. I believe the practical effect here is that the linker will end up re-resolving the weak symbol from the resulting native object files. (This is similar to how we currently handle (non-ODR) weak or linkonce symbols, as we currently don't have a summary resolution that means "discard this symbol".)

tools/gold/gold-plugin.cpp

730 ↗

(On Diff #61155)

It seems more straightforward to just let it be overwritten.

[pcc]

Refresh, rebase on top of D22173, move back to lib/LTO

[tejohnson]

LGTM, with one nit below.

However, I'd like to wait to commit this until:

1. Mehdi signs off
2. D22302 goes in, enabling weak/linkonce resolution for gold. Your patch has the side effect of doing this enabling, and I think it is cleaner to have that functionality added separately from this patch adding the new API.

Mehdi, can you take a look at this patch and D22302 asap - this is getting on the critical path of some fixes and follow on patches I need to add for the distributed backend case (will send a separate update on that).

lib/LTO/Resolution/LTOBackend.cpp

32 ↗

(On Diff #61155)

Ok, maybe rename to LinkerHook and/or add a comment.

[mehdi_amini]

Can you update description / title (it is still marked as "wip" which make me think the revision is not ready).

[pcc]

Can you update description / title (it is still marked as "wip" which make me think the revision is not ready).

Done.

[mehdi_amini]

Can you add one test for the .resolution.txt file with llvm-lto2?

include/llvm/LTO/Config.h

82 ↗

(On Diff #63907)

"will never be called *concurrently* from multiple threads" -> I don't think we want to promise to pin a Module or a task to a given thread.

146 ↗

(On Diff #63907)

You may add that the real reason to have a class here is that you don't want to tie the lifetime of the context to the configuration object and you need to keep a copy of the diagnostic handler alive.
Otherwise it may be easy for someone to nuke this anytime.

include/llvm/LTO/LTO.h

103 ↗

(On Diff #63907)

IIUC the reason we need a context is because we can't get the information we want on an InputFile without a context for now.
Since we have plan to fix this (and this is acknowledged in the FIXME), I'd rather remove the LTO param now and add a Context inside the InputFile itself. It'll:

1. solve any multithreading question about parsing input files.
2. allow to easily get rid of this context later.

321 ↗

(On Diff #63907)

At least, can we wrap them in struct to separate this clearly:

struct {
  unsigned ParallelCodeGenParallelismLevel;
  LTOLLVMContext Ctx;
  std::unique_ptr<Module> CombinedModule;
  IRMover Mover;
} LTOState;

  // These fields are for ThinLTO.
struct {
  ThinBackend Backend;
  ModuleSummaryIndex CombinedIndex;
  MapVector<StringRef, MemoryBufferRef> ModuleMap;
  DenseMap<GlobalValue::GUID, StringRef> PrevailingModuleForGUID;
} ThinLTOState;

It'll make the code more clear on violation (ThinLTO code accessing LTO state or vice-versa)

lib/LTO/LTO.cpp

280 ↗

(On Diff #63907)

This isn't clear to me "the client may want to add symbol definitions to it". Why doesn't the client add a regular LTO module if he needs to add its stuff?

lib/LTO/LTOBackend.cpp

58 ↗

(On Diff #63907)

How is this supposed to work with static archive? It'll write a bunch of file next to the input shared library?
Could we have a separate directory to stuff all these files with ThinLTO?

149 ↗

(On Diff #63907)

Nothing critical but we just already created a TM for the optimization pipeline.

tools/llvm-lto2/llvm-lto2.cpp

2 ↗

(On Diff #63907)

Agree

34 ↗

(On Diff #63907)

"r" is really not very explicit for an option, I think we usually have more self-explaining names.

44 ↗

(On Diff #63907)

Are there incompatible combination? (I don't think so, but just checking).

[tejohnson]

lib/LTO/LTOBackend.cpp

58 ↗

(On Diff #63907)

Yes, it will write the temp files next to the archive library - each constituent bitcode in the archive is given a unique module ID by the gold-plugin (see D20559 claim_file_hook()).

I find it more intuitive to put the saved temp files next to the input object or archive, but open to other suggestions - that should probably change in a separate patch though as this is maintaining the current gold-plugin behavior.

[pcc]

• Address review comments
• Add resolution test based on llvm-lto2

include/llvm/LTO/LTO.h

103 ↗

(On Diff #63907)

Sorry, but I'd prefer not to do this.

solve any multithreading question about parsing input files.

Let's not pretend that we can parse input files in a multithreaded way that allows them to be used later for linking until we can actually do it. As things stand, doing this will probably regress gold plugin performance as we will need to parse the input file an additional time.

allow to easily get rid of this context later.

It doesn't seem like it will be particularly difficult to remove this argument in clients later, it would probably be on the order of a trivial mechanical change.

321 ↗

(On Diff #63907)

Good idea, done.

lib/LTO/LTO.cpp

280 ↗

(On Diff #63907)

Because there may be no regular LTO module available as part of the link. For the gold plugin for example if some of the ThinLTO modules define common symbols then we need to make sure that one of the object files contains the resolved common symbols. In the case where there are no regular LTO modules that can be an object file containing just the common symbols.

lib/LTO/LTOBackend.cpp

149 ↗

(On Diff #63907)

Sure, but it's a little simpler to create this where needed, especially when using parallel code gen. Eventually we might want to just create this once in backend() and thinBackend(), but that would require TargetMachine to be thread safe.

tools/llvm-lto2/llvm-lto2.cpp

34 ↗

(On Diff #63907)

This flag will need to be passed multiple times, and should be familiar to any user of this tool, so it seems appropriate to give it a short name.

44 ↗

(On Diff #63907)

I don't think so either.

[mehdi_amini]

include/llvm/LTO/LTO.h

94 ↗

(On Diff #64068)

I missed how this friend allows access Obj earlier, and it is not great. It is used to create a dependency on having a Module available and on having "GlobalValue *", while we want to be able to make all the resolution without loading IR.
Anytime LTO is accessing an IR construct through an InputFile, there's something that looks like a boundary violation.

As soon as we have a proper symbol table, InputFile should not even expose access to a module or IR at all.

103 ↗

(On Diff #64068)

Let's not pretend that we can parse input files in a multithreaded way that allows them to be used later for linking until we can actually do it

I'm not sure what you mean: this is *already* what we do with libLTO:

The file is lazily loaded in its own context, this is cheap: the IR is not fully parsed but only the list of symbols. It is then reloaded separately, and non lazily, in the LTO context later for the purpose of LTO merging.
If you have performance concern, we need to time this first. I can try to write a test tomorrow.

Right now this choice is penalizing ThinLTO that won't be able to load symbols in parallel and still have to load two times anyway.

lib/LTO/LTO.cpp

280 ↗

(On Diff #64068)

Because there may be no regular LTO module available as part of the link.

Client can just create one then...

Module M;
Lto.add(M);
// done

if some of the ThinLTO modules define common symbols then we need to make sure that one of the object files contains the resolved common symbols.

I don't know what you're referring to here?

lib/LTO/LTOBackend.cpp

150 ↗

(On Diff #64068)

For ThinLTO at least, we don't support parallel codegen, so we don't need TM to be thread safe to be able to have only one TM created in thinBackend() and reused for both opt and codegen.

I even think that it should be already achievable by having:

• backend() and thinbackend() always creating the TM.
• opt() taking the TM as a parameter
• codegen() taking the TM as a parameter
• split_codegen() recreating the TM.

This way you only recreate an extra one in splitcodegen() when parallel codegen is enabled. Unless I missed something this is a net win.

[pcc]

• Avoid creating another TargetMachine
• Add FIXME

include/llvm/LTO/LTO.h

94 ↗

(On Diff #64068)

I agree with you, but until we have a symbol table, there needs to be some way for LTO to access IR entities in the input file.

Added a FIXME for the removal of these friends.

103 ↗

(On Diff #64068)

If you have performance concern, we need to time this first. I can try to write a test tomorrow.

Please let me know what you find. Bear in mind that you are the one advocating for a change here (the status quo in the only current client of this interface, the gold plugin, is a single context) so the burden is on you to justify it. If you wish to make a change, you are welcome to propose it separately from this patch.

lib/LTO/LTO.cpp

280 ↗

(On Diff #64068)

Client can just create one then...

Not as easily as that. Roughly:

if (/* there are no regular LTO objects, oh, looks like we'll need more API surface */) {
  LLVMContext Ctx;
  Module M(Ctx);
  M.setTargetTriple(??? /* yet more API surface */);
  // add common symbols
  std::string BC;
  raw_str_ostream OS(BC);
  WriteBitcodeToFile(M, OS);
  ErrorOr<unique_ptr<InputFile>> F = InputFile::create(BC);
  if (!F) { /* error handling */ }
  Lto.add(std::move(*F));
}

Seems much simpler to always have a module with index 0.

I don't know what you're referring to here?

As I previously mentioned on this code review:

the special handling for common symbols required in the gold plugin (search for addCommons).

[mehdi_amini]

include/llvm/LTO/LTO.h

94 ↗

(On Diff #64198)

I agree with you, but until we have a symbol table, there needs to be some way for LTO to access IR entities in the input file.

It is not clear to me why LTO needs to access any IR entity? If you need access an *IR entity* now, how could we don't need access to them tomorrow (<- read: with a symbol table)?

It seems to me that anywhere you're accessing an IR entity directly through Obj right now is a place that should have an API on the InputFile or the Symbol class (like the one that exists: InputFile::getSourceFileName()).

Assuming "Symbol" is abstracting the entries in the symbol table, for instance:

• Input->Obj->getMemoryBufferRef().getBufferIdentifier(); -> Input->getIdentifier()
• GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl()); -> Symbol *S = Obj->getSymbol(...)
• GV->hasGlobalUnnamedAddr(); -> needs an API on class Symbol
• GV->getName(); -> needs an API on class Symbol
• if (Res.VisibleToRegularObj || (GV && Used.count(GV)) || -> Used contains a GlobalVariable *, how do we go to a symbol table?
• Module &M = Input->Obj->getModule(); -> used for:

for (GlobalVariable &GV : M.globals())
  if (GV.hasAppendingLinkage())
    Keep.push_back(&GV);

which should iterate on Symbols on the InputFile.

• LTO::addRegularLto does some symbol resolution using Obj. At this point you probably want to operate on real IR construct anyway, and that's the point where you leave the "InputFile" to go to the IR. I think InputFile should just expose something like a factory std::unique_ptr<Module> parseModule(/* options */);. If it was just for full LTO, that would solve many of the above items as addSymbolToGlobalRes() could take a Module & instead of operating on the private IRObjectFile inside InputFile. However it won't help for ThinLTO.

GlobalValue *GV = Input->Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
if (Res.Prevailing && GV)
  ThinLto.PrevailingModuleForGUID[GV->getGUID()] =
      MBRef.getBufferIdentifier();

• getGUID() should be implemented on the Symbol directly.

103 ↗

(On Diff #64198)

I'm not sure, it seems easy to turn it around: "you are the one proposing an API that is targeted for performance reason with a specific use case (and client) in mind, so the burden should be on you to justify it if you want this to get it in.", otherwise write a cleaner API.

lib/LTO/LTO.cpp

255 ↗

(On Diff #64198)

Is is possible for GV to be null here?

280 ↗

(On Diff #64198)

Seems much simpler to always have a module with index 0.

"Simpler" from the gold point of view, my point of view is that when adding 1000 ThinLTO modules we'll copy 1000 DataLayout around for no reason.
Which reminds me that I don't expect any object being emitted for this module for a "pure" ThinLTO link.

305 ↗

(On Diff #64198)

(Same question here, can GV be null?)

[mehdi_amini]

include/llvm/LTO/LTO.h

103 ↗

(On Diff #64198)

Running ld64 on the 701 bitcode files that make an LTO build of llc (X86 only), and exiting right before the LTO optimizer starts, i.e. after the LTO merge happens (best of 10 times, on a 4-cores laptop):

• Parallel resolution (dedicated context, double parsing): 7.362s
• Sequential resolution (shared context, single parsing): 8.834s

[pcc]

include/llvm/LTO/LTO.h

94 ↗

(On Diff #64198)

If you need access an *IR entity* now, how could we don't need access to them tomorrow (<- read: with a symbol table)?

The way I see this evolving is that any uses of IR constructs here will be replaced with uses of APIs for accessing the bitcode symbol table (except in places such as LTO::addRegularLto which actually need a Module). The functionality exposed via InputFile and Symbol will remain the primary way by which clients will access the object (i.e. they wouldn't access the symbol table directly).

I would prefer not to expose the properties you propose to expose via InputFile/Symbol because they aren't strictly needed by clients for symbol resolution.

103 ↗

(On Diff #64198)

Thanks. To me it isn't clear how much of that is due to parsing as opposed to parallel resolution, but I can accept that this would simplify clients. So I suppose I can try to make the change you suggested.

lib/LTO/LTO.cpp

255 ↗

(On Diff #64198)

Yes, if the symbol is defined by inline asm.

280 ↗

(On Diff #64198)

I suppose we could avoid creating an empty regular LTO object unless a hook is set.

305 ↗

(On Diff #64198)

Ditto

[mehdi_amini]

include/llvm/LTO/LTO.h

103 ↗

(On Diff #64198)

I chatted with Duncan and thought you may be interested by some history: his recollections of the original motivation for parsing in separate context was not parallelism but the amount of things that leaks to the context, while the modules wouldn't be used if they come from static archives anyway.
Especially at that time global metadata were not lazyloadable.

[pcc]

include/llvm/LTO/LTO.h

103 ↗

(On Diff #64198)

Interesting. LLD originally used multiple contexts, but now uses a single context because of the reversed tradeoff.
http://llvm.org/viewvc/llvm-project?view=revision&revision=267921

the modules wouldn't be used if they come from static archives anyway.

Huh, I would expect the linker to use the archive symbol table to load only the needed modules.

[mehdi_amini]

include/llvm/LTO/LTO.h

103 ↗

(On Diff #64198)

LLD originally used multiple contexts, but now uses a single context because of the reversed tradeoff.

Going from 29.86187751s to 29.814533787s does not justify clobbering any API IMO.

[pcc]

include/llvm/LTO/LTO.h

103 ↗

(On Diff #64198)

Sure. I previously wasn't aware that Rafael had previously measured this.

[pcc]

• Remove LTO argument from InputFile::create
• Only generate code for the regular LTO module when necessary

[mehdi_amini]

lib/LTO/LTO.cpp

323 ↗

(On Diff #64068)

No std::move on return.

tools/llvm-lto2/llvm-lto2.cpp

164 ↗

(On Diff #64068)

Ditto

[pcc]

• No std::move on return

[tejohnson]

Ping - Mehdi, any more comments? I am relying on the follow-on patch D21545 adding support for weak resolution and internalization in the ThinLTO backends.

[mehdi_amini]

include/llvm/LTO/LTO.h

126 ↗

(On Diff #64350)

What about

return Flags & object::BasicSymbolRef::SF_Global ||
          Flags & object::BasicSymbolRef::SF_FormatSpecific;

lib/LTO/LTO.cpp

281 ↗

(On Diff #64350)

This is marked done, but doesn't seem to be?

lib/LTO/LTOBackend.cpp

59 ↗

(On Diff #64350)

Deriving an output temp file from a user-supplied *output* is fine, what does not make sense to me is to write stuff where the *input* files are. This is unlike what we usually do (the static archives could be in a read-only system directory for instance, this is quite common on our setup).

You can maintain whatever behavior Gold currently has by passing an option to addSaveTemps maybe. Right now the API being addSaveTemps(std::string OutputFileName) I don't expect files to be written anywhere unexpected.

Also the doxygen is pretty clear about what should be expected from this API:

/// This is a convenience function that configures this Config object to write
/// temporary files named after the given OutputFileName for each of the LTO
/// phases to disk. A client can use this function to implement -save-temps.

[pcc]

• Refresh
• Address review comments
• Implement linked objects file feature

lib/LTO/LTO.cpp

281 ↗

(On Diff #64350)

See the first three lines of LTO::run.

lib/LTO/LTOBackend.cpp

59 ↗

(On Diff #64350)

I agree with you that we should move to a temp file naming scheme based on output files, however like Teresa I think that should be discussed separately from this patch. I don't think we need a flag either, whatever we decide to do here should apply to all linkers.

Added a FIXME here to change the naming scheme.

[mehdi_amini]

Last comments, we should be able to iterate in-tree after that.

include/llvm/LTO/LTO.h

310 ↗

(On Diff #65465)

Should be RegularLTO I think.

319 ↗

(On Diff #65465)

Similarly, should be ThinLTO

368 ↗

(On Diff #65465)

addRegularLTO

370 ↗

(On Diff #65465)

addThinLTO

373 ↗

(On Diff #65465)

runRegularLTO

374 ↗

(On Diff #65465)

runThinLTO

lib/LTO/LTO.cpp

243 ↗

(On Diff #65465)

HasThinLTOSummary

281 ↗

(On Diff #65465)

I see, I was looking for an empty LTO module (RegularLto.CombinedModule) instead of object (as in produced object).

316 ↗

(On Diff #65465)

Is this test what you intend? It seems reversed (if so, then it's probably not tested).

318 ↗

(On Diff #65465)

This is still gonna copy a few SmallVector + a couple of string + a few other fields. And it's gonna happens for each and every ThinLTO module. This is just not the right place for that. Adding a triple/datalayout for LTO should not be handled here, it is just not the right place. LTO::run could handle it by poking at ThinLto.ModuleMap if needed.

Similarly, RegularLto.CombinedModule should not be created unless explicitly added/requested by the linker (i.e. not call to make_unique<Module>("ld-temp.o", Ctx); in the ctor).

358 ↗

(On Diff #65465)

Doc the above tests.

lib/LTO/LTOBackend.cpp

60 ↗

(On Diff #65465)

It seems we are not looking at this from the same angle: I was looking at it as a *new* API, not a refactoring of whatever the gold plugin was doing. So I can't see "this is gold's behavior" as a valid motivation here (if you want this to be NFC from the gold point of view, the gold-plugin can be patched in the first place).

[tejohnson]

pcc asked me to take over the patch since he is out on vacation this month. Responses below. I have made most of the suggested changes, but not a couple for the reasons stated. Let me see if I can upload the new patch even though pcc created this one.

include/llvm/LTO/LTO.h

310 ↗

(On Diff #65465)

Done

319 ↗

(On Diff #65465)

Done

368 ↗

(On Diff #65465)

Done

370 ↗

(On Diff #65465)

Done

373 ↗

(On Diff #65465)

Done

374 ↗

(On Diff #65465)

Done

lib/LTO/LTO.cpp

243 ↗

(On Diff #65465)

Done

316 ↗

(On Diff #65465)

I think what was meant here was to check if the RegularLTO.CombinedModule's target triple was still empty, and if so set it. I have changed it to that.

318 ↗

(On Diff #65465)

It only does the copy once with the change I made, so there isn't any duplication in the copying anymore. This is the simplest place to set it based on an added ThinLTO module.

I looked at delaying the creation of the CombinedModule, however, it turns out we frequently need it since even if we don't add any regular LTO files, we typically have callback hooks into the linker defined where the linker could add its own resolved symbols to the combined module, in which case it needs to be valid. I don't think we gain much by lazily creating the empty Module.

358 ↗

(On Diff #65465)

Done

lib/LTO/LTOBackend.cpp

60 ↗

(On Diff #65465)

I removed this path so that the OutputFileName is always used, so it now matches the doxygen comment for the API.

[tejohnson]

Taking this over so I can upload new patch.

[tejohnson]

• Address review comments

[mehdi_amini]

lib/LTO/LTO.cpp

318 ↗

(On Diff #67642)

The ", we typically have callback hooks into the linker defined where the linker could add its own resolved symbols to the combined module" is not clear to me at all (LTOCodeGenerator will never do that AFAIK for example).
(I understand that Gold does something with "commons" at that time, but haven't add time to figure why it is needed)

[mehdi_amini]

(Note: I already approved this patch as good enough to be iterated on in-tree, so feel free to commit)

[mehdi_amini]

lib/LTO/LTO.cpp

318 ↗

(On Diff #67642)

To be more explicit: I don't like this behavior because it creates a relatively strong coupling between the linker and the plugin. The linker changes the module in an unpredictable way that can conflict with assumption that the plugin implementation could make. It makes it harder to follow invariant in the plugin implementation, and makes it easy to break the client of the API (the linker).

[tejohnson]

lib/LTO/LTO.cpp

318 ↗

(On Diff #67642)

True, this is the case now because gold is the only user. I went ahead and committed (temporarily, as it turned out, will recommit after fixing some bot failures) so that I can also get pcc's follow-on patch in that I need. But will also work up a patch to make the creation of the combined module lazy. I'm not sure how to address your other concern about the linker changing the module in unpredictable ways, but the API does document that the linker may add symbols to the combined module via the callback hooks.

lib/LTO/LTOBackend.cpp

61 ↗

(On Diff #67642)

I didn't notice that this change was causing an issue since I had old output files around that caused tests to pass, but this caused a bot failure due to missing output files. The problem is not just that the names are different, but that they become difficult to correlate to the corresponding input source name. E.g. in the gold/X86/thinlto_linkonceresolution.ll test, where we have "%gold ... -o %t3.o %t2.o %t.o", the old .opt.bc temp files were named %t.o.4.opt.bc and %t2.o.4.opt.bc, but with the change I made become %t3.o.1.4.opt.bc and %t3.o.2.4.opt.bc. It isn't at all obvious which output file corresponds to which input file (numbering will depend on the ModuleMap iteration order).

What I did was to revert the change I had made, but add a new bool flag parameter on addSaveTemps (UseInputModulePath) that is set to true by gold and will provoke the old behavior. We can iterate on a better solution. Probably pass in a (temp) directory name and output all the files in a tree rooted there (note that you could have same named module identifiers at different paths, so you can't just disambiguate by appending the basename of the input module).

[rogfer01]

llvm/trunk/tools/gold/gold-plugin.cpp

95

I think this one should be size_t as lto::InputFile::Symbol::getCommonSize returns size_t and then the std::max in line 629 fails to build in targets where size_t is unsigned int.

Can you confirm if changing to size_t makes sense here? It does fix the build indeed.

If this is OK I can commit the change.

llvm/trunk/include/llvm/LTO/Config.h

+//===-Config.h - LLVM Link Time Optimizer Configuration -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the lto::Config data structure, which allows clients to
+// configure LTO.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LTO_CONFIG_H
+#define LLVM_LTO_CONFIG_H
+
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/Support/CodeGen.h"
+#include "llvm/Target/TargetOptions.h"
+
+#include <functional>
+
+namespace llvm {
+
+class Error;
+class Module;
+class ModuleSummaryIndex;
+class raw_pwrite_stream;
+
+namespace lto {
+
+/// LTO configuration. A linker can configure LTO by setting fields in this data
+/// structure and passing it to the lto::LTO constructor.
+struct Config {
+  std::string CPU;
+  std::string Features;
+  TargetOptions Options;
+  std::vector<std::string> MAttrs;
+  Reloc::Model RelocModel = Reloc::PIC_;
+  CodeModel::Model CodeModel = CodeModel::Default;
+  CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
+  unsigned OptLevel = 2;
+  bool DisableVerify = false;
+
+  /// Setting this field will replace target triples in input files with this
+  /// triple.
+  std::string OverrideTriple;
+
+  /// Setting this field will replace unspecified target triples in input files
+  /// with this triple.
+  std::string DefaultTriple;
+
+  bool ShouldDiscardValueNames = true;
+  DiagnosticHandlerFunction DiagHandler;
+
+  /// If this field is set, LTO will write input file paths and symbol
+  /// resolutions here in llvm-lto2 command line flag format. This can be
+  /// used for testing and for running the LTO pipeline outside of the linker
+  /// with llvm-lto2.
+  std::unique_ptr<raw_ostream> ResolutionFile;
+
+  /// The following callbacks deal with tasks, which normally represent the
+  /// entire optimization and code generation pipeline for what will become a
+  /// single native object file. Each task has a unique identifier between 0 and
+  /// getMaxTasks()-1, which is supplied to the callback via the Task parameter.
+  /// A task represents the entire pipeline for ThinLTO and regular
+  /// (non-parallel) LTO, but a parallel code generation task will be split into
+  /// N tasks before code generation, where N is the parallelism level.
+  ///
+  /// LTO may decide to stop processing a task at any time, for example if the
+  /// module is empty or if a module hook (see below) returns false. For this
+  /// reason, the client should not expect to receive exactly getMaxTasks()
+  /// native object files.
+
+  /// A module hook may be used by a linker to perform actions during the LTO
+  /// pipeline. For example, a linker may use this function to implement
+  /// -save-temps, or to add its own resolved symbols to the module. If this
+  /// function returns false, any further processing for that task is aborted.
+  ///
+  /// Module hooks must be thread safe with respect to the linker's internal
+  /// data structures. A module hook will never be called concurrently from
+  /// multiple threads with the same task ID, or the same module.
+  ///
+  /// Note that in out-of-process backend scenarios, none of the hooks will be
+  /// called for ThinLTO tasks.
+  typedef std::function<bool(size_t Task, Module &)> ModuleHookFn;
+
+  /// This module hook is called after linking (regular LTO) or loading
+  /// (ThinLTO) the module, before modifying it.
+  ModuleHookFn PreOptModuleHook;
+
+  /// This hook is called after promoting any internal functions
+  /// (ThinLTO-specific).
+  ModuleHookFn PostPromoteModuleHook;
+
+  /// This hook is called after internalizing the module.
+  ModuleHookFn PostInternalizeModuleHook;
+
+  /// This hook is called after importing from other modules (ThinLTO-specific).
+  ModuleHookFn PostImportModuleHook;
+
+  /// This module hook is called after optimization is complete.
+  ModuleHookFn PostOptModuleHook;
+
+  /// This module hook is called before code generation. It is similar to the
+  /// PostOptModuleHook, but for parallel code generation it is called after
+  /// splitting the module.
+  ModuleHookFn PreCodeGenModuleHook;
+
+  /// A combined index hook is called after all per-module indexes have been
+  /// combined (ThinLTO-specific). It can be used to implement -save-temps for
+  /// the combined index.
+  ///
+  /// If this function returns false, any further processing for ThinLTO tasks
+  /// is aborted.
+  ///
+  /// It is called regardless of whether the backend is in-process, although it
+  /// is not called from individual backend processes.
+  typedef std::function<bool(const ModuleSummaryIndex &Index)>
+      CombinedIndexHookFn;
+  CombinedIndexHookFn CombinedIndexHook;
+
+  /// This is a convenience function that configures this Config object to write
+  /// temporary files named after the given OutputFileName for each of the LTO
+  /// phases to disk. A client can use this function to implement -save-temps.
+  ///
+  /// FIXME: Temporary files derived from ThinLTO backends are currently named
+  /// after the input file name, rather than the output file name, when
+  /// UseInputModulePath is set to true.
+  ///
+  /// Specifically, it (1) sets each of the above module hooks and the combined
+  /// index hook to a function that calls the hook function (if any) that was
+  /// present in the appropriate field when the addSaveTemps function was
+  /// called, and writes the module to a bitcode file with a name prefixed by
+  /// the given output file name, and (2) creates a resolution file whose name
+  /// is prefixed by the given output file name and sets ResolutionFile to its
+  /// file handle.
+  Error addSaveTemps(std::string OutputFileName,
+                     bool UseInputModulePath = false);
+};
+
+/// This type defines a stream callback. A stream callback is used to add a
+/// native object that is generated on the fly. The callee must set up and
+/// return a output stream to write the native object to.
+///
+/// Stream callbacks must be thread safe.
+typedef std::function<std::unique_ptr<raw_pwrite_stream>(size_t Task)>
+    AddStreamFn;
+
+/// A derived class of LLVMContext that initializes itself according to a given
+/// Config object. The purpose of this class is to tie ownership of the
+/// diagnostic handler to the context, as opposed to the Config object (which
+/// may be ephemeral).
+struct LTOLLVMContext : LLVMContext {
+  static void funcDiagHandler(const DiagnosticInfo &DI, void *Context) {
+    auto *Fn = static_cast<DiagnosticHandlerFunction *>(Context);
+    (*Fn)(DI);
+  }
+
+  LTOLLVMContext(const Config &C) : DiagHandler(C.DiagHandler) {
+    setDiscardValueNames(C.ShouldDiscardValueNames);
+    enableDebugTypeODRUniquing();
+    setDiagnosticHandler(funcDiagHandler, &DiagHandler, true);
+  }
+  DiagnosticHandlerFunction DiagHandler;
+};
+
+}
+}
+
+#endif

llvm/trunk/include/llvm/LTO/LTO.h

 // to be used both by LTO classes as well as by clients (gold-plugin) that
 // don't utilize the LTO code generator interfaces.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LTO_LTO_H
 #define LLVM_LTO_LTO_H
 
+#include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringSet.h"
+#include "llvm/CodeGen/Analysis.h"
+#include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/LTO/Config.h"
+#include "llvm/Linker/IRMover.h"
+#include "llvm/Object/IRObjectFile.h"
+#include "llvm/Support/thread.h"
+#include "llvm/Target/TargetOptions.h"
+#include "llvm/Transforms/IPO/FunctionImport.h"
 
 namespace llvm {
 
+class Error;
 class LLVMContext;
 class MemoryBufferRef;
 class Module;
+class Target;
+class raw_pwrite_stream;
 
 /// Helper to load a module from bitcode.
 std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
                                              LLVMContext &Context, bool Lazy);
 
 /// Provide a "loader" for the FunctionImporter to access function from other
 /// modules.
 class ModuleLoader {
function_ref<void(StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes)>
         recordNewLinkage);
 
 /// Update the linkages in the given \p Index to mark exported values
 /// as external and non-exported values as internal. The ThinLTO backends
 /// must apply the changes to the Module via thinLTOInternalizeModule.
 void thinLTOInternalizeAndPromoteInIndex(
     ModuleSummaryIndex &Index,
     function_ref<bool(StringRef, GlobalValue::GUID)> isExported);
+
+namespace lto {
+
+class LTO;
+struct SymbolResolution;
+class ThinBackendProc;
+
+/// An input file. This is a wrapper for IRObjectFile that exposes only the
+/// information that an LTO client should need in order to do symbol resolution.
+class InputFile {
+  // FIXME: Remove LTO class friendship once we have bitcode symbol tables.
+  friend LTO;
+  InputFile() = default;
+
+  // FIXME: Remove the LLVMContext once we have bitcode symbol tables.
+  LLVMContext Ctx;
+  std::unique_ptr<object::IRObjectFile> Obj;
+
+public:
+  /// Create an InputFile.
+  static Expected<std::unique_ptr<InputFile>> create(MemoryBufferRef Object);
+
+  class symbol_iterator;
+
+  /// This is a wrapper for object::basic_symbol_iterator that exposes only the
+  /// information that an LTO client should need in order to do symbol
+  /// resolution.
+  ///
+  /// This object is ephemeral; it is only valid as long as an iterator obtained
+  /// from symbols() refers to it.
+  class Symbol {
+    friend symbol_iterator;
+    friend LTO;
+
+    object::basic_symbol_iterator I;
+    const GlobalValue *GV;
+    uint32_t Flags;
+    SmallString<64> Name;
+
+    bool shouldSkip() {
+      return !(Flags & object::BasicSymbolRef::SF_Global) ||
+             (Flags & object::BasicSymbolRef::SF_FormatSpecific);
+    }
+
+    void skip() {
+      const object::SymbolicFile *Obj = I->getObject();
+      auto E = Obj->symbol_end();
+      while (I != E) {
+        Flags = I->getFlags();
+        if (!shouldSkip())
+          break;
+        ++I;
+      }
+      if (I == E)
+        return;
+
+      Name.clear();
+      {
+        raw_svector_ostream OS(Name);
+        I->printName(OS);
+      }
+      GV = cast<object::IRObjectFile>(Obj)->getSymbolGV(I->getRawDataRefImpl());
+    }
+
+  public:
+    Symbol(object::basic_symbol_iterator I) : I(I) { skip(); }
+
+    StringRef getName() const { return Name; }
+    StringRef getIRName() const {
+      if (GV)
+        return GV->getName();
+      return StringRef();
+    }
+    uint32_t getFlags() const { return Flags; }
+    GlobalValue::VisibilityTypes getVisibility() const {
+      if (GV)
+        return GV->getVisibility();
+      return GlobalValue::DefaultVisibility;
+    }
+    bool canBeOmittedFromSymbolTable() const {
+      return GV && llvm::canBeOmittedFromSymbolTable(GV);
+    }
+    Expected<const Comdat *> getComdat() const {
+      const GlobalObject *GO;
+      if (auto *GA = dyn_cast<GlobalAlias>(GV)) {
+        GO = GA->getBaseObject();
+        if (!GO)
+          return make_error<StringError>("Unable to determine comdat of alias!",
+                                         inconvertibleErrorCode());
+      } else {
+        GO = cast<GlobalObject>(GV);
+      }
+      if (GV)
+        return GV->getComdat();
+      return nullptr;
+    }
+    size_t getCommonSize() const {
+      assert(Flags & object::BasicSymbolRef::SF_Common);
+      if (!GV)
+        return 0;
+      return GV->getParent()->getDataLayout().getTypeAllocSize(
+          GV->getType()->getElementType());
+    }
+    unsigned getCommonAlignment() const {
+      assert(Flags & object::BasicSymbolRef::SF_Common);
+      if (!GV)
+        return 0;
+      return GV->getAlignment();
+    }
+  };
+
+  class symbol_iterator {
+    Symbol Sym;
+
+  public:
+    symbol_iterator(object::basic_symbol_iterator I) : Sym(I) {}
+
+    symbol_iterator &operator++() {
+      ++Sym.I;
+      Sym.skip();
+      return *this;
+    }
+
+    symbol_iterator operator++(int) {
+      symbol_iterator I = *this;
+      ++*this;
+      return I;
     }
 
+    const Symbol &operator*() const { return Sym; }
+    const Symbol *operator->() const { return &Sym; }
+
+    bool operator!=(const symbol_iterator &Other) const {
+      return Sym.I != Other.Sym.I;
+    }
+  };
+
+  /// A range over the symbols in this InputFile.
+  iterator_range<symbol_iterator> symbols() {
+    return llvm::make_range(symbol_iterator(Obj->symbol_begin()),
+                            symbol_iterator(Obj->symbol_end()));
+  }
+
+  StringRef getSourceFileName() const {
+    return Obj->getModule().getSourceFileName();
+  }
+};
+
+/// A ThinBackend defines what happens after the thin-link phase during ThinLTO.
+/// The details of this type definition aren't important; clients can only
+/// create a ThinBackend using one of the create*ThinBackend() functions below.
+typedef std::function<std::unique_ptr<ThinBackendProc>(
+    Config &C, ModuleSummaryIndex &CombinedIndex,
+    StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
+    AddStreamFn AddStream)>
+    ThinBackend;
+
+/// This ThinBackend runs the individual backend jobs in-process.
+ThinBackend createInProcessThinBackend(unsigned ParallelismLevel);
+
+/// This ThinBackend writes individual module indexes to files, instead of
+/// running the individual backend jobs. This backend is for distributed builds
+/// where separate processes will invoke the real backends.
+///
+/// To find the path to write the index to, the backend checks if the path has a
+/// prefix of OldPrefix; if so, it replaces that prefix with NewPrefix. It then
+/// appends ".thinlto.bc" and writes the index to that path. If
+/// ShouldEmitImportsFiles is true it also writes a list of imported files to a
+/// similar path with ".imports" appended instead.
+ThinBackend createWriteIndexesThinBackend(std::string OldPrefix,
+                                          std::string NewPrefix,
+                                          bool ShouldEmitImportsFiles,
+                                          std::string LinkedObjectsFile);
+
+/// This class implements a resolution-based interface to LLVM's LTO
+/// functionality. It supports regular LTO, parallel LTO code generation and
+/// ThinLTO. You can use it from a linker in the following way:
+/// - Set hooks and code generation options (see lto::Config struct defined in
+///   Config.h), and use the lto::Config object to create an lto::LTO object.
+/// - Create lto::InputFile objects using lto::InputFile::create(), then use
+///   the symbols() function to enumerate its symbols and compute a resolution
+///   for each symbol (see SymbolResolution below).
+/// - After the linker has visited each input file (and each regular object
+///   file) and computed a resolution for each symbol, take each lto::InputFile
+///   and pass it and an array of symbol resolutions to the add() function.
+/// - Call the getMaxTasks() function to get an upper bound on the number of
+///   native object files that LTO may add to the link.
+/// - Call the run() function. This function will use the supplied AddStream
+///   function to add up to getMaxTasks() native object files to the link.
+class LTO {
+  friend InputFile;
+
+public:
+  /// Create an LTO object. A default constructed LTO object has a reasonable
+  /// production configuration, but you can customize it by passing arguments to
+  /// this constructor.
+  /// FIXME: We do currently require the DiagHandler field to be set in Conf.
+  /// Until that is fixed, a Config argument is required.
+  LTO(Config Conf, ThinBackend Backend = nullptr,
+      unsigned ParallelCodeGenParallelismLevel = 1);
+
+  /// Add an input file to the LTO link, using the provided symbol resolutions.
+  /// The symbol resolutions must appear in the enumeration order given by
+  /// InputFile::symbols().
+  Error add(std::unique_ptr<InputFile> Obj, ArrayRef<SymbolResolution> Res);
+
+  /// Returns an upper bound on the number of tasks that the client may expect.
+  /// This may only be called after all IR object files have been added. For a
+  /// full description of tasks see LTOBackend.h.
+  size_t getMaxTasks() const;
+
+  /// Runs the LTO pipeline. This function calls the supplied AddStream function
+  /// to add native object files to the link.
+  Error run(AddStreamFn AddStream);
+
+private:
+  Config Conf;
+
+  struct RegularLTOState {
+    RegularLTOState(unsigned ParallelCodeGenParallelismLevel, Config &Conf);
+
+    unsigned ParallelCodeGenParallelismLevel;
+    LTOLLVMContext Ctx;
+    bool HasModule = false;
+    std::unique_ptr<Module> CombinedModule;
+    IRMover Mover;
+  } RegularLTO;
+
+  struct ThinLTOState {
+    ThinLTOState(ThinBackend Backend);
+
+    ThinBackend Backend;
+    ModuleSummaryIndex CombinedIndex;
+    MapVector<StringRef, MemoryBufferRef> ModuleMap;
+    DenseMap<GlobalValue::GUID, StringRef> PrevailingModuleForGUID;
+  } ThinLTO;
+
+  // The global resolution for a particular (mangled) symbol name. This is in
+  // particular necessary to track whether each symbol can be internalized.
+  // Because any input file may introduce a new cross-partition reference, we
+  // cannot make any final internalization decisions until all input files have
+  // been added and the client has called run(). During run() we apply
+  // internalization decisions either directly to the module (for regular LTO)
+  // or to the combined index (for ThinLTO).
+  struct GlobalResolution {
+    /// The unmangled name of the global.
+    std::string IRName;
+
+    bool UnnamedAddr = true;
+
+    /// This field keeps track of the partition number of this global. The
+    /// regular LTO object is partition 0, while each ThinLTO object has its own
+    /// partition number from 1 onwards.
+    ///
+    /// Any global that is defined or used by more than one partition, or that
+    /// is referenced externally, may not be internalized.
+    ///
+    /// Partitions generally have a one-to-one correspondence with tasks, except
+    /// that we use partition 0 for all parallel LTO code generation partitions.
+    /// Any partitioning of the combined LTO object is done internally by the
+    /// LTO backend.
+    size_t Partition = Unknown;
+
+    /// Special partition numbers.
+    enum {
+      /// A partition number has not yet been assigned to this global.
+      Unknown = -1ull,
+
+      /// This global is either used by more than one partition or has an
+      /// external reference, and therefore cannot be internalized.
+      External = -2ull,
+    };
+  };
+
+  // Global mapping from mangled symbol names to resolutions.
+  StringMap<GlobalResolution> GlobalResolutions;
+
+  void writeToResolutionFile(InputFile *Input, ArrayRef<SymbolResolution> Res);
+
+  void addSymbolToGlobalRes(object::IRObjectFile *Obj,
+                            SmallPtrSet<GlobalValue *, 8> &Used,
+                            const InputFile::Symbol &Sym, SymbolResolution Res,
+                            size_t Partition);
+
+  Error addRegularLTO(std::unique_ptr<InputFile> Input,
+                      ArrayRef<SymbolResolution> Res);
+  Error addThinLTO(std::unique_ptr<InputFile> Input,
+                   ArrayRef<SymbolResolution> Res);
+
+  Error runRegularLTO(AddStreamFn AddStream);
+  Error runThinLTO(AddStreamFn AddStream);
+
+  mutable bool CalledGetMaxTasks = false;
+};
+
+/// The resolution for a symbol. The linker must provide a SymbolResolution for
+/// each global symbol based on its internal resolution of that symbol.
+struct SymbolResolution {
+  SymbolResolution()
+      : Prevailing(0), FinalDefinitionInLinkageUnit(0), VisibleToRegularObj(0) {
+  }
+  /// The linker has chosen this definition of the symbol.
+  unsigned Prevailing : 1;
+
+  /// The definition of this symbol is unpreemptable at runtime and is known to
+  /// be in this linkage unit.
+  unsigned FinalDefinitionInLinkageUnit : 1;
+
+  /// The definition of this symbol is visible outside of the LTO unit.
+  unsigned VisibleToRegularObj : 1;
+};
+
+} // namespace lto
+} // namespace llvm
+
 #endif

llvm/trunk/include/llvm/LTO/LTOBackend.h

+//===-LTOBackend.h - LLVM Link Time Optimizer Backend ---------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the "backend" phase of LTO, i.e. it performs
+// optimization and code generation on a loaded module. It is generally used
+// internally by the LTO class but can also be used independently, for example
+// to implement a standalone ThinLTO backend.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LTO_LTOBACKEND_H
+#define LLVM_LTO_LTOBACKEND_H
+
+#include "llvm/ADT/MapVector.h"
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/IR/ModuleSummaryIndex.h"
+#include "llvm/LTO/Config.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Target/TargetOptions.h"
+#include "llvm/Transforms/IPO/FunctionImport.h"
+
+namespace llvm {
+
+class Error;
+class Module;
+class Target;
+
+namespace lto {
+
+/// Runs a regular LTO backend.
+Error backend(Config &C, AddStreamFn AddStream,
+              unsigned ParallelCodeGenParallelismLevel,
+              std::unique_ptr<Module> M);
+
+/// Runs a ThinLTO backend.
+Error thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
+                  ModuleSummaryIndex &CombinedIndex,
+                  const FunctionImporter::ImportMapTy &ImportList,
+                  const GVSummaryMapTy &DefinedGlobals,
+                  MapVector<StringRef, MemoryBufferRef> &ModuleMap);
+
+}
+}
+
+#endif

llvm/trunk/lib/LTO/CMakeLists.txt

 else()
   # Not producing a VC revision include.
   set(version_inc)
 endif()
 
 
 add_llvm_library(LLVMLTO
   LTO.cpp
+  LTOBackend.cpp
   LTOModule.cpp
   LTOCodeGenerator.cpp
   UpdateCompilerUsed.cpp
   ThinLTOCodeGenerator.cpp
   ${version_inc}
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/LTO
 )
 
 add_dependencies(LLVMLTO intrinsics_gen)

llvm/trunk/lib/LTO/LLVMBuild.txt

  InstCombine
  Linker
  MC
  ObjCARC
  Object
  Scalar
  Support
  Target
- TransformUtils
+ TransformUtils
 

llvm/trunk/lib/LTO/LTO.cpp

 //===-LTO.cpp - LLVM Link Time Optimizer ----------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements functions and classes used to support LTO.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTO.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/ReaderWriter.h"
+#include "llvm/CodeGen/Analysis.h"
+#include "llvm/IR/AutoUpgrade.h"
+#include "llvm/IR/DiagnosticPrinter.h"
+#include "llvm/IR/LegacyPassManager.h"
+#include "llvm/LTO/LTOBackend.h"
+#include "llvm/Linker/IRMover.h"
+#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
+#include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
+#include "llvm/Target/TargetMachine.h"
+#include "llvm/Target/TargetOptions.h"
+#include "llvm/Transforms/IPO.h"
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/Utils/SplitModule.h"
 
-namespace llvm {
+#include <set>
+
+using namespace llvm;
+using namespace lto;
+using namespace object;
 
 // Simple helper to load a module from bitcode
-std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
-                                             LLVMContext &Context, bool Lazy) {
+std::unique_ptr<Module>
+llvm::loadModuleFromBuffer(const MemoryBufferRef &Buffer, LLVMContext &Context,
+                           bool Lazy) {
   SMDiagnostic Err;
   ErrorOr<std::unique_ptr<Module>> ModuleOrErr(nullptr);
   if (Lazy) {
     ModuleOrErr =
         getLazyBitcodeModule(MemoryBuffer::getMemBuffer(Buffer, false), Context,
                              /* ShouldLazyLoadMetadata */ Lazy);
   } else {
     ModuleOrErr = parseBitcodeFile(Buffer, Context);

 }
 
 // Resolve Weak and LinkOnce values in the \p Index.
 //
 // We'd like to drop these functions if they are no longer referenced in the
 // current module. However there is a chance that another module is still
 // referencing them because of the import. We make sure we always emit at least
 // one copy.
-void thinLTOResolveWeakForLinkerInIndex(
+void llvm::thinLTOResolveWeakForLinkerInIndex(
     ModuleSummaryIndex &Index,
     function_ref<bool(GlobalValue::GUID, const GlobalValueSummary *)>
         isPrevailing,
     function_ref<void(StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes)>
         recordNewLinkage) {
   // We won't optimize the globals that are referenced by an alias for now
   // Ideally we should turn the alias into a global and duplicate the definition
   // when needed.
if (isExported(S->modulePath(), GUID)) {
         S->setLinkage(GlobalValue::ExternalLinkage);
     } else if (!GlobalValue::isLocalLinkage(S->linkage()))
       S->setLinkage(GlobalValue::InternalLinkage);
   }
 }
 
 // Update the linkages in the given \p Index to mark exported values
 // as external and non-exported values as internal.
-void thinLTOInternalizeAndPromoteInIndex(
+void llvm::thinLTOInternalizeAndPromoteInIndex(
     ModuleSummaryIndex &Index,
     function_ref<bool(StringRef, GlobalValue::GUID)> isExported) {
   for (auto &I : Index)
     thinLTOInternalizeAndPromoteGUID(I.second, I.first, isExported);
 }
+
+Expected<std::unique_ptr<InputFile>> InputFile::create(MemoryBufferRef Object) {
+  std::unique_ptr<InputFile> File(new InputFile);
+  std::string Msg;
+  auto DiagHandler = [](const DiagnosticInfo &DI, void *MsgP) {
+    auto *Msg = reinterpret_cast<std::string *>(MsgP);
+    raw_string_ostream OS(*Msg);
+    DiagnosticPrinterRawOStream DP(OS);
+    DI.print(DP);
+  };
+  File->Ctx.setDiagnosticHandler(DiagHandler, static_cast<void *>(&Msg));
+
+  ErrorOr<std::unique_ptr<object::IRObjectFile>> IRObj =
+      IRObjectFile::create(Object, File->Ctx);
+  if (!Msg.empty())
+    return make_error<StringError>(Msg, inconvertibleErrorCode());
+  if (!IRObj)
+    return errorCodeToError(IRObj.getError());
+  File->Obj = std::move(*IRObj);
+
+  File->Ctx.setDiagnosticHandler(nullptr, nullptr);
+
+  return std::move(File);
+}
+
+LTO::RegularLTOState::RegularLTOState(unsigned ParallelCodeGenParallelismLevel,
+                                      Config &Conf)
+    : ParallelCodeGenParallelismLevel(ParallelCodeGenParallelismLevel),
+      Ctx(Conf), CombinedModule(llvm::make_unique<Module>("ld-temp.o", Ctx)),
+      Mover(*CombinedModule) {}
+
+LTO::ThinLTOState::ThinLTOState(ThinBackend Backend) : Backend(Backend) {
+  if (!Backend)
+    this->Backend = createInProcessThinBackend(thread::hardware_concurrency());
+}
+
+LTO::LTO(Config Conf, ThinBackend Backend,
+         unsigned ParallelCodeGenParallelismLevel)
+    : Conf(std::move(Conf)),
+      RegularLTO(ParallelCodeGenParallelismLevel, this->Conf),
+      ThinLTO(Backend) {}
+
+// Add the given symbol to the GlobalResolutions map, and resolve its partition.
+void LTO::addSymbolToGlobalRes(IRObjectFile *Obj,
+                               SmallPtrSet<GlobalValue *, 8> &Used,
+                               const InputFile::Symbol &Sym,
+                               SymbolResolution Res, size_t Partition) {
+  GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
+
+  auto &GlobalRes = GlobalResolutions[Sym.getName()];
+  if (GV) {
+    GlobalRes.UnnamedAddr &= GV->hasGlobalUnnamedAddr();
+    if (Res.Prevailing)
+      GlobalRes.IRName = GV->getName();
+  }
+  if (Res.VisibleToRegularObj || (GV && Used.count(GV)) ||
+      (GlobalRes.Partition != GlobalResolution::Unknown &&
+       GlobalRes.Partition != Partition))
+    GlobalRes.Partition = GlobalResolution::External;
+  else
+    GlobalRes.Partition = Partition;
+}
+
+void LTO::writeToResolutionFile(InputFile *Input,
+                                ArrayRef<SymbolResolution> Res) {
+  StringRef Path = Input->Obj->getMemoryBufferRef().getBufferIdentifier();
+  *Conf.ResolutionFile << Path << '\n';
+  auto ResI = Res.begin();
+  for (const InputFile::Symbol &Sym : Input->symbols()) {
+    assert(ResI != Res.end());
+    SymbolResolution Res = *ResI++;
+
+    *Conf.ResolutionFile << "-r=" << Path << ',' << Sym.getName() << ',';
+    if (Res.Prevailing)
+      *Conf.ResolutionFile << 'p';
+    if (Res.FinalDefinitionInLinkageUnit)
+      *Conf.ResolutionFile << 'l';
+    if (Res.VisibleToRegularObj)
+      *Conf.ResolutionFile << 'x';
+    *Conf.ResolutionFile << '\n';
+  }
+  assert(ResI == Res.end());
+}
+
+Error LTO::add(std::unique_ptr<InputFile> Input,
+               ArrayRef<SymbolResolution> Res) {
+  assert(!CalledGetMaxTasks);
+
+  if (Conf.ResolutionFile)
+    writeToResolutionFile(Input.get(), Res);
+
+  Module &M = Input->Obj->getModule();
+  SmallPtrSet<GlobalValue *, 8> Used;
+  collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
+
+  if (!Conf.OverrideTriple.empty())
+    M.setTargetTriple(Conf.OverrideTriple);
+  else if (M.getTargetTriple().empty())
+    M.setTargetTriple(Conf.DefaultTriple);
+
+  MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
+  bool HasThinLTOSummary = hasGlobalValueSummary(MBRef, Conf.DiagHandler);
+
+  if (HasThinLTOSummary)
+    return addThinLTO(std::move(Input), Res);
+  else
+    return addRegularLTO(std::move(Input), Res);
+}
+
+// Add a regular LTO object to the link.
+Error LTO::addRegularLTO(std::unique_ptr<InputFile> Input,
+                         ArrayRef<SymbolResolution> Res) {
+  RegularLTO.HasModule = true;
+
+  ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
+      IRObjectFile::create(Input->Obj->getMemoryBufferRef(), RegularLTO.Ctx);
+  if (!ObjOrErr)
+    return errorCodeToError(ObjOrErr.getError());
+  std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
+
+  Module &M = Obj->getModule();
+  M.materializeMetadata();
+  UpgradeDebugInfo(M);
+
+  SmallPtrSet<GlobalValue *, 8> Used;
+  collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
+
+  std::vector<GlobalValue *> Keep;
+
+  for (GlobalVariable &GV : M.globals())
+    if (GV.hasAppendingLinkage())
+      Keep.push_back(&GV);
+
+  auto ResI = Res.begin();
+  for (const InputFile::Symbol &Sym :
+       make_range(InputFile::symbol_iterator(Obj->symbol_begin()),
+                  InputFile::symbol_iterator(Obj->symbol_end()))) {
+    assert(ResI != Res.end());
+    SymbolResolution Res = *ResI++;
+    addSymbolToGlobalRes(Obj.get(), Used, Sym, Res, 0);
+
+    GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
+    if (Res.Prevailing && GV) {
+      Keep.push_back(GV);
+      switch (GV->getLinkage()) {
+      default:
+        break;
+      case GlobalValue::LinkOnceAnyLinkage:
+        GV->setLinkage(GlobalValue::WeakAnyLinkage);
+        break;
+      case GlobalValue::LinkOnceODRLinkage:
+        GV->setLinkage(GlobalValue::WeakODRLinkage);
+        break;
+      }
+    }
+
+    // FIXME: use proposed local attribute for FinalDefinitionInLinkageUnit.
+  }
+  assert(ResI == Res.end());
+
+  return RegularLTO.Mover.move(Obj->takeModule(), Keep,
+                               [](GlobalValue &, IRMover::ValueAdder) {});
+}
+
+// Add a ThinLTO object to the link.
+Error LTO::addThinLTO(std::unique_ptr<InputFile> Input,
+                      ArrayRef<SymbolResolution> Res) {
+  Module &M = Input->Obj->getModule();
+  SmallPtrSet<GlobalValue *, 8> Used;
+  collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
+
+  // We need to initialize the target info for the combined regular LTO module
+  // in case we have no regular LTO objects. In that case we still need to build
+  // it as usual because the client may want to add symbol definitions to it.
+  if (RegularLTO.CombinedModule->getTargetTriple().empty()) {
+    RegularLTO.CombinedModule->setTargetTriple(M.getTargetTriple());
+    RegularLTO.CombinedModule->setDataLayout(M.getDataLayout());
+  }
+
+  MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
+  ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>>
+      SummaryObjOrErr =
+          object::ModuleSummaryIndexObjectFile::create(MBRef, Conf.DiagHandler);
+  if (!SummaryObjOrErr)
+    return errorCodeToError(SummaryObjOrErr.getError());
+  ThinLTO.CombinedIndex.mergeFrom((*SummaryObjOrErr)->takeIndex(),
+                                  ThinLTO.ModuleMap.size());
+
+  auto ResI = Res.begin();
+  for (const InputFile::Symbol &Sym : Input->symbols()) {
+    assert(ResI != Res.end());
+    SymbolResolution Res = *ResI++;
+    addSymbolToGlobalRes(Input->Obj.get(), Used, Sym, Res,
+                         ThinLTO.ModuleMap.size() + 1);
+
+    GlobalValue *GV = Input->Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
+    if (Res.Prevailing && GV)
+      ThinLTO.PrevailingModuleForGUID[GV->getGUID()] =
+          MBRef.getBufferIdentifier();
+  }
+  assert(ResI == Res.end());
+
+  ThinLTO.ModuleMap[MBRef.getBufferIdentifier()] = MBRef;
+  return Error();
+}
+
+size_t LTO::getMaxTasks() const {
+  CalledGetMaxTasks = true;
+  return RegularLTO.ParallelCodeGenParallelismLevel + ThinLTO.ModuleMap.size();
+}
+
+Error LTO::run(AddStreamFn AddStream) {
+  // Invoke regular LTO if there was a regular LTO module to start with,
+  // or if there are any hooks that the linker may have used to add
+  // its own resolved symbols to the combined module.
+  if (RegularLTO.HasModule || Conf.PreOptModuleHook ||
+      Conf.PostInternalizeModuleHook || Conf.PostOptModuleHook ||
+      Conf.PreCodeGenModuleHook)
+    if (auto E = runRegularLTO(AddStream))
+      return E;
+  return runThinLTO(AddStream);
+}
+
+Error LTO::runRegularLTO(AddStreamFn AddStream) {
+  if (Conf.PreOptModuleHook &&
+      !Conf.PreOptModuleHook(0, *RegularLTO.CombinedModule))
+    return Error();
+
+  for (const auto &R : GlobalResolutions) {
+    if (R.second.IRName.empty())
+      continue;
+    if (R.second.Partition != 0 &&
+        R.second.Partition != GlobalResolution::External)
+      continue;
+
+    GlobalValue *GV = RegularLTO.CombinedModule->getNamedValue(R.second.IRName);
+    // Ignore symbols defined in other partitions.
+    if (!GV || GV->hasLocalLinkage())
+      continue;
+    GV->setUnnamedAddr(R.second.UnnamedAddr ? GlobalValue::UnnamedAddr::Global
+                                            : GlobalValue::UnnamedAddr::None);
+    if (R.second.Partition == 0)
+      GV->setLinkage(GlobalValue::InternalLinkage);
+  }
+
+  if (Conf.PostInternalizeModuleHook &&
+      !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))
+    return Error();
+
+  return backend(Conf, AddStream, RegularLTO.ParallelCodeGenParallelismLevel,
+                 std::move(RegularLTO.CombinedModule));
+}
+
+/// This class defines the interface to the ThinLTO backend.
+class lto::ThinBackendProc {
+protected:
+  Config &Conf;
+  ModuleSummaryIndex &CombinedIndex;
+  AddStreamFn AddStream;
+  StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries;
+
+public:
+  ThinBackendProc(Config &Conf, ModuleSummaryIndex &CombinedIndex,
+                  AddStreamFn AddStream,
+                  StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries)
+      : Conf(Conf), CombinedIndex(CombinedIndex), AddStream(AddStream),
+        ModuleToDefinedGVSummaries(ModuleToDefinedGVSummaries) {}
+
+  virtual ~ThinBackendProc() {}
+  virtual Error start(size_t Task, MemoryBufferRef MBRef,
+                      StringMap<FunctionImporter::ImportMapTy> &ImportLists,
+                      MapVector<StringRef, MemoryBufferRef> &ModuleMap) = 0;
+  virtual Error wait() = 0;
+};
+
+class InProcessThinBackend : public ThinBackendProc {
+  ThreadPool BackendThreadPool;
+
+  Optional<Error> Err;
+  std::mutex ErrMu;
+
+public:
+  InProcessThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
+                       unsigned ThinLTOParallelismLevel,
+                       StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
+                       AddStreamFn AddStream)
+      : ThinBackendProc(Conf, CombinedIndex, AddStream,
+                        ModuleToDefinedGVSummaries),
+        BackendThreadPool(ThinLTOParallelismLevel) {}
+
+  Error
+  runThinLTOBackendThread(AddStreamFn AddStream, size_t Task,
+                          MemoryBufferRef MBRef,
+                          ModuleSummaryIndex &CombinedIndex,
+                          const FunctionImporter::ImportMapTy &ImportList,
+                          const GVSummaryMapTy &DefinedGlobals,
+                          MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
+    LLVMContext BackendContext;
+
+    ErrorOr<std::unique_ptr<Module>> MOrErr =
+        parseBitcodeFile(MBRef, BackendContext);
+    assert(MOrErr && "Unable to load module in thread?");
+
+    return thinBackend(Conf, Task, AddStream, **MOrErr, CombinedIndex,
+                       ImportList, DefinedGlobals, ModuleMap);
+  }
+
+  Error start(size_t Task, MemoryBufferRef MBRef,
+              StringMap<FunctionImporter::ImportMapTy> &ImportLists,
+              MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
+    StringRef ModulePath = MBRef.getBufferIdentifier();
+    BackendThreadPool.async(
+        [=](MemoryBufferRef MBRef, ModuleSummaryIndex &CombinedIndex,
+            const FunctionImporter::ImportMapTy &ImportList,
+            GVSummaryMapTy &DefinedGlobals,
+            MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
+          Error E =
+              runThinLTOBackendThread(AddStream, Task, MBRef, CombinedIndex,
+                                      ImportList, DefinedGlobals, ModuleMap);
+          if (E) {
+            std::unique_lock<std::mutex> L(ErrMu);
+            if (Err)
+              Err = joinErrors(std::move(*Err), std::move(E));
+            else
+              Err = std::move(E);
+          }
+        },
+        MBRef, std::ref(CombinedIndex), std::ref(ImportLists[ModulePath]),
+        std::ref(ModuleToDefinedGVSummaries[ModulePath]), std::ref(ModuleMap));
+    return Error();
+  }
+
+  Error wait() override {
+    BackendThreadPool.wait();
+    if (Err)
+      return std::move(*Err);
+    else
+      return Error();
+  }
+};
+
+ThinBackend lto::createInProcessThinBackend(unsigned ParallelismLevel) {
+  return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
+             StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
+             AddStreamFn AddStream) {
+    return llvm::make_unique<InProcessThinBackend>(
+        Conf, CombinedIndex, ParallelismLevel, ModuleToDefinedGVSummaries,
+        AddStream);
+  };
+}
+
+class WriteIndexesThinBackend : public ThinBackendProc {
+  std::string OldPrefix, NewPrefix;
+  bool ShouldEmitImportsFiles;
+
+  std::string LinkedObjectsFileName;
+  std::unique_ptr<llvm::raw_fd_ostream> LinkedObjectsFile;
+
+public:
+  WriteIndexesThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
+                          StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
+                          AddStreamFn AddStream, std::string OldPrefix,
+                          std::string NewPrefix, bool ShouldEmitImportsFiles,
+                          std::string LinkedObjectsFileName)
+      : ThinBackendProc(Conf, CombinedIndex, AddStream,
+                        ModuleToDefinedGVSummaries),
+        OldPrefix(OldPrefix), NewPrefix(NewPrefix),
+        ShouldEmitImportsFiles(ShouldEmitImportsFiles),
+        LinkedObjectsFileName(LinkedObjectsFileName) {}
+
+  /// Given the original \p Path to an output file, replace any path
+  /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
+  /// resulting directory if it does not yet exist.
+  std::string getThinLTOOutputFile(const std::string &Path,
+                                   const std::string &OldPrefix,
+                                   const std::string &NewPrefix) {
+    if (OldPrefix.empty() && NewPrefix.empty())
+      return Path;
+    SmallString<128> NewPath(Path);
+    llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
+    StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
+    if (!ParentPath.empty()) {
+      // Make sure the new directory exists, creating it if necessary.
+      if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
+        llvm::errs() << "warning: could not create directory '" << ParentPath
+                     << "': " << EC.message() << '\n';
+    }
+    return NewPath.str();
+  }
+
+  Error start(size_t Task, MemoryBufferRef MBRef,
+              StringMap<FunctionImporter::ImportMapTy> &ImportLists,
+              MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
+    StringRef ModulePath = MBRef.getBufferIdentifier();
+    std::string NewModulePath =
+        getThinLTOOutputFile(ModulePath, OldPrefix, NewPrefix);
+
+    std::error_code EC;
+    if (!LinkedObjectsFileName.empty()) {
+      if (!LinkedObjectsFile) {
+        LinkedObjectsFile = llvm::make_unique<raw_fd_ostream>(
+            LinkedObjectsFileName, EC, sys::fs::OpenFlags::F_None);
+        if (EC)
+          return errorCodeToError(EC);
+      }
+      *LinkedObjectsFile << NewModulePath << '\n';
+    }
+
+    std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
+    gatherImportedSummariesForModule(ModulePath, ModuleToDefinedGVSummaries,
+                                     ImportLists, ModuleToSummariesForIndex);
+
+    raw_fd_ostream OS(NewModulePath + ".thinlto.bc", EC,
+                      sys::fs::OpenFlags::F_None);
+    if (EC)
+      return errorCodeToError(EC);
+    WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
+
+    if (ShouldEmitImportsFiles)
+      return errorCodeToError(EmitImportsFiles(
+          ModulePath, NewModulePath + ".imports", ImportLists));
+    return Error();
+  }
+
+  Error wait() override { return Error(); }
+};
+
+ThinBackend lto::createWriteIndexesThinBackend(std::string OldPrefix,
+                                               std::string NewPrefix,
+                                               bool ShouldEmitImportsFiles,
+                                               std::string LinkedObjectsFile) {
+  return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
+             StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
+             AddStreamFn AddStream) {
+    return llvm::make_unique<WriteIndexesThinBackend>(
+        Conf, CombinedIndex, ModuleToDefinedGVSummaries, AddStream, OldPrefix,
+        NewPrefix, ShouldEmitImportsFiles, LinkedObjectsFile);
+  };
+}
+
+Error LTO::runThinLTO(AddStreamFn AddStream) {
+  if (ThinLTO.ModuleMap.empty())
+    return Error();
+
+  if (Conf.CombinedIndexHook && !Conf.CombinedIndexHook(ThinLTO.CombinedIndex))
+    return Error();
+
+  // Collect for each module the list of function it defines (GUID ->
+  // Summary).
+  StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
+      ModuleToDefinedGVSummaries(ThinLTO.ModuleMap.size());
+  ThinLTO.CombinedIndex.collectDefinedGVSummariesPerModule(
+      ModuleToDefinedGVSummaries);
+
+  StringMap<FunctionImporter::ImportMapTy> ImportLists(
+      ThinLTO.ModuleMap.size());
+  StringMap<FunctionImporter::ExportSetTy> ExportLists(
+      ThinLTO.ModuleMap.size());
+  ComputeCrossModuleImport(ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries,
+                           ImportLists, ExportLists);
+
+  std::set<GlobalValue::GUID> ExportedGUIDs;
+  for (auto &Res : GlobalResolutions) {
+    if (!Res.second.IRName.empty() &&
+        Res.second.Partition == GlobalResolution::External)
+      ExportedGUIDs.insert(GlobalValue::getGUID(Res.second.IRName));
+  }
+
+  auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
+    return ThinLTO.PrevailingModuleForGUID[GUID] == S->modulePath();
+  };
+  auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
+    const auto &ExportList = ExportLists.find(ModuleIdentifier);
+    return (ExportList != ExportLists.end() &&
+            ExportList->second.count(GUID)) ||
+           ExportedGUIDs.count(GUID);
+  };
+  thinLTOInternalizeAndPromoteInIndex(ThinLTO.CombinedIndex, isExported);
+  thinLTOResolveWeakForLinkerInIndex(
+      ThinLTO.CombinedIndex, isPrevailing,
+      [](StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes) {});
+
+  std::unique_ptr<ThinBackendProc> BackendProc = ThinLTO.Backend(
+      Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries, AddStream);
+
+  // Partition numbers for ThinLTO jobs start at 1 (see comments for
+  // GlobalResolution in LTO.h). Task numbers, however, start at
+  // ParallelCodeGenParallelismLevel, as tasks 0 through
+  // ParallelCodeGenParallelismLevel-1 are reserved for parallel code generation
+  // partitions.
+  size_t Task = RegularLTO.ParallelCodeGenParallelismLevel;
+  size_t Partition = 1;
+
+  for (auto &Mod : ThinLTO.ModuleMap) {
+    if (Error E = BackendProc->start(Task, Mod.second, ImportLists,
+                                     ThinLTO.ModuleMap))
+      return E;
+
+    ++Task;
+    ++Partition;
+  }
+
+  return BackendProc->wait();
 }

llvm/trunk/lib/LTO/LTOBackend.cpp

+//===-LTOBackend.cpp - LLVM Link Time Optimizer Backend -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the "backend" phase of LTO, i.e. it performs
+// optimization and code generation on a loaded module. It is generally used
+// internally by the LTO class but can also be used independently, for example
+// to implement a standalone ThinLTO backend.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/LTO/LTOBackend.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
+#include "llvm/Bitcode/ReaderWriter.h"
+#include "llvm/IR/LegacyPassManager.h"
+#include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/ThreadPool.h"
+#include "llvm/Target/TargetMachine.h"
+#include "llvm/Transforms/IPO.h"
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/Utils/FunctionImportUtils.h"
+#include "llvm/Transforms/Utils/SplitModule.h"
+
+using namespace llvm;
+using namespace lto;
+
+Error Config::addSaveTemps(std::string OutputFileName,
+                           bool UseInputModulePath) {
+  ShouldDiscardValueNames = false;
+
+  std::error_code EC;
+  ResolutionFile = llvm::make_unique<raw_fd_ostream>(
+      OutputFileName + ".resolution.txt", EC, sys::fs::OpenFlags::F_Text);
+  if (EC)
+    return errorCodeToError(EC);
+
+  auto setHook = [&](std::string PathSuffix, ModuleHookFn &Hook) {
+    // Keep track of the hook provided by the linker, which also needs to run.
+    ModuleHookFn LinkerHook = Hook;
+    Hook = [=](size_t Task, Module &M) {
+      // If the linker's hook returned false, we need to pass that result
+      // through.
+      if (LinkerHook && !LinkerHook(Task, M))
+        return false;
+
+      std::string PathPrefix;
+      // If this is the combined module (not a ThinLTO backend compile) or the
+      // user hasn't requested using the input module's path, emit to a file
+      // named from the provided OutputFileName with the Task ID appended.
+      if (M.getModuleIdentifier() == "ld-temp.o" || !UseInputModulePath) {
+        PathPrefix = OutputFileName;
+        if (Task != 0)
+          PathPrefix += "." + utostr(Task);
+      } else
+        PathPrefix = M.getModuleIdentifier();
+      std::string Path = PathPrefix + "." + PathSuffix + ".bc";
+      std::error_code EC;
+      raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
+      if (EC) {
+        // Because -save-temps is a debugging feature, we report the error
+        // directly and exit.
+        llvm::errs() << "failed to open " << Path << ": " << EC.message()
+                     << '\n';
+        exit(1);
+      }
+      WriteBitcodeToFile(&M, OS, /*ShouldPreserveUseListOrder=*/false);
+      return true;
+    };
+  };
+
+  setHook("0.preopt", PreOptModuleHook);
+  setHook("1.promote", PostPromoteModuleHook);
+  setHook("2.internalize", PostInternalizeModuleHook);
+  setHook("3.import", PostImportModuleHook);
+  setHook("4.opt", PostOptModuleHook);
+  setHook("5.precodegen", PreCodeGenModuleHook);
+
+  CombinedIndexHook = [=](const ModuleSummaryIndex &Index) {
+    std::string Path = OutputFileName + ".index.bc";
+    std::error_code EC;
+    raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
+    if (EC) {
+      // Because -save-temps is a debugging feature, we report the error
+      // directly and exit.
+      llvm::errs() << "failed to open " << Path << ": " << EC.message() << '\n';
+      exit(1);
+    }
+    WriteIndexToFile(Index, OS);
+    return true;
+  };
+
+  return Error();
+}
+
+namespace {
+
+std::unique_ptr<TargetMachine>
+createTargetMachine(Config &C, StringRef TheTriple, const Target *TheTarget) {
+  SubtargetFeatures Features;
+  Features.getDefaultSubtargetFeatures(Triple(TheTriple));
+  for (const std::string &A : C.MAttrs)
+    Features.AddFeature(A);
+
+  return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
+      TheTriple, C.CPU, Features.getString(), C.Options, C.RelocModel,
+      C.CodeModel, C.CGOptLevel));
+}
+
+bool opt(Config &C, TargetMachine *TM, size_t Task, Module &M, bool IsThinLto) {
+  M.setDataLayout(TM->createDataLayout());
+
+  legacy::PassManager passes;
+  passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
+
+  PassManagerBuilder PMB;
+  PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
+  PMB.Inliner = createFunctionInliningPass();
+  // Unconditionally verify input since it is not verified before this
+  // point and has unknown origin.
+  PMB.VerifyInput = true;
+  PMB.VerifyOutput = !C.DisableVerify;
+  PMB.LoopVectorize = true;
+  PMB.SLPVectorize = true;
+  PMB.OptLevel = C.OptLevel;
+  if (IsThinLto)
+    PMB.populateThinLTOPassManager(passes);
+  else
+    PMB.populateLTOPassManager(passes);
+  passes.run(M);
+
+  if (C.PostOptModuleHook && !C.PostOptModuleHook(Task, M))
+    return false;
+
+  return true;
+}
+
+void codegen(Config &C, TargetMachine *TM, AddStreamFn AddStream, size_t Task,
+             Module &M) {
+  if (C.PreCodeGenModuleHook && !C.PreCodeGenModuleHook(Task, M))
+    return;
+
+  std::unique_ptr<raw_pwrite_stream> OS = AddStream(Task);
+  legacy::PassManager CodeGenPasses;
+  if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
+                              TargetMachine::CGFT_ObjectFile))
+    report_fatal_error("Failed to setup codegen");
+  CodeGenPasses.run(M);
+}
+
+void splitCodeGen(Config &C, TargetMachine *TM, AddStreamFn AddStream,
+                  unsigned ParallelCodeGenParallelismLevel,
+                  std::unique_ptr<Module> M) {
+  ThreadPool CodegenThreadPool(ParallelCodeGenParallelismLevel);
+  unsigned ThreadCount = 0;
+  const Target *T = &TM->getTarget();
+
+  SplitModule(
+      std::move(M), ParallelCodeGenParallelismLevel,
+      [&](std::unique_ptr<Module> MPart) {
+        // We want to clone the module in a new context to multi-thread the
+        // codegen. We do it by serializing partition modules to bitcode
+        // (while still on the main thread, in order to avoid data races) and
+        // spinning up new threads which deserialize the partitions into
+        // separate contexts.
+        // FIXME: Provide a more direct way to do this in LLVM.
+        SmallString<0> BC;
+        raw_svector_ostream BCOS(BC);
+        WriteBitcodeToFile(MPart.get(), BCOS);
+
+        // Enqueue the task
+        CodegenThreadPool.async(
+            [&](const SmallString<0> &BC, unsigned ThreadId) {
+              LTOLLVMContext Ctx(C);
+              ErrorOr<std::unique_ptr<Module>> MOrErr = parseBitcodeFile(
+                  MemoryBufferRef(StringRef(BC.data(), BC.size()), "ld-temp.o"),
+                  Ctx);
+              if (!MOrErr)
+                report_fatal_error("Failed to read bitcode");
+              std::unique_ptr<Module> MPartInCtx = std::move(MOrErr.get());
+
+              std::unique_ptr<TargetMachine> TM =
+                  createTargetMachine(C, MPartInCtx->getTargetTriple(), T);
+              codegen(C, TM.get(), AddStream, ThreadId, *MPartInCtx);
+            },
+            // Pass BC using std::move to ensure that it get moved rather than
+            // copied into the thread's context.
+            std::move(BC), ThreadCount++);
+      },
+      false);
+}
+
+Expected<const Target *> initAndLookupTarget(Config &C, Module &M) {
+  if (!C.OverrideTriple.empty())
+    M.setTargetTriple(C.OverrideTriple);
+  else if (M.getTargetTriple().empty())
+    M.setTargetTriple(C.DefaultTriple);
+
+  std::string Msg;
+  const Target *T = TargetRegistry::lookupTarget(M.getTargetTriple(), Msg);
+  if (!T)
+    return make_error<StringError>(Msg, inconvertibleErrorCode());
+  return T;
+}
+
+}
+
+Error lto::backend(Config &C, AddStreamFn AddStream,
+                   unsigned ParallelCodeGenParallelismLevel,
+                   std::unique_ptr<Module> M) {
+  Expected<const Target *> TOrErr = initAndLookupTarget(C, *M);
+  if (!TOrErr)
+    return TOrErr.takeError();
+
+  std::unique_ptr<TargetMachine> TM =
+      createTargetMachine(C, M->getTargetTriple(), *TOrErr);
+
+  if (!opt(C, TM.get(), 0, *M, /*IsThinLto=*/false))
+    return Error();
+
+  if (ParallelCodeGenParallelismLevel == 1)
+    codegen(C, TM.get(), AddStream, 0, *M);
+  else
+    splitCodeGen(C, TM.get(), AddStream, ParallelCodeGenParallelismLevel,
+                 std::move(M));
+  return Error();
+}
+
+Error lto::thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
+                       ModuleSummaryIndex &CombinedIndex,
+                       const FunctionImporter::ImportMapTy &ImportList,
+                       const GVSummaryMapTy &DefinedGlobals,
+                       MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
+  Expected<const Target *> TOrErr = initAndLookupTarget(C, M);
+  if (!TOrErr)
+    return TOrErr.takeError();
+
+  std::unique_ptr<TargetMachine> TM =
+      createTargetMachine(C, M.getTargetTriple(), *TOrErr);
+
+  if (C.PreOptModuleHook && !C.PreOptModuleHook(Task, M))
+    return Error();
+
+  thinLTOResolveWeakForLinkerModule(M, DefinedGlobals);
+
+  renameModuleForThinLTO(M, CombinedIndex);
+
+  if (C.PostPromoteModuleHook && !C.PostPromoteModuleHook(Task, M))
+    return Error();
+
+  if (!DefinedGlobals.empty())
+    thinLTOInternalizeModule(M, DefinedGlobals);
+
+  if (C.PostInternalizeModuleHook && !C.PostInternalizeModuleHook(Task, M))
+    return Error();
+
+  auto ModuleLoader = [&](StringRef Identifier) {
+    return std::move(getLazyBitcodeModule(MemoryBuffer::getMemBuffer(
+                                              ModuleMap[Identifier], false),
+                                          M.getContext(),
+                                          /*ShouldLazyLoadMetadata=*/true)
+                         .get());
+  };
+
+  FunctionImporter Importer(CombinedIndex, ModuleLoader);
+  Importer.importFunctions(M, ImportList);
+
+  if (C.PostImportModuleHook && !C.PostImportModuleHook(Task, M))
+    return Error();
+
+  if (!opt(C, TM.get(), Task, M, /*IsThinLto=*/true))
+    return Error();
+
+  codegen(C, TM.get(), AddStream, Task, M);
+  return Error();
+}

llvm/trunk/lib/Object/IRObjectFile.cpp

llvm::object::IRObjectFile::create(MemoryBufferRef Object,
 
   ErrorOr<std::unique_ptr<Module>> MOrErr =
       getLazyBitcodeModule(std::move(Buff), Context,
                            /*ShouldLazyLoadMetadata*/ true);
   if (std::error_code EC = MOrErr.getError())
     return EC;
 
   std::unique_ptr<Module> &M = MOrErr.get();
-  return llvm::make_unique<IRObjectFile>(Object, std::move(M));
+  return llvm::make_unique<IRObjectFile>(BCOrErr.get(), std::move(M));
 }

llvm/trunk/test/CMakeLists.txt

set(LLVM_TEST_DEPENDS
           llvm-diff
           llvm-dis
           llvm-dsymutil
           llvm-dwarfdump
           llvm-dwp
           llvm-extract
           llvm-lib
           llvm-link
+          llvm-lto2
           llvm-mc
           llvm-mcmarkup
           llvm-nm
           llvm-objdump
           llvm-pdbdump
           llvm-profdata
           llvm-ranlib
           llvm-readobj

llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll

+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@a = global i32 42

llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll

+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+$c2 = comdat any
+$c1 = comdat any
+
+; This is only present in this file. The linker will keep $c1 from the first
+; file and this will be undefined.
+@will_be_undefined = global i32 1, comdat($c1)
+
+@v1 = weak_odr global i32 41, comdat($c2)
+define weak_odr protected i32 @f1(i8* %this) comdat($c2) {
+bb20:
+  store i8* %this, i8** null
+  br label %bb21
+bb21:
+  ret i32 41
+}
+
+@r21 = global i32* @v1
+@r22 = global i32(i8*)* @f1
+
+@a21 = alias i32, i32* @v1
+@a22 = alias i16, bitcast (i32* @v1 to i16*)
+
+@a23 = alias i32(i8*), i32(i8*)* @f1
+@a24 = alias i16, bitcast (i32(i8*)* @f1 to i16*)
+@a25 = alias i16, i16* @a24

llvm/trunk/test/LTO/Resolution/X86/alias.ll

+; RUN: llvm-as %s -o %t1.o
+; RUN: llvm-as %p/Inputs/alias-1.ll -o %t2.o
+; RUN: llvm-lto2 -o %t3.o %t2.o %t1.o -r %t2.o,a,px -r %t1.o,a, -r %t1.o,b,px -save-temps
+; RUN: llvm-dis < %t3.o.0.preopt.bc -o - | FileCheck %s
+; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
+
+; CHECK-NOT: alias
+; CHECK: @a = global i32 42
+; CHECK-NEXT: @b = global i32 1
+; CHECK-NOT: alias
+
+; RES: 2.o{{$}}
+; RES: {{^}}-r={{.*}}2.o,a,px{{$}}
+; RES: 1.o{{$}}
+; RES: {{^}}-r={{.*}}1.o,b,px{{$}}
+; RES: {{^}}-r={{.*}}1.o,a,{{$}}
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@a = weak alias i32, i32* @b
+@b = global i32 1

llvm/trunk/test/LTO/Resolution/X86/comdat.ll

+; RUN: llvm-as %s -o %t.o
+; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
+; RUN: llvm-lto2 -save-temps -o %t3.o %t.o %t2.o \
+; RUN:  -r=%t.o,f1,plx \
+; RUN:  -r=%t.o,v1,px \
+; RUN:  -r=%t.o,r11,px \
+; RUN:  -r=%t.o,r12,px \
+; RUN:  -r=%t.o,a11,px \
+; RUN:  -r=%t.o,a12,px \
+; RUN:  -r=%t.o,a13,px \
+; RUN:  -r=%t.o,a14,px \
+; RUN:  -r=%t.o,a15,px \
+; RUN:  -r=%t2.o,f1,l \
+; RUN:  -r=%t2.o,will_be_undefined, \
+; RUN:  -r=%t2.o,v1, \
+; RUN:  -r=%t2.o,r21,px \
+; RUN:  -r=%t2.o,r22,px \
+; RUN:  -r=%t2.o,a21,px \
+; RUN:  -r=%t2.o,a22,px \
+; RUN:  -r=%t2.o,a23,px \
+; RUN:  -r=%t2.o,a24,px \
+; RUN:  -r=%t2.o,a25,px
+; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+$c1 = comdat any
+
+@v1 = weak_odr global i32 42, comdat($c1)
+define weak_odr i32 @f1(i8*) comdat($c1) {
+bb10:
+  br label %bb11
+bb11:
+  ret i32 42
+}
+
+@r11 = global i32* @v1
+@r12 = global i32 (i8*)* @f1
+
+@a11 = alias i32, i32* @v1
+@a12 = alias i16, bitcast (i32* @v1 to i16*)
+
+@a13 = alias i32 (i8*), i32 (i8*)* @f1
+@a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
+@a15 = alias i16, i16* @a14
+
+; CHECK: $c1 = comdat any
+; CHECK: $c2 = comdat any
+
+; CHECK-DAG: @v1 = weak_odr global i32 42, comdat($c1)
+
+; CHECK-DAG: @r11 = global i32* @v1{{$}}
+; CHECK-DAG: @r12 = global i32 (i8*)* @f1{{$}}
+
+; CHECK-DAG: @r21 = global i32* @v1{{$}}
+; CHECK-DAG: @r22 = global i32 (i8*)* @f1{{$}}
+
+; CHECK-DAG: @v1.1 = internal global i32 41, comdat($c2)
+
+; CHECK-DAG: @a11 = alias i32, i32* @v1{{$}}
+; CHECK-DAG: @a12 = alias i16, bitcast (i32* @v1 to i16*)
+
+; CHECK-DAG: @a13 = alias i32 (i8*), i32 (i8*)* @f1{{$}}
+; CHECK-DAG: @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
+
+; CHECK-DAG: @a21 = alias i32, i32* @v1.1{{$}}
+; CHECK-DAG: @a22 = alias i16, bitcast (i32* @v1.1 to i16*)
+
+; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
+; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
+
+; CHECK:      define weak_odr i32 @f1(i8*) comdat($c1) {
+; CHECK-NEXT: bb10:
+; CHECK-NEXT:   br label %bb11{{$}}
+; CHECK:      bb11:
+; CHECK-NEXT:   ret i32 42
+; CHECK-NEXT: }
+
+; CHECK:      define internal i32 @f1.2(i8* %this) comdat($c2) {
+; CHECK-NEXT: bb20:
+; CHECK-NEXT:   store i8* %this, i8** null
+; CHECK-NEXT:   br label %bb21
+; CHECK:      bb21:
+; CHECK-NEXT:   ret i32 41
+; CHECK-NEXT: }

llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg

+if not 'X86' in config.root.targets:
+  config.unsupported = True

llvm/trunk/test/lit.cfg

for pattern in [r"\bbugpoint\b(?!-)",
                 r"\bllvm-diff\b",
                 r"\bllvm-dis\b",
                 r"\bllvm-dsymutil\b",
                 r"\bllvm-dwarfdump\b",
                 r"\bllvm-extract\b",
                 r"\bllvm-lib\b",
                 r"\bllvm-link\b",
                 r"\bllvm-lto\b",
+                r"\bllvm-lto2\b",
                 r"\bllvm-mc\b",
                 r"\bllvm-mcmarkup\b",
                 r"\bllvm-nm\b",
                 r"\bllvm-objdump\b",
                 r"\bllvm-pdbdump\b",
                 r"\bllvm-profdata\b",
                 r"\bllvm-ranlib\b",
                 r"\bllvm-readobj\b",

llvm/trunk/test/tools/gold/X86/coff.ll

define void @f() {
   ret void
 }
 
 ; CHECK: define internal void @g() {
 define hidden void @g() {
   ret void
 }
 
-; CHECK: define internal void @h() local_unnamed_addr {
+; CHECK: define internal void @h() {
 define linkonce_odr void @h() local_unnamed_addr {
   ret void
 }

llvm/trunk/test/tools/gold/X86/comdat.ll

-; RUN: llvm-as %s -o %t.o
+; RUN: llvm-as %s -o %t1.o
 ; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
-; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t.o %t2.o \
+; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t1.o %t2.o \
 ; RUN:  -plugin-opt=save-temps
-; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
+; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
+; RUN: llvm-readobj -t %t3.o | FileCheck --check-prefix=OBJ %s
 
 $c1 = comdat any
 
 @v1 = weak_odr global i32 42, comdat($c1)
 define weak_odr i32 @f1(i8*) comdat($c1) {
 bb10:
   br label %bb11
 bb11:
   ret i32 42
 }
 
 @r11 = global i32* @v1
 @r12 = global i32 (i8*)* @f1
 
 @a11 = alias i32, i32* @v1
 @a12 = alias i16, bitcast (i32* @v1 to i16*)
 
 @a13 = alias i32 (i8*), i32 (i8*)* @f1
 @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
 @a15 = alias i16, i16* @a14
 
-; CHECK: $c1 = comdat any
-; CHECK: $c2 = comdat any
+; gold's resolutions should tell us that our $c1 wins, and the other input's $c2
+; wins. f1 is also local due to having protected visibility in the other object.
 
-; CHECK-DAG: @v1 = weak_odr global i32 42, comdat($c1)
-
-; CHECK-DAG: @r11 = global i32* @v1{{$}}
-; CHECK-DAG: @r12 = global i32 (i8*)* @f1{{$}}
-
-; CHECK-DAG: @r21 = global i32* @v1{{$}}
-; CHECK-DAG: @r22 = global i32 (i8*)* @f1{{$}}
-
-; CHECK-DAG: @v1.1 = internal global i32 41, comdat($c2)
-
-; CHECK-DAG: @a11 = alias i32, i32* @v1{{$}}
-; CHECK-DAG: @a12 = alias i16, bitcast (i32* @v1 to i16*)
-
-; CHECK-DAG: @a13 = alias i32 (i8*), i32 (i8*)* @f1{{$}}
-; CHECK-DAG: @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
-
-; CHECK-DAG: @a21 = alias i32, i32* @v1.1{{$}}
-; CHECK-DAG: @a22 = alias i16, bitcast (i32* @v1.1 to i16*)
-
-; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
-; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
-
-; CHECK:      define weak_odr protected i32 @f1(i8*) comdat($c1) {
-; CHECK-NEXT: bb10:
-; CHECK-NEXT:   br label %bb11{{$}}
-; CHECK:      bb11:
-; CHECK-NEXT:   ret i32 42
-; CHECK-NEXT: }
-
-; CHECK:      define internal i32 @f1.2(i8* %this) comdat($c2) {
-; CHECK-NEXT: bb20:
-; CHECK-NEXT:   store i8* %this, i8** null
-; CHECK-NEXT:   br label %bb21
-; CHECK:      bb21:
-; CHECK-NEXT:   ret i32 41
-; CHECK-NEXT: }
+; RES: 1.o,f1,plx{{$}}
+; RES: 1.o,v1,px{{$}}
+; RES: 1.o,r11,px{{$}}
+; RES: 1.o,r12,px{{$}}
+; RES: 1.o,a11,px{{$}}
+; RES: 1.o,a12,px{{$}}
+; RES: 1.o,a13,px{{$}}
+; RES: 1.o,a14,px{{$}}
+; RES: 1.o,a15,px{{$}}
+
+; RES: 2.o,f1,l{{$}}
+; RES: 2.o,will_be_undefined,{{$}}
+; RES: 2.o,v1,{{$}}
+; RES: 2.o,r21,px{{$}}
+; RES: 2.o,r22,px{{$}}
+; RES: 2.o,a21,px{{$}}
+; RES: 2.o,a22,px{{$}}
+; RES: 2.o,a23,px{{$}}
+; RES: 2.o,a24,px{{$}}
+; RES: 2.o,a25,px{{$}}
+
+; f1's protected visibility should be reflected in the DSO.
+
+; OBJ:      Name: f1 (
+; OBJ-NEXT: Value:
+; OBJ-NEXT: Size:
+; OBJ-NEXT: Binding:
+; OBJ-NEXT: Type:
+; OBJ-NEXT: Other [
+; OBJ-NEXT:   STV_PROTECTED
+; OBJ-NEXT: ]

llvm/trunk/test/tools/gold/X86/common.ll

 ; RUN: llvm-as %s -o %t1.o
 ; RUN: llvm-as %p/Inputs/common.ll -o %t2.o
 ; RUN: llvm-as %p/Inputs/common2.ll -o %t2b.o
 ; RUN: llvm-as %p/Inputs/common3.ll -o %t2c.o
 
 @a = common global i16 0, align 8
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    -shared %t1.o %t2.o -o %t3.o
 ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=A
 
 ; Shared library case, we merge @a as common and keep it for the symbol table.
-; A: @a = common global i32 0, align 8
+; A: @a = common global [4 x i8] zeroinitializer, align 8
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    -shared %t1.o %t2b.o -o %t3.o
 ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=B
 
 ; (i16 align 8) + (i8 align 16) = i16 align 16
-; B: @a = common global i16 0, align 16
+; B: @a = common global [2 x i8] zeroinitializer, align 16
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    -shared %t1.o %t2c.o -o %t3.o
 ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=C
 
 ; (i16 align 8) + (i8 align 1) = i16 align 8.
-; C: @a = common global i16 0, align 8
+; C: @a = common global [2 x i8] zeroinitializer, align 8
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    %t1.o %t2.o -o %t3.o
 ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=EXEC %s
 
 ; All IR case, we internalize a after merging.
-; EXEC: @a = internal global i32 0, align 8
+; EXEC: @a = internal global [4 x i8] zeroinitializer, align 8
 
 ; RUN: llc %p/Inputs/common.ll -o %t2native.o -filetype=obj
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    %t1.o %t2native.o -o %t3.o
 ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=MIXED %s
 
 ; Mixed ELF and IR. We keep ours as common so the linker will finish the merge.
-; MIXED: @a = common global i16 0, align 8
+; MIXED: @a = common global [2 x i8] zeroinitializer, align 8

llvm/trunk/test/tools/gold/X86/emit-llvm.ll

 ; RUN: llvm-as %s -o %t.o
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
-; RUN:    --plugin-opt=generate-api-file \
 ; RUN:    -shared %t.o -o %t2.o
 ; RUN: llvm-dis %t2.o -o - | FileCheck %s
-; RUN: FileCheck --check-prefix=API %s < %T/../apifile.txt
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     -m elf_x86_64 --plugin-opt=save-temps \
 ; RUN:    -shared %t.o -o %t3.o
-; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
-; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
-; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
+; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
+; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
+; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
 ; RUN: llvm-nm %t3.o.o | FileCheck --check-prefix=NM %s
 
 ; RUN: rm -f %t4.o
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     -m elf_x86_64 --plugin-opt=disable-output \
 ; RUN:    -shared %t.o -o %t4.o
 ; RUN: not test -a %t4.o
 
 ; NM: T f3
 
 target triple = "x86_64-unknown-linux-gnu"
 
-; CHECK-DAG: @g1 = linkonce_odr constant i32 32
+; CHECK-DAG: @g1 = weak_odr constant i32 32
 @g1 = linkonce_odr constant i32 32
 
-; CHECK-DAG: @g2 = internal local_unnamed_addr constant i32 32
+; CHECK-DAG: @g2 = internal constant i32 32
 @g2 = linkonce_odr local_unnamed_addr constant i32 32
 
 ; CHECK-DAG: @g3 = internal unnamed_addr constant i32 32
 @g3 = linkonce_odr unnamed_addr constant i32 32
 
-; CHECK-DAG: @g4 = linkonce_odr global i32 32
+; CHECK-DAG: @g4 = weak_odr global i32 32
 @g4 = linkonce_odr global i32 32
 
-; CHECK-DAG: @g5 = linkonce_odr local_unnamed_addr global i32 32
+; CHECK-DAG: @g5 = weak_odr global i32 32
 @g5 = linkonce_odr local_unnamed_addr global i32 32
 
 ; CHECK-DAG: @g6 = internal unnamed_addr global i32 32
 @g6 = linkonce_odr unnamed_addr global i32 32
 
 @g7 = extern_weak global i32
 ; CHECK-DAG: @g7 = extern_weak global i32
 

 }
 
 ; CHECK-DAG: define internal void @f4()
 ; OPT2-NOT: @f4
 define linkonce_odr void @f4() local_unnamed_addr {
   ret void
 }
 
-; CHECK-DAG: define linkonce_odr void @f5()
-; OPT-DAG: define linkonce_odr void @f5()
+; CHECK-DAG: define weak_odr void @f5()
+; OPT-DAG: define weak_odr void @f5()
 define linkonce_odr void @f5() {
   ret void
 }
 @g9 = global void()* @f5
 
 ; CHECK-DAG: define internal void @f6() unnamed_addr
 ; OPT-DAG: define internal void @f6() unnamed_addr
 define linkonce_odr void @f6() unnamed_addr {
   ret void
 }
 @g10 = global void()* @f6
 
 define i32* @f7() {
   ret i32* @g7
 }
 
 define i32* @f8() {
   ret i32* @g8
 }
 
-; API: f1 PREVAILING_DEF_IRONLY
-; API: f2 PREVAILING_DEF_IRONLY
-; API: f3 PREVAILING_DEF_IRONLY_EXP
-; API: f4 PREVAILING_DEF_IRONLY_EXP
-; API: f5 PREVAILING_DEF_IRONLY_EXP
-; API: f6 PREVAILING_DEF_IRONLY_EXP
-; API: f7 PREVAILING_DEF_IRONLY_EXP
-; API: f8 PREVAILING_DEF_IRONLY_EXP
-; API: g7 UNDEF
-; API: g8 UNDEF
-; API: g9 PREVAILING_DEF_IRONLY_EXP
-; API: g10 PREVAILING_DEF_IRONLY_EXP
+; RES: .o,f1,pl{{$}}
+; RES: .o,f2,pl{{$}}
+; RES: .o,f3,px{{$}}
+; RES: .o,f4,p{{$}}
+; RES: .o,f5,px{{$}}
+; RES: .o,f6,p{{$}}
+; RES: .o,f7,px{{$}}
+; RES: .o,f8,px{{$}}
+; RES: .o,g1,px{{$}}
+; RES: .o,g2,p{{$}}
+; RES: .o,g3,p{{$}}
+; RES: .o,g4,px{{$}}
+; RES: .o,g5,px{{$}}
+; RES: .o,g6,p{{$}}
+; RES: .o,g7,{{$}}
+; RES: .o,g8,{{$}}
+; RES: .o,g9,px{{$}}
+; RES: .o,g10,px{{$}}

llvm/trunk/test/tools/gold/X86/opt-level.ll

 ; RUN: llvm-as -o %t.bc %s
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
 ; RUN:    -plugin-opt=O0 -r -o %t.o %t.bc
-; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
+; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
 ; RUN:    -plugin-opt=O1 -r -o %t.o %t.bc
-; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
+; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
 ; RUN:    -plugin-opt=O2 -r -o %t.o %t.bc
-; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
+; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
 
 ; CHECK-O0: define internal void @foo(
 ; CHECK-O1: define internal void @foo(
 ; CHECK-O2-NOT: define internal void @foo(
 define internal void @foo() {
   ret void
 }
 

llvm/trunk/test/tools/gold/X86/parallel.ll

 ; RUN: llvm-as -o %t.bc %s
+; RUN: rm -f %t.opt.bc0 %t.opt.bc1 %t.o0 %t.o1
 ; RUN: env LD_PRELOAD=%llvmshlibdir/LLVMgold.so %gold -plugin %llvmshlibdir/LLVMgold.so -u foo -u bar -plugin-opt jobs=2 -plugin-opt save-temps -m elf_x86_64 -o %t %t.bc
-; RUN: llvm-dis %t.opt.bc0 -o - | FileCheck --check-prefix=CHECK-BC0 %s
-; RUN: llvm-dis %t.opt.bc1 -o - | FileCheck --check-prefix=CHECK-BC1 %s
+; RUN: llvm-dis %t.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC0 %s
+; RUN: llvm-dis %t.1.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC1 %s
 ; RUN: llvm-nm %t.o0 | FileCheck --check-prefix=CHECK0 %s
 ; RUN: llvm-nm %t.o1 | FileCheck --check-prefix=CHECK1 %s
 
 target triple = "x86_64-unknown-linux-gnu"
 
 ; CHECK-BC0: define void @foo
 ; CHECK-BC0: declare void @bar
 ; CHECK0-NOT: bar

llvm/trunk/test/tools/gold/X86/slp-vectorize.ll

 ; RUN: llvm-as %s -o %t.o
 
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=save-temps \
 ; RUN:    -shared %t.o -o %t2.o
-; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
+; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
 
 ; test that the vectorizer is run.
 ; CHECK: fadd <4 x float>
 
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @f(float* nocapture %x) {
   %tmp = load float, float* %x, align 4

llvm/trunk/test/tools/gold/X86/start-lib-common.ll

 ; ToT gold (as of 03/2016) honors --start-lib/--end-lib, drops %t2.o and ends up
 ; with (i32 align 4) symbol.
 ; Older gold does not drop %t2.o and ends up with (i32 align 8) symbol. This is
 ; incorrect behavior, but this test does not verify this in order to support
 ; both old and new gold.
 
 ; Check that the common symbol is not dropped completely, which was a regression
 ; in r262676.
-; CHECK: @x = common global i32 0
+; CHECK: @x = common global [4 x i8] zeroinitializer

llvm/trunk/test/tools/gold/X86/strip_names.ll

 ; RUN: llvm-as %s -o %t.o
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=save-temps \
 ; RUN:    -shared %t.o -o %t2.o
-; RUN: llvm-dis %t2.o.bc -o - | FileCheck %s
+; RUN: llvm-dis %t2.o.2.internalize.bc -o - | FileCheck %s
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=emit-llvm \
 ; RUN:    -shared %t.o -o %t2.o
 ; RUN: llvm-dis %t2.o -o - | FileCheck ---check-prefix=NONAME %s
 
 ; CHECK: @GlobalValueName
 ; CHECK: @foo(i32 %in)

llvm/trunk/test/tools/gold/X86/thinlto.ll

 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=thinlto \
 ; RUN:    --plugin-opt=thinlto-index-only \
 ; RUN:    -shared %t.o %t2.o -o %t3
 ; RUN: llvm-bcanalyzer -dump %t.o.thinlto.bc | FileCheck %s --check-prefix=BACKEND1
 ; RUN: llvm-bcanalyzer -dump %t2.o.thinlto.bc | FileCheck %s --check-prefix=BACKEND2
 ; RUN: not test -e %t3
 
-; Ensure gold generates an index as well as a binary by default in ThinLTO mode.
+; Ensure gold generates an index as well as a binary with save-temps in ThinLTO mode.
 ; First force single-threaded mode
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --plugin-opt=save-temps \
 ; RUN:    --plugin-opt=thinlto \
 ; RUN:    --plugin-opt=jobs=1 \
 ; RUN:    -shared %t.o %t2.o -o %t4
-; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
+; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
 ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
 
 ; Next force multi-threaded mode
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --plugin-opt=save-temps \
 ; RUN:    --plugin-opt=thinlto \
 ; RUN:    --plugin-opt=jobs=2 \
 ; RUN:    -shared %t.o %t2.o -o %t4
-; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
+; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
 ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
 
 ; Test --plugin-opt=obj-path to ensure unique object files generated.
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=thinlto \
 ; RUN:    --plugin-opt=jobs=2 \
 ; RUN:    --plugin-opt=obj-path=%t5.o \
 ; RUN:    -shared %t.o %t2.o -o %t4
-; RUN: llvm-nm %t5.o0 | FileCheck %s --check-prefix=NM2
 ; RUN: llvm-nm %t5.o1 | FileCheck %s --check-prefix=NM2
+; RUN: llvm-nm %t5.o2 | FileCheck %s --check-prefix=NM2
 
 ; NM: T f
 ; NM2: T {{f|g}}
 
 ; The backend index for this module contains summaries from itself and
 ; Inputs/thinlto.ll, as it imports from the latter.
 ; BACKEND1: <MODULE_STRTAB_BLOCK
 ; BACKEND1-NEXT: <ENTRY {{.*}} record string = '{{.*}}/test/tools/gold/X86/Output/thinlto.ll.tmp{{.*}}.o'

llvm/trunk/test/tools/gold/X86/thinlto_alias.ll

 ; RUN: opt -module-summary %s -o %t.o
 ; RUN: opt -module-summary %p/Inputs/thinlto_alias.ll -o %t2.o
 
 ; Ensure that a preempted weak symbol that is linked in as a local
 ; copy is handled properly. Specifically, the local copy will be promoted,
 ; and internalization should be able to use the original non-promoted
 ; name to locate the summary (otherwise internalization will abort because
 ; it expects to locate summaries for all definitions).
 ; Note that gold picks the first copy of weakfunc() as the prevailing one,
 ; so listing %t2.o first is sufficient to ensure that this copy is
 ; preempted.
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     --plugin-opt=thinlto \
 ; RUN:     --plugin-opt=save-temps \
 ; RUN:     -o %t3.o %t2.o %t.o
 ; RUN: llvm-nm %t3.o | FileCheck %s
-; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
-; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
+; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+
+; This does not currently pass because the gold plugin now uses the
+; combined summary rather than the IRMover to change the module's linkage
+; during the ThinLTO backend. The internalization step implemented by IRMover
+; for preempted symbols has not yet been implemented for the combined summary.
+; XFAIL: *
 
 ; CHECK-NOT: U f
 ; OPT: define hidden void @weakfunc.llvm.0()
 ; OPT2: define weak void @weakfunc()
 
 target triple = "x86_64-unknown-linux-gnu"
 
 @weakfuncAlias = alias void (...), bitcast (void ()* @weakfunc to void (...)*)
 define weak void @weakfunc() {
 entry:
   ret void
 }

llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll

 ; RUN: opt -module-summary %s -o %t.o
 ; RUN: opt -module-summary %p/Inputs/thinlto_internalize.ll -o %t2.o
 
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     --plugin-opt=thinlto \
 ; RUN:     --plugin-opt=-import-instr-limit=0 \
 ; RUN:     --plugin-opt=save-temps \
 ; RUN:     -o %t3.o %t2.o %t.o
-; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck %s
+; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck %s
 
 ; f() should be internalized and eliminated after inlining
 ; CHECK-NOT: @f()
 
 ; h() should be internalized after promotion, and eliminated after inlining
 ; CHECK-NOT: @h.llvm.
 
 target triple = "x86_64-unknown-linux-gnu"

llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll

 ; RUN: opt -module-summary %s -o %t.o
 ; RUN: opt -module-summary %p/Inputs/thinlto_linkonceresolution.ll -o %t2.o
 
 ; Ensure the plugin ensures that for ThinLTO the prevailing copy of a
 ; linkonce symbol is changed to weak to ensure it is not eliminated.
 ; Note that gold picks the first copy of f() as the prevailing one,
 ; so listing %t2.o first is sufficient to ensure that this copy is
 ; preempted. Also, set the import-instr-limit to 0 to prevent f() from
 ; being imported from %t2.o which hides the problem.
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     --plugin-opt=thinlto \
 ; RUN:     --plugin-opt=-import-instr-limit=0 \
 ; RUN:     --plugin-opt=save-temps \
 ; RUN:     -shared \
 ; RUN:     -o %t3.o %t2.o %t.o
 ; RUN: llvm-nm %t3.o | FileCheck %s
-; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
-; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
+; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
 
 ; Ensure that f() is defined in resulting object file, and also
 ; confirm the weak linkage directly in the saved opt bitcode files.
 ; CHECK-NOT: U f
-; OPT: declare hidden void @f()
+; OPT-NOT: @f()
 ; OPT2: define weak_odr hidden void @f()
 
 target triple = "x86_64-unknown-linux-gnu"
 define i32 @g() {
   call void @f()
   ret i32 0
 }
 define linkonce_odr hidden void @f() {
   ret void
 }

llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll

 ; RUN: opt -module-summary %s -o %t.o
 ; RUN: opt -module-summary %p/Inputs/thinlto_weak_resolution.ll -o %t2.o
 
 ; Verify that prevailing weak for linker symbol is kept.
 ; Note that gold picks the first copy of a function as the prevailing one,
 ; so listing %t.o first is sufficient to ensure that its copies are prevailing.
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:     --plugin-opt=thinlto \
 ; RUN:     --plugin-opt=save-temps \
 ; RUN:     -shared \
 ; RUN:     -o %t3.o %t.o %t2.o
 
 ; RUN: llvm-nm %t3.o | FileCheck %s
 ; CHECK: weakfunc
 
-; All of the preempted functions should have been eliminated (the plugin will
-; not link them in).
-; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+; Most of the preempted functions should have been eliminated (the plugin will
+; set linkage of odr functions to available_externally and linkonce functions
+; are removed by globaldce). FIXME: Need to introduce combined index linkage
+; that means "drop this function" so we can avoid importing linkonce functions
+; and drop weak functions.
+; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
+; OPT2-NOT: @
+; OPT2: @weakfunc
 ; OPT2-NOT: @
 
-; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
+; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
 
 target triple = "x86_64-unknown-linux-gnu"
 
 
 define i32 @main() #0 {
 entry:
   call void @linkonceodralias()
   call void @linkoncealias()

llvm/trunk/test/tools/gold/X86/type-merge2.ll

 ; RUN: llvm-as %s -o %t.o
 ; RUN: llvm-as %p/Inputs/type-merge2.ll -o %t2.o
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=save-temps \
 ; RUN:    -shared %t.o %t2.o -o %t3.o
-; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
+; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
 
 %zed = type { i8 }
 define void @foo()  {
   call void @bar(%zed* null)
   ret void
 }
 declare void @bar(%zed*)
 

llvm/trunk/test/tools/gold/X86/vectorize.ll

 ; RUN: llvm-as %s -o %t.o
 
 ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=save-temps \
 ; RUN:    -shared %t.o -o %t2.o
-; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
+; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
 
 ; test that the vectorizer is run.
 ; CHECK: fadd <4 x float>
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @f(float* nocapture %x, i64 %n) {

llvm/trunk/test/tools/gold/X86/visibility.ll

 ; RUN: llvm-as %s -o %t.o
 ; RUN: llvm-as %p/Inputs/visibility.ll -o %t2.o
 
 ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
 ; RUN:    --plugin-opt=save-temps \
 ; RUN:    -shared %t.o %t2.o -o %t.so
 ; RUN: llvm-readobj -t %t.so | FileCheck %s
-; RUN: llvm-dis %t.so.bc -o - | FileCheck --check-prefix=IR %s
+; RUN: llvm-dis %t.so.2.internalize.bc -o - | FileCheck --check-prefix=IR %s
 
 ; CHECK:      Name: foo
 ; CHECK-NEXT: Value:
 ; CHECK-NEXT: Size:
 ; CHECK-NEXT: Binding: Global
 ; CHECK-NEXT: Type: Function
 ; CHECK-NEXT: Other [
 ; CHECK-NEXT:   STV_PROTECTED
 ; CHECK-NEXT: ]
 
-; IR: define protected void @foo
+; IR: define void @foo
 
 define weak protected void @foo() {
   ret void
 }

llvm/trunk/test/tools/llvm-lto2/errors.ll

+; RUN: llvm-as %s -o %t.bc
+; RUN: not llvm-lto2 -o %t2.o %t.bc 2>&1 | FileCheck --check-prefix=ERR1 %s
+; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,p -r %t.bc,bar,p %t.bc 2>&1 | FileCheck --check-prefix=ERR2 %s
+; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,q %t.bc 2>&1 | FileCheck --check-prefix=ERR3 %s
+; RUN: not llvm-lto2 -o %t2.o -r foo %t.bc 2>&1 | FileCheck --check-prefix=ERR4 %s
+
+; ERR1: missing symbol resolution for {{.*}}.bc,foo
+; ERR2: unused symbol resolution for {{.*}}.bc,bar
+; ERR3: invalid character q in resolution: {{.*}}.bc,foo
+; ERR4: invalid resolution: foo
+@foo = global i32 0

llvm/trunk/tools/gold/gold-plugin.cpp

 //===-- gold-plugin.cpp - Plugin to gold for Link Time Optimization  ------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This is a gold plugin for LLVM. It provides an LLVM implementation of the
 // interface described in http://gcc.gnu.org/wiki/whopr/driver .
 //
 //===----------------------------------------------------------------------===//
 
-#include "llvm/ADT/StringSet.h"
-#include "llvm/Analysis/TargetLibraryInfo.h"
-#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/ReaderWriter.h"
-#include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/CommandFlags.h"
-#include "llvm/CodeGen/ParallelCG.h"
 #include "llvm/Config/config.h" // plugin-api.h requires HAVE_STDINT_H
-#include "llvm/IR/AutoUpgrade.h"
 #include "llvm/IR/Constants.h"
-#include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
-#include "llvm/IR/LLVMContext.h"
-#include "llvm/IR/LegacyPassManager.h"
-#include "llvm/IR/Module.h"
-#include "llvm/IR/Verifier.h"
 #include "llvm/LTO/LTO.h"
-#include "llvm/Linker/IRMover.h"
-#include "llvm/MC/SubtargetFeature.h"
-#include "llvm/Object/IRObjectFile.h"
-#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
-#include "llvm/Support/Host.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
-#include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
-#include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
-#include "llvm/Support/thread.h"
-#include "llvm/Transforms/IPO.h"
-#include "llvm/Transforms/IPO/FunctionImport.h"
-#include "llvm/Transforms/IPO/PassManagerBuilder.h"
-#include "llvm/Transforms/Utils/FunctionImportUtils.h"
-#include "llvm/Transforms/Utils/GlobalStatus.h"
-#include "llvm/Transforms/Utils/ValueMapper.h"
 #include <list>
+#include <map>
 #include <plugin-api.h>
+#include <string>
 #include <system_error>
 #include <utility>
 #include <vector>
 
 // FIXME: remove this declaration when we stop maintaining Ubuntu Quantal and
 // Precise and Debian Wheezy (binutils 2.23 is required)
 #define LDPO_PIE 3
 
 #define LDPT_GET_SYMBOLS_V3 28
 
 using namespace llvm;
+using namespace lto;
 
 static ld_plugin_status discard_message(int level, const char *format, ...) {
   // Die loudly. Recent versions of Gold pass ld_plugin_message as the first
   // callback in the transfer vector. This should never be called.
   abort();
 }
 
 static ld_plugin_release_input_file release_input_file = nullptr;
struct PluginInputFile {
 
   ld_plugin_input_file &file() { return *File; }
 
   PluginInputFile(PluginInputFile &&RHS) = default;
   PluginInputFile &operator=(PluginInputFile &&RHS) = default;
 };
 
 struct ResolutionInfo {
-  uint64_t CommonSize = 0;
-  unsigned CommonAlign = 0;
-  bool IsLinkonceOdr = true;
-  GlobalValue::UnnamedAddr UnnamedAddr = GlobalValue::UnnamedAddr::Global;
-  GlobalValue::VisibilityTypes Visibility = GlobalValue::DefaultVisibility;
-  bool CommonInternal = false;
-  bool UseCommon = false;
+  bool CanOmitFromDynSym = true;
+  bool DefaultVisibility = true;
 };
 
-/// Class to own information used by a task or during its cleanup for a
-/// ThinLTO backend instantiation.
-class ThinLTOTaskInfo {
-  /// The output stream the task will codegen into.
-  std::unique_ptr<raw_fd_ostream> OS;
-
-  /// The file name corresponding to the output stream, used during cleanup.
-  std::string Filename;
-
-  /// Flag indicating whether the output file is a temp file that must be
-  /// added to the cleanup list during cleanup.
-  bool TempOutFile;
-
-public:
-  ThinLTOTaskInfo(std::unique_ptr<raw_fd_ostream> OS, std::string Filename,
-                  bool TempOutFile)
-      : OS(std::move(OS)), Filename(std::move(Filename)),
-        TempOutFile(TempOutFile) {}
-
-  /// Performs task related cleanup activities that must be done
-  /// single-threaded (i.e. call backs to gold).
-  void cleanup();
+struct CommonResolution {
+  bool Prevailing = false;
+  bool VisibleToRegularObj = false;
+  uint64_t Size = 0;
+  unsigned Align = 0;
 };
+
 }
 
 static ld_plugin_add_symbols add_symbols = nullptr;
 static ld_plugin_get_symbols get_symbols = nullptr;
 static ld_plugin_add_input_file add_input_file = nullptr;
 static ld_plugin_set_extra_library_path set_extra_library_path = nullptr;
 static ld_plugin_get_view get_view = nullptr;
+static bool IsExecutable = false;
 static Optional<Reloc::Model> RelocationModel;
 static std::string output_name = "";
 static std::list<claimed_file> Modules;
 static DenseMap<int, void *> FDToLeaderHandle;
 static StringMap<ResolutionInfo> ResInfo;
+static std::map<std::string, CommonResolution> Commons;
 static std::vector<std::string> Cleanup;
 static llvm::TargetOptions TargetOpts;
-static std::string DefaultTriple = sys::getDefaultTargetTriple();
+static size_t MaxTasks;
 
 namespace options {
   enum OutputType {
     OT_NORMAL,
     OT_DISABLE,
     OT_BC_ONLY,
     OT_SAVE_TEMPS
   };
-  static bool generate_api_file = false;
   static OutputType TheOutputType = OT_NORMAL;
   static unsigned OptLevel = 2;
   // Default parallelism of 0 used to indicate that user did not specify.
   // Actual parallelism default value depends on implementation.
   // Currently, code generation defaults to no parallelism, whereas
   // ThinLTO uses the hardware_concurrency as the default.
   static unsigned Parallelism = 0;
 #ifdef NDEBUG
#endif
   // If specified, expects a string of the form "oldprefix:newprefix", and
   // instead of generating these files in the same directory path as the
   // corresponding bitcode file, will use a path formed by replacing the
   // bitcode file's path prefix matching oldprefix with newprefix.
   static std::string thinlto_prefix_replace;
   // Additional options to pass into the code generator.
   // Note: This array will contain all plugin options which are not claimed
   // as plugin exclusive to pass to the code generator.
-  // For example, "generate-api-file" and "as"options are for the plugin
-  // use only and will not be passed.
   static std::vector<const char *> extra;
 
   static void process_plugin_option(const char *opt_)
   {
     if (opt_ == nullptr)
       return;
     llvm::StringRef opt = opt_;
 
-    if (opt == "generate-api-file") {
-      generate_api_file = true;
-    } else if (opt.startswith("mcpu=")) {
+    if (opt.startswith("mcpu=")) {
       mcpu = opt.substr(strlen("mcpu="));
     } else if (opt.startswith("extra-library-path=")) {
       extra_library_path = opt.substr(strlen("extra_library_path="));
     } else if (opt.startswith("mtriple=")) {
       triple = opt.substr(strlen("mtriple="));
     } else if (opt.startswith("obj-path=")) {
       obj_path = opt.substr(strlen("obj-path="));
     } else if (opt == "emit-llvm") {
for (; tv->tv_tag != LDPT_NULL; ++tv) {
     switch (static_cast<int>(tv->tv_tag)) {
     case LDPT_OUTPUT_NAME:
       output_name = tv->tv_u.tv_string;
       break;
     case LDPT_LINKER_OUTPUT:
       switch (tv->tv_u.tv_val) {
       case LDPO_REL: // .o
       case LDPO_DYN: // .so
+        IsExecutable = false;
+        RelocationModel = Reloc::PIC_;
+        break;
       case LDPO_PIE: // position independent executable
+        IsExecutable = true;
         RelocationModel = Reloc::PIC_;
         break;
       case LDPO_EXEC: // .exe
+        IsExecutable = true;
         RelocationModel = Reloc::Static;
         break;
       default:
         message(LDPL_ERROR, "Unknown output file type %d", tv->tv_u.tv_val);
         return LDPS_ERR;
       }
       break;
     case LDPT_OPTION:
ld_plugin_status onload(ld_plugin_tv *tv) {
   if (!release_input_file) {
     message(LDPL_ERROR, "release_input_file not passed to LLVMgold.");
     return LDPS_ERR;
   }
 
   return LDPS_OK;
 }
 
-static const GlobalObject *getBaseObject(const GlobalValue &GV) {
-  if (auto *GA = dyn_cast<GlobalAlias>(&GV))
-    return GA->getBaseObject();
-  return cast<GlobalObject>(&GV);
-}
-
-static bool shouldSkip(uint32_t Symflags) {
-  if (!(Symflags & object::BasicSymbolRef::SF_Global))
-    return true;
-  if (Symflags & object::BasicSymbolRef::SF_FormatSpecific)
-    return true;
-  return false;
-}
-
 static void diagnosticHandler(const DiagnosticInfo &DI) {
   if (const auto *BDI = dyn_cast<BitcodeDiagnosticInfo>(&DI)) {
     std::error_code EC = BDI->getError();
     if (EC == BitcodeError::InvalidBitcodeSignature)
       return;
   }
 
   std::string ErrStorage;
static void diagnosticHandler(const DiagnosticInfo &DI) {
   case DS_Note:
   case DS_Remark:
     Level = LDPL_INFO;
     break;
   }
   message(Level, "LLVM gold plugin: %s",  ErrStorage.c_str());
 }
 
-static void diagnosticHandlerForContext(const DiagnosticInfo &DI,
-                                        void *Context) {
-  diagnosticHandler(DI);
+static void check(Error E, std::string Msg = "LLVM gold plugin") {
+  handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
+    message(LDPL_FATAL, "%s: %s", Msg.c_str(), EIB.message().c_str());
+    return Error::success();
+  });
 }
 
-static GlobalValue::VisibilityTypes
-getMinVisibility(GlobalValue::VisibilityTypes A,
-                 GlobalValue::VisibilityTypes B) {
-  if (A == GlobalValue::HiddenVisibility)
-    return A;
-  if (B == GlobalValue::HiddenVisibility)
-    return B;
-  if (A == GlobalValue::ProtectedVisibility)
-    return A;
-  return B;
+template <typename T> static T check(Expected<T> E) {
+  if (E)
+    return std::move(*E);
+  check(E.takeError());
+  return T();
 }
 
 /// Called by gold to see whether this file is one that our plugin can handle.
 /// We'll try to open it and register all the symbols with add_symbol if
 /// possible.
 static ld_plugin_status claim_file_hook(const ld_plugin_input_file *file,
                                         int *claimed) {
-  LLVMContext Context;
   MemoryBufferRef BufferRef;
   std::unique_ptr<MemoryBuffer> Buffer;
   if (get_view) {
     const void *view;
     if (get_view(file->handle, &view) != LDPS_OK) {
       message(LDPL_ERROR, "Failed to get a view of %s", file->name);
       return LDPS_ERR;
     }
if (get_view) {
     if (std::error_code EC = BufferOrErr.getError()) {
       message(LDPL_ERROR, EC.message().c_str());
       return LDPS_ERR;
     }
     Buffer = std::move(BufferOrErr.get());
     BufferRef = Buffer->getMemBufferRef();
   }
 
-  Context.setDiagnosticHandler(diagnosticHandlerForContext);
-  ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
-      object::IRObjectFile::create(BufferRef, Context);
-  std::error_code EC = ObjOrErr.getError();
+  *claimed = 1;
+
+  Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
+  if (!ObjOrErr) {
+    handleAllErrors(ObjOrErr.takeError(), [&](const ErrorInfoBase &EI) {
+      std::error_code EC = EI.convertToErrorCode();
       if (EC == object::object_error::invalid_file_type ||
           EC == object::object_error::bitcode_section_not_found)
-    return LDPS_OK;
-
-  *claimed = 1;
+        *claimed = 0;
+      else
+        message(LDPL_ERROR,
+                "LLVM gold plugin has failed to create LTO module: %s",
+                EI.message().c_str());
+    });
 
-  if (EC) {
-    message(LDPL_ERROR, "LLVM gold plugin has failed to create LTO module: %s",
-            EC.message().c_str());
-    return LDPS_ERR;
+    return *claimed ? LDPS_ERR : LDPS_OK;
   }
-  std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
+
+  std::unique_ptr<InputFile> Obj = std::move(*ObjOrErr);
 
   Modules.resize(Modules.size() + 1);
   claimed_file &cf = Modules.back();
 
   cf.handle = file->handle;
   // Keep track of the first handle for each file descriptor, since there are
   // multiple in the case of an archive. This is used later in the case of
   // ThinLTO parallel backends to ensure that each file is only opened and
   // released once.
   auto LeaderHandle =
       FDToLeaderHandle.insert(std::make_pair(file->fd, file->handle)).first;
   cf.leader_handle = LeaderHandle->second;
   // Save the filesize since for parallel ThinLTO backends we can only
   // invoke get_input_file once per archive (only for the leader handle).
   cf.filesize = file->filesize;
   // In the case of an archive library, all but the first member must have a
   // non-zero offset, which we can append to the file name to obtain a
   // unique name.
   cf.name = file->name;
   if (file->offset)
     cf.name += ".llvm." + std::to_string(file->offset) + "." +
-               sys::path::filename(Obj->getModule().getSourceFileName()).str();
+               sys::path::filename(Obj->getSourceFileName()).str();
 
   for (auto &Sym : Obj->symbols()) {
     uint32_t Symflags = Sym.getFlags();
-    if (shouldSkip(Symflags))
-      continue;
 
     cf.syms.push_back(ld_plugin_symbol());
     ld_plugin_symbol &sym = cf.syms.back();
     sym.version = nullptr;
+    StringRef Name = Sym.getName();
+    sym.name = strdup(Name.str().c_str());
 
-    SmallString<64> Name;
-    {
-      raw_svector_ostream OS(Name);
-      Sym.printName(OS);
-    }
-    sym.name = strdup(Name.c_str());
-
-    const GlobalValue *GV = Obj->getSymbolGV(Sym.getRawDataRefImpl());
+    ResolutionInfo &Res = ResInfo[Name];
 
-    ResolutionInfo &Res = ResInfo[sym.name];
+    Res.CanOmitFromDynSym &= Sym.canBeOmittedFromSymbolTable();
 
     sym.visibility = LDPV_DEFAULT;
-    if (GV) {
-      Res.UnnamedAddr =
-          GlobalValue::getMinUnnamedAddr(Res.UnnamedAddr, GV->getUnnamedAddr());
-      Res.IsLinkonceOdr &= GV->hasLinkOnceLinkage();
-      Res.Visibility = getMinVisibility(Res.Visibility, GV->getVisibility());
-      switch (GV->getVisibility()) {
-      case GlobalValue::DefaultVisibility:
-        break;
-      case GlobalValue::HiddenVisibility:
-        sym.visibility = LDPV_HIDDEN;
-        break;
-      case GlobalValue::ProtectedVisibility:
-        sym.visibility = LDPV_PROTECTED;
-        break;
-      }
-    }
+    GlobalValue::VisibilityTypes Vis = Sym.getVisibility();
+    if (Vis != GlobalValue::DefaultVisibility)
+      Res.DefaultVisibility = false;
+    switch (Vis) {
 
     if (Symflags & object::BasicSymbolRef::SF_Undefined) {
       sym.def = LDPK_UNDEF;
-      if (GV && GV->hasExternalWeakLinkage())
+      if (Symflags & object::BasicSymbolRef::SF_Weak)
         sym.def = LDPK_WEAKUNDEF;
-    } else {
-      sym.def = LDPK_DEF;
-      if (GV) {
-        assert(!GV->hasExternalWeakLinkage() &&
-               !GV->hasAvailableExternallyLinkage() && "Not a declaration!");
-        if (GV->hasCommonLinkage())
-          sym.def = LDPK_COMMON;
-        else if (GV->isWeakForLinker())
-          sym.def = LDPK_WEAKDEF;
-      }
-    }
+    } else if (Symflags & object::BasicSymbolRef::SF_Common)
+    else if (Symflags & object::BasicSymbolRef::SF_Weak)
+    else
+      sym.def = LDPK_DEF;
 
     sym.size = 0;
     sym.comdat_key = nullptr;
-    if (GV) {
-      const GlobalObject *Base = getBaseObject(*GV);
-      if (!Base)
-        message(LDPL_FATAL, "Unable to determine comdat of alias!");
-      const Comdat *C = Base->getComdat();
-      if (C)
-        sym.comdat_key = strdup(C->getName().str().c_str());
-    }
+    const Comdat *C = check(Sym.getComdat());
 
     sym.resolution = LDPR_UNKNOWN;
   }
 
   if (!cf.syms.empty()) {
     if (add_symbols(cf.handle, cf.syms.size(), cf.syms.data()) != LDPS_OK) {
       message(LDPL_ERROR, "Unable to add symbols!");
       return LDPS_ERR;
     }
   }
 
   return LDPS_OK;
 }
 
-static void internalize(GlobalValue &GV) {
-  if (GV.isDeclarationForLinker())
-    return; // We get here if there is a matching asm definition.
-  if (!GV.hasLocalLinkage())
-    GV.setLinkage(GlobalValue::InternalLinkage);
-}
-
-static const char *getResolutionName(ld_plugin_symbol_resolution R) {
-  switch (R) {
-  case LDPR_UNKNOWN:
-    return "UNKNOWN";
-  case LDPR_UNDEF:
-    return "UNDEF";
-  case LDPR_PREVAILING_DEF:
-    return "PREVAILING_DEF";
-  case LDPR_PREVAILING_DEF_IRONLY:
-    return "PREVAILING_DEF_IRONLY";
-  case LDPR_PREEMPTED_REG:
-    return "PREEMPTED_REG";
-  case LDPR_PREEMPTED_IR:
-    return "PREEMPTED_IR";
-  case LDPR_RESOLVED_IR:
-    return "RESOLVED_IR";
-  case LDPR_RESOLVED_EXEC:
-    return "RESOLVED_EXEC";
-  case LDPR_RESOLVED_DYN:
-    return "RESOLVED_DYN";
-  case LDPR_PREVAILING_DEF_IRONLY_EXP:
-    return "PREVAILING_DEF_IRONLY_EXP";
-  }
-  llvm_unreachable("Unknown resolution");
-}
-
 static void freeSymName(ld_plugin_symbol &Sym) {
   free(Sym.name);
   free(Sym.comdat_key);
   Sym.name = nullptr;
   Sym.comdat_key = nullptr;
 }
 
 /// Helper to get a file's symbols and a view into it via gold callbacks.
 static const void *getSymbolsAndView(claimed_file &F) {
   ld_plugin_status status = get_symbols(F.handle, F.syms.size(), F.syms.data());
   if (status == LDPS_NO_SYMS)
     return nullptr;
 
   if (status != LDPS_OK)
     message(LDPL_FATAL, "Failed to get symbol information");
 
   const void *View;
   if (get_view(F.handle, &View) != LDPS_OK)
     message(LDPL_FATAL, "Failed to get a view of file");
 
   return View;
 }
 
-static std::unique_ptr<ModuleSummaryIndex>
-getModuleSummaryIndexForFile(claimed_file &F) {
-  const void *View = getSymbolsAndView(F);
-  if (!View)
-    return nullptr;
-
+static void addModule(LTO &Lto, claimed_file &F, const void *View) {
   MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), F.name);
+  Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
 
-  // Don't bother trying to build an index if there is no summary information
-  // in this bitcode file.
-  if (!object::ModuleSummaryIndexObjectFile::hasGlobalValueSummaryInMemBuffer(
-          BufferRef, diagnosticHandler))
-    return std::unique_ptr<ModuleSummaryIndex>(nullptr);
-
-  ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>> ObjOrErr =
-      object::ModuleSummaryIndexObjectFile::create(BufferRef,
-                                                   diagnosticHandler);
-
-  if (std::error_code EC = ObjOrErr.getError())
-    message(LDPL_FATAL,
-            "Could not read module summary index bitcode from file : %s",
-            EC.message().c_str());
-
-  object::ModuleSummaryIndexObjectFile &Obj = **ObjOrErr;
-
-  return Obj.takeIndex();
-}
-
-static std::unique_ptr<Module>
-getModuleForFile(LLVMContext &Context, claimed_file &F, const void *View,
-                 StringRef Name, raw_fd_ostream *ApiFile,
-                 StringSet<> &Internalize, std::vector<GlobalValue *> &Keep,
-                 StringMap<unsigned> &Realign) {
-  MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), Name);
-  ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
-      object::IRObjectFile::create(BufferRef, Context);
-
-  if (std::error_code EC = ObjOrErr.getError())
+  if (!ObjOrErr)
     message(LDPL_FATAL, "Could not read bitcode from file : %s",
-            EC.message().c_str());
-
-  object::IRObjectFile &Obj = **ObjOrErr;
-
-  Module &M = Obj.getModule();
-
-  M.materializeMetadata();
-  UpgradeDebugInfo(M);
+            toString(ObjOrErr.takeError()).c_str());
 
-  SmallPtrSet<GlobalValue *, 8> Used;
-  collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
+  InputFile &Obj = **ObjOrErr;
 
   unsigned SymNum = 0;
+  std::vector<SymbolResolution> Resols(F.syms.size());
   for (auto &ObjSym : Obj.symbols()) {
-    GlobalValue *GV = Obj.getSymbolGV(ObjSym.getRawDataRefImpl());
-    if (GV && GV->hasAppendingLinkage())
-      Keep.push_back(GV);
-
-    if (shouldSkip(ObjSym.getFlags()))
-      continue;
     ld_plugin_symbol &Sym = F.syms[SymNum];
+    SymbolResolution &R = Resols[SymNum];
     ++SymNum;
 
     ld_plugin_symbol_resolution Resolution =
         (ld_plugin_symbol_resolution)Sym.resolution;
 
-    if (options::generate_api_file)
-      *ApiFile << Sym.name << ' ' << getResolutionName(Resolution) << '\n';
-
-    if (!GV) {
-      freeSymName(Sym);
-      continue; // Asm symbol.
-    }
-
     ResolutionInfo &Res = ResInfo[Sym.name];
-    if (Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP && !Res.IsLinkonceOdr)
-      Resolution = LDPR_PREVAILING_DEF;
-
-    GV->setUnnamedAddr(Res.UnnamedAddr);
-    GV->setVisibility(Res.Visibility);
-
-    // Override gold's resolution for common symbols. We want the largest
-    // one to win.
-    if (GV->hasCommonLinkage()) {
-      if (Resolution == LDPR_PREVAILING_DEF_IRONLY)
-        Res.CommonInternal = true;
-
-      if (Resolution == LDPR_PREVAILING_DEF_IRONLY ||
-          Resolution == LDPR_PREVAILING_DEF)
-        Res.UseCommon = true;
-
-      const DataLayout &DL = GV->getParent()->getDataLayout();
-      uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
-      unsigned Align = GV->getAlignment();
-
-      if (Res.UseCommon && Size >= Res.CommonSize) {
-        // Take GV.
-        if (Res.CommonInternal)
-          Resolution = LDPR_PREVAILING_DEF_IRONLY;
-        else
-          Resolution = LDPR_PREVAILING_DEF;
-        cast<GlobalVariable>(GV)->setAlignment(
-            std::max(Res.CommonAlign, Align));
-      } else {
-        // Do not take GV, it's smaller than what we already have in the
-        // combined module.
-        Resolution = LDPR_PREEMPTED_IR;
-        if (Align > Res.CommonAlign)
-          // Need to raise the alignment though.
-          Realign[Sym.name] = Align;
-      }
-
-      Res.CommonSize = std::max(Res.CommonSize, Size);
-      Res.CommonAlign = std::max(Res.CommonAlign, Align);
-    }
 
     switch (Resolution) {
     case LDPR_UNKNOWN:
       llvm_unreachable("Unexpected resolution");
 
     case LDPR_RESOLVED_IR:
     case LDPR_RESOLVED_EXEC:
     case LDPR_RESOLVED_DYN:
     case LDPR_PREEMPTED_IR:
     case LDPR_PREEMPTED_REG:
-      break;
-
     case LDPR_UNDEF:
-      if (!GV->isDeclarationForLinker())
-        assert(GV->hasComdat());
       break;
 
-    case LDPR_PREVAILING_DEF_IRONLY: {
-      Keep.push_back(GV);
-      // The IR linker has to be able to map this value to a declaration,
-      // so we can only internalize after linking.
-      if (!Used.count(GV))
-        Internalize.insert(GV->getName());
+    case LDPR_PREVAILING_DEF_IRONLY:
+      R.Prevailing = true;
       break;
-    }
 
     case LDPR_PREVAILING_DEF:
-      Keep.push_back(GV);
-      // There is a non IR use, so we have to force optimizations to keep this.
-      switch (GV->getLinkage()) {
-      default:
-        break;
-      case GlobalValue::LinkOnceAnyLinkage:
-        GV->setLinkage(GlobalValue::WeakAnyLinkage);
-        break;
-      case GlobalValue::LinkOnceODRLinkage:
-        GV->setLinkage(GlobalValue::WeakODRLinkage);
-        break;
-      }
+      R.Prevailing = true;
+      R.VisibleToRegularObj = true;
       break;
 
-    case LDPR_PREVAILING_DEF_IRONLY_EXP: {
-      Keep.push_back(GV);
-      if (canBeOmittedFromSymbolTable(GV))
-        Internalize.insert(GV->getName());
+    case LDPR_PREVAILING_DEF_IRONLY_EXP:
+      R.Prevailing = true;
+      if (!Res.CanOmitFromDynSym)
+        R.VisibleToRegularObj = true;
       break;
     }
-    }
 
-    freeSymName(Sym);
+    if (Resolution != LDPR_RESOLVED_DYN && Resolution != LDPR_UNDEF &&
+        (IsExecutable || !Res.DefaultVisibility))
+      R.FinalDefinitionInLinkageUnit = true;
+
+    if (ObjSym.getFlags() & object::BasicSymbolRef::SF_Common) {
+      // We ignore gold's resolution for common symbols. A common symbol with
+      // the correct size and alignment is added to the module by the pre-opt
+      // module hook if any common symbol prevailed.
+      CommonResolution &CommonRes = Commons[ObjSym.getIRName()];
+      if (R.Prevailing) {
+        CommonRes.Prevailing = true;
+        CommonRes.VisibleToRegularObj = R.VisibleToRegularObj;
+      }
+      CommonRes.Size = std::max(CommonRes.Size, ObjSym.getCommonSize());
+      CommonRes.Align = std::max(CommonRes.Align, ObjSym.getCommonAlignment());
+      R.Prevailing = false;
     }
 
-  return Obj.takeModule();
+    freeSymName(Sym);
   }
 
-static void saveBCFile(StringRef Path, Module &M) {
-  std::error_code EC;
-  raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
-  if (EC)
-    message(LDPL_FATAL, "Failed to write the output file.");
-  WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
+  check(Lto.add(std::move(*ObjOrErr), Resols),
+        std::string("Failed to link module ") + F.name);
 }
 
 static void recordFile(std::string Filename, bool TempOutFile) {
   if (add_input_file(Filename.c_str()) != LDPS_OK)
     message(LDPL_FATAL,
             "Unable to add .o file to the link. File left behind in: %s",
             Filename.c_str());
   if (TempOutFile)
     Cleanup.push_back(Filename.c_str());
 }
 
-void ThinLTOTaskInfo::cleanup() {
-  // Close the output file descriptor before we pass it to gold.
-  OS->close();
-
-  recordFile(Filename, TempOutFile);
-}
-
-namespace {
-/// Class to manage optimization and code generation for a module, possibly
-/// in a thread (ThinLTO).
-class CodeGen {
-  /// The module for which this will generate code.
-  std::unique_ptr<llvm::Module> M;
-
-  /// The output stream to generate code into.
-  raw_fd_ostream *OS;
-
-  /// The task ID when this was invoked in a thread (ThinLTO).
-  int TaskID;
-
-  /// The module summary index for ThinLTO tasks.
-  const ModuleSummaryIndex *CombinedIndex;
-
-  /// The target machine for generating code for this module.
-  std::unique_ptr<TargetMachine> TM;
-
-  /// Filename to use as base when save-temps is enabled, used to get
-  /// a unique and identifiable save-temps output file for each ThinLTO backend.
-  std::string SaveTempsFilename;
-
-  /// Map from a module name to the corresponding buffer holding a view of the
-  /// bitcode provided via the get_view gold callback.
-  StringMap<MemoryBufferRef> *ModuleMap;
-
-  // Functions to import into this module.
-  FunctionImporter::ImportMapTy *ImportList;
-
-  // Map of globals defined in this module to their summary.
-  std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals;
-
-public:
-  /// Constructor used by full LTO.
-  CodeGen(std::unique_ptr<llvm::Module> M)
-      : M(std::move(M)), OS(nullptr), TaskID(-1), CombinedIndex(nullptr),
-        ModuleMap(nullptr) {
-    initTargetMachine();
-  }
-  /// Constructor used by ThinLTO.
-  CodeGen(std::unique_ptr<llvm::Module> M, raw_fd_ostream *OS, int TaskID,
-          const ModuleSummaryIndex *CombinedIndex, std::string Filename,
-          StringMap<MemoryBufferRef> *ModuleMap,
-          FunctionImporter::ImportMapTy *ImportList,
-          std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals)
-      : M(std::move(M)), OS(OS), TaskID(TaskID), CombinedIndex(CombinedIndex),
-        SaveTempsFilename(std::move(Filename)), ModuleMap(ModuleMap),
-        ImportList(ImportList), DefinedGlobals(DefinedGlobals) {
-    assert(options::thinlto == !!CombinedIndex &&
-           "Expected module summary index iff performing ThinLTO");
-    initTargetMachine();
-  }
-
-  /// Invoke LTO passes and the code generator for the module.
-  void runAll();
-
-  /// Invoke the actual code generation to emit Module's object to file.
-  void runCodegenPasses();
-
-private:
-  const Target *TheTarget;
-  std::string TripleStr;
-  std::string FeaturesString;
-  TargetOptions Options;
-
-  /// Create a target machine for the module. Must be unique for each
-  /// module/task.
-  void initTargetMachine();
-
-  std::unique_ptr<TargetMachine> createTargetMachine();
-
-  /// Run all LTO passes on the module.
-  void runLTOPasses();
-
-  /// Sets up output files necessary to perform optional multi-threaded
-  /// split code generation, and invokes the code generation implementation.
-  /// If BCFileName is not empty, saves bitcode for module partitions into
-  /// {BCFileName}0 .. {BCFileName}N.
-  void runSplitCodeGen(const SmallString<128> &BCFilename);
-};
-}
-
-static SubtargetFeatures getFeatures(Triple &TheTriple) {
-  SubtargetFeatures Features;
-  Features.getDefaultSubtargetFeatures(TheTriple);
-  for (const std::string &A : MAttrs)
-    Features.AddFeature(A);
-  return Features;
-}
-
-static CodeGenOpt::Level getCGOptLevel() {
-  switch (options::OptLevel) {
-  case 0:
-    return CodeGenOpt::None;
-  case 1:
-    return CodeGenOpt::Less;
-  case 2:
-    return CodeGenOpt::Default;
-  case 3:
-    return CodeGenOpt::Aggressive;
-  }
-  llvm_unreachable("Invalid optimization level");
-}
-
-void CodeGen::initTargetMachine() {
-  TripleStr = M->getTargetTriple();
-  Triple TheTriple(TripleStr);
-
-  std::string ErrMsg;
-  TheTarget = TargetRegistry::lookupTarget(TripleStr, ErrMsg);
-  if (!TheTarget)
-    message(LDPL_FATAL, "Target not found: %s", ErrMsg.c_str());
-
-  SubtargetFeatures Features = getFeatures(TheTriple);
-  FeaturesString = Features.getString();
-  Options = InitTargetOptionsFromCodeGenFlags();
-
-  // Disable the new X86 relax relocations since gold might not support them.
-  // FIXME: Check the gold version or add a new option to enable them.
-  Options.RelaxELFRelocations = false;
-
-  TM = createTargetMachine();
-}
-
-std::unique_ptr<TargetMachine> CodeGen::createTargetMachine() {
-  CodeGenOpt::Level CGOptLevel = getCGOptLevel();
-
-  return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
-      TripleStr, options::mcpu, FeaturesString, Options, RelocationModel,
-      CodeModel::Default, CGOptLevel));
-}
-
-void CodeGen::runLTOPasses() {
-  M->setDataLayout(TM->createDataLayout());
-
-  if (CombinedIndex) {
-    // Apply summary-based LinkOnce/Weak resolution decisions.
-    thinLTOResolveWeakForLinkerModule(*M, *DefinedGlobals);
-
-    // Apply summary-based internalization decisions. Skip if there are no
-    // defined globals from the summary since not only is it unnecessary, but
-    // if this module did not have a summary section the internalizer will
-    // assert if it finds any definitions in this module that aren't in the
-    // DefinedGlobals set.
-    if (!DefinedGlobals->empty())
-      thinLTOInternalizeModule(*M, *DefinedGlobals);
-
-    // Create a loader that will parse the bitcode from the buffers
-    // in the ModuleMap.
-    ModuleLoader Loader(M->getContext(), *ModuleMap);
-
-    // Perform function importing.
-    FunctionImporter Importer(*CombinedIndex, Loader);
-    Importer.importFunctions(*M, *ImportList);
-  }
-
-  legacy::PassManager passes;
-  passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
-
-  PassManagerBuilder PMB;
-  PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
-  PMB.Inliner = createFunctionInliningPass();
-  // Unconditionally verify input since it is not verified before this
-  // point and has unknown origin.
-  PMB.VerifyInput = true;
-  PMB.VerifyOutput = !options::DisableVerify;
-  PMB.LoopVectorize = true;
-  PMB.SLPVectorize = true;
-  PMB.OptLevel = options::OptLevel;
-  if (options::thinlto)
-    PMB.populateThinLTOPassManager(passes);
-  else
-    PMB.populateLTOPassManager(passes);
-  passes.run(*M);
-}
-
 /// Open a file and return the new file descriptor given a base input
 /// file name, a flag indicating whether a temp file should be generated,
 /// and an optional task id. The new filename generated is
 /// returned in \p NewFilename.
 static int openOutputFile(SmallString<128> InFilename, bool TempOutFile,
                           SmallString<128> &NewFilename, int TaskID = -1) {
   int FD;
   if (TempOutFile) {
if (TempOutFile) {
     std::error_code EC =
         sys::fs::openFileForWrite(NewFilename, FD, sys::fs::F_None);
     if (EC)
       message(LDPL_FATAL, "Could not open file: %s", EC.message().c_str());
   }
   return FD;
 }
 
-void CodeGen::runCodegenPasses() {
-  assert(OS && "Output stream must be set before emitting to file");
-  legacy::PassManager CodeGenPasses;
-  if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
-                              TargetMachine::CGFT_ObjectFile))
-    report_fatal_error("Failed to setup codegen");
-  CodeGenPasses.run(*M);
-}
-
-void CodeGen::runSplitCodeGen(const SmallString<128> &BCFilename) {
-  SmallString<128> Filename;
-  // Note that openOutputFile will append a unique ID for each task
-  if (!options::obj_path.empty())
-    Filename = options::obj_path;
-  else if (options::TheOutputType == options::OT_SAVE_TEMPS)
-    Filename = output_name + ".o";
-
-  // Note that the default parallelism is 1 instead of the
-  // hardware_concurrency, as there are behavioral differences between
-  // parallelism levels (e.g. symbol ordering will be different, and some uses
-  // of inline asm currently have issues with parallelism >1).
-  unsigned int MaxThreads = options::Parallelism ? options::Parallelism : 1;
-
-  std::vector<SmallString<128>> Filenames(MaxThreads);
-  std::vector<SmallString<128>> BCFilenames(MaxThreads);
-  bool TempOutFile = Filename.empty();
-  {
-    // Open a file descriptor for each backend task. This is done in a block
-    // so that the output file descriptors are closed before gold opens them.
-    std::list<llvm::raw_fd_ostream> OSs;
-    std::vector<llvm::raw_pwrite_stream *> OSPtrs(MaxThreads);
-    for (unsigned I = 0; I != MaxThreads; ++I) {
-      int FD = openOutputFile(Filename, TempOutFile, Filenames[I],
-                              // Only append ID if there are multiple tasks.
-                              MaxThreads > 1 ? I : -1);
-      OSs.emplace_back(FD, true);
-      OSPtrs[I] = &OSs.back();
-    }
-
-    std::list<llvm::raw_fd_ostream> BCOSs;
-    std::vector<llvm::raw_pwrite_stream *> BCOSPtrs;
-    if (!BCFilename.empty() && MaxThreads > 1) {
-      for (unsigned I = 0; I != MaxThreads; ++I) {
-        int FD = openOutputFile(BCFilename, false, BCFilenames[I], I);
-        BCOSs.emplace_back(FD, true);
-        BCOSPtrs.push_back(&BCOSs.back());
-      }
-    }
-
-    // Run backend tasks.
-    splitCodeGen(std::move(M), OSPtrs, BCOSPtrs,
-                 [&]() { return createTargetMachine(); });
-  }
-
-  for (auto &Filename : Filenames)
-    recordFile(Filename.c_str(), TempOutFile);
-}
-
-void CodeGen::runAll() {
-  runLTOPasses();
-
-  SmallString<128> OptFilename;
-  if (options::TheOutputType == options::OT_SAVE_TEMPS) {
-    OptFilename = output_name;
-    // If the CodeGen client provided a filename, use it. Always expect
-    // a provided filename if we are in a task (i.e. ThinLTO backend).
-    assert(!SaveTempsFilename.empty() || TaskID == -1);
-    if (!SaveTempsFilename.empty())
-      OptFilename = SaveTempsFilename;
-    OptFilename += ".opt.bc";
-    saveBCFile(OptFilename, *M);
-  }
-
-  // If we are already in a thread (i.e. ThinLTO), just perform
-  // codegen passes directly.
-  if (TaskID >= 0)
-    runCodegenPasses();
-  // Otherwise attempt split code gen.
-  else
-    runSplitCodeGen(OptFilename);
-}
-
-/// Links the module in \p View from file \p F into the combined module
-/// saved in the IRMover \p L.
-static void linkInModule(LLVMContext &Context, IRMover &L, claimed_file &F,
-                         const void *View, StringRef Name,
-                         raw_fd_ostream *ApiFile, StringSet<> &Internalize,
-                         bool SetName = false) {
-  std::vector<GlobalValue *> Keep;
-  StringMap<unsigned> Realign;
-  std::unique_ptr<Module> M = getModuleForFile(Context, F, View, Name, ApiFile,
-                                               Internalize, Keep, Realign);
-  if (!M.get())
-    return;
-  if (!options::triple.empty())
-    M->setTargetTriple(options::triple.c_str());
-  else if (M->getTargetTriple().empty()) {
-    M->setTargetTriple(DefaultTriple);
-  }
-
-  // For ThinLTO we want to propagate the source file name to ensure
-  // we can create the correct global identifiers matching those in the
-  // original module.
-  if (SetName)
-    L.getModule().setSourceFileName(M->getSourceFileName());
-
-  if (Error E = L.move(std::move(M), Keep,
-                       [](GlobalValue &, IRMover::ValueAdder) {})) {
-    handleAllErrors(std::move(E), [&](const llvm::ErrorInfoBase &EIB) {
-      message(LDPL_FATAL, "Failed to link module %s: %s", Name.str().c_str(),
-              EIB.message().c_str());
-    });
-  }
-
-  for (const auto &I : Realign) {
-    GlobalValue *Dst = L.getModule().getNamedValue(I.first());
-    if (!Dst)
+/// Add all required common symbols to M, which is expected to be the first
+/// combined module.
+static void addCommons(Module &M) {
+  for (auto &I : Commons) {
+    if (!I.second.Prevailing)
       continue;
-    cast<GlobalVariable>(Dst)->setAlignment(I.second);
-  }
+    ArrayType *Ty =
+        ArrayType::get(Type::getInt8Ty(M.getContext()), I.second.Size);
+    GlobalVariable *OldGV = M.getNamedGlobal(I.first);
+    auto *GV = new GlobalVariable(M, Ty, false, GlobalValue::CommonLinkage,
+                                  ConstantAggregateZero::get(Ty), "");
+    GV->setAlignment(I.second.Align);
+    if (OldGV) {
+      OldGV->replaceAllUsesWith(ConstantExpr::getBitCast(GV, OldGV->getType()));
+      GV->takeName(OldGV);
+      OldGV->eraseFromParent();
+    } else {
+      GV->setName(I.first);
     }
-
-/// Perform the ThinLTO backend on a single module, invoking the LTO and codegen
-/// pipelines.
-static void thinLTOBackendTask(claimed_file &F, const void *View,
-                               StringRef Name, raw_fd_ostream *ApiFile,
-                               const ModuleSummaryIndex &CombinedIndex,
-                               raw_fd_ostream *OS, unsigned TaskID,
-                               StringMap<MemoryBufferRef> &ModuleMap,
-                               FunctionImporter::ImportMapTy &ImportList,
-                               std::map<GlobalValue::GUID, GlobalValueSummary *> &DefinedGlobals) {
-  // Need to use a separate context for each task
-  LLVMContext Context;
-  Context.setDiscardValueNames(options::TheOutputType !=
-                               options::OT_SAVE_TEMPS);
-  Context.enableDebugTypeODRUniquing(); // Merge debug info types.
-  Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
-
-  std::unique_ptr<llvm::Module> NewModule(new llvm::Module(Name, Context));
-  IRMover L(*NewModule.get());
-
-  StringSet<> Dummy;
-  linkInModule(Context, L, F, View, Name, ApiFile, Dummy, true);
-  if (renameModuleForThinLTO(*NewModule, CombinedIndex))
-    message(LDPL_FATAL, "Failed to rename module for ThinLTO");
-
-  CodeGen codeGen(std::move(NewModule), OS, TaskID, &CombinedIndex, Name,
-                  &ModuleMap, &ImportList, &DefinedGlobals);
-  codeGen.runAll();
-}
-
-/// Launch each module's backend pipeline in a separate task in a thread pool.
-static void
-thinLTOBackends(raw_fd_ostream *ApiFile,
-                const ModuleSummaryIndex &CombinedIndex,
-                StringMap<MemoryBufferRef> &ModuleMap,
-                StringMap<FunctionImporter::ImportMapTy> &ImportLists,
-  StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
-      &ModuleToDefinedGVSummaries) {
-  unsigned TaskCount = 0;
-  std::vector<ThinLTOTaskInfo> Tasks;
-  Tasks.reserve(Modules.size());
-  unsigned int MaxThreads = options::Parallelism
-                                ? options::Parallelism
-                                : thread::hardware_concurrency();
-
-  // Create ThreadPool in nested scope so that threads will be joined
-  // on destruction.
-  {
-    ThreadPool ThinLTOThreadPool(MaxThreads);
-    for (claimed_file &F : Modules) {
-      // Do all the gold callbacks in the main thread, since gold is not thread
-      // safe by default.
-      const void *View = getSymbolsAndView(F);
-      if (!View)
-        continue;
-
-      SmallString<128> Filename;
-      if (!options::obj_path.empty())
-        // Note that openOutputFile will append a unique ID for each task
-        Filename = options::obj_path;
-      else if (options::TheOutputType == options::OT_SAVE_TEMPS) {
-        // Use the input file name so that we get a unique and identifiable
-        // output file for each ThinLTO backend task.
-        Filename = F.name;
-        Filename += ".thinlto.o";
-      }
-      bool TempOutFile = Filename.empty();
-
-      SmallString<128> NewFilename;
-      int FD = openOutputFile(Filename, TempOutFile, NewFilename,
-                              // Only append the TaskID if we will use the
-                              // non-unique obj_path.
-                              !options::obj_path.empty() ? TaskCount : -1);
-      TaskCount++;
-      std::unique_ptr<raw_fd_ostream> OS =
-          llvm::make_unique<raw_fd_ostream>(FD, true);
-
-      // Enqueue the task
-      ThinLTOThreadPool.async(thinLTOBackendTask, std::ref(F), View, F.name,
-                              ApiFile, std::ref(CombinedIndex), OS.get(),
-                              TaskCount, std::ref(ModuleMap),
-                              std::ref(ImportLists[F.name]),
-                              std::ref(ModuleToDefinedGVSummaries[F.name]));
-
-      // Record the information needed by the task or during its cleanup
-      // to a ThinLTOTaskInfo instance. For information needed by the task
-      // the unique_ptr ownership is transferred to the ThinLTOTaskInfo.
-      Tasks.emplace_back(std::move(OS), NewFilename.c_str(), TempOutFile);
-    }
-  }
+    // We may only internalize commons if there is a single LTO task because
+    // other native object files may require the common.
+    if (MaxTasks == 1 && !I.second.VisibleToRegularObj)
+      GV->setLinkage(GlobalValue::InternalLinkage);
 
-  for (auto &Task : Tasks)
-    Task.cleanup();
+static CodeGenOpt::Level getCGOptLevel() {
+  switch (options::OptLevel) {
+  case 0:
+    return CodeGenOpt::None;
+  case 1:
+    return CodeGenOpt::Less;
+  case 2:
+    return CodeGenOpt::Default;
+  case 3:
+    return CodeGenOpt::Aggressive;
+  }
+  llvm_unreachable("Invalid optimization level");
 }
 
 /// Parse the thinlto_prefix_replace option into the \p OldPrefix and
 /// \p NewPrefix strings, if it was specified.
 static void getThinLTOOldAndNewPrefix(std::string &OldPrefix,
                                       std::string &NewPrefix) {
   StringRef PrefixReplace = options::thinlto_prefix_replace;
   assert(PrefixReplace.empty() || PrefixReplace.find(";") != StringRef::npos);
   std::pair<StringRef, StringRef> Split = PrefixReplace.split(";");
   OldPrefix = Split.first.str();
   NewPrefix = Split.second.str();
 }
 
-/// Given the original \p Path to an output file, replace any path
-/// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
-/// resulting directory if it does not yet exist.
-static std::string getThinLTOOutputFile(const std::string &Path,
-                                        const std::string &OldPrefix,
-                                        const std::string &NewPrefix) {
-  if (OldPrefix.empty() && NewPrefix.empty())
-    return Path;
-  SmallString<128> NewPath(Path);
-  llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
-  StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
-  if (!ParentPath.empty()) {
-    // Make sure the new directory exists, creating it if necessary.
-    if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
-      llvm::errs() << "warning: could not create directory '" << ParentPath
-                   << "': " << EC.message() << '\n';
-  }
-  return NewPath.str();
-}
-
-/// Perform ThinLTO link, which creates the combined index file.
-/// Also, either launch backend threads or (under thinlto-index-only)
-/// emit individual index files for distributed backends and exit.
-static ld_plugin_status thinLTOLink(raw_fd_ostream *ApiFile) {
-  // Map from a module name to the corresponding buffer holding a view of the
-  // bitcode provided via the get_view gold callback.
-  StringMap<MemoryBufferRef> ModuleMap;
-  // Map to own RAII objects that manage the file opening and releasing
-  // interfaces with gold.
-  DenseMap<void *, std::unique_ptr<PluginInputFile>> HandleToInputFile;
-
-  // Keep track of symbols that must not be internalized because they
-  // are referenced outside of a single IR module.
-  DenseSet<GlobalValue::GUID> Preserve;
-
-  // Keep track of the prevailing copy for each GUID, for use in resolving
-  // weak linkages.
-  DenseMap<GlobalValue::GUID, const GlobalValueSummary *> PrevailingCopy;
-
-  ModuleSummaryIndex CombinedIndex;
-  uint64_t NextModuleId = 0;
-  for (claimed_file &F : Modules) {
-    if (!HandleToInputFile.count(F.leader_handle))
-      HandleToInputFile.insert(std::make_pair(
-          F.leader_handle, llvm::make_unique<PluginInputFile>(F.handle)));
-    // Pass this into getModuleSummaryIndexForFile
-    const void *View = getSymbolsAndView(F);
-    if (!View)
-      continue;
-
-    MemoryBufferRef ModuleBuffer(StringRef((const char *)View, F.filesize),
-                                 F.name);
-    assert(ModuleMap.find(ModuleBuffer.getBufferIdentifier()) ==
-               ModuleMap.end() &&
-           "Expect unique Buffer Identifier");
-    ModuleMap[ModuleBuffer.getBufferIdentifier()] = ModuleBuffer;
-
-    std::unique_ptr<ModuleSummaryIndex> Index = getModuleSummaryIndexForFile(F);
-
-    // Use gold's symbol resolution information to identify symbols referenced
-    // by more than a single IR module (i.e. referenced by multiple IR modules
-    // or by a non-IR module). Cross references introduced by importing are
-    // checked separately via the export lists. Also track the prevailing copy
-    // for later symbol resolution.
-    for (auto &Sym : F.syms) {
-      ld_plugin_symbol_resolution Resolution =
-          (ld_plugin_symbol_resolution)Sym.resolution;
-      GlobalValue::GUID SymGUID = GlobalValue::getGUID(Sym.name);
-      if (Resolution != LDPR_PREVAILING_DEF_IRONLY)
-        Preserve.insert(SymGUID);
-
-      if (Index && (Resolution == LDPR_PREVAILING_DEF ||
-                    Resolution == LDPR_PREVAILING_DEF_IRONLY ||
-                    Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP))
-        PrevailingCopy[SymGUID] = Index->getGlobalValueSummary(SymGUID);
-    }
-
-    // Skip files without a module summary.
-    if (Index)
-      CombinedIndex.mergeFrom(std::move(Index), ++NextModuleId);
-  }
-
-  // Collect for each module the list of function it defines (GUID ->
-  // Summary).
-  StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
-      ModuleToDefinedGVSummaries(NextModuleId);
-  CombinedIndex.collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
-
-  StringMap<FunctionImporter::ImportMapTy> ImportLists(NextModuleId);
-  StringMap<FunctionImporter::ExportSetTy> ExportLists(NextModuleId);
-  ComputeCrossModuleImport(CombinedIndex, ModuleToDefinedGVSummaries,
-                           ImportLists, ExportLists);
-
-  auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
-    const auto &Prevailing = PrevailingCopy.find(GUID);
-    assert(Prevailing != PrevailingCopy.end());
-    return Prevailing->second == S;
-  };
+static std::unique_ptr<LTO> createLTO() {
+  Config Conf;
+  ThinBackend Backend;
+  unsigned ParallelCodeGenParallelismLevel = 1;
 
-  // Callback for internalization, to prevent internalization of symbols
-  // that were not candidates initially, and those that are being imported
-  // (which introduces new cross references).
-  auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
-    const auto &ExportList = ExportLists.find(ModuleIdentifier);
-    return (ExportList != ExportLists.end() &&
-            ExportList->second.count(GUID)) ||
-           Preserve.count(GUID);
-  };
+  Conf.CPU = options::mcpu;
+  Conf.Options = InitTargetOptionsFromCodeGenFlags();
 
-  thinLTOResolveWeakForLinkerInIndex(
-      CombinedIndex, isPrevailing,
-      [](StringRef ModuleIdentifier, GlobalValue::GUID GUID,
-         GlobalValue::LinkageTypes NewLinkage) {});
-
-  // Use global summary-based analysis to identify symbols that can be
-  // internalized (because they aren't exported or preserved as per callback).
-  // Changes are made in the index, consumed in the ThinLTO backends.
-  thinLTOInternalizeAndPromoteInIndex(CombinedIndex, isExported);
-
-  if (options::thinlto_emit_imports_files && !options::thinlto_index_only)
-    message(LDPL_WARNING,
-            "thinlto-emit-imports-files ignored unless thinlto-index-only");
+  // Disable the new X86 relax relocations since gold might not support them.
+  // FIXME: Check the gold version or add a new option to enable them.
+  Conf.Options.RelaxELFRelocations = false;
 
+  Conf.MAttrs = MAttrs;
+  Conf.RelocModel = *RelocationModel;
+  Conf.CGOptLevel = getCGOptLevel();
+  Conf.DisableVerify = options::DisableVerify;
+  Conf.OptLevel = options::OptLevel;
+  if (options::Parallelism) {
+    if (options::thinlto)
+      Backend = createInProcessThinBackend(options::Parallelism);
+    else
+      ParallelCodeGenParallelismLevel = options::Parallelism;
+  }
   if (options::thinlto_index_only) {
-    // If the thinlto-prefix-replace option was specified, parse it and
-    // extract the old and new prefixes.
     std::string OldPrefix, NewPrefix;
     getThinLTOOldAndNewPrefix(OldPrefix, NewPrefix);
-
-    // If the user requested a list of objects gold included in the link,
-    // create and open the requested file.
-    raw_fd_ostream *ObjFileOS = nullptr;
-    if (!options::thinlto_linked_objects_file.empty()) {
-      std::error_code EC;
-      ObjFileOS = new raw_fd_ostream(options::thinlto_linked_objects_file, EC,
-                                     sys::fs::OpenFlags::F_None);
-      if (EC)
-        message(LDPL_FATAL, "Unable to open %s for writing: %s",
-                options::thinlto_linked_objects_file.c_str(),
-                EC.message().c_str());
-    }
-    // For each input bitcode file, generate an individual index that
-    // contains summaries only for its own global values, and for any that
-    // should be imported.
-    for (claimed_file &F : Modules) {
-      std::error_code EC;
+    Backend = createWriteIndexesThinBackend(
+        OldPrefix, NewPrefix, options::thinlto_emit_imports_files,
+        options::thinlto_linked_objects_file);
 
-      std::string NewModulePath =
-          getThinLTOOutputFile(F.name, OldPrefix, NewPrefix);
+  Conf.OverrideTriple = options::triple;
+  Conf.DefaultTriple = sys::getDefaultTargetTriple();
 
-      if (!options::thinlto_linked_objects_file.empty()) {
-        // If gold included any symbols from ths file in the link, emit path
-        // to the final object file, which should be included in the final
-        // native link.
-        if (get_symbols(F.handle, F.syms.size(), F.syms.data()) !=
-            LDPS_NO_SYMS) {
-          assert(ObjFileOS);
-          *ObjFileOS << NewModulePath << "\n";
-        }
-      }
+  Conf.DiagHandler = diagnosticHandler;
 
-      raw_fd_ostream OS((Twine(NewModulePath) + ".thinlto.bc").str(), EC,
-                        sys::fs::OpenFlags::F_None);
-      if (EC)
-        message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
-                NewModulePath.c_str(), EC.message().c_str());
-      // Build a map of module to the GUIDs and summary objects that should
-      // be written to its index.
-      std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
-      gatherImportedSummariesForModule(F.name, ModuleToDefinedGVSummaries,
-                                       ImportLists, ModuleToSummariesForIndex);
-      WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
-
-      if (options::thinlto_emit_imports_files) {
-        if ((EC = EmitImportsFiles(F.name,
-                                   (Twine(NewModulePath) + ".imports").str(),
-                                   ImportLists)))
-          message(LDPL_FATAL, "Unable to open %s.imports",
-                  NewModulePath.c_str(), EC.message().c_str());
-      }
-    }
+  Conf.PreOptModuleHook = [](size_t Task, Module &M) {
+    if (Task == 0)
+      addCommons(M);
+    return true;
+  };
 
-    if (ObjFileOS)
-      ObjFileOS->close();
+  switch (options::TheOutputType) {
+  case options::OT_NORMAL:
+    break;
 
-    cleanup_hook();
-    exit(0);
-  }
+  case options::OT_DISABLE:
+    Conf.PreOptModuleHook = [](size_t Task, Module &M) { return false; };
+    break;
 
-  // Create OS in nested scope so that it will be closed on destruction.
-  {
+  case options::OT_BC_ONLY:
+    Conf.PostInternalizeModuleHook = [](size_t Task, Module &M) {
       std::error_code EC;
-    raw_fd_ostream OS(output_name + ".thinlto.bc", EC,
-                      sys::fs::OpenFlags::F_None);
+      raw_fd_ostream OS(output_name, EC, sys::fs::OpenFlags::F_None);
       if (EC)
-      message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
-              output_name.data(), EC.message().c_str());
-    WriteIndexToFile(CombinedIndex, OS);
+        message(LDPL_FATAL, "Failed to write the output file.");
+      WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
+      return false;
+    };
+    break;
+
+  case options::OT_SAVE_TEMPS:
+    check(Conf.addSaveTemps(output_name, /* UseInputModulePath */ true));
+    break;
   }
 
-  thinLTOBackends(ApiFile, CombinedIndex, ModuleMap, ImportLists,
-                  ModuleToDefinedGVSummaries);
-  return LDPS_OK;
+  return llvm::make_unique<LTO>(std::move(Conf), Backend,
+                                ParallelCodeGenParallelismLevel);
 }
 
 /// gold informs us that all symbols have been read. At this point, we use
 /// get_symbols to see if any of our definitions have been overridden by a
 /// native object file. Then, perform optimization and codegen.
-static ld_plugin_status allSymbolsReadHook(raw_fd_ostream *ApiFile) {
+static ld_plugin_status allSymbolsReadHook() {
   if (Modules.empty())
     return LDPS_OK;
 
   if (unsigned NumOpts = options::extra.size())
     cl::ParseCommandLineOptions(NumOpts, &options::extra[0]);
 
-  if (options::thinlto)
-    return thinLTOLink(ApiFile);
-
-  LLVMContext Context;
-  Context.setDiscardValueNames(options::TheOutputType !=
-                               options::OT_SAVE_TEMPS);
-  Context.enableDebugTypeODRUniquing(); // Merge debug info types.
-  Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
+  std::unique_ptr<LTO> Lto = createLTO();
 
-  std::unique_ptr<Module> Combined(new Module("ld-temp.o", Context));
-  IRMover L(*Combined);
-
-  StringSet<> Internalize;
   for (claimed_file &F : Modules) {
-    // RAII object to manage the file opening and releasing interfaces with
-    // gold.
     PluginInputFile InputFile(F.handle);
     const void *View = getSymbolsAndView(F);
     if (!View)
       continue;
-    linkInModule(Context, L, F, View, F.name, ApiFile, Internalize);
+    addModule(*Lto, F, View);
   }
 
-  for (const auto &Name : Internalize) {
-    GlobalValue *GV = Combined->getNamedValue(Name.first());
-    if (GV)
-      internalize(*GV);
-  }
+  SmallString<128> Filename;
+  // Note that openOutputFile will append a unique ID for each task
+  if (!options::obj_path.empty())
+    Filename = options::obj_path;
+  else if (options::TheOutputType == options::OT_SAVE_TEMPS)
+    Filename = output_name + ".o";
+  bool SaveTemps = !Filename.empty();
 
-  if (options::TheOutputType == options::OT_DISABLE)
-    return LDPS_OK;
+  MaxTasks = Lto->getMaxTasks();
+  std::vector<uintptr_t> IsTemporary(MaxTasks);
+  std::vector<SmallString<128>> Filenames(MaxTasks);
+
+  auto AddStream = [&](size_t Task) {
+    int FD = openOutputFile(Filename, /*TempOutFile=*/!SaveTemps,
+                            Filenames[Task], MaxTasks > 1 ? Task : -1);
+    IsTemporary[Task] = !SaveTemps;
 
-  if (options::TheOutputType != options::OT_NORMAL) {
-    std::string path;
-    if (options::TheOutputType == options::OT_BC_ONLY)
-      path = output_name;
-    else
-      path = output_name + ".bc";
-    saveBCFile(path, *Combined);
-    if (options::TheOutputType == options::OT_BC_ONLY)
-      return LDPS_OK;
+    return llvm::make_unique<llvm::raw_fd_ostream>(FD, true);
+  };
+
+  check(Lto->run(AddStream));
+
+  if (options::TheOutputType == options::OT_DISABLE ||
+      options::TheOutputType == options::OT_BC_ONLY)
+
+  if (options::thinlto_index_only) {
+    cleanup_hook();
+    exit(0);
   }
 
-  CodeGen codeGen(std::move(Combined));
-  codeGen.runAll();
+  for (unsigned I = 0; I != MaxTasks; ++I)
+    if (!Filenames[I].empty())
+      recordFile(Filenames[I].str(), IsTemporary[I]);
 
   if (!options::extra_library_path.empty() &&
       set_extra_library_path(options::extra_library_path.c_str()) != LDPS_OK)
     message(LDPL_FATAL, "Unable to set the extra library path.");
 
   return LDPS_OK;
 }
 
 static ld_plugin_status all_symbols_read_hook(void) {
-  ld_plugin_status Ret;
-  if (!options::generate_api_file) {
-    Ret = allSymbolsReadHook(nullptr);
-  } else {
-    std::error_code EC;
-    raw_fd_ostream ApiFile("apifile.txt", EC, sys::fs::F_None);
-    if (EC)
-      message(LDPL_FATAL, "Unable to open apifile.txt for writing: %s",
-              EC.message().c_str());
-    Ret = allSymbolsReadHook(&ApiFile);
-  }
-
+  ld_plugin_status Ret = allSymbolsReadHook();
   llvm_shutdown();
 
   if (options::TheOutputType == options::OT_BC_ONLY ||
       options::TheOutputType == options::OT_DISABLE) {
     if (options::TheOutputType == options::OT_DISABLE) {
       // Remove the output file here since ld.bfd creates the output file
       // early.
       std::error_code EC = sys::fs::remove(output_name);

llvm/trunk/tools/llvm-lto2/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  ${LLVM_TARGETS_TO_BUILD}
+  LTO
+  Object
+  Support
+  )
+
+add_llvm_tool(llvm-lto2
+  llvm-lto2.cpp
+  )

llvm/trunk/tools/llvm-lto2/LLVMBuild.txt

+;===- ./tools/llvm-lto2/LLVMBuild.txt --------------------------*- Conf -*--===;
+;
+;                     The LLVM Compiler Infrastructure
+;
+; This file is distributed under the University of Illinois Open Source
+; License. See LICENSE.TXT for details.
+;
+;===------------------------------------------------------------------------===;
+;
+; This is an LLVMBuild description file for the components in this subdirectory.
+;
+; For more information on the LLVMBuild system, please see:
+;
+;   http://llvm.org/docs/LLVMBuild.html
+;
+;===------------------------------------------------------------------------===;
+
+[component_0]
+type = Tool
+name = llvm-lto2
+parent = Tools
+required_libraries = LTO Object all-targets

llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp

+//===-- llvm-lto2: test harness for the resolution-based LTO interface ----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This program takes in a list of bitcode files, links them and performs
+// link-time optimization according to the provided symbol resolutions using the
+// resolution-based LTO interface, and outputs one or more object files.
+//
+// This program is intended to eventually replace llvm-lto which uses the legacy
+// LTO interface.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/LTO/LTO.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/TargetSelect.h"
+
+using namespace llvm;
+using namespace lto;
+using namespace object;
+
+static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
+                                            cl::desc("<input bitcode files>"));
+
+static cl::opt<std::string> OutputFilename("o", cl::Required,
+                                           cl::desc("Output filename"),
+                                           cl::value_desc("filename"));
+
+static cl::opt<bool> SaveTemps("save-temps", cl::desc("Save temporary files"));
+
+static cl::list<std::string> SymbolResolutions(
+    "r",
+    cl::desc("Specify a symbol resolution: filename,symbolname,resolution\n"
+             "where \"resolution\" is a sequence (which may be empty) of the\n"
+             "following characters:\n"
+             " p - prevailing: the linker has chosen this definition of the\n"
+             "     symbol\n"
+             " l - local: the definition of this symbol is unpreemptable at\n"
+             "     runtime and is known to be in this linkage unit\n"
+             " x - externally visible: the definition of this symbol is\n"
+             "     visible outside of the LTO unit\n"
+             "A resolution for each symbol must be specified."),
+    cl::ZeroOrMore);
+
+static void check(Error E, std::string Msg) {
+  if (!E)
+    return;
+  handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
+    errs() << "llvm-lto: " << Msg << ": " << EIB.message().c_str() << '\n';
+  });
+  exit(1);
+}
+
+template <typename T> static T check(Expected<T> E, std::string Msg) {
+  if (E)
+    return std::move(*E);
+  check(E.takeError(), Msg);
+  return T();
+}
+
+static void check(std::error_code EC, std::string Msg) {
+  check(errorCodeToError(EC), Msg);
+}
+
+template <typename T> static T check(ErrorOr<T> E, std::string Msg) {
+  if (E)
+    return std::move(*E);
+  check(E.getError(), Msg);
+  return T();
+}
+
+int main(int argc, char **argv) {
+  InitializeAllTargets();
+  InitializeAllTargetMCs();
+  InitializeAllAsmPrinters();
+  InitializeAllAsmParsers();
+
+  cl::ParseCommandLineOptions(argc, argv, "Resolution-based LTO test harness");
+
+  std::map<std::pair<std::string, std::string>, SymbolResolution>
+      CommandLineResolutions;
+  for (std::string R : SymbolResolutions) {
+    StringRef Rest = R;
+    StringRef FileName, SymbolName;
+    std::tie(FileName, Rest) = Rest.split(',');
+    if (Rest.empty()) {
+      llvm::errs() << "invalid resolution: " << R << '\n';
+      return 1;
+    }
+    std::tie(SymbolName, Rest) = Rest.split(',');
+    SymbolResolution Res;
+    for (char C : Rest) {
+      if (C == 'p')
+        Res.Prevailing = true;
+      else if (C == 'l')
+        Res.FinalDefinitionInLinkageUnit = true;
+      else if (C == 'x')
+        Res.VisibleToRegularObj = true;
+      else
+        llvm::errs() << "invalid character " << C << " in resolution: " << R
+                     << '\n';
+    }
+    CommandLineResolutions[{FileName, SymbolName}] = Res;
+  }
+
+  std::vector<std::unique_ptr<MemoryBuffer>> MBs;
+
+  Config Conf;
+  Conf.DiagHandler = [](const DiagnosticInfo &) {
+    exit(1);
+  };
+
+  if (SaveTemps)
+    check(Conf.addSaveTemps(OutputFilename), "Config::addSaveTemps failed");
+
+  LTO Lto(std::move(Conf));
+
+  bool HasErrors = false;
+  for (std::string F : InputFilenames) {
+    std::unique_ptr<MemoryBuffer> MB = check(MemoryBuffer::getFile(F), F);
+    std::unique_ptr<InputFile> Input =
+        check(InputFile::create(MB->getMemBufferRef()), F);
+
+    std::vector<SymbolResolution> Res;
+    for (const InputFile::Symbol &Sym : Input->symbols()) {
+      auto I = CommandLineResolutions.find({F, Sym.getName()});
+      if (I == CommandLineResolutions.end()) {
+        llvm::errs() << argv[0] << ": missing symbol resolution for " << F
+                     << ',' << Sym.getName() << '\n';
+        HasErrors = true;
+      } else {
+        Res.push_back(I->second);
+        CommandLineResolutions.erase(I);
+      }
+    }
+
+    if (HasErrors)
+      continue;
+
+    MBs.push_back(std::move(MB));
+    check(Lto.add(std::move(Input), Res), F);
+  }
+
+  if (!CommandLineResolutions.empty()) {
+    HasErrors = true;
+    for (auto UnusedRes : CommandLineResolutions)
+      llvm::errs() << argv[0] << ": unused symbol resolution for "
+                   << UnusedRes.first.first << ',' << UnusedRes.first.second
+                   << '\n';
+  }
+  if (HasErrors)
+    return 1;
+
+  auto AddStream = [&](size_t Task) {
+    std::string Path = OutputFilename + "." + utostr(Task);
+    std::error_code EC;
+    auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
+    check(EC, Path);
+    return S;
+  };
+
+  check(Lto.run(AddStream), "LTO::run failed");
+}