libLTO: add a ThinLTOCodeGenerator on the model of LTOCodeGenerator.

[mehdi_amini]

Summary

This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.

[mehdi_amini]

(Remove empty line change)

[tejohnson]

Did an initial pass through, some comments and questions below.

include/llvm-c/lto.h

562

Wraps a single returned object file? (Comment for this and following structure are currently identical)

585

Why "prior to" here and other places?

624

Document return type?

include/llvm/LTO/ThinLTOCodeGenerator.h

53

do you mean "If a symbol is not listed there, it will be optimized away if it is inlined into every usage"?

83

Doxygen comments on these and other members as well?

lib/LTO/ThinLTOCodeGenerator.cpp

65

No, should not be lazy. That is just for reading the combined index.

176

Why don't we want to set an opt level?

249

This case was already handled above.

298

So the linker must place any external symbols with references from other modules in this set, right? I see that you added a comment that suggest that to the thinlto_codegen_add_must_preserve_symbol interface. Might want to add something to that effect here as well.

tools/llvm-lto/llvm-lto.cpp

491

Is this temporary?

503

Why are all these commented out?

tools/lto/lto.cpp

288

Why the FIXME?

[mehdi_amini]

Thanks for the review, I'll fix all of what you found :)

include/llvm-c/lto.h

562

Yes this is a single object, I'll update the comment.

585

"prior to" is an unfortunate copy/paste, I'll remove.

624

I'll add.

include/llvm/LTO/ThinLTOCodeGenerator.h

53

Basically I expect the linker to tell me the list of symbols for which there is a reference from *outside* the ThinLTO modules.

83

Sure!

lib/LTO/ThinLTOCodeGenerator.cpp

65

OK

176

Not sure, we don't have a linker flag to do that, with current full LTO there is none either I think. Maybe passing -mllvm -O3 can have an impact? (I think the -mllvm are supposed to be "debug" flags though)

This is something we'd like to figure out but not completely sure what is the best way (the front-end could encode this information in the bitcode for example).

249

Good catch :)

298

Any symbols with reference from other *non thinlto* modules to be exact.
I'll add a comment here.

tools/llvm-lto/llvm-lto.cpp

491

Yes! Will clean up

503

Some debug code left over, I'll clean it up.

tools/lto/lto.cpp

288

Oh this API should just be removed.
(Early stage of prototyping I was expecting the linker to tell me the exported/preserved symbols *for each individual files*, but I could see any benefit from it later so I reverted to a global list).

[tejohnson]

lib/LTO/ThinLTOCodeGenerator.cpp

298

So an external symbol with a reference from another thinlto module would not be in this set and would be internalized? I realize that you are remembering the original linkage and later setting it back. But what happens in the mean time with the internalize and globalDCE passes? Couldn't they cause one of the symbols referenced from another thinlto module to be internalized and then eliminated, since you don't see those references here?

[mehdi_amini]

lib/LTO/ThinLTOCodeGenerator.cpp

298

You're right!
What has to be inside are functions that are cross-referenced as well.
I was confused because I move the set of preserved symbols from "per module" to be on the code generator (i.e. one for all), but it is the union of the individual sets (if they still existed).

[mehdi_amini]

Update following Teresa's comment

[mehdi_amini]

Cleanup!

[mehdi_amini]

All comments should be addressed!

[mehdi_amini]

More cleanup and a bugfix: cannot run globalopt too early because it breaks ThinLTO.

The issue is that globalopt (after internalize) may turning (former) globals into "unnamed address".
However this happens when optimizing the module, not when the module is used for importing in other modules. The global will be imported as available externally in these modules but won't be available in the source module because the "unamed address".

[tejohnson]

Mostly comment typo and wording suggestions

include/llvm-c/lto.h

580

"Frees the code generator..."

589

typo s/stay/stays/

599

Inconsistent capitalization. Some parameters here and in other interfaces are lower case, some here and other places start with upper case.

613

s/match/matches/

614

maybe "and in future implementations may change" or "and may change in future implementations"

651

Maybe "The intention is to make the bitcode files available for debugging after..."

include/llvm/LTO/ThinLTOCodeGenerator.h

12

s/provide/provides/

45

s/provide/provides/

55

Think this comment is clearer if the last sentence changed to "If a symbol is not listed there, it will be optimized away if it is inlined into every usage."

lib/LTO/ThinLTOCodeGenerator.cpp

150

s/let/let's/ or s/let/so/

151

This will be hard to correlate with the original file. How about using some part of the source file name saved in the module here and in the later opt.bc dump? I.e. use the base name of the path returned by getSourceFileName(), but still add in 'count' to avoid conflicts when the same file name is used at multiple paths.

209

s/let/let's/ or s/let/so/

227

s/possible/possibly/

483

Do we expect to reuse the same code generator multiple times?

[mehdi_amini]

Update with a good candidate to be integrated as a first Proof-of-concept.

I am able to link every target in clang/llvm with this implementation.

Beyong the actual implementation of the CodeGenerator, I am especially interested to
"lock-down" the libLTO API so that we can start to have linker support for it.

[tejohnson]

Some misc comments below.

include/llvm-c/lto.h

666

What is the default?

include/llvm/LTO/ThinLTOCodeGenerator.h

1

Fix file name.

46

How do PruningInterval and Expiration interact? Are they mutually exclusive, or does a module get pruned at the min of these?

99

s/controling/controlling/

lib/LTO/ThinLTOCodeGenerator.cpp

65

Why not lazy load metadata too?

84

Include Module's SourceFileName to get more meaningful dump file names?

96

This is doing more than internalization, so name should probably be adjusted to reflect that. I don't have a good name off the top of my head. Maybe split into 2 and call the first something like resolveLinkOnceAndWeak, and if that returns false then invoke the internalization handling (maybe outline keepIfPreserved and make it return a bool).

104

Should available_externally also be kept in SingleModule mode?

112

I'm confused by the comment. It is already local if private, so no need to internalize or add to Keep, which is consistent with what the code is doing. I just don't understand what the comment about being restrictive is saying.

160

Combine the LinkOnce cases and just pick the appropriate Weak linkage based on the LinkOnce linkage, the handling is identical otherwise.

180

Combine the Weak* cases, the handling is identical.

211

This is doing more than internalization. How about a more generic name such as LinkageOptimization or something like that?

271

This handling is already outlined in the other InternalizeModule() function, invoke here.

317

Does this need to be guarded by lock? Comments for LockM indicate it is just to guard optimization.

329

Why don't we want to do this when there are no preserved or cross referenced symbols? Couldn't the internalization be even more aggressive in that case? Also, wouldn't we at least want the linkonce/weak handling?

343

Needs doxygen class description. Is "EarlyOptimizedModuleMap" more accurate though?

345

When would this not be the case?

352

Out of curiosity, why a StringMap for ModuleMap and an unordered_map from string here? They are both index by the identifier string.

410

crossImportIntoModule?

592

Doesn't instantiating here mean that the map isn't leveraged across threads? Oh, I see that this routine and the above promote() are only invoked from llvm-lto. Is there a reason why llvm-lto doesn't use ThinLTOCodeGenerator::run()?

[mehdi_amini]

Adding Sergei as a reviewer as he expressed interest in the past.

lib/LTO/ThinLTOCodeGenerator.cpp

151

The problem is that with static library, the name is funky. We need to replace parentheses and other special character with something like underscore.

[ygao]

include/llvm-c/lto.h

699

This, and the two functions that ensue, may also need to say LTO_API_VERSION=18?
I think it is more natural to say "a list of global symbols" than "a list of all global symbols", but English is not my first language.

lib/LTO/ThinLTOCodeGenerator.cpp

1

This is probably copied from LTOCodeGenerator.cpp. Can you confirm that the summary line is accurate?

44

Do you need unordered_set here? I did not see unordered_set being used in this file.

[mehdi_amini]

Thanks Gao!

include/llvm-c/lto.h

699

Good catch. Will fix (both).

lib/LTO/ThinLTOCodeGenerator.cpp

44

It used in a more elaborate version of this code that I have locally, but not needed here. Will remove.

[bruno]

Hi Mehdi,

lib/LTO/ThinLTOCodeGenerator.cpp

426

Although not really necessary for this patch, it would be nice to have the flags to disable the inliner and vectorizer like we currently have in full LTO. This is specially useful when investigating differences between full and thin.

[mehdi_amini]

Thanks Teresa for the review. See answers inlined.

include/llvm-c/lto.h

666

No guarantee provided by the API :)

include/llvm/LTO/ThinLTOCodeGenerator.h

46

PruningInterval controls the interval between two check on the cache: i.e. if you set 2h the plugin will not bother checking for cache invalidation for the next two hours.
Cache Expiration is per entry: when pruning you are deleting the entries older than this value.
I documented getter/setter, not enough?

lib/LTO/ThinLTOCodeGenerator.cpp

1

It is not very helpful, but matches LTOCodeGenerator indeed.

65

Aren't they almost immediately materialized anyway? I thought that laziness would have a cost for no real benefit maybe?
(I didn't see a change on the memory profile when lazy loading or not)

84

SourceFileName can fit! Good point.

96

I remove the internalization phase from this patch for now. This will change a lot with the graph in the summary.

104

Good point. Dunno.

112

This is copy/pasted from LTOCodeGenerator::applyRestriction. I read it as "don't need to do anything thing you won't internalize any more something already private"

160

OK, I'll keep this for a future version of the patch, I removed the internalize for now.

317

You need to block "reader" of OptimizedBuffer in case there is already a writer populating it.
(in case of contention on the lock with too many reader there would be a possible mitigation with atomic flags for a fast-path, but not worth it here I think)

329

No preserved or cross referenced symbols means that you will internalize *everything* and then global DCE will remove *everything*. This is of little interest. So I just considered that empty() means the linker didn't provide any information at all. It helps implementing testing with llvm-lto as well.

345

The buffers identifier are supplied by the linker and can be anything:

extern void thinlto_codegen_add_module(thinlto_code_gen_t cg,
                                       const char *identifier, const char *data,
                                       int length);

352

Here there is no good reason I think. I think that StringMap will allocate the entry with the key, so you want a value that is quite small (for rehashing / growing the map).

426

Good point, will update to try using a set of flags as close as possible the LTOCodeGenerator

592

The reason llvm-lto does not use run() is to test steps in isolation.

[mehdi_amini]

Update: taking into account most comments, and removing the internalize phase as it will be very different with the summary-based importing coming next.

[tejohnson]

include/llvm-c/lto.h

668

Is the max cache size set via the space that was available at the start of the compilation, or is the threshold updated so that if something else comes along and eats up some disk space the allowable max cache size is adjusted downward?

include/llvm/LTO/ThinLTOCodeGenerator.h

48

It just wasn't clear how they interact, although your explanation was what I guessed. Maybe for the thinlto_codegen_set_cache_entry_expiration interface add that the module may be removed earlier due to the pruning interval.

lib/LTO/LTOModule.cpp

80

Also for future consider caching value unless it is expected to be called only once per module.

lib/LTO/ThinLTOCodeGenerator.cpp

67

The when lazy metadata linking is enabled, metadata parsing should be postponed until we actually go to do the import and linkInModule during the bulk importing. Since the source module is destroyed at the end of linkInModule, this should reduce the max amount of live metadata to one module at a time, rather than all modules we are importing from. I'm surprised it didn't reduce the max memory usage in a debug build.

If we ever go back to post-pass metadata linking in the function importer (i.e. if we decide to do iterative importing rather than a single bulk import from each module), this will become more critical. However, if we move to summary-only importing decisions it will obviate the need to do this.

Until we go to summary only import decisions, I would think it should reduce the max memory usage as per my first paragraph. Did you see a cost going with lazy metadata loading?

86

Is this a TODO? If so, please add TODO comment.

110

Add doxygyen comment describing ModuleMap.

114

Ok. When you put this support back in your description above ("don't need to do anything thing you won't internalize any more something already private") is better than the comment about restriction.

331

Ok, but I think you still want the linkonce/weak linkage changes? When you put this back this is more reason to split the internalization and linkonce/weak linkage changes into two routines and only invoke the latter if the sets are empty.

359

What is the difference between this ModuleBuffer and the one loaded out of the ModuleMap in the below call to loadModuleFromBuffer? If they are the same, can loadModuleFromBuffer be called a single time and the resulting module optionally saved after?

379

Unfortunately this means that ThinLTOCodeGenerator::run() is currently untested. Consider adding a mode to llvm-lto that does all thinlto steps (thin-action=all?) to test all the steps via this interface.

tools/llvm-lto/llvm-lto.cpp

82

Actual option below is "functionindex". But per the ref graph patch, this is broader than a function index, so I am looking at changing references to function index to something else anyway. So I would suggest changing the actual option below to "thinlto-index".

361

s/mentionned/mentioned/ (here and below for import()).

373

Adding all the modules isn't needed for promotion. Should be able to remove this loop.

398

There is a lot of code duplication between these various functions, consider refactoring possibly in a follow-on patch.

414

Why is the import() step a superset of promote and import, whereas the other steps (e.g. optimize()) only doing one thing?

428

Untested.

484

What if there is more than one occurrence?

[mehdi_amini]

include/llvm-c/lto.h

668

Mmmm, this is yet to be implemented.
I'd say it applies to what we store on disk when the linker is *not running*. I.e. during the link you are allowed to use extra space without any limit.

include/llvm/LTO/ThinLTOCodeGenerator.h

48

Now I'm unsure it was clear enough, because you wrote " the module may be removed earlier due to the pruning interval". A cache entry *can't* be remove earlier. The pruning interval means that will only *check* the entries.

lib/LTO/ThinLTOCodeGenerator.cpp

86

Will do.

110

Will do.

331

Good point.

(I expect this to come back with a very different summary-based implementation)

359

Good point, this is legacy from when there was all the internalization stage, will cleanup!

379

OK.

tools/llvm-lto/llvm-lto.cpp

373

It isn't needed... for now ;)
I wrote this code against my prototype of summary-based importing where the promotion is driven by what the other modules export.
(I'll remove for now).

398

OK

414

Same reason as above: when I wrote it against the pure summary-based importing, it was needed/helpful. I'll remove for now.

428

Any suggestion on how to test that? This is basically like testing opt -O3?

484

I'll add a check and error.

[tejohnson]

include/llvm-c/lto.h

668

I don't really understand what you mean by this, particularly the last sentence about using extra space during the link without any limit. Isn't this set/used during the link (which is running the ThinLTO steps in process)?

include/llvm/LTO/ThinLTOCodeGenerator.h

48

Ah, so a clarifying comment would help then since I misunderstood. It sounds like the modules will be checked for expiration (and pruned from the cache) at the pruning interval. So with the default pruning interval of -1, the expiration is unused? I took these to be separate, complimentary, mechanisms for keeping the cache size down. Another option beyond just documenting the interactions well is to combine these parameters into a single interface that takes both the pruning interval and the expiration.

tools/llvm-lto/llvm-lto.cpp

428

Not sure, maybe just invoke this stage in your test after the importing action to make sure it succeeds (without necessarily checking for any specific optimization)?

[slarin]

Sorry for the late entry... Some of my questions could have been already answered.

include/llvm-c/lto.h

736

Does this handles commons properly?

include/llvm/LTO/ThinLTOCodeGenerator.h

149

Can we theoretically have a mixture of opt levels? -O2/-Os? Should I be able to respect per library opt level?

lib/LTO/ThinLTOCodeGenerator.cpp

68

I do not know if it is relevant at this point, but for what it worth - my IR might have metadata that changes opt/codegen.

82

Default to /tmp?

141

Yes! ...and default should probably be O2...

145

So I assume I should be able to control all of those...

185

Sorry, I miss something - why is this unconditional?

364

In general - don't you want to verify modules as they progress through the stages? I do it in regular LTO and it did help on more than one occasion :)

379

+1 on Teresa's point about llvm-lto.

tools/llvm-lto/llvm-lto.cpp

442

Verify TheModule?

[mehdi_amini]

Thanks for all the comments!
(Please see inline for the discussion)

include/llvm-c/lto.h

736

Can you be more specific? I'm not sure what is specific with common on this aspect?

include/llvm/LTO/ThinLTOCodeGenerator.h

149

Ideally I think we'd want this parameter to be recorded in the bitcode itself, what do you think?
Right now we don't even expose any linker flag for that.

Here it is about the *codegen* opt level though, it won't impact the optimizer.

lib/LTO/ThinLTOCodeGenerator.cpp

68

slarin: how does it play with cross-module importing? If a function is defined in a module compiled with O3 but imported in a module compiled with O2?

82

The way the client is enabling dumping temporaries is by providing a path.
Having a default would mean either always dumping temporaries (we don't want that...) or having a more complicated way of enabling dumping.

145

Yes, but we don't have an interface for these as of today. A serialization of the PMB options in the bitcode would help.

185

As mentioned above, this is conditional in the saveTempBitcode function itself, which starts with

if (SaveTempsDir.empty()) return;

makes sense?

364

We'll always verify the module once at the beginning of the optimizer pipeline, but I guess we could do it more frequently in assert builds.

tools/llvm-lto/llvm-lto.cpp

82

Will do.

442

This is done in optimized()

(see line 143 in ThinLTOCodeGenerator.cpp PMB.VerifyInput = true;)

[slarin]

include/llvm-c/lto.h

736

I probably not fully understand how this list should work... but this is not the proper place to figure it out - please ignore this comment for now.

include/llvm/LTO/ThinLTOCodeGenerator.h

149

Ideally I think we'd want this parameter to be recorded in the bitcode itself, what do you think?

Yes. Certainly.

it is about the *codegen*

...yes. My target has codegen properties that are exposed to a user, which might produce drastically different results if not set properly, but once again, ideally it should come from bitcode.

lib/LTO/ThinLTOCodeGenerator.cpp

68

If I know settings for each module I can handle these situations in platform specific way. Besides rough optimization levels I have different addressing modes used in different modules, and I might chose not to mix certain features at all... At this point my main concern seems to be revolving around general LTO issues (like mixing different optimization scopes into one) and might not be "thin"-lto specific. We had to jump through hoops for regular LTO, and I see very similar set of issues being designed in here as well...

82

I see...

185

Totally

[mehdi_amini]

Taking comment into account

[mehdi_amini]

Thanks for the great review. Hopefully I didn't forget anything with this update.

include/llvm/LTO/ThinLTOCodeGenerator.h

48

Tried to make the doc very explicit, let me know what you think.

lib/LTO/LTOModule.cpp

80

It is supposed to be called once indeed. I think we can update the implementation in the future if needed.

lib/LTO/ThinLTOCodeGenerator.cpp

67

So just tested:

With lazy loading of metadata: getLazyBitcodeModule takes 237ms and materializeMetadata takes 74ms (total 314ms)
without lazy loading of metadata: getLazyBitcodeModule takes 316ms

So no perf diff.

I probably don't see any diff on the memory because most metadata will leak to the context and being free'd with the Module. So there is no real impact on the peak memory (just delaying a little bit).

379

Done, it was valuable :)

[tejohnson]

Thanks for adding the description! I do think that the formula for the new cache size is not right, see suggested fix below.

A couple misc comment typos, but LGTM once these the above is addressed.

include/llvm-c/lto.h

687

s/term/terms/

690

"left over half the available space"? (i.e. add "half") Either that or "left over the free space" (since AvailableSpace = FreeSpace + CacheSize)?

694

This doesn't seem right. I think P should be divided by 100 and not multiplied by it, since it is a percentage. And I think the percentage needs to be multiplied by something, not divided by AvailableSpace?

Should this be:
NewCacheSize = AvailableSpace * P/100

since it is described as the percentage of available space used for the cache.

For this and the above suggestion, any change needs to be replicated below in ThinLTOCodeGenerator.h.

include/llvm/LTO/ThinLTOCodeGenerator.h

105

move 'an' to next line.

lib/LTO/ThinLTOCodeGenerator.cpp

67

Ok, thanks for checking!

[mehdi_amini]

r262977

include/llvm-c/lto.h

 
 /**
  * @defgroup LLVMCLTO LTO
  * @ingroup LLVMC
  *
  * @{
  */
 
-#define LTO_API_VERSION 17
-
+#define LTO_API_VERSION 18
 /**
  * \since prior to LTO_API_VERSION=3
  */
 typedef enum {
     LTO_SYMBOL_ALIGNMENT_MASK              = 0x0000001F, /* log2 of alignment */
     LTO_SYMBOL_PERMISSIONS_MASK            = 0x000000E0,
     LTO_SYMBOL_PERMISSIONS_CODE            = 0x000000A0,
     LTO_SYMBOL_PERMISSIONS_DATA            = 0x000000C0,

 } lto_codegen_model;
 
 /** opaque reference to a loaded object module */
 typedef struct LLVMOpaqueLTOModule *lto_module_t;
 
 /** opaque reference to a code generator */
 typedef struct LLVMOpaqueLTOCodeGenerator *lto_code_gen_t;
 
+/** opaque reference to a thin code generator */
+typedef struct LLVMOpaqueThinLTOCodeGenerator *thinlto_code_gen_t;
+
 #ifdef __cplusplus
 extern "C" {
 #endif
 
 /**
  * Returns a printable string.
  *
  * \since prior to LTO_API_VERSION=3

  * output bitcode.  This should be turned on for all -save-temps output.
  *
  * \since LTO_API_VERSION=15
  */
 extern void
 lto_codegen_set_should_embed_uselists(lto_code_gen_t cg,
                                       lto_bool_t ShouldEmbedUselists);
 
+/**
+ * @}
+ * @defgroup LLVMCLTO ThinLTO
+ * @ingroup LLVMC
+ *
+ * @{
+ */
+
+/**
+ * Type to wrap a single object returned by ThinLTO.
+ *
+ * \since LTO_API_VERSION=18
+ */
+typedef struct {
+  void *Buffer;
+  size_t Size;
+} LTOObjectBuffer;
+
+/**
+ * Instantiates a ThinLTO code generator.
+ * Returns NULL on error (check lto_get_error_message() for details).
+ *
+ *
+ * The ThinLTOCodeGenerator is not intended to be reuse for multiple
+ * compilation: the model is that the client adds modules to the generator and
+ * ask to perform the ThinLTO optimizations / codegen, and finally destroys the
+ * codegenerator.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern thinlto_code_gen_t thinlto_create_codegen();
+
+/**
+ * Frees the generator and all memory it internally allocated.
+ * Upon return the thinlto_code_gen_t is no longer valid.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_dispose(thinlto_code_gen_t cg);
+
+/**
+ * Add a module to a ThinLTO code generator. Identifier has to be unique among
+ * all the modules in a code generator. The data buffer stays owned by the
+ * client, and is expected to be available for the entire lifetime of the
+ * thinlto_code_gen_t it is added to.
+ *
+ * On failure, returns NULL (check lto_get_error_message() for details).
+ *
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_add_module(thinlto_code_gen_t cg,
+                                       const char *identifier, const char *data,
+                                       int length);
+
+/**
+ * Optimize and codegen all the modules added to the codegenerator using
+ * ThinLTO. Resulting objects are accessible using thinlto_module_get_object().
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_process(thinlto_code_gen_t cg);
+
+/**
+ * Returns the number of object files produced by the ThinLTO CodeGenerator.
+ *
+ * It usually matches the number of input files, but this is not a guarantee of
+ * the API and may change in future implementation, so the client should not
+ * assume it.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern unsigned int thinlto_module_get_num_objects(thinlto_code_gen_t cg);
+
+/**
+ * Returns a reference to the ith object file produced by the ThinLTO
+ * CodeGenerator.
+ *
+ * Client should use \p thinlto_module_get_num_objects() to get the number of
+ * available objects.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern LTOObjectBuffer thinlto_module_get_object(thinlto_code_gen_t cg,
+                                                 unsigned int index);
+
+/**
+ * Sets which PIC code model to generate.
+ * Returns true on error (check lto_get_error_message() for details).
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern lto_bool_t thinlto_codegen_set_pic_model(thinlto_code_gen_t cg,
+                                                lto_codegen_model);
+
+/**
+ * Sets the path to a directory to use as a cache storage for incremental build.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_set_cache_dir(thinlto_code_gen_t cg,
+                                          const char *cache_dir);
+
+/**
+ * Sets the cache pruning interval (in seconds). A negative value disable the
+ * pruning (default).
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_set_cache_pruning_interval(thinlto_code_gen_t cg,
+                                                       int interval);
+
+/**
+ * Sets the expiration (in seconds) for an entry in the cache.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_set_cache_entry_expiration(thinlto_code_gen_t cg,
+                                                       unsigned expiration);
+
+/**
+ * Sets the path to a directory to use as a storage for temporary bitcode files.
+ * The intention is to make the bitcode files available for debugging at various
+ * stage of the pipeline.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_set_savetemps_dir(thinlto_code_gen_t cg,
+                                              const char *save_temps_dir);
+
+/**
+ * Sets the cpu to generate code for.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_codegen_set_cpu(thinlto_code_gen_t cg, const char *cpu);
+
+/**
+ * Parse -mllvm style debug options.
+ *
+ * \since LTO_API_VERSION=18
+ */
+extern void thinlto_debug_options(const char *const *options, int number);
+
+/**
+ * Test if a module has support for ThinLTO linking.
+ */
+extern bool lto_module_is_thinlto(lto_module_t mod);
+
+/**
+ * Adds to a list of all global symbols that must exist in the final generated
+ * code. If a function is not listed there, it might be inlined into every usage
+ * and optimized away. For every single module, the functions referenced
+ * from code outside of the ThinLTO modules need to be added here.
+ */
+extern void thinlto_codegen_add_must_preserve_symbol(thinlto_code_gen_t cg,
+                                                     const char *name,
+                                                     int length);
+/**
+ * Adds to a list of all global symbols that are cross-referenced between
+ * ThinLTO files. If the ThinLTO CodeGenerator can ensure that every
+ * references from a ThinLTO module to this symbol is optimized away, then
+ * the symbol can be discarded.
+ */
+extern void thinlto_codegen_add_cross_referenced_symbol(thinlto_code_gen_t cg,
+                                                        const char *name,
+                                                        int length);
+
 #ifdef __cplusplus
 }
 #endif
 
 /**
  * @}
  */
 
 #endif

include/llvm/ADT/STLExtras.h

 
 /// Provide wrappers to std::find which take ranges instead of having to pass
 /// begin/end explicitly.
 template<typename R, class T>
 auto find(R &&Range, const T &val) -> decltype(Range.begin()) {
   return std::find(Range.begin(), Range.end(), val);
 }
 
+/// Provide wrappers to std::find_if which take ranges instead of having to pass
+/// begin/end explicitly.
+template <typename R, class T>
+auto find_if(R &&Range, const T &Pred) -> decltype(Range.begin()) {
+  return std::find_if(Range.begin(), Range.end(), Pred);
+}
+
 //===----------------------------------------------------------------------===//
 //     Extra additions to <memory>
 //===----------------------------------------------------------------------===//
 
 // Implement make_unique according to N3656.
 
 /// \brief Constructs a `new T()` with the given args and returns a
 ///        `unique_ptr<T>` which owns the object.

include/llvm/LTO/LTOModule.h

 
 public:
   ~LTOModule();
 
   /// Returns 'true' if the file or memory contents is LLVM bitcode.
   static bool isBitcodeFile(const void *mem, size_t length);
   static bool isBitcodeFile(const char *path);
 
+  /// Returns 'true' if the Module is produced for ThinLTO.
+  bool isThinLTO();
+
   /// Returns 'true' if the memory buffer is LLVM bitcode for the specified
   /// triple.
   static bool isBitcodeForTarget(MemoryBuffer *memBuffer,
                                  StringRef triplePrefix);
 
   /// Returns a string representing the producer identification stored in the
   /// bitcode, or "" if the bitcode does not contains any.
   ///

include/llvm/LTO/ThinLTOCodeGenerator.h

+//===-LTOCodeGenerator.h - LLVM Link Time Optimizer -----------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file declares the ThinLTOCodeGenerator class, similar to the
+// LTOCodeGenerator but for the ThinLTO scheme. It provides an interface for
+// linker plugin.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LTO_THINLTOCODEGENERATOR_H
+#define LLVM_LTO_THINLTOCODEGENERATOR_H
+
+#include "llvm-c/lto.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
+#include "llvm/ADT/Triple.h"
+#include "llvm/Support/CodeGen.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Target/TargetOptions.h"
+
+#include <string>
+
+namespace llvm {
+class FunctionInfoIndex;
+class LLVMContext;
+class TargetMachine;
+
+/// Helper to gather options relevant to the target machine creation
+struct TargetMachineBuilder {
+  Triple TheTriple;
+  std::string MCpu;
+  std::string MAttr;
+  TargetOptions Options;
+  Reloc::Model RelocModel = Reloc::Default;
+  CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
+
+  std::unique_ptr<TargetMachine> create() const;
+};
+
+/// Cache behavior controls.
+struct CachingOptions {
+  std::string Path;
+  int PruningInterval = -1; // seconds, -1 to disable pruning
+  unsigned int Expiration;  // seconds
+};
+
+/// This class define an interface similar to the LTOCodeGenerator, but adapted
+/// for ThinLTO processing.
+/// The ThinLTOCodeGenerator is not intended to be reuse for multiple
+/// compilation: the model is that the client adds modules to the generator and
+/// ask to perform the ThinLTO optimizations / codegen, and finally destroys the
+/// codegenerator.
+class ThinLTOCodeGenerator {
+public:
+  /// Add given module to the code generator.
+  void addModule(StringRef Identifier, StringRef Data);
+
+  /**
+   * Adds to a list of all global symbols that must exist in the final generated
+   * code. If a symbol is not listed there, it will be optimized away if it is
+   * inlined into every usage.
+   */
+  void preserveSymbol(StringRef Name);
+
+  /**
+   * Adds to a list of all global symbols that are cross-referenced between
+   * ThinLTO files. If the ThinLTO CodeGenerator can ensure that every
+   * references from a ThinLTO module to this symbol is optimized away, then
+   * the symbol can be discarded.
+   */
+  void crossReferenceSymbol(StringRef Name);
+
+  /**
+   * Process all the modules that were added to the code generator in parallel.
+   *
+   * Client can access the resulting object files using getProducedBinaries()
+   */
+  void run();
+
+  /**
+   * Return the "in memory" binaries produced by the code generator.
+   */
+  std::vector<std::unique_ptr<MemoryBuffer>> &getProducedBinaries() {
+    return ProducedBinaries;
+  }
+
+  /**
+   * \defgroup Options setters
+   * @{
+   */
+
+  /**
+   * \defgroup Cache controling options
+   * @{
+   */
+
+  /// Provide a path to a directory where to store the cached files for
+  /// incremental build
+  void setCacheDir(std::string Path) { CacheOptions.Path = std::move(Path); }
+
+  /// Cache policy: interval (seconds) between two prune of the cache. Set to a
+  /// negative value (default) to disable pruning.
+  void setCachePruningInterval(int Interval) {
+    CacheOptions.PruningInterval = Interval;
+  }
+
+  /// Cache policy: expiration (in seconds) for an entry.
+  void setCacheEntryExpiration(unsigned Expiration) {
+    CacheOptions.Expiration = Expiration;
+  }
+
+  /**@}*/
+
+  /// Set the path to a directory where to save temporaries at various stages of
+  /// the processing.
+  void setSaveTempsDir(std::string Path) { SaveTempsDir = std::move(Path); }
+
+  /// CPU to use to initialize the TargetMachine
+  void setCpu(std::string Cpu) { TMBuilder.MCpu = std::move(Cpu); }
+
+  /// Subtarget attributes
+  void setAttr(std::string MAttr) { TMBuilder.MAttr = std::move(MAttr); }
+
+  /// TargetMachine options
+  void setTargetOptions(TargetOptions Options) {
+    TMBuilder.Options = std::move(Options);
+  }
+
+  /// CodeModel
+  void setCodePICModel(Reloc::Model Model) { TMBuilder.RelocModel = Model; }
+
+  /// CodeGen optimization level
+  void setCodeGenOptLevel(CodeGenOpt::Level CGOptLevel) {
+    TMBuilder.CGOptLevel = CGOptLevel;
+  }
+
+  /**@}*/
+
+  /**
+   * \defgroup Set of APIs to run individual stages in isolation.
+   * @{
+   */
+
+  /**
+   * Produce the combined function index from all the bitcode files:
+   * "thin-link".
+   */
+  std::unique_ptr<FunctionInfoIndex> linkCombinedIndex();
+
+  /**
+   * Perform promotion and renaming of exported internal functions.
+   */
+  void promote(Module &Module, FunctionInfoIndex &Index);
+
+  /**
+   * Perform cross-module importing for the module identified by
+   * ModuleIdentifier.
+   */
+  void crossModuleImport(Module &Module, FunctionInfoIndex &Index);
+
+  /**
+   * Perform post-importing ThinLTO optimizations.
+   */
+  void optimize(Module &Module);
+
+  /**
+   * Perform ThinLTO CodeGen.
+   */
+  std::unique_ptr<MemoryBuffer> codegen(Module &Module);
+
+  /**@}*/
+
+private:
+  /// Helper factory to build a TargetMachine
+  TargetMachineBuilder TMBuilder;
+
+  /// Vector holding the in-memory buffer containing the produced binaries.
+  std::vector<std::unique_ptr<MemoryBuffer>> ProducedBinaries;
+
+  /// Vector holding the input buffers containing the bitcode modules to
+  /// process.
+  std::vector<MemoryBufferRef> Modules;
+
+  /// Set of symbols that need to be preserved outside of the set of bitcode
+  /// files.
+  StringSet<> PreservedSymbols;
+
+  /// Set of symbols that are cross-referenced between bitcode files.
+  StringSet<> CrossReferencedSymbols;
+
+  /// Control the caching behavior.
+  CachingOptions CacheOptions;
+
+  /// Path to a directory to save the temporary bitcode files.
+  std::string SaveTempsDir;
+};
+}
+#endif

lib/LTO/CMakeLists.txt

 add_llvm_library(LLVMLTO
   LTOModule.cpp
   LTOCodeGenerator.cpp
+  ThinLTOCodeGenerator.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/LTO
-  )
 
 add_dependencies(LLVMLTO intrinsics_gen)

lib/LTO/LLVMBuild.txt

  InstCombine
  Linker
  MC
  ObjCARC
  Object
  Scalar
  Support
  Target
+ TransformUtils
 No newline at end of file

lib/LTO/LTOModule.cpp

bool LTOModule::isBitcodeFile(const char *Path) {
   if (!BufferOrErr)
     return false;
 
   ErrorOr<MemoryBufferRef> BCData = IRObjectFile::findBitcodeInMemBuffer(
       BufferOrErr.get()->getMemBufferRef());
   return bool(BCData);
 }
 
+bool LTOModule::isThinLTO() {
+  // Right now the detection is only based on the summary presence. We may want
+  // to add a dedicated flag at some point.
+  return hasFunctionSummary(IRFile->getMemoryBufferRef(),
+                            [](const DiagnosticInfo &DI) {
+                              DiagnosticPrinterRawOStream DP(errs());
+                              DI.print(DP);
+                              errs() << '\n';
+                              return;
+                            });
+}
+
 bool LTOModule::isBitcodeForTarget(MemoryBuffer *Buffer,
                                    StringRef TriplePrefix) {
   ErrorOr<MemoryBufferRef> BCOrErr =
       IRObjectFile::findBitcodeInMemBuffer(Buffer->getMemBufferRef());
   if (!BCOrErr)
     return false;
   LLVMContext Context;
   std::string Triple = getBitcodeTargetTriple(*BCOrErr, Context);

lib/LTO/ThinLTOCodeGenerator.cpp

+//===-ThinLTOCodeGenerator.cpp - LLVM Link Time Optimizer -----------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the Thin Link Time Optimization library. This library is
+// intended to be used by linker to optimize code at link time.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/LTO/ThinLTOCodeGenerator.h"
+
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/Statistic.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
+#include "llvm/Bitcode/ReaderWriter.h"
+#include "llvm/Bitcode/BitcodeWriterPass.h"
+#include "llvm/ExecutionEngine/ObjectMemoryBuffer.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/DiagnosticPrinter.h"
+#include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/Mangler.h"
+#include "llvm/IRReader/IRReader.h"
+#include "llvm/Linker/Linker.h"
+#include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Object/FunctionIndexObjectFile.h"
+#include "llvm/Support/raw_sha1_ostream.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/ThreadPool.h"
+#include "llvm/Target/TargetMachine.h"
+#include "llvm/Transforms/IPO.h"
+#include "llvm/Transforms/IPO/FunctionImport.h"
+#include "llvm/Transforms/IPO/PassManagerBuilder.h"
+#include "llvm/Transforms/ObjCARC.h"
+#include "llvm/Transforms/Utils/FunctionImportUtils.h"
+
+#include <unordered_map>
+#include <unordered_set>
+
+using namespace llvm;
+
+namespace {
+
+static void diagnosticHandler(const DiagnosticInfo &DI) {
+  DiagnosticPrinterRawOStream DP(errs());
+  DI.print(DP);
+  errs() << '\n';
+}
+
+// Simple helper to load a module from bitcode
+static std::unique_ptr<Module>
+loadModuleFromBuffer(const MemoryBufferRef &Buffer, LLVMContext &Context,
+                     bool Lazy) {
+  SMDiagnostic Err;
+  ErrorOr<std::unique_ptr<Module>> ModuleOrErr(nullptr);
+  if (Lazy) {
+    ModuleOrErr =
+        getLazyBitcodeModule(MemoryBuffer::getMemBuffer(Buffer, false), Context,
+                             /* ShouldLazyLoadMetadata */ false);
+  } else {
+    ModuleOrErr = parseBitcodeFile(Buffer, Context);
+  }
+  if (std::error_code EC = ModuleOrErr.getError()) {
+    Err = SMDiagnostic(Buffer.getBufferIdentifier(), SourceMgr::DK_Error,
+                       EC.message());
+    Err.print("ThinLTO", errs());
+    report_fatal_error("Can't load module, abort.");
+  }
+  return std::move(ModuleOrErr.get());
+}
+
+// Simple helper to save temporary files for debug.
+static void saveTempBitcode(const Module &TheModule, StringRef TempDir,
+                            unsigned count, StringRef Suffix) {
+  if (TempDir.empty())
+    return;
+  // User asked to save temps, let dump the bitcode file after import.
+  auto SaveTempPath = TempDir + llvm::utostr(count) + Suffix;
+  std::error_code EC;
+  raw_fd_ostream OS(SaveTempPath.str(), EC, sys::fs::F_None);
+  if (EC)
+    report_fatal_error(Twine("Failed to open ") + SaveTempPath +
+                       " to save optimized bitcode\n");
+  WriteBitcodeToFile(&TheModule, OS, true, false);
+}
+
+/// Process a global adjusting its linkage, adding it to the "Keep" set if
+/// needed.
+/// \return true if the global will be internalized.
+static bool InternalizeGlobal(Module &TheModule, GlobalValue &GV,
+                              StringSet<> &Keep, const FunctionInfoIndex &Index,
+                              const StringSet<> &PreservedSymbols,
+                              const StringSet<> &CrossReferencedSymbols,
+                              bool IsFunction) {
+  Mangler Mang;
+  SmallString<128> Buffer;
+
+  if (GV.hasAvailableExternallyLinkage()) {
+    Keep.insert(GV.getName());
+    return false;
+  }
+  // There are no restrictions to apply to declarations.
+  if (GV.isDeclaration())
+    return false;
+
+  // There is nothing more restrictive than private linkage.
+  if (GV.hasPrivateLinkage())
+    return false;
+
+  auto IsFirstDefinitionForLinker = [&](GlobalValue &GV) {
+    assert(!GV.isStrongDefinitionForLinker() && "Expect weak or linkonce");
+    auto It = Index.findFunctionInfoList(GV.getName());
+    assert(It != Index.end());
+
+    // Get the first *linker visible* definition for this global in the summary
+    // list.
+    auto FirstDefForLinker = llvm::find_if(
+        It->second, [](const std::unique_ptr<FunctionInfo> &FuncInfo) {
+          auto Linkage = FuncInfo->functionSummary()->getFunctionLinkage();
+          return !GlobalValue::isAvailableExternallyLinkage(Linkage);
+        });
+    // If \p GV is not the first definition, give up...
+    if ((*FirstDefForLinker)->functionSummary()->modulePath() !=
+        TheModule.getModuleIdentifier())
+      return false;
+    // If there is any strong definition anywhere, do not bother emitting this.
+    if (llvm::any_of(
+            It->second, [](const std::unique_ptr<FunctionInfo> &FuncInfo) {
+              auto Linkage = FuncInfo->functionSummary()->getFunctionLinkage();
+              return !GlobalValue::isAvailableExternallyLinkage(Linkage) &&
+                     !GlobalValue::isWeakForLinker(Linkage);
+            }))
+      return false;
+    return true;
+  };
+  auto Linkage = GV.getLinkage();
+
+  auto HasMultipleCopies = [&](GlobalValue &GV) {
+    return Index.findFunctionInfoList(GV.getName())->second.size() > 1;
+  };
+
+  if (Linkage == GlobalValue::LinkOnceAnyLinkage) {
+    // We need to emit only one of these, the first module will keep
+    // it, but turned into a weak while the others will drop it.
+    if (IsFunction && HasMultipleCopies(GV)) {
+      if (IsFirstDefinitionForLinker(GV))
+        GV.setLinkage(GlobalValue::WeakAnyLinkage);
+      else
+        GV.setLinkage(GlobalValue::AvailableExternallyLinkage);
+    }
+    Keep.insert(GV.getName());
+    return false;
+  }
+  if (Linkage == GlobalValue::LinkOnceODRLinkage) {
+    // We need to emit only one of these, the first module will keep
+    // it, but turned into a weak while the others will drop it.
+    if (IsFunction && HasMultipleCopies(GV)) {
+      if (IsFirstDefinitionForLinker(GV))
+        GV.setLinkage(GlobalValue::WeakODRLinkage);
+      else
+        GV.setLinkage(GlobalValue::AvailableExternallyLinkage);
+    }
+    Keep.insert(GV.getName());
+    return false;
+  }
+  if (Linkage == GlobalValue::WeakAnyLinkage) {
+    // We need to emit only one of these, the first module will keep
+    // it "as is" and the others will drop it
+    if (IsFunction && !IsFirstDefinitionForLinker(GV))
+      GV.setLinkage(GlobalValue::AvailableExternallyLinkage);
+    Keep.insert(GV.getName());
+    return false;
+  }
+  if (Linkage == GlobalValue::WeakODRLinkage) {
+    // We need to emit only one of these, the first module will keep
+    // it "as is" and the others will drop it
+    if (IsFunction && !IsFirstDefinitionForLinker(GV))
+      GV.setLinkage(GlobalValue::AvailableExternallyLinkage);
+    Keep.insert(GV.getName());
+    return false;
+  }
+
+  // Check the list of supplied symbols to preserve because of cross reference
+  // or for symbols that need to be exported.
+  Buffer.clear();
+  Mang.getNameWithPrefix(Buffer, &GV, false);
+  if (PreservedSymbols.count(Buffer) || CrossReferencedSymbols.count(Buffer)) {
+    Keep.insert(GV.getName());
+    return false;
+  }
+  return true;
+}
+
+static void InternalizeModule(Module &TheModule, StringSet<> PreservedSymbols) {
+  // Run the internalize pass and some trivial optimizations
+  legacy::PassManager PM;
+  PM.add(createInternalizePass(std::move(PreservedSymbols)));
+  PM.add(createGlobalDCEPass());
+  PM.add(createIPSCCPPass());
+  PM.add(createGlobalOptimizerPass());
+  PM.run(TheModule);
+}
+
+// Helper function running internalize for ThinLTO
+static void InternalizeModule(Module &TheModule, const FunctionInfoIndex &Index,
+                              const StringSet<> &PreservedSymbols,
+                              const StringSet<> &CrossReferencedSymbols,
+                              bool SingleModule) {
+  // Set of symbol to preserve during Internalize
+  StringSet<> Keep;
+  Mangler Mang;
+  SmallString<128> Buffer;
+
+  // Add to the keep list if needed
+  auto keepIfPreserved = [&](const GlobalValue &GV) {
+    Buffer.clear();
+    Mang.getNameWithPrefix(Buffer, &GV, false);
+    if (PreservedSymbols.count(Buffer) ||
+        CrossReferencedSymbols.count(Buffer)) {
+      Keep.insert(GV.getName());
+    }
+  };
+
+  if (SingleModule) {
+    // Fast-path: single module case
+    for (Function &GV : TheModule)
+      keepIfPreserved(GV);
+    for (GlobalVariable &GV : TheModule.globals())
+      keepIfPreserved(GV);
+    for (GlobalAlias &GV : TheModule.aliases())
+      keepIfPreserved(GV);
+    return InternalizeModule(TheModule, std::move(Keep));
+  }
+
+  // Keep track of the original linkage, needed to be restored
+  // The issue is that even if a function is not accessed from outside the
+  // current module before ThinLTO, it can't be internalized because it
+  // potentially has a caller in the current module and this caller can be
+  // imported in another module.
+  StringMap<std::pair<GlobalValue::LinkageTypes, GlobalValue::VisibilityTypes>>
+      OriginalLinkages;
+
+  auto RememberOriginalLinkage = [&](GlobalValue &GV) {
+    OriginalLinkages[GV.getName()] =
+        std::make_pair(GV.getLinkage(), GV.getVisibility());
+  };
+
+  for (Function &GV : TheModule) {
+    if (InternalizeGlobal(TheModule, GV, Keep, Index, PreservedSymbols,
+                          CrossReferencedSymbols, true))
+      RememberOriginalLinkage(GV);
+  }
+  for (GlobalVariable &GV : TheModule.globals()) {
+    if (InternalizeGlobal(TheModule, GV, Keep, Index, PreservedSymbols,
+                          CrossReferencedSymbols, false))
+      RememberOriginalLinkage(GV);
+  }
+  for (GlobalAlias &GV : TheModule.aliases()) {
+    if (InternalizeGlobal(TheModule, GV, Keep, Index, PreservedSymbols,
+                          CrossReferencedSymbols, false))
+      RememberOriginalLinkage(GV);
+  }
+
+  {
+    // Run the internalize pass and some trivial optimizations
+    legacy::PassManager PM;
+    PM.add(createInternalizePass(std::move(Keep))); // apply scope restrictions
+    PM.add(createGlobalDCEPass());
+    PM.add(createIPSCCPPass());          // IP SCCP
+    PM.add(createGlobalOptimizerPass()); // Optimize out global vars
+    PM.run(TheModule);
+  }
+
+  // Restore the original linkage for function that haven't been eliminated
+  for (auto &GVInfo : OriginalLinkages) {
+    if (auto *GV = TheModule.getNamedValue(GVInfo.first())) {
+      GV->setLinkage(GVInfo.second.first);
+      GV->setVisibility(GVInfo.second.second);
+
+      // Workaround an issue with "common": they can't be marked as constant.
+      // However a common that is internalized is seen as "internal" and no
+      // longer "common" by globalopt, who sets it "constant" if possible.
+      // When we restore the linkage, we need to unset the "constant" part as
+      // well
+      if (GV->hasCommonLinkage() && isa<GlobalVariable>(GV)) {
+        cast<GlobalVariable>(GV)->setConstant(false);
+      }
+    }
+  }
+}
+
+/// This class is a helper to load and do early optimization on a module.
+///
+/// The first thread to ask for this module will trigger the early optimizations
+/// and store the bitcode. Further request will directly load the pre-optimized
+/// bitcode.
+class EarlyOptimizedModule {
+  std::unique_ptr<MemoryBuffer> OptimizedBuffer;
+  /// Lock to guard optimization (i.e. first time this module is requested)
+  std::mutex LockM;
+
+public:
+  std::unique_ptr<Module>
+  load(LLVMContext &Ctx, StringMap<MemoryBufferRef> &ModuleMap,
+       StringRef Identifier, const FunctionInfoIndex &Index,
+       const StringSet<> &PreservedSymbols,
+       const StringSet<> &CrossReferencedSymbols, bool Lazy) {
+    std::lock_guard<std::mutex> Lock(LockM);
+    if (OptimizedBuffer)
+      // optimizations already ran, we have a pre-optimized copy.
+      return loadModuleFromBuffer(OptimizedBuffer->getMemBufferRef(), Ctx,
+                                  Lazy);
+
+    assert(ModuleMap.find(Identifier) != ModuleMap.end() && "Unknown module");
+
+    // "Benchmark"-like optimization: single-source case
+    bool SingleModule = (ModuleMap.size() == 1);
+
+    const MemoryBufferRef &OriginalBuffer = ModuleMap[Identifier];
+    auto TheModule = loadModuleFromBuffer(OriginalBuffer, Ctx, false);
+
+    if (!PreservedSymbols.empty() || !CrossReferencedSymbols.empty())
+      InternalizeModule(*TheModule, Index, PreservedSymbols,
+                        CrossReferencedSymbols, SingleModule);
+    SmallVector<char, 0> OutputBuffer;
+    {
+      // save the bitcode for further request
+      raw_svector_ostream OS(OutputBuffer);
+      WriteBitcodeToFile(TheModule.get(), OS);
+    }
+    OptimizedBuffer =
+        make_unique<ObjectMemoryBuffer>(std::move(OutputBuffer), Identifier);
+    return TheModule;
+  }
+};
+
+class PreOptimizedModuleMap {
+  /// Helper map to find the original Module buffer from its identifier
+  /// !! Relies on having a unique getBufferIdentifier() !!
+  StringMap<MemoryBufferRef> ModuleMap;
+  const FunctionInfoIndex &Index;
+  const StringSet<> &PreservedSymbols;
+  const StringSet<> &CrossReferencedSymbols;
+
+  /// Map of Module Identifier to MemoryBuffer used to load a Module.
+  std::unordered_map<std::string, EarlyOptimizedModule> OptimizedModuleMap;
+
+  /// Serialize access to the map
+  std::mutex LockMap;
+
+  EarlyOptimizedModule &getModuleEntry(StringRef Identifier) {
+    std::lock_guard<std::mutex> Lock(LockMap);
+    return OptimizedModuleMap[Identifier];
+  }
+
+public:
+  PreOptimizedModuleMap(const std::vector<MemoryBufferRef> &Modules,
+                        const FunctionInfoIndex &Index,
+                        const StringSet<> &PreservedSymbols,
+                        const StringSet<> &CrossReferencedSymbols)
+      : Index(Index), PreservedSymbols(PreservedSymbols),
+        CrossReferencedSymbols(CrossReferencedSymbols) {
+    for (auto &ModuleBuffer : Modules) {
+      assert(ModuleMap.find(ModuleBuffer.getBufferIdentifier()) ==
+                 ModuleMap.end() &&
+             "Expect unique Buffer Identifier");
+      ModuleMap[ModuleBuffer.getBufferIdentifier()] = ModuleBuffer;
+    }
+  }
+
+  std::unique_ptr<Module> load(StringRef Identifier, LLVMContext &Ctx,
+                               bool Lazy) {
+    auto &EarlyOptimizedModule = getModuleEntry(Identifier);
+    return EarlyOptimizedModule.load(Ctx, ModuleMap, Identifier, Index,
+                                     PreservedSymbols, CrossReferencedSymbols,
+                                     Lazy);
+  }
+
+  size_t size() { return ModuleMap.size(); }
+};
+
+/// Provide a "loader" for the FunctionImporter to access function from other
+/// modules.
+class ModuleLoader {
+  /// The context that will be used for importing.
+  LLVMContext &Context;
+  PreOptimizedModuleMap &ModuleMap;
+
+public:
+  ModuleLoader(LLVMContext &Context, PreOptimizedModuleMap &ModuleMap)
+      : Context(Context), ModuleMap(ModuleMap) {}
+
+  /// Load a module on demand.
+  std::unique_ptr<Module> operator()(StringRef Identifier) {
+    return ModuleMap.load(Identifier, Context, /* Lazy */ true);
+  }
+};
+
+static void promoteModule(Module &TheModule, const FunctionInfoIndex &Index) {
+  if (renameModuleForThinLTO(TheModule, &Index))
+    report_fatal_error("renameModuleForThinLTO failed");
+}
+
+static void crossImportForModule(Module &TheModule,
+                                 const FunctionInfoIndex &Index,
+                                 PreOptimizedModuleMap &ModuleMap) {
+  ModuleLoader Loader(TheModule.getContext(), ModuleMap);
+  FunctionImporter Importer(Index, Loader);
+  Importer.importFunctions(TheModule);
+}
+
+static void optimizeModule(Module &TheModule, TargetMachine &TM) {
+  // Populate the PassManager
+  PassManagerBuilder PMB;
+  PMB.LibraryInfo = new TargetLibraryInfoImpl(TM.getTargetTriple());
+  PMB.Inliner = createFunctionInliningPass();
+  // FIXME: should get it from the bitcode?
+  PMB.OptLevel = 3;
+  PMB.LoopVectorize = true;
+  PMB.SLPVectorize = true;
+  PMB.VerifyInput = true;
+  PMB.VerifyOutput = false;
+
+  legacy::PassManager PM;
+
+  // Add the TTI (required to inform the vectorizer about register size for
+  // instance)
+  PM.add(createTargetTransformInfoWrapperPass(TM.getTargetIRAnalysis()));
+
+  // Add optimizations
+  PMB.populateThinLTOPassManager(PM);
+  PM.add(createObjCARCContractPass());
+
+  PM.run(TheModule);
+}
+
+std::unique_ptr<MemoryBuffer> codegenModule(Module &TheModule,
+                                            TargetMachine &TM) {
+  SmallVector<char, 128> OutputBuffer;
+
+  // CodeGen
+  {
+    raw_svector_ostream OS(OutputBuffer);
+    legacy::PassManager PM;
+    if (TM.addPassesToEmitFile(PM, OS, TargetMachine::CGFT_ObjectFile,
+                               /* DisableVerify */ true))
+      report_fatal_error("Failed to setup codegen");
+
+    // Run codegen now. resulting binary is in OutputBuffer.
+    PM.run(TheModule);
+  }
+  return make_unique<ObjectMemoryBuffer>(std::move(OutputBuffer));
+}
+
+static std::unique_ptr<MemoryBuffer>
+ProcessThinLTOModule(Module &TheModule, const FunctionInfoIndex &Index,
+                     PreOptimizedModuleMap &ModuleMap, TargetMachine &TM,
+                     CachingOptions CacheOptions, StringRef SaveTempsDir,
+                     unsigned count) {
+
+  // Save temps: after IPO.
+  saveTempBitcode(TheModule, SaveTempsDir, count, ".1.IPO.bc");
+
+  // "Benchmark"-like optimization: single-source case
+  bool SingleModule = (ModuleMap.size() == 1);
+
+  if (!SingleModule) {
+    promoteModule(TheModule, Index);
+
+    // Save temps: after promotion.
+    saveTempBitcode(TheModule, SaveTempsDir, count, ".2.promoted.bc");
+
+    crossImportForModule(TheModule, Index, ModuleMap);
+
+    // Save temps: after cross-module import.
+    saveTempBitcode(TheModule, SaveTempsDir, count, ".3.imported.bc");
+  }
+
+  optimizeModule(TheModule, TM);
+
+  saveTempBitcode(TheModule, SaveTempsDir, count, ".3.opt.bc");
+
+  return codegenModule(TheModule, TM);
+}
+
+} // end anonymous namespace
+
+void ThinLTOCodeGenerator::addModule(StringRef Identifier, StringRef Data) {
+  MemoryBufferRef Buffer(Data, Identifier);
+  if (Modules.empty()) {
+    // First module added, so initialize the triple and some options
+    LLVMContext Context;
+    Triple TheTriple(getBitcodeTargetTriple(Buffer, Context));
+
+    // Set a default CPU for Darwin triples (copied from LTOCodeGenerator).
+    // FIXME this looks pretty terrible...
+    if (TMBuilder.MCpu.empty() && TheTriple.isOSDarwin()) {
+      if (TheTriple.getArch() == llvm::Triple::x86_64)
+        TMBuilder.MCpu = "core2";
+      else if (TheTriple.getArch() == llvm::Triple::x86)
+        TMBuilder.MCpu = "yonah";
+      else if (TheTriple.getArch() == llvm::Triple::aarch64)
+        TMBuilder.MCpu = "cyclone";
+    }
+
+    TMBuilder.TheTriple = std::move(TheTriple);
+  }
+#ifndef NDEBUG
+  else {
+    LLVMContext Context;
+    assert(TMBuilder.TheTriple.str() ==
+               getBitcodeTargetTriple(Buffer, Context) &&
+           "ThinLTO modules with different triple not supported");
+  }
+#endif
+  Modules.push_back(Buffer);
+}
+
+void ThinLTOCodeGenerator::preserveSymbol(StringRef Name) {
+  PreservedSymbols.insert(Name);
+}
+
+void ThinLTOCodeGenerator::crossReferenceSymbol(StringRef Name) {
+  CrossReferencedSymbols.insert(Name);
+}
+
+// TargetMachine factory
+std::unique_ptr<TargetMachine> TargetMachineBuilder::create() const {
+  std::string ErrMsg;
+  const Target *TheTarget =
+      TargetRegistry::lookupTarget(TheTriple.str(), ErrMsg);
+  if (!TheTarget) {
+    report_fatal_error("Can't load target for this Triple");
+  }
+
+  // Use MAttr as the default set of features.
+  SubtargetFeatures Features(MAttr);
+  Features.getDefaultSubtargetFeatures(TheTriple);
+  std::string FeatureStr = Features.getString();
+  return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
+      TheTriple.str(), MCpu, FeatureStr, Options, RelocModel,
+      CodeModel::Default, CGOptLevel));
+}
+
+/**
+ * Produce the combined function index from all the bitcode files:
+ * "thin-link".
+ */
+std::unique_ptr<FunctionInfoIndex> ThinLTOCodeGenerator::linkCombinedIndex() {
+  std::unique_ptr<FunctionInfoIndex> CombinedIndex;
+  uint64_t NextModuleId = 0;
+  for (auto &ModuleBuffer : Modules) {
+    ErrorOr<std::unique_ptr<object::FunctionIndexObjectFile>> ObjOrErr =
+        object::FunctionIndexObjectFile::create(ModuleBuffer, diagnosticHandler,
+                                                false);
+    if (std::error_code EC = ObjOrErr.getError()) {
+      // FIXME diagnose
+      errs() << "error: can't create FunctionIndexObjectFile for buffer: "
+             << EC.message() << "\n";
+      return nullptr;
+    }
+    auto Index = (*ObjOrErr)->takeIndex();
+    if (CombinedIndex) {
+      CombinedIndex->mergeFrom(std::move(Index), ++NextModuleId);
+    } else {
+      CombinedIndex = std::move(Index);
+    }
+  }
+  return CombinedIndex;
+}
+
+/**
+ * Perform promotion and renaming of exported internal functions.
+ */
+void ThinLTOCodeGenerator::promote(Module &TheModule,
+                                   FunctionInfoIndex &Index) {
+  promoteModule(TheModule, Index);
+}
+
+/**
+ * Perform cross-module importing for the module identified by ModuleIdentifier.
+ */
+void ThinLTOCodeGenerator::crossModuleImport(Module &TheModule,
+                                             FunctionInfoIndex &Index) {
+  /// Helper to load a cached module after early global optimizations
+  PreOptimizedModuleMap ModuleMap(Modules, Index, PreservedSymbols,
+                                  CrossReferencedSymbols);
+  crossImportForModule(TheModule, Index, ModuleMap);
+}
+
+/**
+ * Perform post-importing ThinLTO optimizations.
+ */
+void ThinLTOCodeGenerator::optimize(Module &TheModule) {
+  optimizeModule(TheModule, *TMBuilder.create());
+}
+
+/**
+ * Perform ThinLTO CodeGen.
+ */
+std::unique_ptr<MemoryBuffer> ThinLTOCodeGenerator::codegen(Module &TheModule) {
+  return codegenModule(TheModule, *TMBuilder.create());
+}
+
+// Main entry point for the ThinLTO processing
+void ThinLTOCodeGenerator::run() {
+  // Sequential linking phase
+  auto Index = linkCombinedIndex();
+
+  // Save temps: index.
+  if (!SaveTempsDir.empty()) {
+    auto SaveTempPath = SaveTempsDir + "index.bc";
+    std::error_code EC;
+    raw_fd_ostream OS(SaveTempPath, EC, sys::fs::F_None);
+    if (EC)
+      report_fatal_error(Twine("Failed to open ") + SaveTempPath +
+                         " to save optimized bitcode\n");
+    WriteFunctionSummaryToFile(*Index, OS);
+  }
+
+  // Prepare the resulting object vector
+  assert(ProducedBinaries.empty() && "The generator should not be reused");
+  ProducedBinaries.resize(Modules.size());
+
+  /// Helper to load a cached module after early global optimizations
+  PreOptimizedModuleMap ModuleMap(Modules, *Index, PreservedSymbols,
+                                  CrossReferencedSymbols);
+
+  // Parallel optimizer + codegen
+  {
+    ThreadPool Pool;
+    int count = 0;
+    for (auto &ModuleBuffer : Modules) {
+      Pool.async([&](int count) {
+        LLVMContext Context;
+
+        // Save temps: original file.
+        if (SaveTempsDir.empty()) {
+          auto TheModule = loadModuleFromBuffer(ModuleBuffer, Context, false);
+          saveTempBitcode(*TheModule, SaveTempsDir, count, ".0.original.bc");
+        }
+
+        // Parse module now
+        auto TheModule = ModuleMap.load(ModuleBuffer.getBufferIdentifier(),
+                                        Context, /* Lazy */ false);
+
+        ProducedBinaries[count] = ProcessThinLTOModule(
+            *TheModule, *Index, ModuleMap, *TMBuilder.create(), CacheOptions,
+            SaveTempsDir, count);
+      }, count);
+      count++;
+    }
+  }
+
+  // If statistics were requested, print them out now.
+  if (llvm::AreStatisticsEnabled())
+    llvm::PrintStatistics();
+}

test/ThinLTO/Inputs/funcimport.ll

+define i32 @main() #0 {
+entry:
+  call void (...) @weakalias()
+  call void (...) @analias()
+  %call = call i32 (...) @referencestatics()
+  %call1 = call i32 (...) @referenceglobals()
+  %call2 = call i32 (...) @referencecommon()
+  call void (...) @setfuncptr()
+  call void (...) @callfuncptr()
+  call void (...) @callweakfunc()
+  ret i32 0
+}
+
+declare void @weakalias(...) #1
+
+declare void @analias(...) #1
+
+declare i32 @referencestatics(...) #1
+
+declare i32 @referenceglobals(...) #1
+
+declare i32 @referencecommon(...) #1
+
+declare void @setfuncptr(...) #1
+
+declare void @callfuncptr(...) #1
+
+declare void @callweakfunc(...) #1

test/ThinLTO/funcimport.ll

+; Do setup work for all below tests: generate bitcode and combined index
+; RUN: llvm-as -function-summary %s -o %t.bc
+; RUN: llvm-as -function-summary %p/Inputs/funcimport.ll -o %t2.bc
+; RUN: llvm-lto -thinlto-action=thinlink -o %t3.bc %t.bc %t2.bc
+
+; Ensure statics are promoted/renamed correctly from this file (all but
+; constant variable need promotion).
+; RUN: llvm-lto -thinlto-action=promote %t.bc -functionindex=%t3.bc -o - | llvm-dis -o - | FileCheck %s --check-prefix=EXPORTSTATIC
+; EXPORTSTATIC-DAG: @staticvar.llvm.0 = hidden global
+; EXPORTSTATIC-DAG: @staticconstvar = internal unnamed_addr constant
+; EXPORTSTATIC-DAG: @P.llvm.0 = hidden global void ()* null
+; EXPORTSTATIC-DAG: define hidden i32 @staticfunc.llvm.0
+; EXPORTSTATIC-DAG: define hidden void @staticfunc2.llvm.0
+
+; Ensure that both weak alias to an imported function and strong alias to a
+; non-imported function are correctly turned into declarations.
+; Also ensures that alias to a linkonce function is turned into a declaration
+; and that the associated linkonce function is not in the output, as it is
+; lazily linked and never referenced/materialized.
+; RUN: llvm-lto -thinlto-action=import %t2.bc -functionindex=%t3.bc -o - | llvm-dis -o - | FileCheck %s --check-prefix=IMPORTGLOB1
+; IMPORTGLOB1-DAG: define available_externally void @globalfunc1
+; IMPORTGLOB1-DAG: declare void @weakalias
+; IMPORTGLOB1-DAG: declare void @analias
+; IMPORTGLOB1-NOT: @linkoncealias
+; IMPORTGLOB1-NOT: @linkoncefunc
+; IMPORTGLOB1-NOT: declare void @globalfunc2
+
+@globalvar_in_section = global i32 1, align 4
+@globalvar = global i32 1, align 4
+@staticvar = internal global i32 1, align 4
+@staticvar2 = internal global i32 1, align 4
+@staticconstvar = internal unnamed_addr constant [2 x i32] [i32 10, i32 20], align 4
+@commonvar = common global i32 0, align 4
+@P = internal global void ()* null, align 8
+
+@weakalias = weak alias void (...), bitcast (void ()* @globalfunc1 to void (...)*)
+@analias = alias void (...), bitcast (void ()* @globalfunc2 to void (...)*)
+@linkoncealias = alias void (...), bitcast (void ()* @linkoncefunc to void (...)*)
+
+define void @globalfunc1() #0 {
+entry:
+  ret void
+}
+
+define void @globalfunc2() #0 {
+entry:
+  ret void
+}
+
+define linkonce_odr void @linkoncefunc() #0 {
+entry:
+  ret void
+}
+
+define i32 @referencestatics(i32 %i) #0 {
+entry:
+  %i.addr = alloca i32, align 4
+  store i32 %i, i32* %i.addr, align 4
+  %call = call i32 @staticfunc()
+  %0 = load i32, i32* @staticvar, align 4
+  %add = add nsw i32 %call, %0
+  %1 = load i32, i32* %i.addr, align 4
+  %idxprom = sext i32 %1 to i64
+  %arrayidx = getelementptr inbounds [2 x i32], [2 x i32]* @staticconstvar, i64 0, i64 %idxprom
+  %2 = load i32, i32* %arrayidx, align 4
+  %add1 = add nsw i32 %add, %2
+  ret i32 %add1
+}
+
+define i32 @referenceglobals(i32 %i) #0 {
+entry:
+  %i.addr = alloca i32, align 4
+  store i32 %i, i32* %i.addr, align 4
+  call void @globalfunc1()
+  %0 = load i32, i32* @globalvar, align 4
+  ret i32 %0
+}
+
+define i32 @referencecommon(i32 %i) #0 {
+entry:
+  %i.addr = alloca i32, align 4
+  store i32 %i, i32* %i.addr, align 4
+  %0 = load i32, i32* @commonvar, align 4
+  ret i32 %0
+}
+
+define void @setfuncptr() #0 {
+entry:
+  store void ()* @staticfunc2, void ()** @P, align 8
+  ret void
+}
+
+define void @callfuncptr() #0 {
+entry:
+  %0 = load void ()*, void ()** @P, align 8
+  call void %0()
+  ret void
+}
+
+@weakvar = weak global i32 1, align 4
+define weak void @weakfunc() #0 {
+entry:
+  ret void
+}
+
+define void @callweakfunc() #0 {
+entry:
+  call void @weakfunc()
+  ret void
+}
+
+define internal i32 @staticfunc() #0 {
+entry:
+  ret i32 1
+}
+
+define internal void @staticfunc2() #0 {
+entry:
+  %0 = load i32, i32* @staticvar2, align 4
+  ret void
+}

tools/llvm-lto/llvm-lto.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/StringSet.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IRReader/IRReader.h"
 #include "llvm/LTO/LTOCodeGenerator.h"
+#include "llvm/LTO/ThinLTOCodeGenerator.h"
 #include "llvm/LTO/LTOModule.h"
 #include "llvm/Object/FunctionIndexObjectFile.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/Signals.h"
+#include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/ToolOutputFile.h"
 #include "llvm/Support/raw_ostream.h"
 #include <list>
 
 using namespace llvm;
 
 static cl::opt<char>

 static cl::opt<bool>
 UseDiagnosticHandler("use-diagnostic-handler", cl::init(false),
   cl::desc("Use a diagnostic handler to test the handler interface"));
 
 static cl::opt<bool>
     ThinLTO("thinlto", cl::init(false),
             cl::desc("Only write combined global index for ThinLTO backends"));
 
+enum ThinLTOModes { THINLINK, THINPROMOTE, THINIMPORT, THINOPT, THINCODEGEN };
+
+cl::opt<ThinLTOModes> ThinLTOMode(
+    "thinlto-action", cl::desc("Perform a single ThinLTO stage:"),
+    cl::values(
+        clEnumValN(
+            THINLINK, "thinlink",
+            "ThinLink: produces the index by linking only the summaries."),
+        clEnumValN(THINPROMOTE, "promote",
+                   "Perform pre-import promotion (requires -thinlto-index)."),
+        clEnumValN(THINIMPORT, "import", "Perform both promotion and "
+                                         "cross-module importing (requires "
+                                         "-thinlto-index)."),
+        clEnumValN(THINOPT, "optimize", "Perform ThinLTO optimizations."),
+        clEnumValN(THINCODEGEN, "codegen", "CodeGen (expected to match llc)"),
+        clEnumValEnd));
+
+static cl::opt<std::string>
+    ThinLTOIndex("functionindex",
+                 cl::desc("Provide the index produced by a ThinLink, required "
+                          "to perform the promotion and/or importing."));
+
 static cl::opt<bool>
 SaveModuleFile("save-merged-module", cl::init(false),
                cl::desc("Write merged LTO module to file before CodeGen"));
 
 static cl::list<std::string>
 InputFilenames(cl::Positional, cl::OneOrMore,
   cl::desc("<input bitcode files>"));
 
static void createCombinedFunctionIndex() {
   assert(!OutputFilename.empty());
   raw_fd_ostream OS(OutputFilename + ".thinlto.bc", EC,
                     sys::fs::OpenFlags::F_None);
   error(EC, "error opening the file '" + OutputFilename + ".thinlto.bc'");
   WriteFunctionSummaryToFile(CombinedIndex, OS);
   OS.close();
 }
 
+namespace thinlto {
+
+std::vector<std::unique_ptr<MemoryBuffer>>
+loadAllFilesForIndex(const FunctionInfoIndex &Index) {
+  std::vector<std::unique_ptr<MemoryBuffer>> InputBuffers;
+
+  for (auto &ModPath : Index.modPathStringEntries()) {
+    const auto &Filename = ModPath.first();
+    auto CurrentActivity = "loading file '" + Filename + "'";
+    errs() << CurrentActivity << "\n";
+    auto InputOrErr = MemoryBuffer::getFile(Filename);
+    error(InputOrErr, "error " + CurrentActivity);
+    InputBuffers.push_back(std::move(*InputOrErr));
+  }
+  return InputBuffers;
+}
+
+std::unique_ptr<FunctionInfoIndex> loadCombinedIndex() {
+  if (ThinLTOIndex.empty())
+    report_fatal_error("Missing -thinlto-index for ThinLTO promotion stage");
+  auto CurrentActivity = "loading file '" + ThinLTOIndex + "'";
+  ErrorOr<std::unique_ptr<FunctionInfoIndex>> IndexOrErr =
+      llvm::getFunctionIndexForFile(ThinLTOIndex, diagnosticHandler);
+  error(IndexOrErr, "error " + CurrentActivity);
+  return std::move(IndexOrErr.get());
+}
+
+static std::unique_ptr<Module> loadModule(StringRef Filename,
+                                          LLVMContext &Ctx) {
+  SMDiagnostic Err;
+  std::unique_ptr<Module> M(parseIRFile(Filename, Err, Ctx));
+  if (!M) {
+    Err.print("llvm-lto", errs());
+    report_fatal_error("Can't load module for file " + Filename);
+  }
+  return M;
+}
+
+class ThinLTOProcessing {
+public:
+  ThinLTOCodeGenerator ThinGenerator;
+
+  ThinLTOProcessing(const TargetOptions &Options) {
+    ThinGenerator.setCodePICModel(RelocModel);
+    ThinGenerator.setTargetOptions(Options);
+  }
+
+  void run() {
+    switch (ThinLTOMode) {
+    case THINLINK:
+      return thinLink();
+    case THINPROMOTE:
+      return promote();
+    case THINIMPORT:
+      return import();
+    case THINOPT:
+      return optimize();
+    case THINCODEGEN:
+      return codegen();
+    }
+  }
+
+private:
+  /// Load the input files, create the combined index, and write it out.
+  void thinLink() {
+    // Perform "ThinLink": just produce the index
+    if (OutputFilename.empty())
+      report_fatal_error(
+          "OutputFilename is necessary to store the combined index.\n");
+
+    LLVMContext Ctx;
+    std::vector<std::unique_ptr<MemoryBuffer>> InputBuffers;
+    for (unsigned i = 0; i < InputFilenames.size(); ++i) {
+      auto &Filename = InputFilenames[i];
+      StringRef CurrentActivity = "loading file '" + Filename + "'";
+      auto InputOrErr = MemoryBuffer::getFile(Filename);
+      error(InputOrErr, "error " + CurrentActivity);
+      InputBuffers.push_back(std::move(*InputOrErr));
+      ThinGenerator.addModule(Filename, InputBuffers.back()->getBuffer());
+    }
+
+    auto CombinedIndex = ThinGenerator.linkCombinedIndex();
+    std::error_code EC;
+    raw_fd_ostream OS(OutputFilename, EC, sys::fs::OpenFlags::F_None);
+    error(EC, "error opening the file '" + OutputFilename + "'");
+    WriteFunctionSummaryToFile(*CombinedIndex, OS);
+    OS.close();
+    return;
+  }
+
+  /// Load the combined index from disk, then load every file referenced by
+  /// the index and add them to the generator, finally perform the promotion
+  /// on the files mentionned on the command line (these must match the index
+  /// content).
+  void promote() {
+    if (InputFilenames.size() != 1 && !OutputFilename.empty())
+      report_fatal_error("Can't handle a single output filename and multiple "
+                         "input files, do not provide an output filename and "
+                         "the output files will be suffixed from the input "
+                         "ones.");
+
+    auto Index = loadCombinedIndex();
+    auto InputBuffers = loadAllFilesForIndex(*Index);
+    for (auto &MemBuffer : InputBuffers)
+      ThinGenerator.addModule(MemBuffer->getBufferIdentifier(),
+                              MemBuffer->getBuffer());
+
+    for (auto &Filename : InputFilenames) {
+      LLVMContext Ctx;
+      auto TheModule = loadModule(Filename, Ctx);
+
+      ThinGenerator.promote(*TheModule, *Index);
+
+      std::string OutputName = OutputFilename;
+      if (OutputName.empty()) {
+        OutputName = Filename + ".thinlto.promoted.bc";
+      }
+      std::error_code EC;
+      raw_fd_ostream OS(OutputName.c_str(), EC, sys::fs::F_None);
+      error(EC, "Failed to open " + OutputName + " to save bitcode\n");
+      WriteBitcodeToFile(TheModule.get(), OS, true, false);
+    }
+  }
+
+  /// Load the combined index from disk, then load every file referenced by
+  /// the index and add them to the generator, then performs the promotion and
+  /// cross module importing on the files mentionned on the command line
+  /// (these must match the index content).
+  void import() {
+    if (InputFilenames.size() != 1 && !OutputFilename.empty())
+      report_fatal_error("Can't handle a single output filename and multiple "
+                         "input files, do not provide an output filename and "
+                         "the output files will be suffixed from the input "
+                         "ones.");
+
+    auto Index = loadCombinedIndex();
+    auto InputBuffers = loadAllFilesForIndex(*Index);
+    for (auto &MemBuffer : InputBuffers)
+      ThinGenerator.addModule(MemBuffer->getBufferIdentifier(),
+                              MemBuffer->getBuffer());
+
+    for (auto &Filename : InputFilenames) {
+      LLVMContext Ctx;
+      auto TheModule = loadModule(Filename, Ctx);
+
+      ThinGenerator.promote(*TheModule, *Index);
+      ThinGenerator.crossModuleImport(*TheModule, *Index);
+
+      std::string OutputName = OutputFilename;
+      if (OutputName.empty()) {
+        OutputName = Filename + ".thinlto.imported.bc";
+      }
+      std::error_code EC;
+      raw_fd_ostream OS(OutputName.c_str(), EC, sys::fs::F_None);
+      error(EC, "Failed to open " + OutputName + " to save bitcode\n");
+      WriteBitcodeToFile(TheModule.get(), OS, true, false);
+    }
+  }
+
+  void optimize() {
+    if (InputFilenames.size() != 1 && !OutputFilename.empty())
+      report_fatal_error("Can't handle a single output filename and multiple "
+                         "input files, do not provide an output filename and "
+                         "the output files will be suffixed from the input "
+                         "ones.");
+    if (!ThinLTOIndex.empty())
+      errs() << "Warning: -thinlto-index ignored for optimize stage";
+
+    for (auto &Filename : InputFilenames) {
+      LLVMContext Ctx;
+      auto TheModule = loadModule(Filename, Ctx);
+
+      ThinGenerator.optimize(*TheModule);
+
+      std::string OutputName = OutputFilename;
+      if (OutputName.empty()) {
+        OutputName = Filename + ".thinlto.imported.bc";
+      }
+      std::error_code EC;
+      raw_fd_ostream OS(OutputName.c_str(), EC, sys::fs::F_None);
+      error(EC, "Failed to open " + OutputName + " to save bitcode\n");
+      WriteBitcodeToFile(TheModule.get(), OS, true, false);
+    }
+  }
+
+  void codegen() { report_fatal_error("codegen unimplemented yet"); }
+};
+
+} // namespace thinlto
+
 int main(int argc, char **argv) {
   // Print a stack trace if we signal out.
   sys::PrintStackTraceOnErrorSignal();
   PrettyStackTraceProgram X(argc, argv);
 
   llvm_shutdown_obj Y; // Call llvm_shutdown() on exit.
   cl::ParseCommandLineOptions(argc, argv, "llvm LTO linker\n");
 
int main(int argc, char **argv) {
   // set up the TargetOptions for the machine
   TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
 
   if (ListSymbolsOnly) {
     listSymbols(Options);
     return 0;
   }
 
+  if (ThinLTOMode.getNumOccurrences()) {
+    thinlto::ThinLTOProcessing ThinLTOProcessor(Options);
+    ThinLTOProcessor.run();
+    return 0;
+  }
+
   if (ThinLTO) {
     createCombinedFunctionIndex();
     return 0;
   }
 
   unsigned BaseArg = 0;
 
   LLVMContext Context;
   Context.setDiagnosticHandler(diagnosticHandlerWithContext, nullptr, true);
 
   LTOCodeGenerator CodeGen(Context);
 
   if (UseDiagnosticHandler)
     CodeGen.setDiagnosticHandler(handleDiagnostics, nullptr);
 
   CodeGen.setCodePICModel(RelocModel);
 
   CodeGen.setDebugInfo(LTO_DEBUG_MODEL_DWARF);
   CodeGen.setTargetOptions(Options);
   CodeGen.setShouldRestoreGlobalsLinkage(RestoreGlobalsLinkage);
 
   llvm::StringSet<llvm::MallocAllocator> DSOSymbolsSet;

tools/lto/lto.cpp

 #include "llvm-c/lto.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/LTO/LTOCodeGenerator.h"
 #include "llvm/LTO/LTOModule.h"
+#include "llvm/LTO/ThinLTOCodeGenerator.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/raw_ostream.h"
 
 // extra command-line flags needed for LTOCodeGenerator
 static cl::opt<char>
 OptLevel("O",
struct LibLTOCodeGenerator : LTOCodeGenerator {
 
   std::unique_ptr<MemoryBuffer> NativeObjectFile;
   std::unique_ptr<LLVMContext> OwnedContext;
 };
 
 }
 
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(LibLTOCodeGenerator, lto_code_gen_t)
+DEFINE_SIMPLE_CONVERSION_FUNCTIONS(ThinLTOCodeGenerator, thinlto_code_gen_t)
 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(LTOModule, lto_module_t)
 
 // Convert the subtarget features into a string to pass to LTOCodeGenerator.
 static void lto_add_attrs(lto_code_gen_t cg) {
   LTOCodeGenerator *CG = unwrap(cg);
   if (MAttrs.size()) {
     std::string attrs;
     for (unsigned i = 0; i < MAttrs.size(); ++i) {
const char* lto_module_get_target_triple(lto_module_t mod) {
   return unwrap(mod)->getTargetTriple().c_str();
 }
 
 void lto_module_set_target_triple(lto_module_t mod, const char *triple) {
   return unwrap(mod)->setTargetTriple(triple);
 }
 
 unsigned int lto_module_get_num_symbols(lto_module_t mod) {
   return unwrap(mod)->getSymbolCount();
 }
 
 const char* lto_module_get_symbol_name(lto_module_t mod, unsigned int index) {
   return unwrap(mod)->getSymbolName(index);
 }
 
 lto_symbol_attributes lto_module_get_symbol_attribute(lto_module_t mod,
                                                       unsigned int index) {
void lto_codegen_set_should_internalize(lto_code_gen_t cg,
                                         bool ShouldInternalize) {
   unwrap(cg)->setShouldInternalize(ShouldInternalize);
 }
 
 void lto_codegen_set_should_embed_uselists(lto_code_gen_t cg,
                                            lto_bool_t ShouldEmbedUselists) {
   unwrap(cg)->setShouldEmbedUselists(ShouldEmbedUselists);
 }
+
+// ThinLTO API below
+
+thinlto_code_gen_t thinlto_create_codegen() {
+  lto_initialize();
+  ThinLTOCodeGenerator *CodeGen = new ThinLTOCodeGenerator();
+  CodeGen->setTargetOptions(InitTargetOptionsFromCodeGenFlags());
+
+  return wrap(CodeGen);
+}
+
+void thinlto_codegen_dispose(thinlto_code_gen_t cg) { delete unwrap(cg); }
+
+void thinlto_codegen_add_module(thinlto_code_gen_t cg, const char *Identifier,
+                                const char *Data, int Length) {
+  unwrap(cg)->addModule(Identifier, StringRef(Data, Length));
+}
+
+void thinlto_codegen_process(thinlto_code_gen_t cg) { unwrap(cg)->run(); }
+
+unsigned int thinlto_module_get_num_objects(thinlto_code_gen_t cg) {
+  return unwrap(cg)->getProducedBinaries().size();
+}
+LTOObjectBuffer thinlto_module_get_object(thinlto_code_gen_t cg,
+                                          unsigned int index) {
+  assert(index < unwrap(cg)->getProducedBinaries().size() && "Index overflow");
+  auto &MemBuffer = unwrap(cg)->getProducedBinaries()[index];
+  return LTOObjectBuffer{(void *)MemBuffer->getBufferStart(),
+                         MemBuffer->getBufferSize()};
+}
+
+void thinlto_debug_options(const char *const *options, int number) {
+  // if options were requested, set them
+  if (number && options) {
+    std::vector<const char *> CodegenArgv(1, "libLTO");
+    for (auto Arg : ArrayRef<const char *>(options, number))
+      CodegenArgv.push_back(Arg);
+    cl::ParseCommandLineOptions(CodegenArgv.size(), CodegenArgv.data());
+  }
+}
+
+bool lto_module_is_thinlto(lto_module_t mod) {
+  return unwrap(mod)->isThinLTO();
+}
+
+void thinlto_codegen_add_must_preserve_symbol(thinlto_code_gen_t cg,
+                                              const char *Name, int Length) {
+  unwrap(cg)->preserveSymbol(StringRef(Name, Length));
+}
+
+void thinlto_codegen_add_cross_referenced_symbol(thinlto_code_gen_t cg,
+                                                 const char *Name, int Length) {
+  unwrap(cg)->crossReferenceSymbol(StringRef(Name, Length));
+}
+
+void thinlto_codegen_set_cpu(thinlto_code_gen_t cg, const char *cpu) {
+  return unwrap(cg)->setCpu(cpu);
+}
+
+void thinlto_codegen_set_cache_dir(thinlto_code_gen_t cg,
+                                   const char *cache_dir) {
+  return unwrap(cg)->setCacheDir(cache_dir);
+}
+
+void thinlto_codegen_set_cache_pruning_interval(thinlto_code_gen_t cg,
+                                                int interval) {
+  return unwrap(cg)->setCachePruningInterval(interval);
+}
+
+void thinlto_codegen_set_cache_entry_expiration(thinlto_code_gen_t cg,
+                                                unsigned expiration) {
+  return unwrap(cg)->setCacheEntryExpiration(expiration);
+}
+
+void thinlto_codegen_set_savetemps_dir(thinlto_code_gen_t cg,
+                                       const char *save_temps_dir) {
+  return unwrap(cg)->setSaveTempsDir(save_temps_dir);
+}
+
+lto_bool_t thinlto_codegen_set_pic_model(thinlto_code_gen_t cg,
+                                         lto_codegen_model model) {
+  switch (model) {
+  case LTO_CODEGEN_PIC_MODEL_STATIC:
+    unwrap(cg)->setCodePICModel(Reloc::Static);
+    return false;
+  case LTO_CODEGEN_PIC_MODEL_DYNAMIC:
+    unwrap(cg)->setCodePICModel(Reloc::PIC_);
+    return false;
+  case LTO_CODEGEN_PIC_MODEL_DYNAMIC_NO_PIC:
+    unwrap(cg)->setCodePICModel(Reloc::DynamicNoPIC);
+    return false;
+  case LTO_CODEGEN_PIC_MODEL_DEFAULT:
+    unwrap(cg)->setCodePICModel(Reloc::Default);
+    return false;
+  }
+  sLastErrorString = "Unknown PIC model";
+  return true;
+}

tools/lto/lto.exports

 lto_codegen_compile_optimized
 lto_codegen_set_should_internalize
 lto_codegen_set_should_embed_uselists
 LLVMCreateDisasm
 LLVMCreateDisasmCPU
 LLVMDisasmDispose
 LLVMDisasmInstruction
 LLVMSetDisasmOptions
+thinlto_create_codegen
+thinlto_codegen_dispose
+thinlto_codegen_add_module
+thinlto_codegen_process
+thinlto_module_get_num_objects
+thinlto_module_get_object
+thinlto_codegen_set_pic_model
+thinlto_codegen_set_cache_dir
+thinlto_codegen_set_cache_pruning_interval
+thinlto_codegen_set_cache_entry_expiration
+thinlto_codegen_set_savetemps_dir
+thinlto_codegen_set_cpu
+thinlto_debug_options
+lto_module_is_thinlto
+thinlto_codegen_add_must_preserve_symbol
+thinlto_codegen_add_cross_referenced_symbol