This is an archive of the discontinued LLVM Phabricator instance.

LLVM support for -fthinlto option.
AbandonedPublic

Authored by tejohnson on Aug 10 2015, 9:39 AM.

Details

Summary

Add basic LLVM support for the -fthinlto option, which will
be passed to LLVM via the EmitThinLTOIndex code gen option. This will
be used to guard the ThinLTO bitcode writing in WriteModule shown in
http://reviews.llvm.org/D11722.

Diff Detail

Event Timeline

tejohnson updated this revision to Diff 31676.Aug 10 2015, 9:39 AM
tejohnson retitled this revision from to LLVM support for -fthinlto option, and gold support for thinlto plugin option..
tejohnson updated this object.
tejohnson added reviewers: rafael, dexonsmith.
tejohnson added a subscriber: llvm-commits.
pcc added a subscriber: pcc.Aug 10 2015, 3:00 PM
pcc added inline comments.
tools/gold/gold-plugin.cpp
132 ↗(On Diff #31676)

Do we need a new plugin option? Would it not be possible to determine whether ThinLTO is needed by reading the intermediate files?

tejohnson added inline comments.Aug 11 2015, 2:06 PM
tools/gold/gold-plugin.cpp
132 ↗(On Diff #31676)

Thanks for the suggestion. Yes, we could do that. It does mean (for the bitcode representation I proposed last week) that the MODULE_BLOCK would first need to be scanned for a THINLTO_BLOCK nested directly within it. I.e. each of the top level blocks and records nested directly within the module block need to be skipped through. So a bit more compile time. I'm not sure how much this matters in practice though.

I experimented with a large source file (>400K loc, >51K functions), and measured how long it took to skip all blocks/records and just check whether it contained a thinlto block. It was negligible (~0.1s), so I think this is the way to go. I will update the patch to remove the change to gold-plugin.cpp and the change to pass in the new plugin option.

tejohnson updated this revision to Diff 31865.Aug 12 2015, 6:29 AM
tejohnson retitled this revision from LLVM support for -fthinlto option, and gold support for thinlto plugin option. to LLVM support for -fthinlto option..
tejohnson updated this object.

Removed the gold plugin option.

It seems a bit intrusive to make the BitcodeWriter aware of LTO. It seems you want it to handle your specific information at the end of a module block right?
Can we have a generic hook mechanism? For instance, we might provide a std::function callback instead of this ad-hoc boolean.

include/llvm/Bitcode/BitcodeWriterPass.h
37

Update doxygen?

include/llvm/Bitcode/ReaderWriter.h
74

ditto.

tejohnson marked 2 inline comments as done.Aug 17 2015, 7:59 AM
In D11907#224874, @joker.eph wrote:

It seems a bit intrusive to make the BitcodeWriter aware of LTO. It seems you want it to handle your specific information at the end of a module block right?
Can we have a generic hook mechanism? For instance, we might provide a std::function callback instead of this ad-hoc boolean.

Interesting idea. The boolean is passed down from clang (based on the -fthinlto option) when it calls createBitcodeWriterPass. I'm trying to envision how a callback would work - I guess save the bool passed from clang on the WriteBitcodePass object (as it is already done in this patch), and then have a static WriteBitcodePass method that returns the bool value? But then we would need to pass this function into WriteBitcodeToFile as a std::function, and that would have to be passed down to WriteModule (instead of the bool) and invoked there. Am I missing a better way to do this via a callback? It seems like we end up either passing down a bool or a std::function callback.

include/llvm/Bitcode/BitcodeWriterPass.h
37

Will upload new patch in a min that updates doxygen.

tejohnson updated this revision to Diff 32299.Aug 17 2015, 8:01 AM
tejohnson marked an inline comment as done.

Updated doxygen as per review comments.