LLVM support for -fthinlto option.

[tejohnson]

Summary

Add basic LLVM support for the -fthinlto option, which will
be passed to LLVM via the EmitThinLTOIndex code gen option. This will
be used to guard the ThinLTO bitcode writing in WriteModule shown in
http://reviews.llvm.org/D11722.

[pcc]

tools/gold/gold-plugin.cpp

132

Do we need a new plugin option? Would it not be possible to determine whether ThinLTO is needed by reading the intermediate files?

[tejohnson]

tools/gold/gold-plugin.cpp

132

Thanks for the suggestion. Yes, we could do that. It does mean (for the bitcode representation I proposed last week) that the MODULE_BLOCK would first need to be scanned for a THINLTO_BLOCK nested directly within it. I.e. each of the top level blocks and records nested directly within the module block need to be skipped through. So a bit more compile time. I'm not sure how much this matters in practice though.

I experimented with a large source file (>400K loc, >51K functions), and measured how long it took to skip all blocks/records and just check whether it contained a thinlto block. It was negligible (~0.1s), so I think this is the way to go. I will update the patch to remove the change to gold-plugin.cpp and the change to pass in the new plugin option.

[tejohnson]

Removed the gold plugin option.

[mehdi_amini]

It seems a bit intrusive to make the BitcodeWriter aware of LTO. It seems you want it to handle your specific information at the end of a module block right?
Can we have a generic hook mechanism? For instance, we might provide a std::function callback instead of this ad-hoc boolean.

include/llvm/Bitcode/BitcodeWriterPass.h

34

Update doxygen?

include/llvm/Bitcode/ReaderWriter.h

71

ditto.

[tejohnson]

In D11907#224874, @joker.eph wrote:

It seems a bit intrusive to make the BitcodeWriter aware of LTO. It seems you want it to handle your specific information at the end of a module block right?
Can we have a generic hook mechanism? For instance, we might provide a std::function callback instead of this ad-hoc boolean.

Interesting idea. The boolean is passed down from clang (based on the -fthinlto option) when it calls createBitcodeWriterPass. I'm trying to envision how a callback would work - I guess save the bool passed from clang on the WriteBitcodePass object (as it is already done in this patch), and then have a static WriteBitcodePass method that returns the bool value? But then we would need to pass this function into WriteBitcodeToFile as a std::function, and that would have to be passed down to WriteModule (instead of the bool) and invoked there. Am I missing a better way to do this via a callback? It seems like we end up either passing down a bool or a std::function callback.

include/llvm/Bitcode/BitcodeWriterPass.h

34

Will upload new patch in a min that updates doxygen.

[tejohnson]

Updated doxygen as per review comments.

[tejohnson]

Obsolete. New patch is http://reviews.llvm.org/D13107

include/llvm/Bitcode/BitcodeWriterPass.h

 
 /// \brief Create and return a pass that writes the module to the specified
 /// ostream. Note that this pass is designed for use with the legacy pass
 /// manager.
 ///
 /// If \c ShouldPreserveUseListOrder, encode use-list order so it can be
 /// reproduced when deserialized.
 ModulePass *createBitcodeWriterPass(raw_ostream &Str,
-                                    bool ShouldPreserveUseListOrder = false);
+                                    bool ShouldPreserveUseListOrder = false,
+                                    bool EmitThinLTOIndex = false);
 
 /// \brief Pass for writing a module of IR out to a bitcode file.
 ///
 /// Note that this is intended for use with the new pass manager. To construct
 /// a pass for the legacy pass manager, use the function above.
 class BitcodeWriterPass {
   raw_ostream &OS;
   bool ShouldPreserveUseListOrder;
+  bool EmitThinLTOIndex;
 
 public:
   /// \brief Construct a bitcode writer pass around a particular output stream.
   ///
   /// If \c ShouldPreserveUseListOrder, encode use-list order so it can be
   /// reproduced when deserialized.
   explicit BitcodeWriterPass(raw_ostream &OS,
-                             bool ShouldPreserveUseListOrder = false)
-      : OS(OS), ShouldPreserveUseListOrder(ShouldPreserveUseListOrder) {}
+                             bool ShouldPreserveUseListOrder = false,
+                             bool EmitThinLTOIndex = false)
+      : OS(OS), ShouldPreserveUseListOrder(ShouldPreserveUseListOrder),
+        EmitThinLTOIndex(EmitThinLTOIndex) {}
 
   /// \brief Run the bitcode writer pass, and output the module to the selected
   /// output stream.
   PreservedAnalyses run(Module &M);
 
   static StringRef name() { return "BitcodeWriterPass"; }
 };
 
 }
 
 #endif

include/llvm/Bitcode/ReaderWriter.h

namespace llvm {
   ///
   /// For streams where it matters, the given stream should be in "binary"
   /// mode.
   ///
   /// If \c ShouldPreserveUseListOrder, encode the use-list order for each \a
   /// Value in \c M.  These will be reconstructed exactly when \a M is
   /// deserialized.
   void WriteBitcodeToFile(const Module *M, raw_ostream &Out,
-                          bool ShouldPreserveUseListOrder = false);
+                          bool ShouldPreserveUseListOrder = false,
+                          bool EmitThinLTOIndex = false);
 
   /// isBitcodeWrapper - Return true if the given bytes are the magic bytes
   /// for an LLVM IR bitcode wrapper.
   ///
   inline bool isBitcodeWrapper(const unsigned char *BufPtr,
                                const unsigned char *BufEnd) {
     // See if you can find the hidden message in the magic bytes :-).
     // (Hint: it's a little-endian encoding.)

lib/Bitcode/Writer/BitcodeWriter.cpp

if (Stream.EmitBlockInfoAbbrev(bitc::FUNCTION_BLOCK_ID, Abbv) !=
       llvm_unreachable("Unexpected abbrev ordering!");
   }
 
   Stream.ExitBlock();
 }
 
 /// WriteModule - Emit the specified module to the bitstream.
 static void WriteModule(const Module *M, BitstreamWriter &Stream,
-                        bool ShouldPreserveUseListOrder) {
+                        bool ShouldPreserveUseListOrder,
+                        bool LLVM_ATTRIBUTE_UNUSED EmitThinLTOIndex) {
   Stream.EnterSubblock(bitc::MODULE_BLOCK_ID, 3);
 
   SmallVector<unsigned, 1> Vals;
   unsigned CurVersion = 1;
   Vals.push_back(CurVersion);
   Stream.EmitRecord(bitc::MODULE_CODE_VERSION, Vals);
 
   // Analyze the module, enumerating globals, functions, etc.
static void WriteModule(const Module *M, BitstreamWriter &Stream,
   if (VE.shouldPreserveUseListOrder())
     WriteUseListBlock(nullptr, VE, Stream);
 
   // Emit function bodies.
   for (Module::const_iterator F = M->begin(), E = M->end(); F != E; ++F)
     if (!F->isDeclaration())
       WriteFunction(*F, VE, Stream);
 
+  // TODO: If EmitThinLTOIndex, then emit the ThinLTO index in bitcode
+  // using information saved during function writing.
+
   Stream.ExitBlock();
 }
 
 /// EmitDarwinBCHeader - If generating a bc file on darwin, we have to emit a
 /// header and trailer to make it compatible with the system archiver.  To do
 /// this we emit the following header, and then emit a trailer that pads the
 /// file out to be a multiple of 16 bytes.
 ///
static void EmitDarwinBCHeaderAndTrailer(SmallVectorImpl<char> &Buffer,
   // If the file is not a multiple of 16 bytes, insert dummy padding.
   while (Buffer.size() & 15)
     Buffer.push_back(0);
 }
 
 /// WriteBitcodeToFile - Write the specified module to the specified output
 /// stream.
 void llvm::WriteBitcodeToFile(const Module *M, raw_ostream &Out,
-                              bool ShouldPreserveUseListOrder) {
+                              bool ShouldPreserveUseListOrder,
+                              bool EmitThinLTOIndex) {
   SmallVector<char, 0> Buffer;
   Buffer.reserve(256*1024);
 
   // If this is darwin or another generic macho target, reserve space for the
   // header.
   Triple TT(M->getTargetTriple());
   if (TT.isOSDarwin())
     Buffer.insert(Buffer.begin(), DarwinBCHeaderSize, 0);
 
   // Emit the module into the buffer.
   {
     BitstreamWriter Stream(Buffer);
 
     // Emit the file header.
     Stream.Emit((unsigned)'B', 8);
     Stream.Emit((unsigned)'C', 8);
     Stream.Emit(0x0, 4);
     Stream.Emit(0xC, 4);
     Stream.Emit(0xE, 4);
     Stream.Emit(0xD, 4);
 
     // Emit the module.
-    WriteModule(M, Stream, ShouldPreserveUseListOrder);
+    WriteModule(M, Stream, ShouldPreserveUseListOrder, EmitThinLTOIndex);
   }
 
   if (TT.isOSDarwin())
     EmitDarwinBCHeaderAndTrailer(Buffer, TT);
 
   // Write the generated bitstream to "Out".
   Out.write((char*)&Buffer.front(), Buffer.size());
 }

lib/Bitcode/Writer/BitcodeWriterPass.cpp

 #include "llvm/Bitcode/BitcodeWriterPass.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/Pass.h"
 using namespace llvm;
 
 PreservedAnalyses BitcodeWriterPass::run(Module &M) {
-  WriteBitcodeToFile(&M, OS, ShouldPreserveUseListOrder);
+  WriteBitcodeToFile(&M, OS, ShouldPreserveUseListOrder, EmitThinLTOIndex);
   return PreservedAnalyses::all();
 }
 
 namespace {
   class WriteBitcodePass : public ModulePass {
     raw_ostream &OS; // raw_ostream to print on
     bool ShouldPreserveUseListOrder;
+    bool EmitThinLTOIndex;
 
   public:
     static char ID; // Pass identification, replacement for typeid
-    explicit WriteBitcodePass(raw_ostream &o, bool ShouldPreserveUseListOrder)
+    explicit WriteBitcodePass(raw_ostream &o, bool ShouldPreserveUseListOrder,
+                              bool EmitThinLTOIndex)
         : ModulePass(ID), OS(o),
-          ShouldPreserveUseListOrder(ShouldPreserveUseListOrder) {}
+          ShouldPreserveUseListOrder(ShouldPreserveUseListOrder),
+          EmitThinLTOIndex(EmitThinLTOIndex) {}
 
     const char *getPassName() const override { return "Bitcode Writer"; }
 
     bool runOnModule(Module &M) override {
-      WriteBitcodeToFile(&M, OS, ShouldPreserveUseListOrder);
+      WriteBitcodeToFile(&M, OS, ShouldPreserveUseListOrder, EmitThinLTOIndex);
       return false;
     }
   };
 }
 
 char WriteBitcodePass::ID = 0;
 
 ModulePass *llvm::createBitcodeWriterPass(raw_ostream &Str,
-                                          bool ShouldPreserveUseListOrder) {
-  return new WriteBitcodePass(Str, ShouldPreserveUseListOrder);
+                                          bool ShouldPreserveUseListOrder,
+                                          bool EmitThinLTOIndex) {
+  return new WriteBitcodePass(Str, ShouldPreserveUseListOrder,
+                              EmitThinLTOIndex);
 }

tools/gold/gold-plugin.cpp

namespace options {
   };
   static bool generate_api_file = false;
   static OutputType TheOutputType = OT_NORMAL;
   static unsigned OptLevel = 2;
   static std::string obj_path;
   static std::string extra_library_path;
   static std::string triple;
   static std::string mcpu;
+  // TODO: When the thinlto plugin option is specified, only read
+  // the ThinLTO information from intermediate files and write a combined
+  // ThinLTO global index.
+  static bool thinlto = false;
   // Additional options to pass into the code generator.
   // Note: This array will contain all plugin options which are not claimed
   // as plugin exclusive to pass to the code generator.
   // For example, "generate-api-file" and "as"options are for the plugin
   // use only and will not be passed.
   static std::vector<const char *> extra;
 
   static void process_plugin_option(const char *opt_)
static void process_plugin_option(const char *opt_)
     } else if (opt.startswith("obj-path=")) {
       obj_path = opt.substr(strlen("obj-path="));
     } else if (opt == "emit-llvm") {
       TheOutputType = OT_BC_ONLY;
     } else if (opt == "save-temps") {
       TheOutputType = OT_SAVE_TEMPS;
     } else if (opt == "disable-output") {
       TheOutputType = OT_DISABLE;
+    } else if (opt == "thinlto") {
+      thinlto = true;
     } else if (opt.size() == 2 && opt[0] == 'O') {
       if (opt[1] < '0' || opt[1] > '3')
         report_fatal_error("Optimization level must be between 0 and 3");
       OptLevel = opt[1] - '0';
     } else {
       // Save this option to pass to the code generator.
       // ParseCommandLineOptions() expects argv[0] to be program name. Lazily
       // add that.