Bitcode: Introduce BitcodeWriter interface.

[pcc]

Summary

This interface allows clients to write multiple modules to a single
bitcode file. Also introduce the llvm-join utility which can be used
to create a bitcode file containing multiple modules.

[mehdi_amini]

It'd be nice to add the opposite feature of llvm-join to llvm-extract: being able to list/extract the modules in a single file.

[pcc]

Yes, that's blocked on an appropriate reader interface though (waiting on a response to my RFC).

[pcc]

By which I mean "RFC [Bitcode]: Moving block info block state"

[pcc]

Refresh

[mehdi_amini]

The name llvm-join is quite close from llvm-link, but I don't have a better suggestion right now.

llvm/test/Bitcode/multi-module.ll

6

Now that we have D26778 you could update this test I think!

[mehdi_amini]

llvm-combine, llvm-fuse, llvm-merge are alternative names I can think about in the same idea, but that does not distinguish from llvm-link either.
What about llvm-append?

[silvas]

Can we just add an llvm-bcutil and add subcommands to it? This could probably subsume llvm-extract and llvm-bcanalyzer someday too (and we could switch on argv[0] to imitate them if necessary). Maybe we could grow one of those two into llvm-bcutil?

[silvas]

Maybe something like
llvm-bcutil cat (or concat as peter suggested)
llvm-bcutil uncat
llvm-bcutil split-module (current llvm-split?)
llvm-bcutil extract-function (current llvm-extract?)
llvm-bcutil extract-module (for D26778/llvm-modextract?)
llvm-bcutil analyze (current llvm-bcanalyzer?)
llvm-bcutil link-modules (current llvm-link?)
...

We don't have to get there overnight, but having a nice place to put these things with low boilerplate would be good IMO. We shouldn't have significant overhead or boilerplate for adding testing utilities or utilities that are handy for us as developers.

[mehdi_amini]

In your list, there are some that are IR level tools, it seems strange to me to put them under a "bcutil" umbrella.

[silvas]

In D26179#605127, @mehdi_amini wrote:

In your list, there are some that are IR level tools, it seems strange to me to put them under a "bcutil" umbrella.

I thought about that, but I think the only one that doesn't actually materialize a Module in memory is llvm-bcanalyzer. So in some sense they are all about more than just bitcode. Also, colloquially, "bc file" is often used when IR is meant ("can you send me a bc file of that so I can debug the issue?"; often, textual IR would be fine in those contexts), so I don't think it's that confusing. I agree it's slightly confusing though.

[mehdi_amini]

I'd like llvm-modextract and llvm-join (or llvm-concat) to operate without materializing IR. It is blocked on updating the bitcode to not have absolute reference from the beginning of the file.

[silvas]

In D26179#605129, @mehdi_amini wrote:

I'd like llvm-modextract and llvm-join (or llvm-concat) to operate without materializing IR. It is blocked on updating the bitcode to not have absolute reference from the beginning of the file.

oh, nice!

[pcc]

• Change bitcode format to correspond with r288098

[pcc]

• http://clang.llvm.org/demo/cathead.png has decreed that the tool should be renamed llvm-cat

[mehdi_amini]

llvm/include/llvm/Bitcode/BitcodeWriter.h

35 ↗

(On Diff #79502)

\brief is only useful if there is more than one sentence.

llvm/tools/llvm-cat/llvm-cat.cpp

46 ↗

(On Diff #79502)

So, if I followed correctly, we should be able to do now as well:

{
  SmallVector<char, 0> Buffer;
  BitcodeWriter Writer(Buffer);
}
for (std::string InputFilename : InputFilenames) {
  std::unique_ptr<MemoryBuffer> MB =
      ExitOnErr(errorOrToExpected(MemoryBuffer::getFileOrSTDIN(InputFilename)));
  std::vector<BitcodeModule> Mods = ExitOnErr(getBitcodeModuleList(*MB));
  for (auto &BitcodeMod : Mods)  
    Buffer.insert(Buffer.end(), BitcodeMod.getBuffer().begin(), BitcodeMod.getBuffer().end());
}

Can we put this behind an option in llvm-cat? I'm not sure sure it is worth creating llvm-binarycat for this.

(Also it makes me notice that your current implementation wouldn't handle input files that contains multiple modules)

[pcc]

• Documentation tweak; llvm-cat -b considered harmless (for now)

[pcc]

llvm/tools/llvm-cat/llvm-cat.cpp

46 ↗

(On Diff #79502)

Seems reasonable enough for now. Let's split it out into llvm-bcutil binarycat once we get around to that though.

[mehdi_amini]

LGTM. Thanks!

[mehdi_amini]

(Make sure to update the commit message with llvm-cat instead of llvm-join)

llvm/include/llvm/Bitcode/BitstreamWriter.h

public:
   //===--------------------------------------------------------------------===//
   // BlockInfo Block Emission
   //===--------------------------------------------------------------------===//
 
   /// EnterBlockInfoBlock - Start emitting the BLOCKINFO_BLOCK.
   void EnterBlockInfoBlock() {
     EnterSubblock(bitc::BLOCKINFO_BLOCK_ID, 2);
     BlockInfoCurBID = ~0U;
+    BlockInfoRecords.clear();
   }
 private:
   /// SwitchToBlockID - If we aren't already talking about the specified block
   /// ID, emit a BLOCKINFO_CODE_SETBID record.
   void SwitchToBlockID(unsigned BlockID) {
     if (BlockInfoCurBID == BlockID) return;
     SmallVector<unsigned, 2> V;
     V.push_back(BlockID);

llvm/include/llvm/Bitcode/ReaderWriter.h

namespace llvm {
   hasGlobalValueSummary(MemoryBufferRef Buffer,
                         const DiagnosticHandlerFunction &DiagnosticHandler);
 
   /// Parse the specified bitcode buffer, returning the module summary index.
   ErrorOr<std::unique_ptr<ModuleSummaryIndex>>
   getModuleSummaryIndex(MemoryBufferRef Buffer,
                         const DiagnosticHandlerFunction &DiagnosticHandler);
 
+  class BitcodeWriter {
+    SmallVectorImpl<char> &Buffer;
+    std::unique_ptr<BitstreamWriter> Stream;
+    uint64_t BitcodeStartBit;
+
+   public:
+    /// Create a BitcodeWriter that writes to Buffer.
+    BitcodeWriter(SmallVectorImpl<char> &Buffer);
+
+    ~BitcodeWriter();
+
+    /// \brief Write the specified module to the buffer specified at
+    /// construction time.
+    ///
+    /// If \c ShouldPreserveUseListOrder, encode the use-list order for each \a
+    /// Value in \c M.  These will be reconstructed exactly when \a M is
+    /// deserialized.
+    ///
+    /// If \c Index is supplied, the bitcode will contain the summary index
+    /// (currently for use in ThinLTO optimization).
+    ///
+    /// \p GenerateHash enables hashing the Module and including the hash in the
+    /// bitcode (currently for use in ThinLTO incremental build).
+    void writeModule(const Module *M, bool ShouldPreserveUseListOrder = false,
+                     const ModuleSummaryIndex *Index = nullptr,
+                     bool GenerateHash = false);
+  };
+
   /// \brief Write the specified module to the specified raw output stream.
   ///
   /// For streams where it matters, the given stream should be in "binary"
   /// mode.
   ///
   /// If \c ShouldPreserveUseListOrder, encode the use-list order for each \a
   /// Value in \c M.  These will be reconstructed exactly when \a M is
   /// deserialized.

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

enum {
   FUNCTION_INST_CAST_ABBREV,
   FUNCTION_INST_RET_VOID_ABBREV,
   FUNCTION_INST_RET_VAL_ABBREV,
   FUNCTION_INST_UNREACHABLE_ABBREV,
   FUNCTION_INST_GEP_ABBREV,
 };
 
 /// Abstract class to manage the bitcode writing, subclassed for each bitcode
-/// file type. Owns the BitstreamWriter, and includes the main entry point for
-/// writing.
-class BitcodeWriter {
+/// file type.
+class BitcodeWriterBase {
 protected:
-  /// Pointer to the buffer allocated by caller for bitcode writing.
-  const SmallVectorImpl<char> &Buffer;
-
-  /// The stream created and owned by the BitodeWriter.
-  BitstreamWriter Stream;
+  /// The stream created and owned by the client.
+  BitstreamWriter &Stream;
 
   /// Saves the offset of the VSTOffset record that must eventually be
   /// backpatched with the offset of the actual VST.
   uint64_t VSTOffsetPlaceholder = 0;
 
 public:
-  /// Constructs a BitcodeWriter object, and initializes a BitstreamRecord,
-  /// writing to the provided \p Buffer.
-  BitcodeWriter(SmallVectorImpl<char> &Buffer)
-      : Buffer(Buffer), Stream(Buffer) {}
-
-  virtual ~BitcodeWriter() = default;
-
-  /// Main entry point to write the bitcode file, which writes the bitcode
-  /// header and will then invoke the virtual writeBlocks() method.
-  void write();
-
-private:
-  /// Derived classes must implement this to write the corresponding blocks for
-  /// that bitcode file type.
-  virtual void writeBlocks() = 0;
+  /// Constructs a BitcodeWriterBase object that writes to the provided
+  /// \p Stream.
+  BitcodeWriterBase(BitstreamWriter &Stream) : Stream(Stream) {}
 
 protected:
   bool hasVSTOffsetPlaceholder() { return VSTOffsetPlaceholder != 0; }
   void writeValueSymbolTableForwardDecl();
   void writeBitcodeHeader();
 };
 
 /// Class to manage the bitcode writing for a module.
-class ModuleBitcodeWriter : public BitcodeWriter {
+class ModuleBitcodeWriter : public BitcodeWriterBase {
+  /// Pointer to the buffer allocated by caller for bitcode writing.
+  const SmallVectorImpl<char> &Buffer;
+
   /// The Module to write to bitcode.
   const Module &M;
 
   /// Enumerates ids for all values in the module.
   ValueEnumerator VE;
 
   /// Optional per-module index to write for ThinLTO.
   const ModuleSummaryIndex *Index;
 
   /// True if a module hash record should be written.
   bool GenerateHash;
 
   /// The start bit of the module block, for use in generating a module hash
-  uint64_t BitcodeStartBit = 0;
+  uint64_t BitcodeStartBit;
 
   /// Map that holds the correspondence between GUIDs in the summary index,
   /// that came from indirect call profiles, and a value id generated by this
   /// class to use in the VST and summary block records.
   std::map<GlobalValue::GUID, unsigned> GUIDToValueIdMap;
 
   /// Tracks the last value id recorded in the GUIDToValueMap.
   unsigned GlobalValueId;
 
 public:
   /// Constructs a ModuleBitcodeWriter object for the given Module,
   /// writing to the provided \p Buffer.
   ModuleBitcodeWriter(const Module *M, SmallVectorImpl<char> &Buffer,
-                      bool ShouldPreserveUseListOrder,
-                      const ModuleSummaryIndex *Index, bool GenerateHash)
-      : BitcodeWriter(Buffer), M(*M), VE(*M, ShouldPreserveUseListOrder),
-        Index(Index), GenerateHash(GenerateHash) {
-    // Save the start bit of the actual bitcode, in case there is space
-    // saved at the start for the darwin header above. The reader stream
-    // will start at the bitcode, and we need the offset of the VST
-    // to line up.
-    BitcodeStartBit = Stream.GetCurrentBitNo();
-
+                      BitstreamWriter &Stream, bool ShouldPreserveUseListOrder,
+                      const ModuleSummaryIndex *Index, bool GenerateHash,
+                      uint64_t BitcodeStartBit)
+      : BitcodeWriterBase(Stream), Buffer(Buffer), M(*M),
+        VE(*M, ShouldPreserveUseListOrder), Index(Index),
+        GenerateHash(GenerateHash), BitcodeStartBit(BitcodeStartBit) {
     // Assign ValueIds to any callee values in the index that came from
     // indirect call profiles and were recorded as a GUID not a Value*
     // (which would have been assigned an ID by the ValueEnumerator).
     // The starting ValueId is just after the number of values in the
     // ValueEnumerator, so that they can be emitted in the VST.
     GlobalValueId = VE.getValues().size();
     if (!Index)
       return;
     for (const auto &GUIDSummaryLists : *Index)
       // Examine all summaries for this GUID.
       for (auto &Summary : GUIDSummaryLists.second)
         if (auto FS = dyn_cast<FunctionSummary>(Summary.get()))
           // For each call in the function summary, see if the call
           // is to a GUID (which means it is for an indirect call,
           // otherwise we would have a Value for it). If so, synthesize
           // a value id.
           for (auto &CallEdge : FS->calls())
             if (CallEdge.first.isGUID())
               assignValueId(CallEdge.first.getGUID());
   }
 
-private:
-  /// Main entry point for writing a module to bitcode, invoked by
-  /// BitcodeWriter::write() after it writes the header.
-  void writeBlocks() override;
-
-  /// Create the "IDENTIFICATION_BLOCK_ID" containing a single string with the
-  /// current llvm version, and a record for the epoch number.
-  void writeIdentificationBlock();
-
   /// Emit the current module to the bitstream.
-  void writeModule();
+  void write();
 
+private:
   uint64_t bitcodeStartBit() { return BitcodeStartBit; }
 
-  void writeStringRecord(unsigned Code, StringRef Str, unsigned AbbrevToUse);
   void writeAttributeGroupTable();
   void writeAttributeTable();
   void writeTypeTable();
   void writeComdats();
   void writeModuleInfo();
   void writeValueAsMetadata(const ValueAsMetadata *MD,
                             SmallVectorImpl<uint64_t> &Record);
   void writeMDTuple(const MDTuple *N, SmallVectorImpl<uint64_t> &Record,
unsigned getValueId(ValueInfo VI) {
     if (VI.isGUID())
       return getValueId(VI.getGUID());
     return VE.getValueID(VI.getValue());
   }
   std::map<GlobalValue::GUID, unsigned> &valueIds() { return GUIDToValueIdMap; }
 };
 
 /// Class to manage the bitcode writing for a combined index.
-class IndexBitcodeWriter : public BitcodeWriter {
+class IndexBitcodeWriter : public BitcodeWriterBase {
   /// The combined index to write to bitcode.
   const ModuleSummaryIndex &Index;
 
   /// When writing a subset of the index for distributed backends, client
   /// provides a map of modules to the corresponding GUIDs/summaries to write.
   const std::map<std::string, GVSummaryMapTy> *ModuleToSummariesForIndex;
 
   /// Map that holds the correspondence between the GUID used in the combined
   /// index and a value id generated by this class to use in references.
   std::map<GlobalValue::GUID, unsigned> GUIDToValueIdMap;
 
   /// Tracks the last value id recorded in the GUIDToValueMap.
   unsigned GlobalValueId = 0;
 
 public:
   /// Constructs a IndexBitcodeWriter object for the given combined index,
   /// writing to the provided \p Buffer. When writing a subset of the index
   /// for a distributed backend, provide a \p ModuleToSummariesForIndex map.
-  IndexBitcodeWriter(SmallVectorImpl<char> &Buffer,
-                     const ModuleSummaryIndex &Index,
+  IndexBitcodeWriter(BitstreamWriter &Stream, const ModuleSummaryIndex &Index,
                      const std::map<std::string, GVSummaryMapTy>
                          *ModuleToSummariesForIndex = nullptr)
-      : BitcodeWriter(Buffer), Index(Index),
+      : BitcodeWriterBase(Stream), Index(Index),
         ModuleToSummariesForIndex(ModuleToSummariesForIndex) {
     // Assign unique value ids to all summaries to be written, for use
     // in writing out the call graph edges. Save the mapping from GUID
     // to the new global value id to use when writing those edges, which
     // are currently saved in the index in terms of GUID.
     for (const auto &I : *this)
       GUIDToValueIdMap[I.first] = ++GlobalValueId;
   }
public:
     }
   };
 
   /// Obtain the start iterator over the summaries to be written.
   iterator begin() { return iterator(*this, /*IsAtEnd=*/false); }
   /// Obtain the end iterator over the summaries to be written.
   iterator end() { return iterator(*this, /*IsAtEnd=*/true); }
 
-private:
-  /// Main entry point for writing a combined index to bitcode, invoked by
-  /// BitcodeWriter::write() after it writes the header.
-  void writeBlocks() override;
+  /// Main entry point for writing a combined index to bitcode.
+  void write();
 
+private:
   void writeIndex();
   void writeModStrings();
   void writeCombinedValueSymbolTable();
   void writeCombinedGlobalValueSummary();
 
   /// Indicates whether the provided \p ModulePath should be written into
   /// the module string table, e.g. if full index written or if it is in
   /// the provided subset.

 static unsigned getEncodedSynchScope(SynchronizationScope SynchScope) {
   switch (SynchScope) {
   case SingleThread: return bitc::SYNCHSCOPE_SINGLETHREAD;
   case CrossThread: return bitc::SYNCHSCOPE_CROSSTHREAD;
   }
   llvm_unreachable("Invalid synch scope");
 }
 
-void ModuleBitcodeWriter::writeStringRecord(unsigned Code, StringRef Str,
-                                            unsigned AbbrevToUse) {
+static void writeStringRecord(BitstreamWriter &Stream, unsigned Code,
+                              StringRef Str, unsigned AbbrevToUse) {
   SmallVector<unsigned, 64> Vals;
 
   // Code: [strchar x N]
   for (unsigned i = 0, e = Str.size(); i != e; ++i) {
     if (AbbrevToUse && !BitCodeAbbrevOp::isChar6(Str[i]))
       AbbrevToUse = 0;
     Vals.push_back(Str[i]);
   }
case Type::StructTyID: {
           Code = bitc::TYPE_CODE_OPAQUE;
         } else {
           Code = bitc::TYPE_CODE_STRUCT_NAMED;
           AbbrevToUse = StructNamedAbbrev;
         }
 
         // Emit the name if it is present.
         if (!ST->getName().empty())
-          writeStringRecord(bitc::TYPE_CODE_STRUCT_NAME, ST->getName(),
+          writeStringRecord(Stream, bitc::TYPE_CODE_STRUCT_NAME, ST->getName(),
                             StructNameAbbrev);
       }
       break;
     }
     case Type::ArrayTyID: {
       ArrayType *AT = cast<ArrayType>(T);
       // ARRAY: [numelts, eltty]
       Code = bitc::TYPE_CODE_ARRAY;
for (const Comdat *C : VE.getComdats()) {
     Stream.EmitRecord(bitc::MODULE_CODE_COMDAT, Vals, /*AbbrevToUse=*/0);
     Vals.clear();
   }
 }
 
 /// Write a record that will eventually hold the word offset of the
 /// module-level VST. For now the offset is 0, which will be backpatched
 /// after the real VST is written. Saves the bit offset to backpatch.
-void BitcodeWriter::writeValueSymbolTableForwardDecl() {
+void BitcodeWriterBase::writeValueSymbolTableForwardDecl() {
   // Write a placeholder value in for the offset of the real VST,
   // which is written after the function blocks so that it can include
   // the offset of each function. The placeholder offset will be
   // updated when the real VST is written.
   BitCodeAbbrev *Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::MODULE_CODE_VSTOFFSET));
   // Blocks are 32-bit aligned, so we can use a 32-bit word offset to
   // hold the real VST offset. Must use fixed instead of VBR as we don't

 }
 
 /// Emit top-level description of module, including target triple, inline asm,
 /// descriptors for global variables, and function prototype info.
 /// Returns the bit offset to backpatch with the location of the real VST.
 void ModuleBitcodeWriter::writeModuleInfo() {
   // Emit various pieces of data attached to a module.
   if (!M.getTargetTriple().empty())
-    writeStringRecord(bitc::MODULE_CODE_TRIPLE, M.getTargetTriple(),
+    writeStringRecord(Stream, bitc::MODULE_CODE_TRIPLE, M.getTargetTriple(),
                       0 /*TODO*/);
   const std::string &DL = M.getDataLayoutStr();
   if (!DL.empty())
-    writeStringRecord(bitc::MODULE_CODE_DATALAYOUT, DL, 0 /*TODO*/);
+    writeStringRecord(Stream, bitc::MODULE_CODE_DATALAYOUT, DL, 0 /*TODO*/);
   if (!M.getModuleInlineAsm().empty())
-    writeStringRecord(bitc::MODULE_CODE_ASM, M.getModuleInlineAsm(),
+    writeStringRecord(Stream, bitc::MODULE_CODE_ASM, M.getModuleInlineAsm(),
                       0 /*TODO*/);
 
   // Emit information about sections and GC, computing how many there are. Also
   // compute the maximum alignment value.
   std::map<std::string, unsigned> SectionMap;
   std::map<std::string, unsigned> GCMap;
   unsigned MaxAlignment = 0;
   unsigned MaxGlobalType = 0;
   for (const GlobalValue &GV : M.globals()) {
     MaxAlignment = std::max(MaxAlignment, GV.getAlignment());
     MaxGlobalType = std::max(MaxGlobalType, VE.getTypeID(GV.getValueType()));
     if (GV.hasSection()) {
       // Give section names unique ID's.
       unsigned &Entry = SectionMap[GV.getSection()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_SECTIONNAME, GV.getSection(),
+        writeStringRecord(Stream, bitc::MODULE_CODE_SECTIONNAME, GV.getSection(),
                           0 /*TODO*/);
         Entry = SectionMap.size();
       }
     }
   }
   for (const Function &F : M) {
     MaxAlignment = std::max(MaxAlignment, F.getAlignment());
     if (F.hasSection()) {
       // Give section names unique ID's.
       unsigned &Entry = SectionMap[F.getSection()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_SECTIONNAME, F.getSection(),
+        writeStringRecord(Stream, bitc::MODULE_CODE_SECTIONNAME, F.getSection(),
                           0 /*TODO*/);
         Entry = SectionMap.size();
       }
     }
     if (F.hasGC()) {
       // Same for GC names.
       unsigned &Entry = GCMap[F.getGC()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_GCNAME, F.getGC(), 0 /*TODO*/);
+        writeStringRecord(Stream, bitc::MODULE_CODE_GCNAME, F.getGC(),
+                          0 /*TODO*/);
         Entry = GCMap.size();
       }
     }
   }
 
   // Emit abbrev for globals, now that we know # sections and max alignment.
   unsigned SimpleGVarAbbrev = 0;
   if (!M.global_empty()) {
for (auto *AS : Aliases) {
     Stream.EmitRecord(bitc::FS_COMBINED_ALIAS, NameVals, FSAliasAbbrev);
     NameVals.clear();
     MaybeEmitOriginalName(*AS);
   }
 
   Stream.ExitBlock();
 }
 
-void ModuleBitcodeWriter::writeIdentificationBlock() {
+/// Create the "IDENTIFICATION_BLOCK_ID" containing a single string with the
+/// current llvm version, and a record for the epoch number.
+void writeIdentificationBlock(BitstreamWriter &Stream) {
   Stream.EnterSubblock(bitc::IDENTIFICATION_BLOCK_ID, 5);
 
   // Write the "user readable" string identifying the bitcode producer
   BitCodeAbbrev *Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::IDENTIFICATION_CODE_STRING));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Array));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Char6));
   auto StringAbbrev = Stream.EmitAbbrev(Abbv);
-  writeStringRecord(bitc::IDENTIFICATION_CODE_STRING,
+  writeStringRecord(Stream, bitc::IDENTIFICATION_CODE_STRING,
                     "LLVM" LLVM_VERSION_STRING, StringAbbrev);
 
   // Write the epoch version
   Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::IDENTIFICATION_CODE_EPOCH));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));
   auto EpochAbbrev = Stream.EmitAbbrev(Abbv);
   SmallVector<unsigned, 1> Vals = {bitc::BITCODE_CURRENT_EPOCH};
SubHash |= LShift(Hash[Pos + 1], 16) | LShift(Hash[Pos + 2], 8) |
                (unsigned)(unsigned char)Hash[Pos + 3];
     Vals.push_back(SubHash);
   }
 
   // Emit the finished record.
   Stream.EmitRecord(bitc::MODULE_CODE_HASH, Vals);
 }
 
-void BitcodeWriter::write() {
-  // Emit the file header first.
-  writeBitcodeHeader();
-
-  writeBlocks();
-}
-
-void ModuleBitcodeWriter::writeBlocks() {
-  writeIdentificationBlock();
-  writeModule();
-}
-
-void IndexBitcodeWriter::writeBlocks() {
-  // Index contains only a single outer (module) block.
-  writeIndex();
-}
-
-void ModuleBitcodeWriter::writeModule() {
+void ModuleBitcodeWriter::write() {
   Stream.EnterSubblock(bitc::MODULE_BLOCK_ID, 3);
   size_t BlockStartPos = Buffer.size();
 
   SmallVector<unsigned, 1> Vals;
   unsigned CurVersion = 1;
   Vals.push_back(CurVersion);
   Stream.EmitRecord(bitc::MODULE_CODE_VERSION, Vals);
 
static void emitDarwinBCHeaderAndTrailer(SmallVectorImpl<char> &Buffer,
   writeInt32ToBuffer(CPUType, Buffer, Position);
 
   // If the file is not a multiple of 16 bytes, insert dummy padding.
   while (Buffer.size() & 15)
     Buffer.push_back(0);
 }
 
 /// Helper to write the header common to all bitcode files.
-void BitcodeWriter::writeBitcodeHeader() {
+static void writeBitcodeHeader(BitstreamWriter &Stream) {
   // Emit the file header.
   Stream.Emit((unsigned)'B', 8);
   Stream.Emit((unsigned)'C', 8);
   Stream.Emit(0x0, 4);
   Stream.Emit(0xC, 4);
   Stream.Emit(0xE, 4);
   Stream.Emit(0xD, 4);
 }
 
+BitcodeWriter::BitcodeWriter(SmallVectorImpl<char> &Buffer)
+    : Buffer(Buffer), Stream(new BitstreamWriter(Buffer)) {
+  // Save the start bit of the actual bitcode, in case there is space
+  // saved at the start for the darwin header above. The reader stream
+  // will start at the bitcode, and we need the offset of the VST
+  // to line up.
+  BitcodeStartBit = Stream->GetCurrentBitNo();
+
+  writeBitcodeHeader(*Stream);
+  writeIdentificationBlock(*Stream);
+}
+
+BitcodeWriter::~BitcodeWriter() = default;
+
+void BitcodeWriter::writeModule(const Module *M,
+                                bool ShouldPreserveUseListOrder,
+                                const ModuleSummaryIndex *Index,
+                                bool GenerateHash) {
+  ModuleBitcodeWriter ModuleWriter(M, Buffer, *Stream,
+                                   ShouldPreserveUseListOrder, Index,
+                                   GenerateHash, BitcodeStartBit);
+  ModuleWriter.write();
+}
+
 /// WriteBitcodeToFile - Write the specified module to the specified output
 /// stream.
 void llvm::WriteBitcodeToFile(const Module *M, raw_ostream &Out,
                               bool ShouldPreserveUseListOrder,
                               const ModuleSummaryIndex *Index,
                               bool GenerateHash) {
   SmallVector<char, 0> Buffer;
   Buffer.reserve(256*1024);
 
   // If this is darwin or another generic macho target, reserve space for the
   // header.
   Triple TT(M->getTargetTriple());
   if (TT.isOSDarwin() || TT.isOSBinFormatMachO())
     Buffer.insert(Buffer.begin(), BWH_HeaderSize, 0);
 
-  // Emit the module into the buffer.
-  ModuleBitcodeWriter ModuleWriter(M, Buffer, ShouldPreserveUseListOrder, Index,
-                                   GenerateHash);
-  ModuleWriter.write();
+  BitcodeWriter Writer(Buffer);
+  Writer.writeModule(M, ShouldPreserveUseListOrder, Index, GenerateHash);
 
   if (TT.isOSDarwin() || TT.isOSBinFormatMachO())
     emitDarwinBCHeaderAndTrailer(Buffer, TT);
 
   // Write the generated bitstream to "Out".
   Out.write((char*)&Buffer.front(), Buffer.size());
 }
 
-void IndexBitcodeWriter::writeIndex() {
+void IndexBitcodeWriter::write() {
   Stream.EnterSubblock(bitc::MODULE_BLOCK_ID, 3);
 
   SmallVector<unsigned, 1> Vals;
   unsigned CurVersion = 1;
   Vals.push_back(CurVersion);
   Stream.EmitRecord(bitc::MODULE_CODE_VERSION, Vals);
 
   // If we have a VST, write the VSTOFFSET record placeholder.

 // writing the combined index file for ThinLTO. When writing a subset of the
 // index for a distributed backend, provide a \p ModuleToSummariesForIndex map.
 void llvm::WriteIndexToFile(
     const ModuleSummaryIndex &Index, raw_ostream &Out,
     const std::map<std::string, GVSummaryMapTy> *ModuleToSummariesForIndex) {
   SmallVector<char, 0> Buffer;
   Buffer.reserve(256 * 1024);
 
-  IndexBitcodeWriter IndexWriter(Buffer, Index, ModuleToSummariesForIndex);
+  BitstreamWriter Stream(Buffer);
+  writeBitcodeHeader(Stream);
+
+  IndexBitcodeWriter IndexWriter(Stream, Index, ModuleToSummariesForIndex);
   IndexWriter.write();
 
   Out.write((char *)&Buffer.front(), Buffer.size());
 }

llvm/test/Bitcode/Inputs/multi-module.ll

+define void @f2() {
+  ret void
+}

llvm/test/Bitcode/multi-module.ll

+; RUN: llvm-join -o %t %s %S/Inputs/multi-module.ll
+; RUN: llvm-dis -o - %t | FileCheck %s
+
+; FIXME: Make llvm-bcanalyzer understand multiple block info blocks.
+; For now we can at least check that the bitcode contains multiple modules.
+; RUN: not llvm-bcanalyzer -dump %t | FileCheck --check-prefix=BCA %s
+
+; BCA: <MODULE_BLOCK
+; BCA: <MODULE_BLOCK
+
+; CHECK: define void @f1()
+define void @f1() {
+  ret void
+}

llvm/test/CMakeLists.txt

set(LLVM_TEST_DEPENDS
           llvm-cov
           llvm-cxxdump
           llvm-diff
           llvm-dis
           llvm-dsymutil
           llvm-dwarfdump
           llvm-dwp
           llvm-extract
+          llvm-join
           llvm-lib
           llvm-link
           llvm-lto2
           llvm-mc
           llvm-mcmarkup
           llvm-nm
           llvm-objdump
           llvm-opt-report

llvm/tools/LLVMBuild.txt

  llvm-bcanalyzer
  llvm-cov
  llvm-diff
  llvm-dis
  llvm-dwarfdump
  llvm-dwp
  llvm-extract
  llvm-jitlistener
+ llvm-join
  llvm-link
  llvm-lto
  llvm-mc
  llvm-mcmarkup
  llvm-nm
  llvm-objdump
  llvm-pdbdump
  llvm-profdata

llvm/tools/llvm-join/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  IRReader
+  BitWriter
+  Core
+  Support
+  )
+
+add_llvm_tool(llvm-join
+  llvm-join.cpp
+  )

llvm/tools/llvm-join/LLVMBuild.txt

-;===- ./tools/LLVMBuild.txt ------------------------------------*- Conf -*--===;
+;===- ./tools/llvm-join/LLVMBuild.txt --------------------------*- Conf -*--===;
 ;
 ;                     The LLVM Compiler Infrastructure
 ;
 ; This file is distributed under the University of Illinois Open Source
 ; License. See LICENSE.TXT for details.
 ;
 ;===------------------------------------------------------------------------===;
 ;
 ; This is an LLVMBuild description file for the components in this subdirectory.
 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
-[common]
-subdirectories =
- bugpoint
- dsymutil
- llc
- lli
- llvm-ar
- llvm-as
- llvm-bcanalyzer
- llvm-cov
- llvm-diff
- llvm-dis
- llvm-dwarfdump
- llvm-dwp
- llvm-extract
- llvm-jitlistener
- llvm-link
- llvm-lto
- llvm-mc
- llvm-mcmarkup
- llvm-nm
- llvm-objdump
- llvm-pdbdump
- llvm-profdata
- llvm-rtdyld
- llvm-size
- llvm-split
- opt
- verify-uselistorder
-
 [component_0]
-type = Group
-name = Tools
-parent = $ROOT
+type = Tool
+name = llvm-join
+parent = Tools
+required_libraries = AsmParser BitWriter

llvm/tools/llvm-join/llvm-join.cpp

+//===-- llvm-join.cpp - LLVM module join utility --------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This program is for testing features that rely on multi-module bitcode files.
+// It takes a list of input modules and uses them to create a multi-module
+// bitcode file.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/ReaderWriter.h"
+#include "llvm/IRReader/IRReader.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+
+using namespace llvm;
+
+static cl::opt<std::string> OutputFilename("o", cl::Required,
+                                           cl::desc("Output filename"),
+                                           cl::value_desc("filename"));
+
+static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
+                                            cl::desc("<input  files>"));
+
+int main(int argc, char **argv) {
+  cl::ParseCommandLineOptions(argc, argv, "Module joiner");
+
+  LLVMContext Context;
+
+  SmallVector<char, 0> Buffer;
+  BitcodeWriter Writer(Buffer);
+  for (std::string InputFilename : InputFilenames) {
+    SMDiagnostic Err;
+    std::unique_ptr<Module> M = parseIRFile(InputFilename, Err, Context);
+    if (!M) {
+      Err.print(argv[0], errs());
+      return 1;
+    }
+    Writer.writeModule(M.get());
+  }
+
+  std::error_code EC;
+  raw_fd_ostream OS(OutputFilename, EC, sys::fs::OpenFlags::F_None);
+  if (EC) {
+    llvm::errs() << argv[0] << ": cannot open " << OutputFilename
+                 << " for writing: " << EC.message();
+    return 1;
+  }
+
+  OS.write(Buffer.data(), Buffer.size());
+  return 0;
+}