Bitcode: Introduce BitcodeWriter interface.

[pcc]

Summary

This interface allows clients to write multiple modules to a single
bitcode file. Also introduce the llvm-join utility which can be used
to create a bitcode file containing multiple modules.

[mehdi_amini]

It'd be nice to add the opposite feature of llvm-join to llvm-extract: being able to list/extract the modules in a single file.

[pcc]

Yes, that's blocked on an appropriate reader interface though (waiting on a response to my RFC).

[pcc]

By which I mean "RFC [Bitcode]: Moving block info block state"

[pcc]

Refresh

[mehdi_amini]

The name llvm-join is quite close from llvm-link, but I don't have a better suggestion right now.

llvm/test/Bitcode/multi-module.ll

6

Now that we have D26778 you could update this test I think!

[mehdi_amini]

llvm-combine, llvm-fuse, llvm-merge are alternative names I can think about in the same idea, but that does not distinguish from llvm-link either.
What about llvm-append?

[silvas]

Can we just add an llvm-bcutil and add subcommands to it? This could probably subsume llvm-extract and llvm-bcanalyzer someday too (and we could switch on argv[0] to imitate them if necessary). Maybe we could grow one of those two into llvm-bcutil?

[silvas]

Maybe something like
llvm-bcutil cat (or concat as peter suggested)
llvm-bcutil uncat
llvm-bcutil split-module (current llvm-split?)
llvm-bcutil extract-function (current llvm-extract?)
llvm-bcutil extract-module (for D26778/llvm-modextract?)
llvm-bcutil analyze (current llvm-bcanalyzer?)
llvm-bcutil link-modules (current llvm-link?)
...

We don't have to get there overnight, but having a nice place to put these things with low boilerplate would be good IMO. We shouldn't have significant overhead or boilerplate for adding testing utilities or utilities that are handy for us as developers.

[mehdi_amini]

In your list, there are some that are IR level tools, it seems strange to me to put them under a "bcutil" umbrella.

[silvas]

In D26179#605127, @mehdi_amini wrote:

In your list, there are some that are IR level tools, it seems strange to me to put them under a "bcutil" umbrella.

I thought about that, but I think the only one that doesn't actually materialize a Module in memory is llvm-bcanalyzer. So in some sense they are all about more than just bitcode. Also, colloquially, "bc file" is often used when IR is meant ("can you send me a bc file of that so I can debug the issue?"; often, textual IR would be fine in those contexts), so I don't think it's that confusing. I agree it's slightly confusing though.

[mehdi_amini]

I'd like llvm-modextract and llvm-join (or llvm-concat) to operate without materializing IR. It is blocked on updating the bitcode to not have absolute reference from the beginning of the file.

[silvas]

In D26179#605129, @mehdi_amini wrote:

I'd like llvm-modextract and llvm-join (or llvm-concat) to operate without materializing IR. It is blocked on updating the bitcode to not have absolute reference from the beginning of the file.

oh, nice!

[pcc]

• Change bitcode format to correspond with r288098

[pcc]

• http://clang.llvm.org/demo/cathead.png has decreed that the tool should be renamed llvm-cat

[mehdi_amini]

llvm/include/llvm/Bitcode/BitcodeWriter.h

35

\brief is only useful if there is more than one sentence.

llvm/tools/llvm-cat/llvm-cat.cpp

47

So, if I followed correctly, we should be able to do now as well:

{
  SmallVector<char, 0> Buffer;
  BitcodeWriter Writer(Buffer);
}
for (std::string InputFilename : InputFilenames) {
  std::unique_ptr<MemoryBuffer> MB =
      ExitOnErr(errorOrToExpected(MemoryBuffer::getFileOrSTDIN(InputFilename)));
  std::vector<BitcodeModule> Mods = ExitOnErr(getBitcodeModuleList(*MB));
  for (auto &BitcodeMod : Mods)  
    Buffer.insert(Buffer.end(), BitcodeMod.getBuffer().begin(), BitcodeMod.getBuffer().end());
}

Can we put this behind an option in llvm-cat? I'm not sure sure it is worth creating llvm-binarycat for this.

(Also it makes me notice that your current implementation wouldn't handle input files that contains multiple modules)

[pcc]

• Documentation tweak; llvm-cat -b considered harmless (for now)

[pcc]

llvm/tools/llvm-cat/llvm-cat.cpp

47

Seems reasonable enough for now. Let's split it out into llvm-bcutil binarycat once we get around to that though.

[mehdi_amini]

LGTM. Thanks!

[mehdi_amini]

(Make sure to update the commit message with llvm-cat instead of llvm-join)

llvm/include/llvm/Bitcode/BitcodeReader.h

class BitcodeModule {
     friend Expected<std::vector<BitcodeModule>>
     getBitcodeModuleList(MemoryBufferRef Buffer);
 
     Expected<std::unique_ptr<Module>>
     getModuleImpl(LLVMContext &Context, bool MaterializeAll,
                   bool ShouldLazyLoadMetadata);
 
   public:
+    ArrayRef<uint8_t> getBuffer() const { return Buffer; }
+
     /// Read the bitcode module and prepare for lazy deserialization of function
     /// bodies. If ShouldLazyLoadMetadata is true, lazily load metadata as well.
     Expected<std::unique_ptr<Module>>
     getLazyModule(LLVMContext &Context, bool ShouldLazyLoadMetadata);
 
     /// Read the entire bitcode module and return it.
     Expected<std::unique_ptr<Module>> parseModule(LLVMContext &Context);
   };

llvm/include/llvm/Bitcode/BitcodeWriter.h

 
 #ifndef LLVM_BITCODE_BITCODEWRITER_H
 #define LLVM_BITCODE_BITCODEWRITER_H
 
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include <string>
 
 namespace llvm {
+  class BitstreamWriter;
   class Module;
   class raw_ostream;
 
+  class BitcodeWriter {
+    SmallVectorImpl<char> &Buffer;
+    std::unique_ptr<BitstreamWriter> Stream;
+
+   public:
+    /// Create a BitcodeWriter that writes to Buffer.
+    BitcodeWriter(SmallVectorImpl<char> &Buffer);
+
+    ~BitcodeWriter();
+
+    /// Write the specified module to the buffer specified at construction time.
+    ///
+    /// If \c ShouldPreserveUseListOrder, encode the use-list order for each \a
+    /// Value in \c M.  These will be reconstructed exactly when \a M is
+    /// deserialized.
+    ///
+    /// If \c Index is supplied, the bitcode will contain the summary index
+    /// (currently for use in ThinLTO optimization).
+    ///
+    /// \p GenerateHash enables hashing the Module and including the hash in the
+    /// bitcode (currently for use in ThinLTO incremental build).
+    void writeModule(const Module *M, bool ShouldPreserveUseListOrder = false,
+                     const ModuleSummaryIndex *Index = nullptr,
+                     bool GenerateHash = false);
+  };
+
   /// \brief Write the specified module to the specified raw output stream.
   ///
   /// For streams where it matters, the given stream should be in "binary"
   /// mode.
   ///
   /// If \c ShouldPreserveUseListOrder, encode the use-list order for each \a
   /// Value in \c M.  These will be reconstructed exactly when \a M is
   /// deserialized.

llvm/include/llvm/Bitcode/BitstreamWriter.h

public:
   //===--------------------------------------------------------------------===//
   // BlockInfo Block Emission
   //===--------------------------------------------------------------------===//
 
   /// EnterBlockInfoBlock - Start emitting the BLOCKINFO_BLOCK.
   void EnterBlockInfoBlock() {
     EnterSubblock(bitc::BLOCKINFO_BLOCK_ID, 2);
     BlockInfoCurBID = ~0U;
+    BlockInfoRecords.clear();
   }
 private:
   /// SwitchToBlockID - If we aren't already talking about the specified block
   /// ID, emit a BLOCKINFO_CODE_SETBID record.
   void SwitchToBlockID(unsigned BlockID) {
     if (BlockInfoCurBID == BlockID) return;
     SmallVector<unsigned, 2> V;
     V.push_back(BlockID);

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

enum {
   FUNCTION_INST_CAST_ABBREV,
   FUNCTION_INST_RET_VOID_ABBREV,
   FUNCTION_INST_RET_VAL_ABBREV,
   FUNCTION_INST_UNREACHABLE_ABBREV,
   FUNCTION_INST_GEP_ABBREV,
 };
 
 /// Abstract class to manage the bitcode writing, subclassed for each bitcode
-/// file type. Owns the BitstreamWriter, and includes the main entry point for
-/// writing.
-class BitcodeWriter {
+/// file type.
+class BitcodeWriterBase {
 protected:
-  /// Pointer to the buffer allocated by caller for bitcode writing.
-  const SmallVectorImpl<char> &Buffer;
-
-  /// The stream created and owned by the BitodeWriter.
-  BitstreamWriter Stream;
+  /// The stream created and owned by the client.
+  BitstreamWriter &Stream;
 
   /// Saves the offset of the VSTOffset record that must eventually be
   /// backpatched with the offset of the actual VST.
   uint64_t VSTOffsetPlaceholder = 0;
 
 public:
-  /// Constructs a BitcodeWriter object, and initializes a BitstreamRecord,
-  /// writing to the provided \p Buffer.
-  BitcodeWriter(SmallVectorImpl<char> &Buffer)
-      : Buffer(Buffer), Stream(Buffer) {}
-
-  virtual ~BitcodeWriter() = default;
-
-  /// Main entry point to write the bitcode file, which writes the bitcode
-  /// header and will then invoke the virtual writeBlocks() method.
-  void write();
-
-private:
-  /// Derived classes must implement this to write the corresponding blocks for
-  /// that bitcode file type.
-  virtual void writeBlocks() = 0;
+  /// Constructs a BitcodeWriterBase object that writes to the provided
+  /// \p Stream.
+  BitcodeWriterBase(BitstreamWriter &Stream) : Stream(Stream) {}
 
 protected:
   bool hasVSTOffsetPlaceholder() { return VSTOffsetPlaceholder != 0; }
   void writeValueSymbolTableForwardDecl();
   void writeBitcodeHeader();
 };
 
 /// Class to manage the bitcode writing for a module.
-class ModuleBitcodeWriter : public BitcodeWriter {
+class ModuleBitcodeWriter : public BitcodeWriterBase {
+  /// Pointer to the buffer allocated by caller for bitcode writing.
+  const SmallVectorImpl<char> &Buffer;
+
   /// The Module to write to bitcode.
   const Module &M;
 
   /// Enumerates ids for all values in the module.
   ValueEnumerator VE;
 
   /// Optional per-module index to write for ThinLTO.
   const ModuleSummaryIndex *Index;
 
   /// True if a module hash record should be written.
   bool GenerateHash;
 
-  /// The start bit of the module block, for use in generating a module hash
-  uint64_t BitcodeStartBit = 0;
+  /// The start bit of the identification block.
+  uint64_t BitcodeStartBit;
 
   /// Map that holds the correspondence between GUIDs in the summary index,
   /// that came from indirect call profiles, and a value id generated by this
   /// class to use in the VST and summary block records.
   std::map<GlobalValue::GUID, unsigned> GUIDToValueIdMap;
 
   /// Tracks the last value id recorded in the GUIDToValueMap.
   unsigned GlobalValueId;
 
 public:
   /// Constructs a ModuleBitcodeWriter object for the given Module,
   /// writing to the provided \p Buffer.
   ModuleBitcodeWriter(const Module *M, SmallVectorImpl<char> &Buffer,
-                      bool ShouldPreserveUseListOrder,
+                      BitstreamWriter &Stream, bool ShouldPreserveUseListOrder,
                       const ModuleSummaryIndex *Index, bool GenerateHash)
-      : BitcodeWriter(Buffer), M(*M), VE(*M, ShouldPreserveUseListOrder),
-        Index(Index), GenerateHash(GenerateHash) {
-    // Save the start bit of the actual bitcode, in case there is space
-    // saved at the start for the darwin header above. The reader stream
-    // will start at the bitcode, and we need the offset of the VST
-    // to line up.
-    BitcodeStartBit = Stream.GetCurrentBitNo();
-
+      : BitcodeWriterBase(Stream), Buffer(Buffer), M(*M),
+        VE(*M, ShouldPreserveUseListOrder), Index(Index),
+        GenerateHash(GenerateHash), BitcodeStartBit(Stream.GetCurrentBitNo()) {
     // Assign ValueIds to any callee values in the index that came from
     // indirect call profiles and were recorded as a GUID not a Value*
     // (which would have been assigned an ID by the ValueEnumerator).
     // The starting ValueId is just after the number of values in the
     // ValueEnumerator, so that they can be emitted in the VST.
     GlobalValueId = VE.getValues().size();
     if (!Index)
       return;
     for (const auto &GUIDSummaryLists : *Index)
       // Examine all summaries for this GUID.
       for (auto &Summary : GUIDSummaryLists.second)
         if (auto FS = dyn_cast<FunctionSummary>(Summary.get()))
           // For each call in the function summary, see if the call
           // is to a GUID (which means it is for an indirect call,
           // otherwise we would have a Value for it). If so, synthesize
           // a value id.
           for (auto &CallEdge : FS->calls())
             if (CallEdge.first.isGUID())
               assignValueId(CallEdge.first.getGUID());
   }
 
-private:
-  /// Main entry point for writing a module to bitcode, invoked by
-  /// BitcodeWriter::write() after it writes the header.
-  void writeBlocks() override;
-
-  /// Create the "IDENTIFICATION_BLOCK_ID" containing a single string with the
-  /// current llvm version, and a record for the epoch number.
-  void writeIdentificationBlock();
-
   /// Emit the current module to the bitstream.
-  void writeModule();
+  void write();
 
+private:
   uint64_t bitcodeStartBit() { return BitcodeStartBit; }
 
-  void writeStringRecord(unsigned Code, StringRef Str, unsigned AbbrevToUse);
   void writeAttributeGroupTable();
   void writeAttributeTable();
   void writeTypeTable();
   void writeComdats();
   void writeModuleInfo();
   void writeValueAsMetadata(const ValueAsMetadata *MD,
                             SmallVectorImpl<uint64_t> &Record);
   void writeMDTuple(const MDTuple *N, SmallVectorImpl<uint64_t> &Record,
unsigned getValueId(ValueInfo VI) {
     if (VI.isGUID())
       return getValueId(VI.getGUID());
     return VE.getValueID(VI.getValue());
   }
   std::map<GlobalValue::GUID, unsigned> &valueIds() { return GUIDToValueIdMap; }
 };
 
 /// Class to manage the bitcode writing for a combined index.
-class IndexBitcodeWriter : public BitcodeWriter {
+class IndexBitcodeWriter : public BitcodeWriterBase {
   /// The combined index to write to bitcode.
   const ModuleSummaryIndex &Index;
 
   /// When writing a subset of the index for distributed backends, client
   /// provides a map of modules to the corresponding GUIDs/summaries to write.
   const std::map<std::string, GVSummaryMapTy> *ModuleToSummariesForIndex;
 
   /// Map that holds the correspondence between the GUID used in the combined
   /// index and a value id generated by this class to use in references.
   std::map<GlobalValue::GUID, unsigned> GUIDToValueIdMap;
 
   /// Tracks the last value id recorded in the GUIDToValueMap.
   unsigned GlobalValueId = 0;
 
 public:
   /// Constructs a IndexBitcodeWriter object for the given combined index,
   /// writing to the provided \p Buffer. When writing a subset of the index
   /// for a distributed backend, provide a \p ModuleToSummariesForIndex map.
-  IndexBitcodeWriter(SmallVectorImpl<char> &Buffer,
-                     const ModuleSummaryIndex &Index,
+  IndexBitcodeWriter(BitstreamWriter &Stream, const ModuleSummaryIndex &Index,
                      const std::map<std::string, GVSummaryMapTy>
                          *ModuleToSummariesForIndex = nullptr)
-      : BitcodeWriter(Buffer), Index(Index),
+      : BitcodeWriterBase(Stream), Index(Index),
         ModuleToSummariesForIndex(ModuleToSummariesForIndex) {
     // Assign unique value ids to all summaries to be written, for use
     // in writing out the call graph edges. Save the mapping from GUID
     // to the new global value id to use when writing those edges, which
     // are currently saved in the index in terms of GUID.
     for (const auto &I : *this)
       GUIDToValueIdMap[I.first] = ++GlobalValueId;
   }
public:
     }
   };
 
   /// Obtain the start iterator over the summaries to be written.
   iterator begin() { return iterator(*this, /*IsAtEnd=*/false); }
   /// Obtain the end iterator over the summaries to be written.
   iterator end() { return iterator(*this, /*IsAtEnd=*/true); }
 
-private:
-  /// Main entry point for writing a combined index to bitcode, invoked by
-  /// BitcodeWriter::write() after it writes the header.
-  void writeBlocks() override;
+  /// Main entry point for writing a combined index to bitcode.
+  void write();
 
+private:
   void writeIndex();
   void writeModStrings();
   void writeCombinedValueSymbolTable();
   void writeCombinedGlobalValueSummary();
 
   /// Indicates whether the provided \p ModulePath should be written into
   /// the module string table, e.g. if full index written or if it is in
   /// the provided subset.

 static unsigned getEncodedSynchScope(SynchronizationScope SynchScope) {
   switch (SynchScope) {
   case SingleThread: return bitc::SYNCHSCOPE_SINGLETHREAD;
   case CrossThread: return bitc::SYNCHSCOPE_CROSSTHREAD;
   }
   llvm_unreachable("Invalid synch scope");
 }
 
-void ModuleBitcodeWriter::writeStringRecord(unsigned Code, StringRef Str,
-                                            unsigned AbbrevToUse) {
+static void writeStringRecord(BitstreamWriter &Stream, unsigned Code,
+                              StringRef Str, unsigned AbbrevToUse) {
   SmallVector<unsigned, 64> Vals;
 
   // Code: [strchar x N]
   for (unsigned i = 0, e = Str.size(); i != e; ++i) {
     if (AbbrevToUse && !BitCodeAbbrevOp::isChar6(Str[i]))
       AbbrevToUse = 0;
     Vals.push_back(Str[i]);
   }
case Type::StructTyID: {
           Code = bitc::TYPE_CODE_OPAQUE;
         } else {
           Code = bitc::TYPE_CODE_STRUCT_NAMED;
           AbbrevToUse = StructNamedAbbrev;
         }
 
         // Emit the name if it is present.
         if (!ST->getName().empty())
-          writeStringRecord(bitc::TYPE_CODE_STRUCT_NAME, ST->getName(),
+          writeStringRecord(Stream, bitc::TYPE_CODE_STRUCT_NAME, ST->getName(),
                             StructNameAbbrev);
       }
       break;
     }
     case Type::ArrayTyID: {
       ArrayType *AT = cast<ArrayType>(T);
       // ARRAY: [numelts, eltty]
       Code = bitc::TYPE_CODE_ARRAY;
for (const Comdat *C : VE.getComdats()) {
     Stream.EmitRecord(bitc::MODULE_CODE_COMDAT, Vals, /*AbbrevToUse=*/0);
     Vals.clear();
   }
 }
 
 /// Write a record that will eventually hold the word offset of the
 /// module-level VST. For now the offset is 0, which will be backpatched
 /// after the real VST is written. Saves the bit offset to backpatch.
-void BitcodeWriter::writeValueSymbolTableForwardDecl() {
+void BitcodeWriterBase::writeValueSymbolTableForwardDecl() {
   // Write a placeholder value in for the offset of the real VST,
   // which is written after the function blocks so that it can include
   // the offset of each function. The placeholder offset will be
   // updated when the real VST is written.
   BitCodeAbbrev *Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::MODULE_CODE_VSTOFFSET));
   // Blocks are 32-bit aligned, so we can use a 32-bit word offset to
   // hold the real VST offset. Must use fixed instead of VBR as we don't

 }
 
 /// Emit top-level description of module, including target triple, inline asm,
 /// descriptors for global variables, and function prototype info.
 /// Returns the bit offset to backpatch with the location of the real VST.
 void ModuleBitcodeWriter::writeModuleInfo() {
   // Emit various pieces of data attached to a module.
   if (!M.getTargetTriple().empty())
-    writeStringRecord(bitc::MODULE_CODE_TRIPLE, M.getTargetTriple(),
+    writeStringRecord(Stream, bitc::MODULE_CODE_TRIPLE, M.getTargetTriple(),
                       0 /*TODO*/);
   const std::string &DL = M.getDataLayoutStr();
   if (!DL.empty())
-    writeStringRecord(bitc::MODULE_CODE_DATALAYOUT, DL, 0 /*TODO*/);
+    writeStringRecord(Stream, bitc::MODULE_CODE_DATALAYOUT, DL, 0 /*TODO*/);
   if (!M.getModuleInlineAsm().empty())
-    writeStringRecord(bitc::MODULE_CODE_ASM, M.getModuleInlineAsm(),
+    writeStringRecord(Stream, bitc::MODULE_CODE_ASM, M.getModuleInlineAsm(),
                       0 /*TODO*/);
 
   // Emit information about sections and GC, computing how many there are. Also
   // compute the maximum alignment value.
   std::map<std::string, unsigned> SectionMap;
   std::map<std::string, unsigned> GCMap;
   unsigned MaxAlignment = 0;
   unsigned MaxGlobalType = 0;
   for (const GlobalValue &GV : M.globals()) {
     MaxAlignment = std::max(MaxAlignment, GV.getAlignment());
     MaxGlobalType = std::max(MaxGlobalType, VE.getTypeID(GV.getValueType()));
     if (GV.hasSection()) {
       // Give section names unique ID's.
       unsigned &Entry = SectionMap[GV.getSection()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_SECTIONNAME, GV.getSection(),
+        writeStringRecord(Stream, bitc::MODULE_CODE_SECTIONNAME, GV.getSection(),
                           0 /*TODO*/);
         Entry = SectionMap.size();
       }
     }
   }
   for (const Function &F : M) {
     MaxAlignment = std::max(MaxAlignment, F.getAlignment());
     if (F.hasSection()) {
       // Give section names unique ID's.
       unsigned &Entry = SectionMap[F.getSection()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_SECTIONNAME, F.getSection(),
+        writeStringRecord(Stream, bitc::MODULE_CODE_SECTIONNAME, F.getSection(),
                           0 /*TODO*/);
         Entry = SectionMap.size();
       }
     }
     if (F.hasGC()) {
       // Same for GC names.
       unsigned &Entry = GCMap[F.getGC()];
       if (!Entry) {
-        writeStringRecord(bitc::MODULE_CODE_GCNAME, F.getGC(), 0 /*TODO*/);
+        writeStringRecord(Stream, bitc::MODULE_CODE_GCNAME, F.getGC(),
+                          0 /*TODO*/);
         Entry = GCMap.size();
       }
     }
   }
 
   // Emit abbrev for globals, now that we know # sections and max alignment.
   unsigned SimpleGVarAbbrev = 0;
   if (!M.global_empty()) {
if (VST.empty()) {
     assert(!IsModuleLevel || !hasVSTOffsetPlaceholder());
     return;
   }
 
   if (IsModuleLevel && hasVSTOffsetPlaceholder()) {
     // Get the offset of the VST we are writing, and backpatch it into
     // the VST forward declaration record.
     uint64_t VSTOffset = Stream.GetCurrentBitNo();
-    // The BitcodeStartBit was the stream offset of the actual bitcode
-    // (e.g. excluding any initial darwin header).
+    // The BitcodeStartBit was the stream offset of the identification block.
     VSTOffset -= bitcodeStartBit();
     assert((VSTOffset & 31) == 0 && "VST block not 32-bit aligned");
-    Stream.BackpatchWord(VSTOffsetPlaceholder, VSTOffset / 32);
+    // Note that we add 1 here because the offset is relative to one word
+    // before the start of the identification block, which was historically
+    // always the start of the regular bitcode header.
+    Stream.BackpatchWord(VSTOffsetPlaceholder, VSTOffset / 32 + 1);
   }
 
   Stream.EnterSubblock(bitc::VALUE_SYMTAB_BLOCK_ID, 4);
 
   // For the module-level VST, add abbrev Ids for the VST_CODE_FNENTRY
   // records, which are not used in the per-function VSTs.
   unsigned FnEntry8BitAbbrev;
   unsigned FnEntry7BitAbbrev;
if (isa<BasicBlock>(Name.getValue())) {
       // contain any Function symbols.
       assert(FunctionToBitcodeIndex);
       assert(hasVSTOffsetPlaceholder());
 
       // Save the word offset of the function (from the start of the
       // actual bitcode written to the stream).
       uint64_t BitcodeIndex = (*FunctionToBitcodeIndex)[F] - bitcodeStartBit();
       assert((BitcodeIndex & 31) == 0 && "function block not 32-bit aligned");
-      NameVals.push_back(BitcodeIndex / 32);
+      // Note that we add 1 here because the offset is relative to one word
+      // before the start of the identification block, which was historically
+      // always the start of the regular bitcode header.
+      NameVals.push_back(BitcodeIndex / 32 + 1);
 
       Code = bitc::VST_CODE_FNENTRY;
       AbbrevToUse = FnEntry8BitAbbrev;
       if (Bits == SE_Char6)
         AbbrevToUse = FnEntry6BitAbbrev;
       else if (Bits == SE_Fixed7)
         AbbrevToUse = FnEntry7BitAbbrev;
     } else {
for (auto *AS : Aliases) {
     Stream.EmitRecord(bitc::FS_COMBINED_ALIAS, NameVals, FSAliasAbbrev);
     NameVals.clear();
     MaybeEmitOriginalName(*AS);
   }
 
   Stream.ExitBlock();
 }
 
-void ModuleBitcodeWriter::writeIdentificationBlock() {
+/// Create the "IDENTIFICATION_BLOCK_ID" containing a single string with the
+/// current llvm version, and a record for the epoch number.
+void writeIdentificationBlock(BitstreamWriter &Stream) {
   Stream.EnterSubblock(bitc::IDENTIFICATION_BLOCK_ID, 5);
 
   // Write the "user readable" string identifying the bitcode producer
   BitCodeAbbrev *Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::IDENTIFICATION_CODE_STRING));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Array));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Char6));
   auto StringAbbrev = Stream.EmitAbbrev(Abbv);
-  writeStringRecord(bitc::IDENTIFICATION_CODE_STRING,
+  writeStringRecord(Stream, bitc::IDENTIFICATION_CODE_STRING,
                     "LLVM" LLVM_VERSION_STRING, StringAbbrev);
 
   // Write the epoch version
   Abbv = new BitCodeAbbrev();
   Abbv->Add(BitCodeAbbrevOp(bitc::IDENTIFICATION_CODE_EPOCH));
   Abbv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));
   auto EpochAbbrev = Stream.EmitAbbrev(Abbv);
   SmallVector<unsigned, 1> Vals = {bitc::BITCODE_CURRENT_EPOCH};
void ModuleBitcodeWriter::writeModuleHash(size_t BlockStartPos) {
   for (int Pos = 0; Pos < 20; Pos += 4) {
     Vals[Pos / 4] = support::endian::read32be(Hash.data() + Pos);
   }
 
   // Emit the finished record.
   Stream.EmitRecord(bitc::MODULE_CODE_HASH, Vals);
 }
 
-void BitcodeWriter::write() {
-  // Emit the file header first.
-  writeBitcodeHeader();
-
-  writeBlocks();
-}
-
-void ModuleBitcodeWriter::writeBlocks() {
-  writeIdentificationBlock();
-  writeModule();
-}
-
-void IndexBitcodeWriter::writeBlocks() {
-  // Index contains only a single outer (module) block.
-  writeIndex();
-}
+void ModuleBitcodeWriter::write() {
+  writeIdentificationBlock(Stream);
 
-void ModuleBitcodeWriter::writeModule() {
   Stream.EnterSubblock(bitc::MODULE_BLOCK_ID, 3);
   size_t BlockStartPos = Buffer.size();
 
   SmallVector<unsigned, 1> Vals;
   unsigned CurVersion = 1;
   Vals.push_back(CurVersion);
   Stream.EmitRecord(bitc::MODULE_CODE_VERSION, Vals);
 
static void emitDarwinBCHeaderAndTrailer(SmallVectorImpl<char> &Buffer,
   writeInt32ToBuffer(CPUType, Buffer, Position);
 
   // If the file is not a multiple of 16 bytes, insert dummy padding.
   while (Buffer.size() & 15)
     Buffer.push_back(0);
 }
 
 /// Helper to write the header common to all bitcode files.
-void BitcodeWriter::writeBitcodeHeader() {
+static void writeBitcodeHeader(BitstreamWriter &Stream) {
   // Emit the file header.
   Stream.Emit((unsigned)'B', 8);
   Stream.Emit((unsigned)'C', 8);
   Stream.Emit(0x0, 4);
   Stream.Emit(0xC, 4);
   Stream.Emit(0xE, 4);
   Stream.Emit(0xD, 4);
 }
 
+BitcodeWriter::BitcodeWriter(SmallVectorImpl<char> &Buffer)
+    : Buffer(Buffer), Stream(new BitstreamWriter(Buffer)) {
+  writeBitcodeHeader(*Stream);
+}
+
+BitcodeWriter::~BitcodeWriter() = default;
+
+void BitcodeWriter::writeModule(const Module *M,
+                                bool ShouldPreserveUseListOrder,
+                                const ModuleSummaryIndex *Index,
+                                bool GenerateHash) {
+  ModuleBitcodeWriter ModuleWriter(
+      M, Buffer, *Stream, ShouldPreserveUseListOrder, Index, GenerateHash);
+  ModuleWriter.write();
+}
+
 /// WriteBitcodeToFile - Write the specified module to the specified output
 /// stream.
 void llvm::WriteBitcodeToFile(const Module *M, raw_ostream &Out,
                               bool ShouldPreserveUseListOrder,
                               const ModuleSummaryIndex *Index,
                               bool GenerateHash) {
   SmallVector<char, 0> Buffer;
   Buffer.reserve(256*1024);
 
   // If this is darwin or another generic macho target, reserve space for the
   // header.
   Triple TT(M->getTargetTriple());
   if (TT.isOSDarwin() || TT.isOSBinFormatMachO())
     Buffer.insert(Buffer.begin(), BWH_HeaderSize, 0);
 
-  // Emit the module into the buffer.
-  ModuleBitcodeWriter ModuleWriter(M, Buffer, ShouldPreserveUseListOrder, Index,
-                                   GenerateHash);
-  ModuleWriter.write();
+  BitcodeWriter Writer(Buffer);
+  Writer.writeModule(M, ShouldPreserveUseListOrder, Index, GenerateHash);
 
   if (TT.isOSDarwin() || TT.isOSBinFormatMachO())
     emitDarwinBCHeaderAndTrailer(Buffer, TT);
 
   // Write the generated bitstream to "Out".
   Out.write((char*)&Buffer.front(), Buffer.size());
 }
 
-void IndexBitcodeWriter::writeIndex() {
+void IndexBitcodeWriter::write() {
   Stream.EnterSubblock(bitc::MODULE_BLOCK_ID, 3);
 
   SmallVector<unsigned, 1> Vals;
   unsigned CurVersion = 1;
   Vals.push_back(CurVersion);
   Stream.EmitRecord(bitc::MODULE_CODE_VERSION, Vals);
 
   // If we have a VST, write the VSTOFFSET record placeholder.

 // writing the combined index file for ThinLTO. When writing a subset of the
 // index for a distributed backend, provide a \p ModuleToSummariesForIndex map.
 void llvm::WriteIndexToFile(
     const ModuleSummaryIndex &Index, raw_ostream &Out,
     const std::map<std::string, GVSummaryMapTy> *ModuleToSummariesForIndex) {
   SmallVector<char, 0> Buffer;
   Buffer.reserve(256 * 1024);
 
-  IndexBitcodeWriter IndexWriter(Buffer, Index, ModuleToSummariesForIndex);
+  BitstreamWriter Stream(Buffer);
+  writeBitcodeHeader(Stream);
+
+  IndexBitcodeWriter IndexWriter(Stream, Index, ModuleToSummariesForIndex);
   IndexWriter.write();
 
   Out.write((char *)&Buffer.front(), Buffer.size());
 }

llvm/test/Bitcode/Inputs/multi-module.ll

+define void @f2() {
+  ret void
+}

llvm/test/Bitcode/multi-module.ll

+; RUN: llvm-cat -o %t %s %S/Inputs/multi-module.ll
+; RUN: not llvm-dis -o - %t 2>&1 | FileCheck --check-prefix=ERROR %s
+; ERROR: Expected a single module
+
+; FIXME: Introduce a tool for extracting modules from bitcode and use it here.
+; For now we can at least check that the bitcode contains multiple modules.
+; RUN: llvm-bcanalyzer -dump %t | FileCheck --check-prefix=BCA %s
+
+; RUN: llvm-as -o %t1 %s
+; RUN: llvm-as -o %t2 %S/Inputs/multi-module.ll
+; RUN: llvm-cat -o %t %t1 %t2
+; RUN: not llvm-dis -o - %t 2>&1 | FileCheck --check-prefix=ERROR %s
+; RUN: llvm-bcanalyzer -dump %t | FileCheck --check-prefix=BCA %s
+
+; RUN: llvm-cat -b -o %t %t1 %t2
+; RUN: not llvm-dis -o - %t 2>&1 | FileCheck --check-prefix=ERROR %s
+; RUN: llvm-bcanalyzer -dump %t | FileCheck --check-prefix=BCA %s
+
+; RUN: llvm-cat -b -o %t3 %t %t
+; RUN: not llvm-dis -o - %t3 2>&1 | FileCheck --check-prefix=ERROR %s
+; RUN: llvm-bcanalyzer -dump %t3 | FileCheck --check-prefix=BCA4 %s
+
+; BCA: <IDENTIFICATION_BLOCK
+; BCA: <MODULE_BLOCK
+; BCA: <IDENTIFICATION_BLOCK
+; BCA: <MODULE_BLOCK
+
+; BCA4: <IDENTIFICATION_BLOCK
+; BCA4: <MODULE_BLOCK
+; BCA4: <IDENTIFICATION_BLOCK
+; BCA4: <MODULE_BLOCK
+; BCA4: <IDENTIFICATION_BLOCK
+; BCA4: <MODULE_BLOCK
+; BCA4: <IDENTIFICATION_BLOCK
+; BCA4: <MODULE_BLOCK
+
+define void @f1() {
+  ret void
+}

llvm/test/CMakeLists.txt

set(LLVM_TEST_DEPENDS
           count
           llc
           lli
           lli-child-target
           llvm-ar
           llvm-as
           llvm-bcanalyzer
           llvm-c-test
+          llvm-cat
           llvm-cxxfilt
           llvm-config
           llvm-cov
           llvm-cxxdump
           llvm-diff
           llvm-dis
           llvm-dsymutil
           llvm-dwarfdump

llvm/tools/LLVMBuild.txt

 subdirectories =
  bugpoint
  dsymutil
  llc
  lli
  llvm-ar
  llvm-as
  llvm-bcanalyzer
+ llvm-cat
  llvm-cov
  llvm-diff
  llvm-dis
  llvm-dwarfdump
  llvm-dwp
  llvm-extract
  llvm-jitlistener
  llvm-link

llvm/tools/llvm-cat/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  IRReader
+  BitWriter
+  Core
+  Support
+  )
+
+add_llvm_tool(llvm-cat
+  llvm-cat.cpp
+  )

llvm/tools/llvm-cat/LLVMBuild.txt

-;===- ./tools/LLVMBuild.txt ------------------------------------*- Conf -*--===;
+;===- ./tools/llvm-cat/LLVMBuild.txt ---------------------------*- Conf -*--===;
 ;
 ;                     The LLVM Compiler Infrastructure
 ;
 ; This file is distributed under the University of Illinois Open Source
 ; License. See LICENSE.TXT for details.
 ;
 ;===------------------------------------------------------------------------===;
 ;
 ; This is an LLVMBuild description file for the components in this subdirectory.
 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
-[common]
-subdirectories =
- bugpoint
- dsymutil
- llc
- lli
- llvm-ar
- llvm-as
- llvm-bcanalyzer
- llvm-cov
- llvm-diff
- llvm-dis
- llvm-dwarfdump
- llvm-dwp
- llvm-extract
- llvm-jitlistener
- llvm-link
- llvm-lto
- llvm-mc
- llvm-mcmarkup
- llvm-nm
- llvm-objdump
- llvm-pdbdump
- llvm-profdata
- llvm-rtdyld
- llvm-size
- llvm-split
- opt
- verify-uselistorder
-
 [component_0]
-type = Group
-name = Tools
-parent = $ROOT
+type = Tool
+name = llvm-cat
+parent = Tools
+required_libraries = AsmParser BitWriter

llvm/tools/llvm-cat/llvm-cat.cpp

+//===-- llvm-cat.cpp - LLVM module concatenation utility ------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This program is for testing features that rely on multi-module bitcode files.
+// It takes a list of input modules and uses them to create a multi-module
+// bitcode file.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeReader.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/IRReader/IRReader.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+
+using namespace llvm;
+
+static cl::opt<bool>
+    BinaryCat("b", cl::desc("Whether to perform binary concatenation"));
+
+static cl::opt<std::string> OutputFilename("o", cl::Required,
+                                           cl::desc("Output filename"),
+                                           cl::value_desc("filename"));
+
+static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
+                                            cl::desc("<input  files>"));
+
+int main(int argc, char **argv) {
+  cl::ParseCommandLineOptions(argc, argv, "Module concatenation");
+
+  ExitOnError ExitOnErr("llvm-cat: ");
+  LLVMContext Context;
+
+  SmallVector<char, 0> Buffer;
+  BitcodeWriter Writer(Buffer);
+  if (BinaryCat) {
+    for (std::string InputFilename : InputFilenames) {
+      std::unique_ptr<MemoryBuffer> MB = ExitOnErr(
+          errorOrToExpected(MemoryBuffer::getFileOrSTDIN(InputFilename)));
+      std::vector<BitcodeModule> Mods = ExitOnErr(getBitcodeModuleList(*MB));
+      for (auto &BitcodeMod : Mods)
+        Buffer.insert(Buffer.end(), BitcodeMod.getBuffer().begin(),
+                      BitcodeMod.getBuffer().end());
+    }
+  } else {
+    for (std::string InputFilename : InputFilenames) {
+      SMDiagnostic Err;
+      std::unique_ptr<Module> M = parseIRFile(InputFilename, Err, Context);
+      if (!M) {
+        Err.print(argv[0], errs());
+        return 1;
+      }
+      Writer.writeModule(M.get());
+    }
+  }
+
+  std::error_code EC;
+  raw_fd_ostream OS(OutputFilename, EC, sys::fs::OpenFlags::F_None);
+  if (EC) {
+    llvm::errs() << argv[0] << ": cannot open " << OutputFilename
+                 << " for writing: " << EC.message();
+    return 1;
+  }
+
+  OS.write(Buffer.data(), Buffer.size());
+  return 0;
+}