[Remarks] Add an LLVM-bitstream-based remark serializer

[thegameg]

Summary

Add a new serializer, using a binary format based on the LLVM bitstream format.

This format provides a way to serialize the remarks in two modes:

1. Separate mode: the metadata is separate from the remark entries.
2. Standalone mode: the metadata and the remark entries are in the same file.

The format contains:

• a meta block: container version, container type, string table, external file path, remark version
• a remark block: type, remark name, pass name, function name, debug file, debug line, debug column, hotness, arguments (key, value, debug file, debug line, debug column)

A string table is required for this format, which will be dumped in the meta block to be consumed before parsing the remark blocks.

On clang itself, we noticed a size reduction of 13.4x compared to YAML, and a compile-time reduction of between 1.7% and 3.5% on CTMark.

[paquette]

llvm/include/llvm/Remarks/BitstreamRemarkContainer.h

33–35

A couple things:

1. Is remark metadata explicitly documented anywhere? The comment in BitstreamRemarkSerializer.h seems to be about half way there. Maybe it would be good to expand a bit upon that, and then reference that file here? (I think it might be worth adding to the LLVM docs on remarks eventually.)

2. The "separate" point only talks about SeparateRemarksFile?

53

extra comma

56

Can we have a documentation comment on this?

64

A documentation comment would be useful here as well.

llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h

63–64

Is it possible to verify this with a test?

E.g. add a debug dumper for the available abbrev IDs and do something similar to llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir?

77

Grammar nit: Setup is a noun. I would use set up here. English is weird.

82

This, and the other ones should probably say "set up" like setupMetaBlockInfo

llvm/lib/Remarks/BitstreamRemarkSerializer.cpp

24–27

Whenever this function is called, there's a call to Bitstream.EmitRecord following it. Would it make sense to pull that into the function and give it a more descriptive name?

31–33

This is a pretty common pattern as well. Could this be pulled out into a function?

[thegameg]

llvm/include/llvm/Remarks/BitstreamRemarkContainer.h

33–35

1. Yes, I documented it right above the SeparateRemarksMeta enum entry. I agree that I should add something to the LLVM docs.
2. Actually, it only talks about SeparateRemarksMeta. I will make this more explicit, but the idea for the separate model is:
   • the meta part will eventually go into a section. It will contain one BLOCK_META with some records.
   • the remark part will go into a separate file (with its path saved in the meta part). It will also contain one BLOCK_META (for more versioning) and multiple BLOCK_REMARK blocks.

llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h

63–64

Nothing obvious comes to mind right now for iterating on the available abbrev IDs without clobbering the code with debug stuff. Technically, the BitstreamWriter will assert if the abbrev id's width is bigger than what specified when entering the block.

[thegameg]

• Address Jessica's comments.
• Rename BLOCK_* to *_BLOCK_ID.
• Teach llvm-bcanalyzer to recognize the magic number.
• Add a unit test to check for the values that shouldn't change without a version change.

[paquette]

llvm/lib/Remarks/BitstreamRemarkSerializer.cpp

343

This should only run when DidSetUp == true, correct?

Would it make sense to move DidSetUp into the Helper? Then you could set it in setupBlockInfo and either assert or throw an error if DidSetUp == false in appropriate Helper functions.

llvm/test/CodeGen/X86/remarks-bitstream.ll

11 ↗

(On Diff #206982)

Are these outputs guaranteed to be stable? Judging by the opt runline, it looks like this might rely on inlining behaviour + remark formatting?

llvm/tools/llvm-bcanalyzer/llvm-bcanalyzer.cpp

975 ↗

(On Diff #206982)

"Bitstream" seems a little redundant here. Just "LLVM Remarks" instead?

[thegameg]

llvm/lib/Remarks/BitstreamRemarkSerializer.cpp

343

Correct.

I wouldn't want to move DidSetUp into the Helper because I intend to use the Helper in other places, like the generation of the remark section where emitRemarkBlock won't be called.

Maybe I should just assert(DidSetUp == true) to avoid code getting in between the initialization and the emission of the remark?

llvm/test/CodeGen/X86/remarks-bitstream.ll

11 ↗

(On Diff #206982)

You're right, they rely on inlining behavior and on the emission of the remarks from the passes. Ideally we would have unit tests for this, but I'm afraid at that point we'll have to check for byte-by-byte correctness, while here we can sort of have a textual check.

Maybe we should move the whole analysis behavior from llvm-bcanalyzer to lib/Bitcode? That way we can have unit tests that come from actual Remark objects and can at least compare strings instead of raw bytes.

Let me know if you have any ideas for testing this differently.

[thegameg]

This adds a lot more testing.

1. BitstreamRemarksFormatTest.cpp: test for the stability of the versions and IDs.
2. BitstreamRemarksSerializerTest.cpp: don't depend on actual codegen to generate remarks. Manually create remark objects that get serialized then analyzed by the bitcode analyzer. This tests the multiple remark container types for the metadata blocks.
3. Gets rid of the codegen test: llvm/test/CodeGen/X86/remarks-bitstream.ll.

[thegameg]

Add documentation.

[paquette]

LGTM!

llvm/docs/Remarks.rst

``Basic options``
 
     .. option:: -pass-remarks-format=<format>
 
       Specifies the output format of the serialized remarks.
 
       Supported formats:
 
       * :ref:`yaml <yamlremarks>` (default)
+      * :ref:`bitstream <bitstreamremarks>`
 
 ``Content configuration``
 
     .. option:: -pass-remarks-filter=<regex>
 
       Only passes whose name match the given (POSIX) regular expression will be
       serialized to the final output.
 

 
 If a ``DebugLoc`` entry is specified within an ``arg`` entry, the following
 fields are required:
 
 * ``<arg-file>``
 * ``<arg-line>``
 * ``<arg-column>``
 
+.. _bitstreamremarks:
+
+LLVM bitstream remarks
+======================
+
+This format is using :doc:`LLVM bitstream <BitCodeFormat>` to serialize remarks
+and their associated metadata.
+
+A bitstream remark stream can be identified by the magic number ``"RMRK"`` that
+is placed at the very beginning.
+
+The format for serializing remarks is composed of two different block types:
+
+.. _bitstreamremarksmetablock:
+
+META_BLOCK
+----------
+
+The block providing information about the rest of the content in the stream.
+
+Exactly one block is expected. Having multiple metadata blocks is an error.
+
+This block can contain the following records:
+
+.. _bitstreamremarksrecordmetacontainerinfo:
+
+``RECORD_META_CONTAINER_INFO``
+
+    The container version and type.
+
+    Version: u32
+
+    Type:    u2
+
+.. _bitstreamremarksrecordmetaremarkversion:
+
+``RECORD_META_REMARK_VERSION``
+
+    The version of the remark entries. This can change independently from the
+    container version.
+
+    Version: u32
+
+.. _bitstreamremarksrecordmetastrtab:
+
+``RECORD_META_STRTAB``
+
+    The string table used by the remark entries. The format of the string table
+    is a sequence of strings separated by ``\0``.
+
+.. _bitstreamremarksrecordmetaexternalfile:
+
+``RECORD_META_EXTERNAL_FILE``
+
+    The external remark file path that contains the remark blocks associated
+    with this metadata. This is an absolute path.
+
+.. _bitstreamremarksremarkblock:
+
+REMARK_BLOCK
+------------
+
+The block describing a remark entry.
+
+0 or more blocks per file are allowed. Each block will depend on the
+:ref:`META_BLOCK <bitstreamremarksmetablock>` in order to be parsed correctly.
+
+This block can contain the following records:
+
+``RECORD_REMARK_HEADER``
+
+    The header of the remark. This contains all the mandatory information about
+    a remark.
+
+    +---------------+---------------------------+
+    | Type          | u3                        |
+    +---------------+---------------------------+
+    | Remark name   | VBR6 (string table index) |
+    +---------------+---------------------------+
+    | Pass name     | VBR6 (string table index) |
+    +---------------+---------------------------+
+    | Function name | VBR6 (string table index) |
+    +---------------+---------------------------+
+
+``RECORD_REMARK_DEBUG_LOC``
+
+    The source location for the corresponding remark. This record is optional.
+
+    +--------+---------------------------+
+    | File   | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Line   | u32                       |
+    +--------+---------------------------+
+    | Column | u32                       |
+    +--------+---------------------------+
+
+``RECORD_REMARK_HOTNESS``
+
+    The hotness of the remark. This record is optional.
+
+    +---------------+---------------------+
+    | Hotness | VBR8 (string table index) |
+    +---------------+---------------------+
+
+``RECORD_REMARK_ARG_WITH_DEBUGLOC``
+
+    A remark argument with an associated debug location.
+
+    +--------+---------------------------+
+    | Key    | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Value  | VBR7 (string table index) |
+    +--------+---------------------------+
+    | File   | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Line   | u32                       |
+    +--------+---------------------------+
+    | Column | u32                       |
+    +--------+---------------------------+
+
+``RECORD_REMARK_ARG_WITHOUT_DEBUGLOC``
+
+    A remark argument with an associated debug location.
+
+    +--------+---------------------------+
+    | Key    | VBR7 (string table index) |
+    +--------+---------------------------+
+    | Value  | VBR7 (string table index) |
+    +--------+---------------------------+
+
+The remark container
+--------------------
+
+Bitstream remarks are designed to be used in two different modes:
+
+``The separate mode``
+
+    The separate mode is the mode that is typically used during compilation. It
+    provides a way to serialize the remark entries to a stream while some
+    metadata is kept in memory to be emitted in the product of the compilation
+    (typically, an object file).
+
+``The standalone mode``
+
+    The standalone mode is typically stored and used after the distribution of
+    a program. It contains all the information that allows the parsing of all
+    the remarks without having any external dependencies.
+
+In order to support multiple modes, the format introduces the concept of a
+bitstream remark container type.
+
+.. _bitstreamremarksseparateremarksmeta:
+
+``SeparateRemarksMeta: the metadata emitted separately``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
+    * :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`
+
+    Typically, this is emitted in a section in the object files, allowing
+    clients to retrieve remarks and their associated metadata directly from
+    intermediate products.
+
+``SeparateRemarksFile: the remark entries emitted separately``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
+
+    This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+
+    Typically, this is emitted in a side-file alongside an object file, and is
+    made to be able to stream to without increasing the memory consumption of
+    the compiler. This is referenced by the :ref:`RECORD_META_EXTERNAL_FILE
+    <bitstreamremarksrecordmetaexternalfile>` entry in the
+    :ref:`SeparateRemarksMeta <bitstreamremarksseparateremarksmeta>` container.
+
+When the parser tries to parse a container that contains the metadata for the
+separate remarks, it should parse the version and type, then keep the string
+table in memory while opening the external file, validating its metadata and
+parsing the remark entries.
+
+The container versions from the separate container should match in order to
+have a well-formed file.
+
+``Standalone: the metadata and the remark entries emitted together``
+
+    This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:
+
+    * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`
+    * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`
+    * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`
+
+    This container type expects 0 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.
+
+A complete output of :program:`llvm-bcanalyzer` on the different container types:
+
+``SeparateRemarksMeta``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=13 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=5 op1=0/>
+      <String table codeid=3 abbrevid=5/> blob data = 'pass\\x00key\\x00value\\x00'
+      <External File codeid=4 abbrevid=6/> blob data = '/path/to/file/name'
+    </Meta>
+
+``SeparateRemarksFile``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=3 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=0 op1=1/>
+      <Remark version codeid=2 abbrevid=5 op0=0/>
+    </Meta>
+    <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
+      <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>
+      <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
+      <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
+      <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
+    </Remark>
+
+``Standalone``
+
+.. code-block:: none
+
+    <BLOCKINFO_BLOCK/>
+    <Meta BlockID=8 NumWords=15 BlockCodeSize=3>
+      <Container info codeid=1 abbrevid=4 op0=5 op1=2/>
+      <Remark version codeid=2 abbrevid=5 op0=30/>
+      <String table codeid=3 abbrevid=6/> blob data = 'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x00'
+    </Meta>
+    <Remark BlockID=9 NumWords=8 BlockCodeSize=4>
+      <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>
+      <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>
+      <Remark hotness codeid=7 abbrevid=6 op0=999999999/>
+      <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 op3=11 op4=66/>
+    </Remark>
+
 opt-viewer
 ==========
 
 The ``opt-viewer`` directory contains a collection of tools that visualize and
 summarize serialized remarks.
 
 .. _optviewerpy:
 

llvm/include/llvm/Bitcode/BitcodeAnalyzer.h

 namespace llvm {
 
 /// CurStreamTypeType - A type for CurStreamType
 enum CurStreamTypeType {
   UnknownBitstream,
   LLVMIRBitstream,
   ClangSerializedASTBitstream,
   ClangSerializedDiagnosticsBitstream,
+  LLVMBitstreamRemarks
 };
 
 struct BCDumpOptions {
   /// The stream.
   raw_ostream &OS;
   /// Print per-code histogram.
   bool Histogram = false;
   /// Don't emit numeric info in dump if symbolic info is available.

llvm/include/llvm/IR/RemarkStreamer.h

struct RemarkSetupPatternError : RemarkSetupErrorInfo<RemarkSetupPatternError> {
   using RemarkSetupErrorInfo<RemarkSetupPatternError>::RemarkSetupErrorInfo;
 };
 
 struct RemarkSetupFormatError : RemarkSetupErrorInfo<RemarkSetupFormatError> {
   static char ID;
   using RemarkSetupErrorInfo<RemarkSetupFormatError>::RemarkSetupErrorInfo;
 };
 
-enum class RemarksSerializerFormat { Unknown, YAML };
+enum class RemarksSerializerFormat { Unknown, YAML, Bitstream };
 
 Expected<RemarksSerializerFormat> parseSerializerFormat(StringRef Format);
 
 /// Setup optimization remarks.
 Expected<std::unique_ptr<ToolOutputFile>>
 setupOptimizationRemarks(LLVMContext &Context, StringRef RemarksFilename,
                          StringRef RemarksPasses, StringRef RemarksFormat,
                          bool RemarksWithHotness,
                          unsigned RemarksHotnessThreshold = 0);
 
 } // end namespace llvm
 
 #endif // LLVM_IR_REMARKSTREAMER_H

llvm/include/llvm/Remarks/BitstreamRemarkContainer.h

+//===-- BitstreamRemarkContainer.h - Container for remarks --------------*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides declarations for things used in the various types of
+// remark containers.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_REMARK_CONTAINER_H
+#define LLVM_REMARKS_REMARK_CONTAINER_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Bitstream/BitCodes.h"
+#include <cstdint>
+
+namespace llvm {
+namespace remarks {
+
+/// The current version of the remark container.
+/// Note: this is different from the version of the remark entry.
+constexpr uint64_t CurrentContainerVersion = 0;
+/// The magic number used for identifying remark blocks.
+constexpr StringRef Magic("RMRK", 4);
+
+/// Type of the remark container.
+/// The remark container has two modes:
+/// * separate: the metadata is separate from the remarks and points to the
+///             auxiliary file that contains the remarks.
+/// * standalone: the metadata and the remarks are emitted together.
+enum class BitstreamRemarkContainerType {
+  /// The metadata emitted separately.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * String table
+  /// * External file
+  SeparateRemarksMeta,
+  /// The remarks emitted separately.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * Remark version
+  SeparateRemarksFile,
+  /// Everything is emitted together.
+  /// This will contain the following:
+  /// * Container version and type
+  /// * Remark version
+  /// * String table
+  Standalone
+};
+
+/// The possible blocks that will be encountered in a bitstream remark
+/// container.
+enum BlockIDs {
+  /// The metadata block is mandatory. It should always come after the
+  /// BLOCKINFO_BLOCK, and contains metadata that should be used when parsing
+  /// REMARK_BLOCKs.
+  /// There should always be only one META_BLOCK.
+  META_BLOCK_ID = bitc::FIRST_APPLICATION_BLOCKID,
+  /// One remark entry is represented using a REMARK_BLOCK. There can be
+  /// multiple REMARK_BLOCKs in the same file.
+  REMARK_BLOCK_ID
+};
+
+constexpr StringRef MetaBlockName = StringRef("Meta", 4);
+constexpr StringRef RemarkBlockName = StringRef("Remark", 6);
+
+/// The possible records that can be encountered in the previously described
+/// blocks.
+enum RecordIDs {
+  // Meta block records.
+  RECORD_META_CONTAINER_INFO = 1,
+  RECORD_META_REMARK_VERSION,
+  RECORD_META_STRTAB,
+  RECORD_META_EXTERNAL_FILE,
+  // Remark block records.
+  RECORD_REMARK_HEADER,
+  RECORD_REMARK_DEBUG_LOC,
+  RECORD_REMARK_HOTNESS,
+  RECORD_REMARK_ARG_WITH_DEBUGLOC,
+  RECORD_REMARK_ARG_WITHOUT_DEBUGLOC,
+  // Helpers.
+  RECORD_FIRST = RECORD_META_CONTAINER_INFO,
+  RECORD_LAST = RECORD_REMARK_ARG_WITHOUT_DEBUGLOC
+};
+
+constexpr StringRef MetaContainerInfoName = StringRef("Container info", 14);
+constexpr StringRef MetaRemarkVersionName = StringRef("Remark version", 14);
+constexpr StringRef MetaStrTabName = StringRef("String table", 12);
+constexpr StringRef MetaExternalFileName = StringRef("External File", 13);
+constexpr StringRef RemarkHeaderName = StringRef("Remark header", 13);
+constexpr StringRef RemarkDebugLocName = StringRef("Remark debug location", 21);
+constexpr StringRef RemarkHotnessName = StringRef("Remark hotness", 14);
+constexpr StringRef RemarkArgWithDebugLocName =
+    StringRef("Argument with debug location", 28);
+constexpr StringRef RemarkArgWithoutDebugLocName = StringRef("Argument", 8);
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_REMARKS_REMARK_CONTAINER_H */

llvm/include/llvm/Remarks/BitstreamRemarkSerializer.h

+//===-- BitstreamRemarkSerializer.h - Bitstream serializer ------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides an implementation of the serializer using the LLVM
+// Bitstream format.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H
+#define LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H
+
+#include "llvm/Bitstream/BitstreamWriter.h"
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "llvm/Remarks/RemarkSerializer.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+namespace remarks {
+
+/// Serialize the remarks to LLVM bitstream.
+/// This class provides ways to emit remarks in the LLVM bitstream format and
+/// its associated metadata.
+///
+/// * The separate model:
+///   Separate meta:    | Container info
+///                     | String table
+///                     | External file
+///
+///   Separate remarks: | Container info
+///                     | Remark version
+///                     | Remark0
+///                     | Remark1
+///                     | Remark2
+///                     | ...
+///
+/// * The standalone model: | Container info
+///                         | String table
+///                         | Remark version
+///                         | Remark0
+///                         | Remark1
+///                         | Remark2
+///                         | ...
+///
+struct BitstreamSerializerHelper {
+  /// Buffer used for encoding the bitstream before writing it to the final
+  /// stream.
+  SmallVector<char, 1024> Encoded;
+  /// Buffer used to construct records and pass to the bitstream writer.
+  SmallVector<uint64_t, 64> R;
+  /// The Bitstream writer.
+  BitstreamWriter Bitstream;
+  /// The type of the container we are serializing.
+  BitstreamRemarkContainerType ContainerType;
+
+  /// Abbrev IDs initialized in the block info block.
+  /// Note: depending on the container type, some IDs might be uninitialized.
+  /// Warning: When adding more abbrev IDs, make sure to update the
+  /// BlockCodeSize (in the call to EnterSubblock).
+  uint64_t RecordMetaContainerInfoAbbrevID = 0;
+  uint64_t RecordMetaRemarkVersionAbbrevID = 0;
+  uint64_t RecordMetaStrTabAbbrevID = 0;
+  uint64_t RecordMetaExternalFileAbbrevID = 0;
+  uint64_t RecordRemarkHeaderAbbrevID = 0;
+  uint64_t RecordRemarkDebugLocAbbrevID = 0;
+  uint64_t RecordRemarkHotnessAbbrevID = 0;
+  uint64_t RecordRemarkArgWithDebugLocAbbrevID = 0;
+  uint64_t RecordRemarkArgWithoutDebugLocAbbrevID = 0;
+
+  BitstreamSerializerHelper(BitstreamRemarkContainerType ContainerType);
+
+  /// Set up the necessary block info entries according to the container type.
+  void setupBlockInfo();
+
+  /// Set up the block info for the metadata block.
+  void setupMetaBlockInfo();
+  /// The remark version in the metadata block.
+  void setupMetaRemarkVersion();
+  void emitMetaRemarkVersion(uint64_t RemarkVersion);
+  /// The strtab in the metadata block.
+  void setupMetaStrTab();
+  void emitMetaStrTab(const StringTable &StrTab);
+  /// The external file in the metadata block.
+  void setupMetaExternalFile();
+  void emitMetaExternalFile(StringRef Filename);
+
+  /// The block info for the remarks block.
+  void setupRemarkBlockInfo();
+
+  /// Emit the metadata for the remarks.
+  void emitMetaBlock(uint64_t ContainerVersion,
+                     Optional<uint64_t> RemarkVersion,
+                     Optional<const StringTable *> StrTab = None,
+                     Optional<StringRef> Filename = None);
+
+  /// Emit a remark block. The string table is required.
+  void emitRemarkBlock(const Remark &Remark, StringTable &StrTab);
+  /// Finalize the writing to \p OS.
+  void flushToStream(raw_ostream &OS);
+  /// Finalize the writing to a buffer.
+  /// The contents of the buffer remain valid for the lifetime of the object.
+  /// Any call to any other function in this class will invalidate the buffer.
+  StringRef getBuffer();
+};
+
+/// Implementation of the remark serializer using LLVM bitstream.
+struct BitstreamSerializer : public Serializer {
+  /// The file should contain:
+  /// 1) The block info block that describes how to read the blocks.
+  /// 2) The metadata block that contains various information about the remarks
+  ///    in the file.
+  /// 3) A number of remark blocks.
+
+  /// We need to set up 1) and 2) first, so that we can emit 3) after. This flag
+  /// is used to emit the first two blocks only once.
+  bool DidSetUp = false;
+  /// The helper to emit bitstream.
+  BitstreamSerializerHelper Helper;
+
+  BitstreamSerializer(raw_ostream &OS);
+
+  /// Emit a remark to the stream. This also emits the metadata associated to
+  /// the remarks and assumes the container type is
+  /// BitstreamRemarkContainerType::SeparateRemarksFile.
+  /// This writes the serialized output to the provided stream.
+  void emit(const Remark &Remark) override;
+};
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H */

llvm/include/llvm/Remarks/Remark.h

 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/CBindingWrapping.h"
 #include <string>
 
 namespace llvm {
 namespace remarks {
 
-constexpr uint64_t Version = 0;
-constexpr StringRef Magic("REMARKS", 7);
+/// The current version of the remark entry.
+constexpr uint64_t CurrentRemarkVersion = 0;
 
 /// The debug location used to track a remark back to the source file.
 struct RemarkLocation {
   /// Absolute path of the source file corresponding to this remark.
   StringRef SourceFilePath;
   unsigned SourceLine;
   unsigned SourceColumn;
 };

llvm/lib/Bitcode/Reader/BitcodeAnalyzer.cpp

if (Signature[2] == 'C' && Signature[3] == 'H')
       return ClangSerializedASTBitstream;
   } else if (Signature[0] == 'D' && Signature[1] == 'I') {
     if (Error Err = tryRead(Signature[2], 8))
       return std::move(Err);
     if (Error Err = tryRead(Signature[3], 8))
       return std::move(Err);
     if (Signature[2] == 'A' && Signature[3] == 'G')
       return ClangSerializedDiagnosticsBitstream;
+  } else if (Signature[0] == 'R' && Signature[1] == 'M') {
+    if (Error Err = tryRead(Signature[2], 8))
+      return std::move(Err);
+    if (Error Err = tryRead(Signature[3], 8))
+      return std::move(Err);
+    if (Signature[2] == 'R' && Signature[3] == 'K')
+      return LLVMBitstreamRemarks;
   } else {
     if (Error Err = tryRead(Signature[2], 4))
       return std::move(Err);
     if (Error Err = tryRead(Signature[3], 4))
       return std::move(Err);
     if (Error Err = tryRead(Signature[4], 4))
       return std::move(Err);
     if (Error Err = tryRead(Signature[5], 4))
case LLVMIRBitstream:
     O.OS << "LLVM IR\n";
     break;
   case ClangSerializedASTBitstream:
     O.OS << "Clang Serialized AST\n";
     break;
   case ClangSerializedDiagnosticsBitstream:
     O.OS << "Clang Serialized Diagnostics\n";
     break;
+  case LLVMBitstreamRemarks:
+    O.OS << "LLVM Remarks\n";
+    break;
   }
   O.OS << "  # Toplevel Blocks: " << NumTopBlocks << "\n";
   O.OS << "\n";
 
   // Emit per-block stats.
   O.OS << "Per-block Summary:\n";
   for (std::map<unsigned, PerBlockIDStats>::iterator I = BlockIDStats.begin(),
                                                      E = BlockIDStats.end();

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

void AsmPrinter::emitRemarksSection(Module &M) {
   const remarks::Serializer &Serializer = RS->getSerializer();
 
   // Switch to the right section: .remarks/__remarks.
   MCSection *RemarksSection =
       OutContext.getObjectFileInfo()->getRemarksSection();
   OutStreamer->SwitchSection(RemarksSection);
 
   // Emit the magic number.
-  OutStreamer->EmitBytes(remarks::Magic);
+  OutStreamer->EmitBytes("REMARKS");
   // Explicitly emit a '\0'.
   OutStreamer->EmitIntValue(/*Value=*/0, /*Size=*/1);
 
   // Emit the version number: little-endian uint64_t.
   // The version number is located at the offset 0x0 in the section.
   std::array<char, 8> Version;
-  support::endian::write64le(Version.data(), remarks::Version);
+  support::endian::write64le(Version.data(), remarks::CurrentRemarkVersion);
   OutStreamer->EmitBinaryData(StringRef(Version.data(), Version.size()));
 
   // Emit the string table in the section.
   // Note: we need to use the streamer here to emit it in the section. We can't
   // just use the serialize function with a raw_ostream because of the way
   // MCStreamers work.
   uint64_t StrTabSize =
       Serializer.StrTab ? Serializer.StrTab->SerializedSize : 0;

llvm/lib/IR/RemarkStreamer.cpp

 // LLVMContext.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/IR/RemarkStreamer.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalValue.h"
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
 
 using namespace llvm;
 
 RemarkStreamer::RemarkStreamer(StringRef Filename,
                                std::unique_ptr<remarks::Serializer> Serializer)
     : Filename(Filename), PassFilter(), Serializer(std::move(Serializer)) {
   assert(!Filename.empty() && "This needs to be a real filename.");
 }

 static std::unique_ptr<remarks::Serializer>
 formatToSerializer(RemarksSerializerFormat RemarksFormat, raw_ostream &OS) {
   switch (RemarksFormat) {
   default:
     llvm_unreachable("Unknown remark serializer format.");
     return nullptr;
   case RemarksSerializerFormat::YAML:
     return llvm::make_unique<remarks::YAMLSerializer>(OS);
+  case RemarksSerializerFormat::Bitstream:
+    return llvm::make_unique<remarks::BitstreamSerializer>(OS);
   };
 }
 
 Expected<RemarksSerializerFormat>
 llvm::parseSerializerFormat(StringRef StrFormat) {
   auto Format = StringSwitch<RemarksSerializerFormat>(StrFormat)
                     .Cases("", "yaml", RemarksSerializerFormat::YAML)
+                    .Case("bitstream", RemarksSerializerFormat::Bitstream)
                     .Default(RemarksSerializerFormat::Unknown);
 
   if (Format == RemarksSerializerFormat::Unknown)
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark serializer format: '%s'",
                              StrFormat.data());
 
   return Format;

llvm/lib/Remarks/BitstreamRemarkSerializer.cpp

+//===- BitstreamRemarkSerializer.cpp --------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides the implementation of the LLVM bitstream remark serializer
+// using LLVM's bitstream writer.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
+
+using namespace llvm;
+using namespace llvm::remarks;
+
+BitstreamSerializerHelper::BitstreamSerializerHelper(
+    BitstreamRemarkContainerType ContainerType)
+    : Encoded(), R(), Bitstream(Encoded), ContainerType(ContainerType) {}
+
+static void push(SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  for (const char C : Str)
+    R.push_back(C);
+}
+
+static void setRecordName(unsigned RecordID, BitstreamWriter &Bitstream,
+                          SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  R.clear();
+  R.push_back(RecordID);
+  push(R, Str);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETRECORDNAME, R);
+}
+
+static void initBlock(unsigned BlockID, BitstreamWriter &Bitstream,
+                      SmallVectorImpl<uint64_t> &R, StringRef Str) {
+  R.clear();
+  R.push_back(BlockID);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETBID, R);
+
+  R.clear();
+  push(R, Str);
+  Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_BLOCKNAME, R);
+}
+
+void BitstreamSerializerHelper::setupMetaBlockInfo() {
+  // Setup the metadata block.
+  initBlock(META_BLOCK_ID, Bitstream, R, MetaBlockName);
+
+  // The container information.
+  setRecordName(RECORD_META_CONTAINER_INFO, Bitstream, R,
+                MetaContainerInfoName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_CONTAINER_INFO));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version.
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2));  // Type.
+  RecordMetaContainerInfoAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::setupMetaRemarkVersion() {
+  setRecordName(RECORD_META_REMARK_VERSION, Bitstream, R,
+                MetaRemarkVersionName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_REMARK_VERSION));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version.
+  RecordMetaRemarkVersionAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaRemarkVersion(uint64_t RemarkVersion) {
+  // The remark version is emitted only if we emit remarks.
+  R.clear();
+  R.push_back(RECORD_META_REMARK_VERSION);
+  R.push_back(RemarkVersion);
+  Bitstream.EmitRecordWithAbbrev(RecordMetaRemarkVersionAbbrevID, R);
+}
+
+void BitstreamSerializerHelper::setupMetaStrTab() {
+  setRecordName(RECORD_META_STRTAB, Bitstream, R, MetaStrTabName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_STRTAB));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Raw table.
+  RecordMetaStrTabAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaStrTab(const StringTable &StrTab) {
+  // The string table is not emitted if we emit remarks separately.
+  R.clear();
+  R.push_back(RECORD_META_STRTAB);
+
+  // Serialize to a blob.
+  std::string Buf;
+  raw_string_ostream OS(Buf);
+  StrTab.serialize(OS);
+  StringRef Blob = OS.str();
+  Bitstream.EmitRecordWithBlob(RecordMetaStrTabAbbrevID, R, Blob);
+}
+
+void BitstreamSerializerHelper::setupMetaExternalFile() {
+  setRecordName(RECORD_META_EXTERNAL_FILE, Bitstream, R, MetaExternalFileName);
+
+  auto Abbrev = std::make_shared<BitCodeAbbrev>();
+  Abbrev->Add(BitCodeAbbrevOp(RECORD_META_EXTERNAL_FILE));
+  Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Filename.
+  RecordMetaExternalFileAbbrevID =
+      Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev);
+}
+
+void BitstreamSerializerHelper::emitMetaExternalFile(StringRef Filename) {
+  // The external file is emitted only if we emit the separate metadata.
+  R.clear();
+  R.push_back(RECORD_META_EXTERNAL_FILE);
+  Bitstream.EmitRecordWithBlob(RecordMetaExternalFileAbbrevID, R, Filename);
+}
+
+void BitstreamSerializerHelper::setupRemarkBlockInfo() {
+  // Setup the remark block.
+  initBlock(REMARK_BLOCK_ID, Bitstream, R, RemarkBlockName);
+
+  // The header of a remark.
+  {
+    setRecordName(RECORD_REMARK_HEADER, Bitstream, R, RemarkHeaderName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HEADER));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Type
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));   // Remark Name
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6));   // Pass name
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8));   // Function name
+    RecordRemarkHeaderAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // The location of a remark.
+  {
+    setRecordName(RECORD_REMARK_DEBUG_LOC, Bitstream, R, RemarkDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_DEBUG_LOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // File
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column
+    RecordRemarkDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // The hotness of a remark.
+  {
+    setRecordName(RECORD_REMARK_HOTNESS, Bitstream, R, RemarkHotnessName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HOTNESS));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Hotness
+    RecordRemarkHotnessAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // An argument entry with a debug location attached.
+  {
+    setRecordName(RECORD_REMARK_ARG_WITH_DEBUGLOC, Bitstream, R,
+                  RemarkArgWithDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITH_DEBUGLOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // Key
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // Value
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7));    // File
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column
+    RecordRemarkArgWithDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+
+  // An argument entry with no debug location attached.
+  {
+    setRecordName(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, Bitstream, R,
+                  RemarkArgWithoutDebugLocName);
+
+    auto Abbrev = std::make_shared<BitCodeAbbrev>();
+    Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC));
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Key
+    Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Value
+    RecordRemarkArgWithoutDebugLocAbbrevID =
+        Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev);
+  }
+}
+
+void BitstreamSerializerHelper::setupBlockInfo() {
+  // Emit magic number.
+  for (const char C : Magic)
+    Bitstream.Emit(static_cast<unsigned>(C), 8);
+
+  Bitstream.EnterBlockInfoBlock();
+
+  // Setup the main metadata. Depending on the container type, we'll setup the
+  // required records next.
+  setupMetaBlockInfo();
+
+  switch (ContainerType) {
+  case BitstreamRemarkContainerType::SeparateRemarksMeta:
+    // Needs a string table that the separate remark file is using.
+    setupMetaStrTab();
+    // Needs to know where the external remarks file is.
+    setupMetaExternalFile();
+    break;
+  case BitstreamRemarkContainerType::SeparateRemarksFile:
+    // Contains remarks: emit the version.
+    setupMetaRemarkVersion();
+    // Contains remarks: emit the remark abbrevs.
+    setupRemarkBlockInfo();
+    break;
+  case BitstreamRemarkContainerType::Standalone:
+    // Contains remarks: emit the version.
+    setupMetaRemarkVersion();
+    // Needs a string table.
+    setupMetaStrTab();
+    // Contains remarks: emit the remark abbrevs.
+    setupRemarkBlockInfo();
+    break;
+  }
+
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::emitMetaBlock(
+    uint64_t ContainerVersion, Optional<uint64_t> RemarkVersion,
+    Optional<const StringTable *> StrTab, Optional<StringRef> Filename) {
+  // Emit the meta block
+  Bitstream.EnterSubblock(META_BLOCK_ID, 3);
+
+  // The container version and type.
+  R.clear();
+  R.push_back(RECORD_META_CONTAINER_INFO);
+  R.push_back(ContainerVersion);
+  R.push_back(static_cast<uint64_t>(ContainerType));
+  Bitstream.EmitRecordWithAbbrev(RecordMetaContainerInfoAbbrevID, R);
+
+  switch (ContainerType) {
+  case BitstreamRemarkContainerType::SeparateRemarksMeta:
+    assert(StrTab != None && *StrTab != nullptr);
+    emitMetaStrTab(**StrTab);
+    assert(Filename != None);
+    emitMetaExternalFile(*Filename);
+    break;
+  case BitstreamRemarkContainerType::SeparateRemarksFile:
+    assert(RemarkVersion != None);
+    emitMetaRemarkVersion(*RemarkVersion);
+    break;
+  case BitstreamRemarkContainerType::Standalone:
+    assert(RemarkVersion != None);
+    emitMetaRemarkVersion(*RemarkVersion);
+    assert(StrTab != None && *StrTab != nullptr);
+    emitMetaStrTab(**StrTab);
+    break;
+  }
+
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::emitRemarkBlock(const Remark &Remark,
+                                                StringTable &StrTab) {
+  Bitstream.EnterSubblock(REMARK_BLOCK_ID, 4);
+
+  R.clear();
+  R.push_back(RECORD_REMARK_HEADER);
+  R.push_back(static_cast<uint64_t>(Remark.RemarkType));
+  R.push_back(StrTab.add(Remark.RemarkName).first);
+  R.push_back(StrTab.add(Remark.PassName).first);
+  R.push_back(StrTab.add(Remark.FunctionName).first);
+  Bitstream.EmitRecordWithAbbrev(RecordRemarkHeaderAbbrevID, R);
+
+  if (const Optional<RemarkLocation> &Loc = Remark.Loc) {
+    R.clear();
+    R.push_back(RECORD_REMARK_DEBUG_LOC);
+    R.push_back(StrTab.add(Loc->SourceFilePath).first);
+    R.push_back(Loc->SourceLine);
+    R.push_back(Loc->SourceColumn);
+    Bitstream.EmitRecordWithAbbrev(RecordRemarkDebugLocAbbrevID, R);
+  }
+
+  if (Optional<uint64_t> Hotness = Remark.Hotness) {
+    R.clear();
+    R.push_back(RECORD_REMARK_HOTNESS);
+    R.push_back(*Hotness);
+    Bitstream.EmitRecordWithAbbrev(RecordRemarkHotnessAbbrevID, R);
+  }
+
+  for (const Argument &Arg : Remark.Args) {
+    R.clear();
+    unsigned Key = StrTab.add(Arg.Key).first;
+    unsigned Val = StrTab.add(Arg.Val).first;
+    bool HasDebugLoc = Arg.Loc != None;
+    R.push_back(HasDebugLoc ? RECORD_REMARK_ARG_WITH_DEBUGLOC
+                            : RECORD_REMARK_ARG_WITHOUT_DEBUGLOC);
+    R.push_back(Key);
+    R.push_back(Val);
+    if (HasDebugLoc) {
+      R.push_back(StrTab.add(Arg.Loc->SourceFilePath).first);
+      R.push_back(Arg.Loc->SourceLine);
+      R.push_back(Arg.Loc->SourceColumn);
+    }
+    Bitstream.EmitRecordWithAbbrev(HasDebugLoc
+                                       ? RecordRemarkArgWithDebugLocAbbrevID
+                                       : RecordRemarkArgWithoutDebugLocAbbrevID,
+                                   R);
+  }
+  Bitstream.ExitBlock();
+}
+
+void BitstreamSerializerHelper::flushToStream(raw_ostream &OS) {
+  OS.write(Encoded.data(), Encoded.size());
+  Encoded.clear();
+}
+
+StringRef BitstreamSerializerHelper::getBuffer() {
+  return StringRef(Encoded.data(), Encoded.size());
+}
+
+BitstreamSerializer::BitstreamSerializer(raw_ostream &OS)
+    : Serializer(OS),
+      Helper(BitstreamRemarkContainerType::SeparateRemarksFile) {
+  // We always use a string table with bitstream.
+  StrTab.emplace();
+}
+
+void BitstreamSerializer::emit(const Remark &Remark) {
+  if (!DidSetUp) {
+    Helper.setupBlockInfo();
+    Helper.emitMetaBlock(/*ContainerVersion=*/CurrentContainerVersion,
+                         /*RemarkVersion=*/CurrentRemarkVersion,
+                         /*StrTab=*/&*StrTab,
+                         /*Filename=*/None);
+    DidSetUp = true;
+  }
+
+  assert(DidSetUp &&
+         "The Block info block and the meta block were not emitted yet.");
+  Helper.emitRemarkBlock(Remark, *StrTab);
+
+  Helper.flushToStream(OS);
+}

llvm/lib/Remarks/CMakeLists.txt

 add_llvm_library(LLVMRemarks
+  BitstreamRemarkSerializer.cpp
   Remark.cpp
   RemarkParser.cpp
   RemarkStringTable.cpp
   YAMLRemarkParser.cpp
   YAMLRemarkSerializer.cpp
 )

llvm/lib/Remarks/RemarkStringTable.cpp

std::pair<unsigned, StringRef> StringTable::add(StringRef Str) {
   // If it's a new string, add it to the final size.
   if (KV.second)
     SerializedSize += KV.first->first().size() + 1; // +1 for the '\0'
   // Can be either NextID or the previous ID if the string is already there.
   return {KV.first->second, KV.first->first()};
 }
 
 void StringTable::serialize(raw_ostream &OS) const {
-  // Emit the number of strings.
-  uint64_t StrTabSize = SerializedSize;
-  support::endian::write(OS, StrTabSize, support::little);
   // Emit the sequence of strings.
   for (StringRef Str : serialize()) {
     OS << Str;
     // Explicitly emit a '\0'.
     OS.write('\0');
   }
 }
 
 std::vector<StringRef> StringTable::serialize() const {
   std::vector<StringRef> Strings{StrTab.size()};
   for (const auto &KV : StrTab)
     Strings[KV.second] = KV.first();
   return Strings;
 }

llvm/test/Bitcode/stream-types.c

 // Tests that llvm-bcanalyzer recognizes the correct "stream type" for various
 // common bitstream formats.
 
 // RUN: llvm-bcanalyzer -dump %s.ast | FileCheck %s -check-prefix=CHECK-AST
 // CHECK-AST: Stream type: Clang Serialized AST
 
 // RUN: llvm-bcanalyzer -dump %s.dia | FileCheck %s -check-prefix=CHECK-DIAG
 // CHECK-DIAG: Stream type: Clang Serialized Diagnostics
 
 // RUN: not llvm-bcanalyzer -dump %s.ast.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE
 // RUN: not llvm-bcanalyzer -dump %s.dia.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE
 // CHECK-INCOMPLETE: Bitcode stream should be a multiple of 4 bytes in length
+
+// RUN: llvm-bcanalyzer -dump %s.opt.bitstream | FileCheck %s -check-prefix=CHECK-REMARKS
+// CHECK-REMARKS: Stream type: LLVM Remarks

llvm/unittests/Remarks/BitstreamRemarksFormatTest.cpp

+//===- unittest/Support/BitstreamRemarksFormatTest.cpp - BitCodes tests ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+// The goal for this test is to observe test failures and carefully update the
+// constants when they change.
+
+// This should not change over time.
+TEST(BitstreamRemarksFormat, Magic) { EXPECT_EQ(remarks::Magic, "RMRK"); }
+
+// This should be updated whenever any of the tests below are modified.
+TEST(BitstreamRemarksFormat, ContainerVersion) {
+  EXPECT_EQ(remarks::CurrentContainerVersion, 0UL);
+}
+
+// The values of the current blocks should not change over time.
+// When adding new blocks, make sure to append them to the enum.
+TEST(BitstreamRemarksFormat, BlockIDs) {
+  EXPECT_EQ(remarks::META_BLOCK_ID, 8);
+  EXPECT_EQ(remarks::REMARK_BLOCK_ID, 9);
+}
+
+// The values of the current records should not change over time.
+// When adding new records, make sure to append them to the enum.
+TEST(BitstreamRemarksFormat, RecordIDs) {
+  EXPECT_EQ(remarks::RECORD_FIRST, 1);
+  EXPECT_EQ(remarks::RECORD_META_CONTAINER_INFO, 1);
+  EXPECT_EQ(remarks::RECORD_META_REMARK_VERSION, 2);
+  EXPECT_EQ(remarks::RECORD_META_STRTAB, 3);
+  EXPECT_EQ(remarks::RECORD_META_EXTERNAL_FILE, 4);
+  EXPECT_EQ(remarks::RECORD_REMARK_HEADER, 5);
+  EXPECT_EQ(remarks::RECORD_REMARK_DEBUG_LOC, 6);
+  EXPECT_EQ(remarks::RECORD_REMARK_HOTNESS, 7);
+  EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITH_DEBUGLOC, 8);
+  EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, 9);
+  EXPECT_EQ(remarks::RECORD_LAST, 9);
+}

llvm/unittests/Remarks/BitstreamRemarksSerializerTest.cpp

+//===- unittest/Support/BitstreamRemarksSerializerTest.cpp ----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkSerializer.h"
+#include "llvm/Support/raw_ostream.h"
+#include "llvm/Bitcode/BitcodeAnalyzer.h"
+#include "gtest/gtest.h"
+#include <string>
+
+using namespace llvm;
+
+static void checkAnalyze(StringRef Input, StringRef Expected) {
+  std::string OutputBuf;
+  raw_string_ostream OutputOS(OutputBuf);
+  BCDumpOptions O(OutputOS);
+  O.ShowBinaryBlobs = true;
+  BitcodeAnalyzer BA(Input);
+  EXPECT_FALSE(BA.analyze(O)); // Expect no errors.
+  EXPECT_EQ(OutputOS.str(), Expected);
+}
+
+static void check(const remarks::Remark &R, StringRef ExpectedStrTab,
+                  StringRef Expected) {
+  // Emit the remark.
+  std::string InputBuf;
+  raw_string_ostream InputOS(InputBuf);
+  remarks::BitstreamSerializer Serializer(InputOS);
+  Serializer.emit(R);
+
+  // Analyze the serialized remark.
+  checkAnalyze(InputOS.str(), Expected);
+
+  // Analyze the string table.
+  std::string StrTabBuf;
+  raw_string_ostream StrTabOS(StrTabBuf);
+  Serializer.StrTab->serialize(StrTabOS);
+  EXPECT_EQ(StrTabOS.str(), ExpectedStrTab);
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileNoOptionals) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  check(R, StringRef("remark\0pass\0function\0", 21),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=1 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  check(R, StringRef("remark\0pass\0function\0path\0", 26),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=4 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileHotness) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Hotness.emplace(999999999);
+  check(R, StringRef("remark\0pass\0function\0", 21),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=3 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgNoDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  SmallVector<remarks::Argument, 1> Args;
+  Args.emplace_back();
+  Args.back().Key = "key";
+  Args.back().Val = "value";
+  R.Args = Args;
+  check(R, StringRef("remark\0pass\0function\0key\0value\0", 31),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=2 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Argument codeid=9 abbrevid=8 op0=3 op1=4/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgDebugLoc) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  SmallVector<remarks::Argument, 1> Args;
+  Args.emplace_back();
+  Args.back().Key = "key";
+  Args.back().Val = "value";
+  Args.back().Loc.emplace();
+  Args.back().Loc->SourceFilePath = "path";
+  Args.back().Loc->SourceLine = 99;
+  Args.back().Loc->SourceColumn = 55;
+  R.Args = Args;
+  check(R, StringRef("remark\0pass\0function\0key\0value\0path\0", 36),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=4 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Argument with debug location codeid=8 abbrevid=7 op0=3 op1=4 op2=5 "
+        "op3=99 op4=55/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkFileAll) {
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  R.Hotness.emplace(999999999);
+  SmallVector<remarks::Argument, 1> Args;
+  Args.emplace_back();
+  Args.back().Key = "key";
+  Args.back().Val = "value";
+  Args.back().Loc.emplace();
+  Args.back().Loc->SourceFilePath = "argpath";
+  Args.back().Loc->SourceLine = 11;
+  Args.back().Loc->SourceColumn = 66;
+  R.Args = Args;
+  check(R, StringRef("remark\0pass\0function\0path\0key\0value\0argpath\0", 44),
+        "<BLOCKINFO_BLOCK/>\n"
+        "<Meta BlockID=8 NumWords=3 BlockCodeSize=3>\n"
+        "  <Container info codeid=1 abbrevid=4 op0=0 op1=1/>\n"
+        "  <Remark version codeid=2 abbrevid=5 op0=0/>\n"
+        "</Meta>\n"
+        "<Remark BlockID=9 NumWords=8 BlockCodeSize=4>\n"
+        "  <Remark header codeid=5 abbrevid=4 op0=2 op1=0 op2=1 op3=2/>\n"
+        "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+        "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+        "  <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 "
+        "op3=11 op4=66/>\n"
+        "</Remark>\n");
+}
+
+TEST(BitstreamRemarkSerializer, SeparateRemarkMeta) {
+  remarks::StringTable StrTab;
+  StrTab.add("pass");
+  StrTab.add("key");
+  StrTab.add("value");
+
+  remarks::BitstreamSerializerHelper MetaSerializer(
+      remarks::BitstreamRemarkContainerType::SeparateRemarksMeta);
+  MetaSerializer.setupBlockInfo();
+  MetaSerializer.emitMetaBlock(5, /*RemarkVersion=*/None, &StrTab,
+                               StringRef("/path/to/file/name"));
+  checkAnalyze(MetaSerializer.getBuffer(),
+               "<BLOCKINFO_BLOCK/>\n"
+               "<Meta BlockID=8 NumWords=13 BlockCodeSize=3>\n"
+               "  <Container info codeid=1 abbrevid=4 op0=5 op1=0/>\n"
+               "  <String table codeid=3 abbrevid=5/> blob data = "
+               "'pass\\x00key\\x00value\\x00'\n"
+               "  <External File codeid=4 abbrevid=6/> blob data = "
+               "'/path/to/file/name'\n"
+               "</Meta>\n");
+}
+
+TEST(BitstreamRemarkSerializer, Standalone) {
+  // Pre-populate the string table.
+  remarks::StringTable StrTab;
+  StrTab.add("pass");
+  StrTab.add("remark");
+  StrTab.add("function");
+  StrTab.add("path");
+  StrTab.add("key");
+  StrTab.add("value");
+  StrTab.add("argpath");
+
+  // Emit the metadata.
+  remarks::BitstreamSerializerHelper Serializer(
+      remarks::BitstreamRemarkContainerType::Standalone);
+  Serializer.setupBlockInfo();
+  Serializer.emitMetaBlock(/*ContainerVersion=*/5, /*RemarkVersion=*/30,
+                           &StrTab);
+  remarks::Remark R;
+  R.RemarkType = remarks::Type::Missed;
+  R.PassName = "pass";
+  R.RemarkName = "remark";
+  R.FunctionName = "function";
+  R.Loc.emplace();
+  R.Loc->SourceFilePath = "path";
+  R.Loc->SourceLine = 99;
+  R.Loc->SourceColumn = 55;
+  R.Hotness.emplace(999999999);
+  SmallVector<remarks::Argument, 1> Args;
+  Args.emplace_back();
+  Args.back().Key = "key";
+  Args.back().Val = "value";
+  Args.back().Loc.emplace();
+  Args.back().Loc->SourceFilePath = "argpath";
+  Args.back().Loc->SourceLine = 11;
+  Args.back().Loc->SourceColumn = 66;
+  R.Args = Args;
+  // Emit the remark.
+  Serializer.emitRemarkBlock(R, StrTab);
+
+  checkAnalyze(
+      Serializer.getBuffer(),
+      "<BLOCKINFO_BLOCK/>\n"
+      "<Meta BlockID=8 NumWords=15 BlockCodeSize=3>\n"
+      "  <Container info codeid=1 abbrevid=4 op0=5 op1=2/>\n"
+      "  <Remark version codeid=2 abbrevid=5 op0=30/>\n"
+      "  <String table codeid=3 abbrevid=6/> blob data = "
+      "'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x0"
+      "0'\n"
+      "</Meta>\n"
+      "<Remark BlockID=9 NumWords=8 BlockCodeSize=4>\n"
+      "  <Remark header codeid=5 abbrevid=4 op0=2 op1=1 op2=0 op3=2/>\n"
+      "  <Remark debug location codeid=6 abbrevid=5 op0=3 op1=99 op2=55/>\n"
+      "  <Remark hotness codeid=7 abbrevid=6 op0=999999999/>\n"
+      "  <Argument with debug location codeid=8 abbrevid=7 op0=4 op1=5 op2=6 "
+      "op3=11 op4=66/>\n"
+      "</Remark>\n");
+}

llvm/unittests/Remarks/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
+  BitReader
   Remarks
   Support
   )
 
 add_llvm_unittest(RemarksTests
+  BitstreamRemarksFormatTest.cpp
+  BitstreamRemarksSerializerTest.cpp
   RemarksStrTabParsingTest.cpp
   YAMLRemarksParsingTest.cpp
   )