[Remarks] Add parser for bitstream remarks

[thegameg]

Summary

The bitstream remark serializer landed in r367372.

This adds a bitstream remark parser that parser bitstream remark files to llvm::remarks::Remark objects through the llvm::remarks::RemarkParser interface.

A few interesting things to point out:

• There are parsing helpers to parse the different types of blocks
• The main parsing helper allows us to parse remark metadata and open an external file containing the encoded remarks
• This adds a dependency from the Remarks library to the BitstreamReader library
• The testing strategy is to create a remark entry through YAML, parse it, serialize it to bitstream, parse that back and compare the objects.
• There are close to no tests for malformed bitstream remarks, due to the lack of textual format for the bitstream format.
• This adds a new C API for parsing bitstream remarks: LLVMRemarkParserCreateBitstream.
• This bumps the REMARKS_API_VERSION to 1.

[JDevlieghere]

I really like your code style, useful comments, good abstractions, and great error handling. This was fun to review. A few things inline but overall this LGTM!

llvm/lib/Remarks/BitstreamRemarkParser.cpp

44

Looks like this comment needs to move down one line or Blob should be declared after the SmallVector?

91

Same as previous comment.

194

Is this actually reachable?

510

It's a little unfortunate that this pattern is repeated here but I don't have a good alternative. I considered suggesting putting it behind a macro, but given that there's only three instances that might be overkill and actually hurt readability.

llvm/lib/Remarks/BitstreamRemarkParser.h

67

I think these helpers could be private? Doing so would help convey what's part of the interface and what's not. The same applies to the other structs.

[thegameg]

Address Jonas' comments. Thanks for the review!

llvm/lib/Remarks/BitstreamRemarkParser.cpp

194

I was imagining the case where we have something like a bunch of record entries followed by an end of file with no END_BLOCK. After the last successfully parsed record, we continue, then the loop condition is false, and we end up with an unterminated block because the stream is at the end, so we need to report an error.

510

I agree, I thought of a macro but I came to the same conclusion as you.

llvm/include/llvm-c/Remarks.h

 
 /**
  * @defgroup LLVMCREMARKS Remarks
  * @ingroup LLVMC
  *
  * @{
  */
 
-#define REMARKS_API_VERSION 0
+// 0 -> 1: Bitstream remarks support.
+#define REMARKS_API_VERSION 1
 
 /**
  * The type of the emitted remark.
  */
 enum LLVMRemarkType {
   LLVMRemarkTypeUnknown,
   LLVMRemarkTypePassed,
   LLVMRemarkTypeMissed,

  * leaking resources.
  *
  * \since REMARKS_API_VERSION=0
  */
 extern LLVMRemarkParserRef LLVMRemarkParserCreateYAML(const void *Buf,
                                                       uint64_t Size);
 
 /**
+ * Creates a remark parser that can be used to parse the buffer located in \p
+ * Buf of size \p Size bytes.
+ *
+ * \p Buf cannot be `NULL`.
+ *
+ * This function should be paired with LLVMRemarkParserDispose() to avoid
+ * leaking resources.
+ *
+ * \since REMARKS_API_VERSION=1
+ */
+extern LLVMRemarkParserRef LLVMRemarkParserCreateBitstream(const void *Buf,
+                                                           uint64_t Size);
+
+/**
  * Returns the next remark in the file.
  *
  * The value pointed to by the return value needs to be disposed using a call to
  * LLVMRemarkEntryDispose().
  *
  * All the entries in the returned value that are of LLVMRemarkStringRef type
  * will become invalidated once a call to LLVMRemarkParserDispose is made.
  *

llvm/include/llvm/Bitstream/BitstreamReader.h

public:
   using SimpleBitstreamCursor::GetCurrentBitNo;
   using SimpleBitstreamCursor::getCurrentByteNo;
   using SimpleBitstreamCursor::getPointerToByte;
   using SimpleBitstreamCursor::JumpToBit;
   using SimpleBitstreamCursor::Read;
   using SimpleBitstreamCursor::ReadVBR;
   using SimpleBitstreamCursor::ReadVBR64;
   using SimpleBitstreamCursor::SizeInBytes;
+  using SimpleBitstreamCursor::skipToEnd;
 
   /// Return the number of bits used to encode an abbrev #.
   unsigned getAbbrevIDWidth() const { return CurCodeSize; }
 
   /// Flags that modify the behavior of advance().
   enum {
     /// If this flag is used, the advance() method does not automatically pop
     /// the block scope when the end of a block is reached.

llvm/include/llvm/Remarks/BitstreamRemarkParser.h

+//===-- BitstreamRemarkParser.h - Bitstream parser --------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides an implementation of the remark parser using the LLVM
+// Bitstream format.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_REMARKS_BITSTREAM_REMARK_PARSER_H
+#define LLVM_REMARKS_BITSTREAM_REMARK_PARSER_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Bitstream/BitstreamReader.h"
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "llvm/Remarks/Remark.h"
+#include "llvm/Remarks/RemarkParser.h"
+#include "llvm/Support/Error.h"
+#include <array>
+
+namespace llvm {
+namespace remarks {
+
+/// Helper to parse a META_BLOCK for a bitstream remark container.
+struct BitstreamMetaParserHelper {
+  /// The Bitstream reader.
+  BitstreamCursor &Stream;
+  /// Reference to the storage for the block info.
+  BitstreamBlockInfo &BlockInfo;
+  /// The parsed content: depending on the container type, some fields might be
+  /// empty.
+  Optional<uint64_t> ContainerVersion;
+  Optional<uint8_t> ContainerType;
+  Optional<StringRef> StrTabBuf;
+  Optional<StringRef> ExternalFilePath;
+  Optional<uint64_t> RemarkVersion;
+
+  /// Continue parsing with \p Stream. \p Stream is expected to contain a
+  /// ENTER_SUBBLOCK to the META_BLOCK at the current position.
+  /// \p Stream is expected to have a BLOCKINFO_BLOCK set.
+  BitstreamMetaParserHelper(BitstreamCursor &Stream,
+                            BitstreamBlockInfo &BlockInfo);
+
+  /// Parse the META_BLOCK and fill the available entries.
+  /// This helper does not check for the validity of the fields.
+  Error parse();
+};
+
+/// Helper to parse a REMARK_BLOCK for a bitstream remark container.
+struct BitstreamRemarkParserHelper {
+  /// The Bitstream reader.
+  BitstreamCursor &Stream;
+  /// The parsed content: depending on the remark, some fields might be empty.
+  Optional<uint8_t> Type;
+  Optional<uint64_t> RemarkNameIdx;
+  Optional<uint64_t> PassNameIdx;
+  Optional<uint64_t> FunctionNameIdx;
+  Optional<uint64_t> SourceFileNameIdx;
+  Optional<uint32_t> SourceLine;
+  Optional<uint32_t> SourceColumn;
+  Optional<uint64_t> Hotness;
+  struct Argument {
+    Optional<uint64_t> KeyIdx;
+    Optional<uint64_t> ValueIdx;
+    Optional<uint64_t> SourceFileNameIdx;
+    Optional<uint32_t> SourceLine;
+    Optional<uint32_t> SourceColumn;
+  };
+  Optional<ArrayRef<Argument>> Args;
+  /// Avoid re-allocating a vector every time.
+  SmallVector<Argument, 8> TmpArgs;
+
+  /// Continue parsing with \p Stream. \p Stream is expected to contain a
+  /// ENTER_SUBBLOCK to the REMARK_BLOCK at the current position.
+  /// \p Stream is expected to have a BLOCKINFO_BLOCK set and to have already
+  /// parsed the META_BLOCK.
+  BitstreamRemarkParserHelper(BitstreamCursor &Stream);
+
+  /// Parse the REMARK_BLOCK and fill the available entries.
+  /// This helper does not check for the validity of the fields.
+  Error parse();
+};
+
+/// Helper to parse any bitstream remark container.
+struct BitstreamParserHelper {
+  /// The Bitstream reader.
+  BitstreamCursor Stream;
+  /// The block info block.
+  BitstreamBlockInfo BlockInfo;
+  /// Start parsing at \p Buffer.
+  BitstreamParserHelper(StringRef Buffer);
+  /// Parse the magic number.
+  Expected<std::array<char, 4>> parseMagic();
+  /// Parse the block info block containing all the abbrevs.
+  /// This needs to be called before calling any other parsing function.
+  Error parseBlockInfoBlock();
+  /// Return true if the next block is a META_BLOCK. This function does not move
+  /// the cursor.
+  Expected<bool> isMetaBlock();
+  /// Return true if the next block is a REMARK_BLOCK. This function does not
+  /// move the cursor.
+  Expected<bool> isRemarkBlock();
+  /// Return true if the parser reached the end of the stream.
+  bool atEndOfStream() { return Stream.AtEndOfStream(); }
+  /// Jump to the end of the stream, skipping everything.
+  void skipToEnd() { return Stream.skipToEnd(); }
+};
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_REMARKS_BITSTREAM_REMARK_PARSER_H */

llvm/lib/Remarks/BitstreamRemarkParser.h

+//===-- BitstreamRemarkParser.h - Parser for Bitstream remarks --*- C++/-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides the impementation of the Bitstream remark parser.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_REMARKS_BITSTREAM_REMARK_PARSER_H
+#define LLVM_LIB_REMARKS_BITSTREAM_REMARK_PARSER_H
+
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Remarks/BitstreamRemarkParser.h"
+#include "llvm/Remarks/RemarkFormat.h"
+#include "llvm/Remarks/RemarkParser.h"
+#include "llvm/Support/raw_ostream.h"
+#include <memory>
+#include <string>
+
+namespace llvm {
+namespace remarks {
+/// Parses and holds the state of the latest parsed remark.
+struct BitstreamRemarkParser : public RemarkParser {
+  /// The buffer to parse.
+  BitstreamParserHelper ParserHelper;
+  /// The string table used for parsing strings.
+  Optional<ParsedStringTable> StrTab;
+  /// Temporary remark buffer used when the remarks are stored separately.
+  std::unique_ptr<MemoryBuffer> TmpRemarkBuffer;
+  /// The common metadata used to decide how to parse the buffer.
+  /// This is filled when parsing the metadata block.
+  uint64_t ContainerVersion;
+  uint64_t RemarkVersion;
+  BitstreamRemarkContainerType ContainerType;
+  /// Wether the parser is ready to parse remarks.
+  bool ReadyToParseRemarks = false;
+
+  /// Create a parser that expects to find a string table embedded in the
+  /// stream.
+  BitstreamRemarkParser(StringRef Buf)
+      : RemarkParser(Format::Bitstream), ParserHelper(Buf) {}
+
+  /// Create a parser that uses a pre-parsed string table.
+  BitstreamRemarkParser(StringRef Buf, ParsedStringTable StrTab)
+      : RemarkParser(Format::Bitstream), ParserHelper(Buf),
+        StrTab(std::move(StrTab)) {}
+
+  Expected<std::unique_ptr<Remark>> next() override;
+
+  static bool classof(const RemarkParser *P) {
+    return P->ParserFormat == Format::Bitstream;
+  }
+
+  /// Parse and process the metadata of the buffer.
+  Error parseMeta();
+
+  /// Parse a Bitstream remark.
+  Expected<std::unique_ptr<Remark>> parseRemark();
+
+private:
+  /// Helper functions.
+  Error processCommonMeta(BitstreamMetaParserHelper &Helper);
+  Error processStandaloneMeta(BitstreamMetaParserHelper &Helper);
+  Error processSeparateRemarksFileMeta(BitstreamMetaParserHelper &Helper);
+  Error processSeparateRemarksMetaMeta(BitstreamMetaParserHelper &Helper);
+  Expected<std::unique_ptr<Remark>>
+  processRemark(BitstreamRemarkParserHelper &Helper);
+  Error processExternalFilePath(Optional<StringRef> ExternalFilePath);
+};
+
+Expected<std::unique_ptr<BitstreamRemarkParser>>
+createBitstreamParserFromMeta(StringRef Buf,
+                              Optional<ParsedStringTable> StrTab = None);
+
+} // end namespace remarks
+} // end namespace llvm
+
+#endif /* LLVM_LIB_REMARKS_BITSTREAM_REMARK_PARSER_H */

llvm/lib/Remarks/BitstreamRemarkParser.cpp

+//===- BitstreamRemarkParser.cpp ------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file provides utility methods used by clients that want to use the
+// parser for remark diagnostics in LLVM.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Remarks/BitstreamRemarkParser.h"
+#include "BitstreamRemarkParser.h"
+#include "llvm/Remarks/BitstreamRemarkContainer.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+using namespace llvm::remarks;
+
+static Error unknownRecord(const char *BlockName, unsigned RecordID) {
+  return createStringError(
+      std::make_error_code(std::errc::illegal_byte_sequence),
+      "Error while parsing %s: unknown record entry (%lu).", BlockName,
+      RecordID);
+}
+
+static Error malformedRecord(const char *BlockName, const char *RecordName) {
+  return createStringError(
+      std::make_error_code(std::errc::illegal_byte_sequence),
+      "Error while parsing %s: malformed record entry (%s).", BlockName,
+      RecordName);
+}
+
+BitstreamMetaParserHelper::BitstreamMetaParserHelper(
+    BitstreamCursor &Stream, BitstreamBlockInfo &BlockInfo)
+    : Stream(Stream), BlockInfo(BlockInfo) {}
+
+/// Parse a record and fill in the fields in the parser.
+static Error parseRecord(BitstreamMetaParserHelper &Parser, unsigned Code) {
+  BitstreamCursor &Stream = Parser.Stream;
+  // Note: 2 is used here because it's the max number of fields we have per
+  // record.
+  SmallVector<uint64_t, 2> Record;
+  StringRef Blob;
+  Expected<unsigned> RecordID = Stream.readRecord(Code, Record, &Blob);
+  if (!RecordID)
+    return RecordID.takeError();
+
+  switch (*RecordID) {
+  case RECORD_META_CONTAINER_INFO: {
+    if (Record.size() != 2)
+      return malformedRecord("BLOCK_META", "RECORD_META_CONTAINER_INFO");
+    Parser.ContainerVersion = Record[0];
+    Parser.ContainerType = Record[1];
+    break;
+  }
+  case RECORD_META_REMARK_VERSION: {
+    if (Record.size() != 1)
+      return malformedRecord("BLOCK_META", "RECORD_META_REMARK_VERSION");
+    Parser.RemarkVersion = Record[0];
+    break;
+  }
+  case RECORD_META_STRTAB: {
+    if (Record.size() != 0)
+      return malformedRecord("BLOCK_META", "RECORD_META_STRTAB");
+    Parser.StrTabBuf = Blob;
+    break;
+  }
+  case RECORD_META_EXTERNAL_FILE: {
+    if (Record.size() != 0)
+      return malformedRecord("BLOCK_META", "RECORD_META_EXTERNAL_FILE");
+    Parser.ExternalFilePath = Blob;
+    break;
+  }
+  default:
+    return unknownRecord("BLOCK_META", *RecordID);
+  }
+  return Error::success();
+}
+
+BitstreamRemarkParserHelper::BitstreamRemarkParserHelper(
+    BitstreamCursor &Stream)
+    : Stream(Stream) {}
+
+/// Parse a record and fill in the fields in the parser.
+static Error parseRecord(BitstreamRemarkParserHelper &Parser, unsigned Code) {
+  BitstreamCursor &Stream = Parser.Stream;
+  // Note: 5 is used here because it's the max number of fields we have per
+  // record.
+  SmallVector<uint64_t, 5> Record;
+  StringRef Blob;
+  Expected<unsigned> RecordID = Stream.readRecord(Code, Record, &Blob);
+  if (!RecordID)
+    return RecordID.takeError();
+
+  switch (*RecordID) {
+  case RECORD_REMARK_HEADER: {
+    if (Record.size() != 4)
+      return malformedRecord("BLOCK_REMARK", "RECORD_REMARK_HEADER");
+    Parser.Type = Record[0];
+    Parser.RemarkNameIdx = Record[1];
+    Parser.PassNameIdx = Record[2];
+    Parser.FunctionNameIdx = Record[3];
+    break;
+  }
+  case RECORD_REMARK_DEBUG_LOC: {
+    if (Record.size() != 3)
+      return malformedRecord("BLOCK_REMARK", "RECORD_REMARK_DEBUG_LOC");
+    Parser.SourceFileNameIdx = Record[0];
+    Parser.SourceLine = Record[1];
+    Parser.SourceColumn = Record[2];
+    break;
+  }
+  case RECORD_REMARK_HOTNESS: {
+    if (Record.size() != 1)
+      return malformedRecord("BLOCK_REMARK", "RECORD_REMARK_HOTNESS");
+    Parser.Hotness = Record[0];
+    break;
+  }
+  case RECORD_REMARK_ARG_WITH_DEBUGLOC: {
+    if (Record.size() != 5)
+      return malformedRecord("BLOCK_REMARK", "RECORD_REMARK_ARG_WITH_DEBUGLOC");
+    // Create a temporary argument. Use that as a valid memory location for this
+    // argument entry.
+    Parser.TmpArgs.emplace_back();
+    Parser.TmpArgs.back().KeyIdx = Record[0];
+    Parser.TmpArgs.back().ValueIdx = Record[1];
+    Parser.TmpArgs.back().SourceFileNameIdx = Record[2];
+    Parser.TmpArgs.back().SourceLine = Record[3];
+    Parser.TmpArgs.back().SourceColumn = Record[4];
+    Parser.Args =
+        ArrayRef<BitstreamRemarkParserHelper::Argument>(Parser.TmpArgs);
+    break;
+  }
+  case RECORD_REMARK_ARG_WITHOUT_DEBUGLOC: {
+    if (Record.size() != 2)
+      return malformedRecord("BLOCK_REMARK",
+                             "RECORD_REMARK_ARG_WITHOUT_DEBUGLOC");
+    // Create a temporary argument. Use that as a valid memory location for this
+    // argument entry.
+    Parser.TmpArgs.emplace_back();
+    Parser.TmpArgs.back().KeyIdx = Record[0];
+    Parser.TmpArgs.back().ValueIdx = Record[1];
+    Parser.Args =
+        ArrayRef<BitstreamRemarkParserHelper::Argument>(Parser.TmpArgs);
+    break;
+  }
+  default:
+    return unknownRecord("BLOCK_REMARK", *RecordID);
+  }
+  return Error::success();
+}
+
+template <typename T>
+static Error parseBlock(T &ParserHelper, unsigned BlockID,
+                        const char *BlockName) {
+  BitstreamCursor &Stream = ParserHelper.Stream;
+  Expected<BitstreamEntry> Next = Stream.advance();
+  if (!Next)
+    return Next.takeError();
+  if (Next->Kind != BitstreamEntry::SubBlock || Next->ID != BlockID)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing %s: expecting [ENTER_SUBBLOCK, %s, ...].",
+        BlockName, BlockName);
+  if (Stream.EnterSubBlock(BlockID))
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while entering %s.", BlockName);
+
+  // Stop when there is nothing to read anymore or when we encounter an
+  // END_BLOCK.
+  while (!Stream.AtEndOfStream()) {
+    Expected<BitstreamEntry> Next = Stream.advance();
+    if (!Next)
+      return Next.takeError();
+    switch (Next->Kind) {
+    case BitstreamEntry::EndBlock:
+      return Error::success();
+    case BitstreamEntry::Error:
+    case BitstreamEntry::SubBlock:
+      return createStringError(
+          std::make_error_code(std::errc::illegal_byte_sequence),
+          "Error while parsing %s: expecting records.", BlockName);
+    case BitstreamEntry::Record:
+      if (Error E = parseRecord(ParserHelper, Next->ID))
+        return E;
+      continue;
+    }
+  }
+  // If we're here, it means we didn't get an END_BLOCK yet, but we're at the
+  // end of the stream. In this case, error.
+  return createStringError(
+      std::make_error_code(std::errc::illegal_byte_sequence),
+      "Error while parsing %s: unterminated block.", BlockName);
+}
+
+Error BitstreamMetaParserHelper::parse() {
+  return parseBlock(*this, META_BLOCK_ID, "META_BLOCK");
+}
+
+Error BitstreamRemarkParserHelper::parse() {
+  return parseBlock(*this, REMARK_BLOCK_ID, "REMARK_BLOCK");
+}
+
+BitstreamParserHelper::BitstreamParserHelper(StringRef Buffer)
+    : Stream(Buffer) {}
+
+Expected<std::array<char, 4>> BitstreamParserHelper::parseMagic() {
+  std::array<char, 4> Result;
+  for (unsigned i = 0; i < 4; ++i)
+    if (Expected<unsigned> R = Stream.Read(8))
+      Result[i] = *R;
+    else
+      return R.takeError();
+  return Result;
+}
+
+Error BitstreamParserHelper::parseBlockInfoBlock() {
+  Expected<BitstreamEntry> Next = Stream.advance();
+  if (!Next)
+    return Next.takeError();
+  if (Next->Kind != BitstreamEntry::SubBlock ||
+      Next->ID != llvm::bitc::BLOCKINFO_BLOCK_ID)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCKINFO_BLOCK: expecting [ENTER_SUBBLOCK, "
+        "BLOCKINFO_BLOCK, ...].");
+
+  Expected<Optional<BitstreamBlockInfo>> MaybeBlockInfo =
+      Stream.ReadBlockInfoBlock();
+  if (!MaybeBlockInfo)
+    return MaybeBlockInfo.takeError();
+
+  if (!*MaybeBlockInfo)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCKINFO_BLOCK.");
+
+  BlockInfo = **MaybeBlockInfo;
+
+  Stream.setBlockInfo(&BlockInfo);
+  return Error::success();
+}
+
+static Expected<bool> isBlock(BitstreamCursor &Stream, unsigned BlockID) {
+  bool Result = false;
+  uint64_t PreviousBitNo = Stream.GetCurrentBitNo();
+  Expected<BitstreamEntry> Next = Stream.advance();
+  if (!Next)
+    return Next.takeError();
+  switch (Next->Kind) {
+  case BitstreamEntry::SubBlock:
+    // Check for the block id.
+    Result = Next->ID == BlockID;
+    break;
+  case BitstreamEntry::Error:
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Unexpected error while parsing bitstream.");
+  default:
+    Result = false;
+    break;
+  }
+  if (Error E = Stream.JumpToBit(PreviousBitNo))
+    return std::move(E);
+  return Result;
+}
+
+Expected<bool> BitstreamParserHelper::isMetaBlock() {
+  return isBlock(Stream, META_BLOCK_ID);
+}
+
+Expected<bool> BitstreamParserHelper::isRemarkBlock() {
+  return isBlock(Stream, META_BLOCK_ID);
+}
+
+static Error validateMagicNumber(StringRef Magic) {
+  if (Magic != remarks::ContainerMagic)
+    return createStringError(std::make_error_code(std::errc::invalid_argument),
+                             "Unknown magic number: expecting %s, got %.4s.",
+                             remarks::ContainerMagic.data(), Magic.data());
+  return Error::success();
+}
+
+static Error advanceToMetaBlock(BitstreamParserHelper &Helper) {
+  Expected<std::array<char, 4>> Magic = Helper.parseMagic();
+  if (!Magic)
+    return Magic.takeError();
+  if (Error E = validateMagicNumber(StringRef(Magic->data(), Magic->size())))
+    return E;
+  if (Error E = Helper.parseBlockInfoBlock())
+    return E;
+  Expected<bool> isMetaBlock = Helper.isMetaBlock();
+  if (!isMetaBlock)
+    return isMetaBlock.takeError();
+  if (!*isMetaBlock)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Expecting META_BLOCK after the BLOCKINFO_BLOCK.");
+  return Error::success();
+}
+
+Expected<std::unique_ptr<BitstreamRemarkParser>>
+remarks::createBitstreamParserFromMeta(StringRef Buf,
+                                       Optional<ParsedStringTable> StrTab) {
+  BitstreamParserHelper Helper(Buf);
+  Expected<std::array<char, 4>> Magic = Helper.parseMagic();
+  if (!Magic)
+    return Magic.takeError();
+
+  if (Error E = validateMagicNumber(StringRef(Magic->data(), Magic->size())))
+    return std::move(E);
+
+  return StrTab
+             ? std::make_unique<BitstreamRemarkParser>(Buf, std::move(*StrTab))
+             : std::make_unique<BitstreamRemarkParser>(Buf);
+}
+
+Expected<std::unique_ptr<Remark>> BitstreamRemarkParser::next() {
+  if (ParserHelper.atEndOfStream())
+    return make_error<EndOfFileError>();
+
+  if (!ReadyToParseRemarks) {
+    if (Error E = parseMeta())
+      return std::move(E);
+    ReadyToParseRemarks = true;
+  }
+
+  return parseRemark();
+}
+
+Error BitstreamRemarkParser::parseMeta() {
+  // Advance and to the meta block.
+  if (Error E = advanceToMetaBlock(ParserHelper))
+    return E;
+
+  BitstreamMetaParserHelper MetaHelper(ParserHelper.Stream,
+                                       ParserHelper.BlockInfo);
+  if (Error E = MetaHelper.parse())
+    return E;
+
+  if (Error E = processCommonMeta(MetaHelper))
+    return E;
+
+  switch (ContainerType) {
+  case BitstreamRemarkContainerType::Standalone:
+    return processStandaloneMeta(MetaHelper);
+  case BitstreamRemarkContainerType::SeparateRemarksFile:
+    return processSeparateRemarksFileMeta(MetaHelper);
+  case BitstreamRemarkContainerType::SeparateRemarksMeta:
+    return processSeparateRemarksMetaMeta(MetaHelper);
+  }
+}
+
+Error BitstreamRemarkParser::processCommonMeta(
+    BitstreamMetaParserHelper &MetaHelper) {
+  if (Optional<uint64_t> Version = MetaHelper.ContainerVersion)
+    ContainerVersion = *Version;
+  else
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_META: missing container version.");
+
+  if (Optional<uint8_t> Type = MetaHelper.ContainerType) {
+    if (*Type < static_cast<uint8_t>(BitstreamRemarkContainerType::First) ||
+        *Type > static_cast<uint8_t>(BitstreamRemarkContainerType::Last))
+      return createStringError(
+          std::make_error_code(std::errc::illegal_byte_sequence),
+          "Error while parsing BLOCK_META: invalid container type.");
+
+    ContainerType = static_cast<BitstreamRemarkContainerType>(*Type);
+  } else
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_META: missing container type.");
+
+  return Error::success();
+}
+
+static Error processStrTab(BitstreamRemarkParser &P,
+                           Optional<StringRef> StrTabBuf) {
+  if (!StrTabBuf)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_META: missing string table.");
+  // Parse and assign the string table.
+  P.StrTab.emplace(*StrTabBuf);
+  return Error::success();
+}
+
+static Error processRemarkVersion(BitstreamRemarkParser &P,
+                                  Optional<uint64_t> RemarkVersion) {
+  if (!RemarkVersion)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_META: missing remark version.");
+  P.RemarkVersion = *RemarkVersion;
+  return Error::success();
+}
+
+Error BitstreamRemarkParser::processExternalFilePath(
+    Optional<StringRef> ExternalFilePath) {
+  if (!ExternalFilePath)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_META: missing external file path.");
+
+  // External file: open the external file, parse it, check if its metadata
+  // matches the one from the separate metadata, then replace the current parser
+  // with the one parsing the remarks.
+  ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr =
+      MemoryBuffer::getFile(*ExternalFilePath);
+  if (std::error_code EC = BufferOrErr.getError())
+    return errorCodeToError(EC);
+  TmpRemarkBuffer = std::move(*BufferOrErr);
+
+  // Create a separate parser used for parsing the separate file.
+  ParserHelper = BitstreamParserHelper(TmpRemarkBuffer->getBuffer());
+  // Advance and check until we can parse the meta block.
+  if (Error E = advanceToMetaBlock(ParserHelper))
+    return E;
+  // Parse the meta from the separate file.
+  // Note: here we overwrite the BlockInfo with the one from the file. This will
+  // be used to parse the rest of the file.
+  BitstreamMetaParserHelper SeparateMetaHelper(ParserHelper.Stream,
+                                               ParserHelper.BlockInfo);
+  if (Error E = SeparateMetaHelper.parse())
+    return E;
+
+  uint64_t PreviousContainerVersion = ContainerVersion;
+  if (Error E = processCommonMeta(SeparateMetaHelper))
+    return E;
+
+  if (ContainerType != BitstreamRemarkContainerType::SeparateRemarksFile)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing external file's BLOCK_META: wrong container "
+        "type.");
+
+  if (PreviousContainerVersion != ContainerVersion)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing external file's BLOCK_META: mismatching versions: "
+        "original meta: %lu, external file meta: %lu.",
+        PreviousContainerVersion, ContainerVersion);
+
+  // Process the meta from the separate file.
+  return processSeparateRemarksFileMeta(SeparateMetaHelper);
+}
+
+Error BitstreamRemarkParser::processStandaloneMeta(
+    BitstreamMetaParserHelper &Helper) {
+  if (Error E = processStrTab(*this, Helper.StrTabBuf))
+    return E;
+  return processRemarkVersion(*this, Helper.RemarkVersion);
+}
+
+Error BitstreamRemarkParser::processSeparateRemarksFileMeta(
+    BitstreamMetaParserHelper &Helper) {
+  return processRemarkVersion(*this, Helper.RemarkVersion);
+}
+
+Error BitstreamRemarkParser::processSeparateRemarksMetaMeta(
+    BitstreamMetaParserHelper &Helper) {
+  if (Error E = processStrTab(*this, Helper.StrTabBuf))
+    return E;
+  return processExternalFilePath(Helper.ExternalFilePath);
+}
+
+Expected<std::unique_ptr<Remark>> BitstreamRemarkParser::parseRemark() {
+  BitstreamRemarkParserHelper RemarkHelper(ParserHelper.Stream);
+  if (Error E = RemarkHelper.parse())
+    return std::move(E);
+
+  return processRemark(RemarkHelper);
+}
+
+Expected<std::unique_ptr<Remark>>
+BitstreamRemarkParser::processRemark(BitstreamRemarkParserHelper &Helper) {
+  std::unique_ptr<Remark> Result = std::make_unique<Remark>();
+  Remark &R = *Result;
+
+  if (StrTab == None)
+    return createStringError(
+        std::make_error_code(std::errc::invalid_argument),
+        "Error while parsing BLOCK_REMARK: missing string table.");
+
+  if (!Helper.Type)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_REMARK: missing remark type.");
+
+  if (*Helper.Type < static_cast<uint8_t>(Type::First) ||
+      *Helper.Type > static_cast<uint8_t>(Type::Last))
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_REMARK: unknown remark type.");
+
+  R.RemarkType = static_cast<Type>(*Helper.Type);
+
+  if (!Helper.RemarkNameIdx)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_REMARK: missing remark name.");
+
+  if (Expected<StringRef> RemarkName = (*StrTab)[*Helper.RemarkNameIdx])
+    R.RemarkName = *RemarkName;
+  else
+    return RemarkName.takeError();
+
+  if (!Helper.PassNameIdx)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_REMARK: missing remark pass.");
+
+  if (Expected<StringRef> PassName = (*StrTab)[*Helper.PassNameIdx])
+    R.PassName = *PassName;
+  else
+    return PassName.takeError();
+
+  if (!Helper.FunctionNameIdx)
+    return createStringError(
+        std::make_error_code(std::errc::illegal_byte_sequence),
+        "Error while parsing BLOCK_REMARK: missing remark function name.");
+  if (Expected<StringRef> FunctionName = (*StrTab)[*Helper.FunctionNameIdx])
+    R.FunctionName = *FunctionName;
+  else
+    return FunctionName.takeError();
+
+  if (Helper.SourceFileNameIdx && Helper.SourceLine && Helper.SourceColumn) {
+    Expected<StringRef> SourceFileName = (*StrTab)[*Helper.SourceFileNameIdx];
+    if (!SourceFileName)
+      return SourceFileName.takeError();
+    R.Loc.emplace();
+    R.Loc->SourceFilePath = *SourceFileName;
+    R.Loc->SourceLine = *Helper.SourceLine;
+    R.Loc->SourceColumn = *Helper.SourceColumn;
+  }
+
+  if (Helper.Hotness)
+    R.Hotness = *Helper.Hotness;
+
+  if (!Helper.Args)
+    return std::move(Result);
+
+  for (const BitstreamRemarkParserHelper::Argument &Arg : *Helper.Args) {
+    if (!Arg.KeyIdx)
+      return createStringError(
+          std::make_error_code(std::errc::illegal_byte_sequence),
+          "Error while parsing BLOCK_REMARK: missing key in remark argument.");
+    if (!Arg.ValueIdx)
+      return createStringError(
+          std::make_error_code(std::errc::illegal_byte_sequence),
+          "Error while parsing BLOCK_REMARK: missing value in remark "
+          "argument.");
+
+    // We have at least a key and a value, create an entry.
+    R.Args.emplace_back();
+
+    if (Expected<StringRef> Key = (*StrTab)[*Arg.KeyIdx])
+      R.Args.back().Key = *Key;
+    else
+      return Key.takeError();
+
+    if (Expected<StringRef> Value = (*StrTab)[*Arg.ValueIdx])
+      R.Args.back().Val = *Value;
+    else
+      return Value.takeError();
+
+    if (Arg.SourceFileNameIdx && Arg.SourceLine && Arg.SourceColumn) {
+      if (Expected<StringRef> SourceFileName =
+              (*StrTab)[*Arg.SourceFileNameIdx]) {
+        R.Args.back().Loc.emplace();
+        R.Args.back().Loc->SourceFilePath = *SourceFileName;
+        R.Args.back().Loc->SourceLine = *Arg.SourceLine;
+        R.Args.back().Loc->SourceColumn = *Arg.SourceColumn;
+      } else
+        return SourceFileName.takeError();
+    }
+  }
+
+  return std::move(Result);
+}

llvm/lib/Remarks/CMakeLists.txt

 add_llvm_library(LLVMRemarks
+  BitstreamRemarkParser.cpp
   BitstreamRemarkSerializer.cpp
   Remark.cpp
   RemarkFormat.cpp
   RemarkParser.cpp
   RemarkSerializer.cpp
   RemarkStringTable.cpp
   YAMLRemarkParser.cpp
   YAMLRemarkSerializer.cpp
 )

llvm/lib/Remarks/LLVMBuild.txt

 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [component_0]
 type = Library
 name = Remarks
 parent = Libraries
-required_libraries = Support
+required_libraries = BitstreamReader Support

llvm/lib/Remarks/RemarkParser.cpp

 //===- RemarkParser.cpp --------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file provides utility methods used by clients that want to use the
 // parser for remark diagnostics in LLVM.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Remarks/RemarkParser.h"
+#include "BitstreamRemarkParser.h"
 #include "YAMLRemarkParser.h"
 #include "llvm-c/Remarks.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Support/CBindingWrapping.h"
 
 using namespace llvm;
 using namespace llvm::remarks;
 
llvm::remarks::createRemarkParser(Format ParserFormat, StringRef Buf) {
   switch (ParserFormat) {
   case Format::YAML:
     return std::make_unique<YAMLRemarkParser>(Buf);
   case Format::YAMLStrTab:
     return createStringError(
         std::make_error_code(std::errc::invalid_argument),
         "The YAML with string table format requires a parsed string table.");
   case Format::Bitstream:
-    return createStringError(std::make_error_code(std::errc::invalid_argument),
-                             "Parsing bitstream remarks is not supported.");
+    return std::make_unique<BitstreamRemarkParser>(Buf);
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");
   }
   llvm_unreachable("unhandled ParseFormat");
 }
 
 Expected<std::unique_ptr<RemarkParser>>
 llvm::remarks::createRemarkParser(Format ParserFormat, StringRef Buf,
                                   ParsedStringTable StrTab) {
   switch (ParserFormat) {
   case Format::YAML:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "The YAML format can't be used with a string "
                              "table. Use yaml-strtab instead.");
   case Format::YAMLStrTab:
     return std::make_unique<YAMLStrTabRemarkParser>(Buf, std::move(StrTab));
   case Format::Bitstream:
-    return createStringError(std::make_error_code(std::errc::invalid_argument),
-                             "Parsing bitstream remarks is not supported.");
+    return std::make_unique<BitstreamRemarkParser>(Buf, std::move(StrTab));
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");
   }
   llvm_unreachable("unhandled ParseFormat");
 }
 
 Expected<std::unique_ptr<RemarkParser>>
 llvm::remarks::createRemarkParserFromMeta(Format ParserFormat, StringRef Buf,
                                           Optional<ParsedStringTable> StrTab) {
   switch (ParserFormat) {
   // Depending on the metadata, the format can be either yaml or yaml-strtab,
   // regardless of the input argument.
   case Format::YAML:
   case Format::YAMLStrTab:
     return createYAMLParserFromMeta(Buf, std::move(StrTab));
   case Format::Bitstream:
-    return createStringError(std::make_error_code(std::errc::invalid_argument),
-                             "Parsing bitstream remarks is not supported.");
+    return createBitstreamParserFromMeta(Buf, std::move(StrTab));
   case Format::Unknown:
     return createStringError(std::make_error_code(std::errc::invalid_argument),
                              "Unknown remark parser format.");
   }
   llvm_unreachable("unhandled ParseFormat");
 }
 
 namespace {

 DEFINE_SIMPLE_CONVERSION_FUNCTIONS(CParser, LLVMRemarkParserRef)
 
 extern "C" LLVMRemarkParserRef LLVMRemarkParserCreateYAML(const void *Buf,
                                                           uint64_t Size) {
   return wrap(new CParser(Format::YAML,
                           StringRef(static_cast<const char *>(Buf), Size)));
 }
 
+extern "C" LLVMRemarkParserRef LLVMRemarkParserCreateBitstream(const void *Buf,
+                                                               uint64_t Size) {
+  return wrap(new CParser(Format::Bitstream,
+                          StringRef(static_cast<const char *>(Buf), Size)));
+}
+
 extern "C" LLVMRemarkEntryRef
 LLVMRemarkParserGetNext(LLVMRemarkParserRef Parser) {
   CParser &TheCParser = *unwrap(Parser);
   remarks::RemarkParser &TheParser = *TheCParser.TheParser;
 
   Expected<std::unique_ptr<Remark>> MaybeRemark = TheParser.next();
   if (Error E = MaybeRemark.takeError()) {
     if (E.isA<EndOfFileError>()) {

llvm/tools/remarks-shlib/Remarks.exports

 LLVMRemarkEntryGetRemarkName
 LLVMRemarkEntryGetFunctionName
 LLVMRemarkEntryGetDebugLoc
 LLVMRemarkEntryGetHotness
 LLVMRemarkEntryGetNumArgs
 LLVMRemarkEntryGetFirstArg
 LLVMRemarkEntryGetNextArg
 LLVMRemarkParserCreateYAML
+LLVMRemarkParserCreateBitstream
 LLVMRemarkParserGetNext
 LLVMRemarkParserHasError
 LLVMRemarkParserGetErrorMessage
 LLVMRemarkParserDispose
 LLVMRemarkVersion

llvm/unittests/Remarks/BitstreamRemarksParsingTest.cpp

+//===- unittests/Support/BitstreamRemarksParsingTest.cpp - Parsing tests --===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm-c/Remarks.h"
+#include "llvm/Remarks/Remark.h"
+#include "llvm/Remarks/RemarkParser.h"
+#include "llvm/Remarks/RemarkSerializer.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+template <size_t N> void parseGood(const char (&Buf)[N]) {
+  // 1. Parse the YAML remark -> FromYAMLRemark
+  // 2. Serialize it to bitstream -> BSStream
+  // 3. Parse it back -> FromBSRemark
+  // 4. Compare the remark objects
+  //
+  // This testing methodology has the drawback of relying on both the YAML
+  // remark parser and the bitstream remark serializer. It does simplify
+  // testing a lot, since working directly with bitstream is not that easy.
+
+  // 1.
+  Expected<std::unique_ptr<remarks::RemarkParser>> MaybeParser =
+      remarks::createRemarkParser(remarks::Format::YAML, {Buf, N - 1});
+  EXPECT_FALSE(errorToBool(MaybeParser.takeError()));
+  EXPECT_TRUE(*MaybeParser != nullptr);
+
+  std::unique_ptr<remarks::Remark> FromYAMLRemark = nullptr;
+  remarks::RemarkParser &Parser = **MaybeParser;
+  Expected<std::unique_ptr<remarks::Remark>> Remark = Parser.next();
+  EXPECT_FALSE(errorToBool(Remark.takeError())); // Check for parsing errors.
+  EXPECT_TRUE(*Remark != nullptr);               // At least one remark.
+  // Keep the previous remark around.
+  FromYAMLRemark = std::move(*Remark);
+  Remark = Parser.next();
+  Error E = Remark.takeError();
+  EXPECT_TRUE(E.isA<remarks::EndOfFileError>());
+  EXPECT_TRUE(errorToBool(std::move(E))); // Check for parsing errors.
+
+  // 2.
+  remarks::StringTable BSStrTab;
+  BSStrTab.internalize(*FromYAMLRemark);
+  std::string BSBuf;
+  raw_string_ostream BSStream(BSBuf);
+  Expected<std::unique_ptr<remarks::RemarkSerializer>> BSSerializer =
+      remarks::createRemarkSerializer(remarks::Format::Bitstream,
+                                      remarks::SerializerMode::Standalone,
+                                      BSStream, std::move(BSStrTab));
+  EXPECT_FALSE(errorToBool(BSSerializer.takeError()));
+  (*BSSerializer)->emit(*FromYAMLRemark);
+
+  // 3.
+  Expected<std::unique_ptr<remarks::RemarkParser>> MaybeBSParser =
+      remarks::createRemarkParser(remarks::Format::Bitstream, BSStream.str());
+  EXPECT_FALSE(errorToBool(MaybeBSParser.takeError()));
+  EXPECT_TRUE(*MaybeBSParser != nullptr);
+
+  std::unique_ptr<remarks::Remark> FromBSRemark = nullptr;
+  remarks::RemarkParser &BSParser = **MaybeBSParser;
+  Expected<std::unique_ptr<remarks::Remark>> BSRemark = BSParser.next();
+  EXPECT_FALSE(errorToBool(BSRemark.takeError())); // Check for parsing errors.
+  EXPECT_TRUE(*BSRemark != nullptr);               // At least one remark.
+  // Keep the previous remark around.
+  FromBSRemark = std::move(*BSRemark);
+  BSRemark = BSParser.next();
+  Error BSE = BSRemark.takeError();
+  EXPECT_TRUE(BSE.isA<remarks::EndOfFileError>());
+  EXPECT_TRUE(errorToBool(std::move(BSE))); // Check for parsing errors.
+
+  EXPECT_EQ(*FromYAMLRemark, *FromBSRemark);
+}
+
+TEST(BitstreamRemarks, ParsingGood) {
+  parseGood("\n"
+            "--- !Missed\n"
+            "Pass: inline\n"
+            "Name: NoDefinition\n"
+            "DebugLoc: { File: file.c, Line: 3, Column: 12 }\n"
+            "Function: foo\n"
+            "Args:\n"
+            "  - Callee: bar\n"
+            "  - String: ' will not be inlined into '\n"
+            "  - Caller: foo\n"
+            "    DebugLoc: { File: file.c, Line: 2, Column: 0 }\n"
+            "  - String: ' because its definition is unavailable'\n"
+            "");
+
+  // No debug loc should also pass.
+  parseGood("\n"
+            "--- !Missed\n"
+            "Pass: inline\n"
+            "Name: NoDefinition\n"
+            "Function: foo\n"
+            "Args:\n"
+            "  - Callee: bar\n"
+            "  - String: ' will not be inlined into '\n"
+            "  - Caller: foo\n"
+            "    DebugLoc: { File: file.c, Line: 2, Column: 0 }\n"
+            "  - String: ' because its definition is unavailable'\n"
+            "");
+
+  // No args is also ok.
+  parseGood("\n"
+            "--- !Missed\n"
+            "Pass: inline\n"
+            "Name: NoDefinition\n"
+            "DebugLoc: { File: file.c, Line: 3, Column: 12 }\n"
+            "Function: foo\n"
+            "");
+}
+
+// Mandatory common part of a remark.
+#define COMMON_REMARK "\nPass: inline\nName: NoDefinition\nFunction: foo\n\n"
+// Test all the types.
+TEST(BitstreamRemarks, ParsingTypes) {
+  // Type: Passed
+  parseGood("--- !Passed" COMMON_REMARK);
+  // Type: Missed
+  parseGood("--- !Missed" COMMON_REMARK);
+  // Type: Analysis
+  parseGood("--- !Analysis" COMMON_REMARK);
+  // Type: AnalysisFPCommute
+  parseGood("--- !AnalysisFPCommute" COMMON_REMARK);
+  // Type: AnalysisAliasing
+  parseGood("--- !AnalysisAliasing" COMMON_REMARK);
+  // Type: Failure
+  parseGood("--- !Failure" COMMON_REMARK);
+}
+#undef COMMON_REMARK
+
+static inline StringRef checkStr(StringRef Str, unsigned ExpectedLen) {
+  const char *StrData = Str.data();
+  unsigned StrLen = Str.size();
+  EXPECT_EQ(StrLen, ExpectedLen);
+  return StringRef(StrData, StrLen);
+}
+
+TEST(BitstreamRemarks, Contents) {
+  StringRef Buf = "\n"
+                  "--- !Missed\n"
+                  "Pass: inline\n"
+                  "Name: NoDefinition\n"
+                  "DebugLoc: { File: file.c, Line: 3, Column: 12 }\n"
+                  "Function: foo\n"
+                  "Hotness: 4\n"
+                  "Args:\n"
+                  "  - Callee: bar\n"
+                  "  - String: ' will not be inlined into '\n"
+                  "  - Caller: foo\n"
+                  "    DebugLoc: { File: file.c, Line: 2, Column: 0 }\n"
+                  "  - String: ' because its definition is unavailable'\n"
+                  "\n";
+
+  Expected<std::unique_ptr<remarks::RemarkParser>> MaybeParser =
+      remarks::createRemarkParser(remarks::Format::YAML, Buf);
+  EXPECT_FALSE(errorToBool(MaybeParser.takeError()));
+  EXPECT_TRUE(*MaybeParser != nullptr);
+
+  remarks::RemarkParser &Parser = **MaybeParser;
+  Expected<std::unique_ptr<remarks::Remark>> MaybeRemark = Parser.next();
+  EXPECT_FALSE(
+      errorToBool(MaybeRemark.takeError())); // Check for parsing errors.
+  EXPECT_TRUE(*MaybeRemark != nullptr);      // At least one remark.
+
+  const remarks::Remark &Remark = **MaybeRemark;
+  EXPECT_EQ(Remark.RemarkType, remarks::Type::Missed);
+  EXPECT_EQ(checkStr(Remark.PassName, 6), "inline");
+  EXPECT_EQ(checkStr(Remark.RemarkName, 12), "NoDefinition");
+  EXPECT_EQ(checkStr(Remark.FunctionName, 3), "foo");
+  EXPECT_TRUE(Remark.Loc);
+  const remarks::RemarkLocation &RL = *Remark.Loc;
+  EXPECT_EQ(checkStr(RL.SourceFilePath, 6), "file.c");
+  EXPECT_EQ(RL.SourceLine, 3U);
+  EXPECT_EQ(RL.SourceColumn, 12U);
+  EXPECT_TRUE(Remark.Hotness);
+  EXPECT_EQ(*Remark.Hotness, 4U);
+  EXPECT_EQ(Remark.Args.size(), 4U);
+
+  unsigned ArgID = 0;
+  for (const remarks::Argument &Arg : Remark.Args) {
+    switch (ArgID) {
+    case 0:
+      EXPECT_EQ(checkStr(Arg.Key, 6), "Callee");
+      EXPECT_EQ(checkStr(Arg.Val, 3), "bar");
+      EXPECT_FALSE(Arg.Loc);
+      break;
+    case 1:
+      EXPECT_EQ(checkStr(Arg.Key, 6), "String");
+      EXPECT_EQ(checkStr(Arg.Val, 26), " will not be inlined into ");
+      EXPECT_FALSE(Arg.Loc);
+      break;
+    case 2: {
+      EXPECT_EQ(checkStr(Arg.Key, 6), "Caller");
+      EXPECT_EQ(checkStr(Arg.Val, 3), "foo");
+      EXPECT_TRUE(Arg.Loc);
+      const remarks::RemarkLocation &RL = *Arg.Loc;
+      EXPECT_EQ(checkStr(RL.SourceFilePath, 6), "file.c");
+      EXPECT_EQ(RL.SourceLine, 2U);
+      EXPECT_EQ(RL.SourceColumn, 0U);
+      break;
+    }
+    case 3:
+      EXPECT_EQ(checkStr(Arg.Key, 6), "String");
+      EXPECT_EQ(checkStr(Arg.Val, 38),
+                " because its definition is unavailable");
+      EXPECT_FALSE(Arg.Loc);
+      break;
+    default:
+      break;
+    }
+    ++ArgID;
+  }
+
+  MaybeRemark = Parser.next();
+  Error E = MaybeRemark.takeError();
+  EXPECT_TRUE(E.isA<remarks::EndOfFileError>());
+  EXPECT_TRUE(errorToBool(std::move(E))); // Check for parsing errors.
+}
+
+static inline StringRef checkStr(LLVMRemarkStringRef Str,
+                                 unsigned ExpectedLen) {
+  const char *StrData = LLVMRemarkStringGetData(Str);
+  unsigned StrLen = LLVMRemarkStringGetLen(Str);
+  EXPECT_EQ(StrLen, ExpectedLen);
+  return StringRef(StrData, StrLen);
+}
+
+TEST(BitstreamRemarks, ContentsCAPI) {
+  remarks::StringTable BSStrTab;
+  remarks::Remark ToSerializeRemark;
+  ToSerializeRemark.RemarkType = remarks::Type::Missed;
+  ToSerializeRemark.PassName = "inline";
+  ToSerializeRemark.RemarkName = "NoDefinition";
+  ToSerializeRemark.FunctionName = "foo";
+  ToSerializeRemark.Loc = remarks::RemarkLocation{"file.c", 3, 12};
+  ToSerializeRemark.Hotness = 0;
+  ToSerializeRemark.Args.emplace_back();
+  ToSerializeRemark.Args.back().Key = "Callee";
+  ToSerializeRemark.Args.back().Val = "bar";
+  ToSerializeRemark.Args.emplace_back();
+  ToSerializeRemark.Args.back().Key = "String";
+  ToSerializeRemark.Args.back().Val = " will not be inlined into ";
+  ToSerializeRemark.Args.emplace_back();
+  ToSerializeRemark.Args.back().Key = "Caller";
+  ToSerializeRemark.Args.back().Val = "foo";
+  ToSerializeRemark.Args.back().Loc = remarks::RemarkLocation{"file.c", 2, 0};
+  ToSerializeRemark.Args.emplace_back();
+  ToSerializeRemark.Args.back().Key = "String";
+  ToSerializeRemark.Args.back().Val = " because its definition is unavailable";
+  BSStrTab.internalize(ToSerializeRemark);
+  std::string BSBuf;
+  raw_string_ostream BSStream(BSBuf);
+  Expected<std::unique_ptr<remarks::RemarkSerializer>> BSSerializer =
+      remarks::createRemarkSerializer(remarks::Format::Bitstream,
+                                      remarks::SerializerMode::Standalone,
+                                      BSStream, std::move(BSStrTab));
+  EXPECT_FALSE(errorToBool(BSSerializer.takeError()));
+  (*BSSerializer)->emit(ToSerializeRemark);
+
+  StringRef Buf = BSStream.str();
+  LLVMRemarkParserRef Parser =
+      LLVMRemarkParserCreateBitstream(Buf.data(), Buf.size());
+  LLVMRemarkEntryRef Remark = LLVMRemarkParserGetNext(Parser);
+  EXPECT_FALSE(Remark == nullptr);
+  EXPECT_EQ(LLVMRemarkEntryGetType(Remark), LLVMRemarkTypeMissed);
+  EXPECT_EQ(checkStr(LLVMRemarkEntryGetPassName(Remark), 6), "inline");
+  EXPECT_EQ(checkStr(LLVMRemarkEntryGetRemarkName(Remark), 12), "NoDefinition");
+  EXPECT_EQ(checkStr(LLVMRemarkEntryGetFunctionName(Remark), 3), "foo");
+  LLVMRemarkDebugLocRef DL = LLVMRemarkEntryGetDebugLoc(Remark);
+  EXPECT_EQ(checkStr(LLVMRemarkDebugLocGetSourceFilePath(DL), 6), "file.c");
+  EXPECT_EQ(LLVMRemarkDebugLocGetSourceLine(DL), 3U);
+  EXPECT_EQ(LLVMRemarkDebugLocGetSourceColumn(DL), 12U);
+  EXPECT_EQ(LLVMRemarkEntryGetHotness(Remark), 0U);
+  EXPECT_EQ(LLVMRemarkEntryGetNumArgs(Remark), 4U);
+
+  unsigned ArgID = 0;
+  LLVMRemarkArgRef Arg = LLVMRemarkEntryGetFirstArg(Remark);
+  do {
+    switch (ArgID) {
+    case 0:
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetKey(Arg), 6), "Callee");
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetValue(Arg), 3), "bar");
+      EXPECT_EQ(LLVMRemarkArgGetDebugLoc(Arg), nullptr);
+      break;
+    case 1:
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetKey(Arg), 6), "String");
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetValue(Arg), 26),
+                " will not be inlined into ");
+      EXPECT_EQ(LLVMRemarkArgGetDebugLoc(Arg), nullptr);
+      break;
+    case 2: {
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetKey(Arg), 6), "Caller");
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetValue(Arg), 3), "foo");
+      LLVMRemarkDebugLocRef DL = LLVMRemarkArgGetDebugLoc(Arg);
+      EXPECT_EQ(checkStr(LLVMRemarkDebugLocGetSourceFilePath(DL), 6), "file.c");
+      EXPECT_EQ(LLVMRemarkDebugLocGetSourceLine(DL), 2U);
+      EXPECT_EQ(LLVMRemarkDebugLocGetSourceColumn(DL), 0U);
+      break;
+    }
+    case 3:
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetKey(Arg), 6), "String");
+      EXPECT_EQ(checkStr(LLVMRemarkArgGetValue(Arg), 38),
+                " because its definition is unavailable");
+      EXPECT_EQ(LLVMRemarkArgGetDebugLoc(Arg), nullptr);
+      break;
+    default:
+      break;
+    }
+    ++ArgID;
+  } while ((Arg = LLVMRemarkEntryGetNextArg(Arg, Remark)));
+
+  LLVMRemarkEntryDispose(Remark);
+
+  EXPECT_EQ(LLVMRemarkParserGetNext(Parser), nullptr);
+
+  EXPECT_FALSE(LLVMRemarkParserHasError(Parser));
+  LLVMRemarkParserDispose(Parser);
+}
+
+static void parseBad(StringRef Input, const char *ErrorMsg) {
+  Expected<std::unique_ptr<remarks::RemarkParser>> MaybeBSParser =
+      remarks::createRemarkParser(remarks::Format::Bitstream, Input);
+  EXPECT_FALSE(errorToBool(MaybeBSParser.takeError()));
+  EXPECT_TRUE(*MaybeBSParser != nullptr);
+
+  remarks::RemarkParser &BSParser = **MaybeBSParser;
+  Expected<std::unique_ptr<remarks::Remark>> BSRemark = BSParser.next();
+  EXPECT_EQ(ErrorMsg, toString(BSRemark.takeError())); // Expect an error.
+}
+
+TEST(BitstreamRemarks, ParsingEmpty) {
+  parseBad(StringRef(), "End of file reached.");
+}
+
+TEST(BitstreamRemarks, ParsingBadMagic) {
+  parseBad("KRMR", "Unknown magic number: expecting RMRK, got KRMR.");
+}
+
+// Testing malformed bitstream is not easy. We would need to replace bytes in
+// the stream to create malformed and unknown records and blocks. There is no
+// textual format for bitstream that can be decoded, modified and encoded
+// back.
+
+// FIXME: Add tests for the following error messages:
+// * Error while parsing META_BLOCK: malformed record entry
+// (RECORD_META_CONTAINER_INFO).
+// * Error while parsing META_BLOCK: malformed record entry
+// (RECORD_META_REMARK_VERSION).
+// * Error while parsing META_BLOCK: malformed record entry
+// (RECORD_META_STRTAB).
+// * Error while parsing META_BLOCK: malformed record entry
+// (RECORD_META_EXTERNAL_FILE).
+// * Error while parsing META_BLOCK: unknown record entry (NUM).
+// * Error while parsing REMARK_BLOCK: malformed record entry
+// (RECORD_REMARK_HEADER).
+// * Error while parsing REMARK_BLOCK: malformed record entry
+// (RECORD_REMARK_DEBUG_LOC).
+// * Error while parsing REMARK_BLOCK: malformed record entry
+// (RECORD_REMARK_HOTNESS).
+// * Error while parsing REMARK_BLOCK: malformed record entry
+// (RECORD_REMARK_ARG_WITH_DEBUGLOC).
+// * Error while parsing REMARK_BLOCK: malformed record entry
+// (RECORD_REMARK_ARG_WITHOUT_DEBUGLOC).
+// * Error while parsing REMARK_BLOCK: unknown record entry (NUM).
+// * Error while parsing META_BLOCK: expecting [ENTER_SUBBLOCO, META_BLOCK,
+// ...].
+// * Error while entering META_BLOCK.
+// * Error while parsing META_BLOCK: expecting records.
+// * Error while parsing META_BLOCK: unterminated block.
+// * Error while parsing REMARK_BLOCK: expecting [ENTER_SUBBLOCO, REMARK_BLOCK,
+// ...].
+// * Error while entering REMARK_BLOCK.
+// * Error while parsing REMARK_BLOCK: expecting records.
+// * Error while parsing REMARK_BLOCK: unterminated block.
+// * Error while parsing BLOCKINFO_BLOCK: expecting [ENTER_SUBBLOCK,
+// BLOCKINFO_BLOCK, ...].
+// * Error while parsing BLOCKINFO_BLOCK.
+// * Unexpected error while parsing bitstream.
+// * Expecting META_BLOCK after the BLOCKINFO_BLOCK.
+// * Error while parsing BLOCK_META: missing container version.
+// * Error while parsing BLOCK_META: invalid container type.
+// * Error while parsing BLOCK_META: missing container type.
+// * Error while parsing BLOCK_META: missing string table.
+// * Error while parsing BLOCK_META: missing remark version.
+// * Error while parsing BLOCK_META: missing external file path.
+// * Error while parsing external file's BLOCK_META: wrong container type.
+// * Error while parsing external file's BLOCK_META: mismatching versions:
+// original meta: NUM, external file meta: NUM.
+// * Error while parsing BLOCK_REMARK: missing string table.
+// * Error while parsing BLOCK_REMARK: missing remark type.
+// * Error while parsing BLOCK_REMARK: unknown remark type.
+// * Error while parsing BLOCK_REMARK: missing remark name.
+// * Error while parsing BLOCK_REMARK: missing remark pass.
+// * Error while parsing BLOCK_REMARK: missing remark function name.
+// * Error while parsing BLOCK_REMARK: missing key in remark argument.
+// * Error while parsing BLOCK_REMARK: missing value in remark argument.

llvm/unittests/Remarks/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   BitReader
   Remarks
   Support
   )
 
 add_llvm_unittest(RemarksTests
   BitstreamRemarksFormatTest.cpp
+  BitstreamRemarksParsingTest.cpp
   BitstreamRemarksSerializerTest.cpp
   RemarksAPITest.cpp
   RemarksStrTabParsingTest.cpp
   YAMLRemarksParsingTest.cpp
   YAMLRemarksSerializerTest.cpp
   )