Index: llvm/trunk/docs/Remarks.rst =================================================================== --- llvm/trunk/docs/Remarks.rst +++ llvm/trunk/docs/Remarks.rst @@ -113,6 +113,7 @@ * :ref:`yaml ` (default) * :ref:`yaml-strtab ` + * :ref:`bitstream ` ``Content configuration`` @@ -260,6 +261,250 @@ In case the metadata only acts as a header to the remarks, the file path can be omitted. +.. _bitstreamremarks: + +LLVM bitstream remarks +====================== + +This format is using :doc:`LLVM bitstream ` to serialize remarks +and their associated metadata. + +A bitstream remark stream can be identified by the magic number ``"RMRK"`` that +is placed at the very beginning. + +The format for serializing remarks is composed of two different block types: + +.. _bitstreamremarksmetablock: + +META_BLOCK +---------- + +The block providing information about the rest of the content in the stream. + +Exactly one block is expected. Having multiple metadata blocks is an error. + +This block can contain the following records: + +.. _bitstreamremarksrecordmetacontainerinfo: + +``RECORD_META_CONTAINER_INFO`` + + The container version and type. + + Version: u32 + + Type: u2 + +.. _bitstreamremarksrecordmetaremarkversion: + +``RECORD_META_REMARK_VERSION`` + + The version of the remark entries. This can change independently from the + container version. + + Version: u32 + +.. _bitstreamremarksrecordmetastrtab: + +``RECORD_META_STRTAB`` + + The string table used by the remark entries. The format of the string table + is a sequence of strings separated by ``\0``. + +.. _bitstreamremarksrecordmetaexternalfile: + +``RECORD_META_EXTERNAL_FILE`` + + The external remark file path that contains the remark blocks associated + with this metadata. This is an absolute path. + +.. _bitstreamremarksremarkblock: + +REMARK_BLOCK +------------ + +The block describing a remark entry. + +0 or more blocks per file are allowed. Each block will depend on the +:ref:`META_BLOCK ` in order to be parsed correctly. + +This block can contain the following records: + +``RECORD_REMARK_HEADER`` + + The header of the remark. This contains all the mandatory information about + a remark. + + +---------------+---------------------------+ + | Type | u3 | + +---------------+---------------------------+ + | Remark name | VBR6 (string table index) | + +---------------+---------------------------+ + | Pass name | VBR6 (string table index) | + +---------------+---------------------------+ + | Function name | VBR6 (string table index) | + +---------------+---------------------------+ + +``RECORD_REMARK_DEBUG_LOC`` + + The source location for the corresponding remark. This record is optional. + + +--------+---------------------------+ + | File | VBR7 (string table index) | + +--------+---------------------------+ + | Line | u32 | + +--------+---------------------------+ + | Column | u32 | + +--------+---------------------------+ + +``RECORD_REMARK_HOTNESS`` + + The hotness of the remark. This record is optional. + + +---------------+---------------------+ + | Hotness | VBR8 (string table index) | + +---------------+---------------------+ + +``RECORD_REMARK_ARG_WITH_DEBUGLOC`` + + A remark argument with an associated debug location. + + +--------+---------------------------+ + | Key | VBR7 (string table index) | + +--------+---------------------------+ + | Value | VBR7 (string table index) | + +--------+---------------------------+ + | File | VBR7 (string table index) | + +--------+---------------------------+ + | Line | u32 | + +--------+---------------------------+ + | Column | u32 | + +--------+---------------------------+ + +``RECORD_REMARK_ARG_WITHOUT_DEBUGLOC`` + + A remark argument with an associated debug location. + + +--------+---------------------------+ + | Key | VBR7 (string table index) | + +--------+---------------------------+ + | Value | VBR7 (string table index) | + +--------+---------------------------+ + +The remark container +-------------------- + +Bitstream remarks are designed to be used in two different modes: + +``The separate mode`` + + The separate mode is the mode that is typically used during compilation. It + provides a way to serialize the remark entries to a stream while some + metadata is kept in memory to be emitted in the product of the compilation + (typically, an object file). + +``The standalone mode`` + + The standalone mode is typically stored and used after the distribution of + a program. It contains all the information that allows the parsing of all + the remarks without having any external dependencies. + +In order to support multiple modes, the format introduces the concept of a +bitstream remark container type. + +.. _bitstreamremarksseparateremarksmeta: + +``SeparateRemarksMeta: the metadata emitted separately`` + + This container type expects only a :ref:`META_BLOCK ` containing only: + + * :ref:`RECORD_META_CONTAINER_INFO ` + * :ref:`RECORD_META_STRTAB ` + * :ref:`RECORD_META_EXTERNAL_FILE ` + + Typically, this is emitted in a section in the object files, allowing + clients to retrieve remarks and their associated metadata directly from + intermediate products. + +``SeparateRemarksFile: the remark entries emitted separately`` + + This container type expects only a :ref:`META_BLOCK ` containing only: + + * :ref:`RECORD_META_CONTAINER_INFO ` + * :ref:`RECORD_META_REMARK_VERSION ` + + This container type expects 0 or more :ref:`REMARK_BLOCK `. + + Typically, this is emitted in a side-file alongside an object file, and is + made to be able to stream to without increasing the memory consumption of + the compiler. This is referenced by the :ref:`RECORD_META_EXTERNAL_FILE + ` entry in the + :ref:`SeparateRemarksMeta ` container. + +When the parser tries to parse a container that contains the metadata for the +separate remarks, it should parse the version and type, then keep the string +table in memory while opening the external file, validating its metadata and +parsing the remark entries. + +The container versions from the separate container should match in order to +have a well-formed file. + +``Standalone: the metadata and the remark entries emitted together`` + + This container type expects only a :ref:`META_BLOCK ` containing only: + + * :ref:`RECORD_META_CONTAINER_INFO ` + * :ref:`RECORD_META_REMARK_VERSION ` + * :ref:`RECORD_META_STRTAB ` + + This container type expects 0 or more :ref:`REMARK_BLOCK `. + +A complete output of :program:`llvm-bcanalyzer` on the different container types: + +``SeparateRemarksMeta`` + +.. code-block:: none + + + + + blob data = 'pass\\x00key\\x00value\\x00' + blob data = '/path/to/file/name' + + +``SeparateRemarksFile`` + +.. code-block:: none + + + + + + + + + + + + + +``Standalone`` + +.. code-block:: none + + + + + + blob data = 'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x00' + + + + + + + + opt-viewer ========== Index: llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h =================================================================== --- llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h +++ llvm/trunk/include/llvm/Bitcode/BitcodeAnalyzer.h @@ -30,6 +30,7 @@ LLVMIRBitstream, ClangSerializedASTBitstream, ClangSerializedDiagnosticsBitstream, + LLVMBitstreamRemarks }; struct BCDumpOptions { Index: llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h =================================================================== --- llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h +++ llvm/trunk/include/llvm/Remarks/BitstreamRemarkContainer.h @@ -0,0 +1,106 @@ +//===-- BitstreamRemarkContainer.h - Container for remarks --------------*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +// +// This file provides declarations for things used in the various types of +// remark containers. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_REMARKS_REMARK_CONTAINER_H +#define LLVM_REMARKS_REMARK_CONTAINER_H + +#include "llvm/ADT/StringRef.h" +#include "llvm/Bitstream/BitCodes.h" +#include + +namespace llvm { +namespace remarks { + +/// The current version of the remark container. +/// Note: this is different from the version of the remark entry. +constexpr uint64_t CurrentContainerVersion = 0; +/// The magic number used for identifying remark blocks. +constexpr StringRef ContainerMagic("RMRK", 4); + +/// Type of the remark container. +/// The remark container has two modes: +/// * separate: the metadata is separate from the remarks and points to the +/// auxiliary file that contains the remarks. +/// * standalone: the metadata and the remarks are emitted together. +enum class BitstreamRemarkContainerType { + /// The metadata emitted separately. + /// This will contain the following: + /// * Container version and type + /// * String table + /// * External file + SeparateRemarksMeta, + /// The remarks emitted separately. + /// This will contain the following: + /// * Container version and type + /// * Remark version + SeparateRemarksFile, + /// Everything is emitted together. + /// This will contain the following: + /// * Container version and type + /// * Remark version + /// * String table + Standalone, + First = SeparateRemarksMeta, + Last = Standalone, +}; + +/// The possible blocks that will be encountered in a bitstream remark +/// container. +enum BlockIDs { + /// The metadata block is mandatory. It should always come after the + /// BLOCKINFO_BLOCK, and contains metadata that should be used when parsing + /// REMARK_BLOCKs. + /// There should always be only one META_BLOCK. + META_BLOCK_ID = bitc::FIRST_APPLICATION_BLOCKID, + /// One remark entry is represented using a REMARK_BLOCK. There can be + /// multiple REMARK_BLOCKs in the same file. + REMARK_BLOCK_ID +}; + +constexpr StringRef MetaBlockName = StringRef("Meta", 4); +constexpr StringRef RemarkBlockName = StringRef("Remark", 6); + +/// The possible records that can be encountered in the previously described +/// blocks. +enum RecordIDs { + // Meta block records. + RECORD_META_CONTAINER_INFO = 1, + RECORD_META_REMARK_VERSION, + RECORD_META_STRTAB, + RECORD_META_EXTERNAL_FILE, + // Remark block records. + RECORD_REMARK_HEADER, + RECORD_REMARK_DEBUG_LOC, + RECORD_REMARK_HOTNESS, + RECORD_REMARK_ARG_WITH_DEBUGLOC, + RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, + // Helpers. + RECORD_FIRST = RECORD_META_CONTAINER_INFO, + RECORD_LAST = RECORD_REMARK_ARG_WITHOUT_DEBUGLOC +}; + +constexpr StringRef MetaContainerInfoName = StringRef("Container info", 14); +constexpr StringRef MetaRemarkVersionName = StringRef("Remark version", 14); +constexpr StringRef MetaStrTabName = StringRef("String table", 12); +constexpr StringRef MetaExternalFileName = StringRef("External File", 13); +constexpr StringRef RemarkHeaderName = StringRef("Remark header", 13); +constexpr StringRef RemarkDebugLocName = StringRef("Remark debug location", 21); +constexpr StringRef RemarkHotnessName = StringRef("Remark hotness", 14); +constexpr StringRef RemarkArgWithDebugLocName = + StringRef("Argument with debug location", 28); +constexpr StringRef RemarkArgWithoutDebugLocName = StringRef("Argument", 8); + +} // end namespace remarks +} // end namespace llvm + +#endif /* LLVM_REMARKS_REMARK_CONTAINER_H */ Index: llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h =================================================================== --- llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h +++ llvm/trunk/include/llvm/Remarks/BitstreamRemarkSerializer.h @@ -0,0 +1,188 @@ +//===-- BitstreamRemarkSerializer.h - Bitstream serializer ------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +// +// This file provides an implementation of the serializer using the LLVM +// Bitstream format. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H +#define LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H + +#include "llvm/Bitstream/BitstreamWriter.h" +#include "llvm/Remarks/BitstreamRemarkContainer.h" +#include "llvm/Remarks/RemarkSerializer.h" +#include "llvm/Support/raw_ostream.h" + +namespace llvm { +namespace remarks { + +/// Serialize the remarks to LLVM bitstream. +/// This class provides ways to emit remarks in the LLVM bitstream format and +/// its associated metadata. +/// +/// * The separate model: +/// Separate meta: | Container info +/// | String table +/// | External file +/// +/// Separate remarks: | Container info +/// | Remark version +/// | Remark0 +/// | Remark1 +/// | Remark2 +/// | ... +/// +/// * The standalone model: | Container info +/// | String table +/// | Remark version +/// | Remark0 +/// | Remark1 +/// | Remark2 +/// | ... +/// +struct BitstreamSerializerHelper { + /// Buffer used for encoding the bitstream before writing it to the final + /// stream. + SmallVector Encoded; + /// Buffer used to construct records and pass to the bitstream writer. + SmallVector R; + /// The Bitstream writer. + BitstreamWriter Bitstream; + /// The type of the container we are serializing. + BitstreamRemarkContainerType ContainerType; + + /// Abbrev IDs initialized in the block info block. + /// Note: depending on the container type, some IDs might be uninitialized. + /// Warning: When adding more abbrev IDs, make sure to update the + /// BlockCodeSize (in the call to EnterSubblock). + uint64_t RecordMetaContainerInfoAbbrevID = 0; + uint64_t RecordMetaRemarkVersionAbbrevID = 0; + uint64_t RecordMetaStrTabAbbrevID = 0; + uint64_t RecordMetaExternalFileAbbrevID = 0; + uint64_t RecordRemarkHeaderAbbrevID = 0; + uint64_t RecordRemarkDebugLocAbbrevID = 0; + uint64_t RecordRemarkHotnessAbbrevID = 0; + uint64_t RecordRemarkArgWithDebugLocAbbrevID = 0; + uint64_t RecordRemarkArgWithoutDebugLocAbbrevID = 0; + + BitstreamSerializerHelper(BitstreamRemarkContainerType ContainerType); + + // Disable copy and move: Bitstream points to Encoded, which needs special + // handling during copy/move, but moving the vectors is probably useless + // anyway. + BitstreamSerializerHelper(const BitstreamSerializerHelper &) = delete; + BitstreamSerializerHelper & + operator=(const BitstreamSerializerHelper &) = delete; + BitstreamSerializerHelper(BitstreamSerializerHelper &&) = delete; + BitstreamSerializerHelper &operator=(BitstreamSerializerHelper &&) = delete; + + /// Set up the necessary block info entries according to the container type. + void setupBlockInfo(); + + /// Set up the block info for the metadata block. + void setupMetaBlockInfo(); + /// The remark version in the metadata block. + void setupMetaRemarkVersion(); + void emitMetaRemarkVersion(uint64_t RemarkVersion); + /// The strtab in the metadata block. + void setupMetaStrTab(); + void emitMetaStrTab(const StringTable &StrTab); + /// The external file in the metadata block. + void setupMetaExternalFile(); + void emitMetaExternalFile(StringRef Filename); + + /// The block info for the remarks block. + void setupRemarkBlockInfo(); + + /// Emit the metadata for the remarks. + void emitMetaBlock(uint64_t ContainerVersion, + Optional RemarkVersion, + Optional StrTab = None, + Optional Filename = None); + + /// Emit a remark block. The string table is required. + void emitRemarkBlock(const Remark &Remark, StringTable &StrTab); + /// Finalize the writing to \p OS. + void flushToStream(raw_ostream &OS); + /// Finalize the writing to a buffer. + /// The contents of the buffer remain valid for the lifetime of the object. + /// Any call to any other function in this class will invalidate the buffer. + StringRef getBuffer(); +}; + +/// Implementation of the remark serializer using LLVM bitstream. +struct BitstreamSerializer : public RemarkSerializer { + /// The file should contain: + /// 1) The block info block that describes how to read the blocks. + /// 2) The metadata block that contains various information about the remarks + /// in the file. + /// 3) A number of remark blocks. + + /// We need to set up 1) and 2) first, so that we can emit 3) after. This flag + /// is used to emit the first two blocks only once. + bool DidSetUp = false; + /// The helper to emit bitstream. + BitstreamSerializerHelper Helper; + + /// Construct a serializer that will create its own string table. + BitstreamSerializer(raw_ostream &OS, SerializerMode Mode); + /// Construct a serializer with a pre-filled string table. + BitstreamSerializer(raw_ostream &OS, SerializerMode Mode, StringTable StrTab); + + /// Emit a remark to the stream. This also emits the metadata associated to + /// the remarks based on the SerializerMode specified at construction. + /// This writes the serialized output to the provided stream. + void emit(const Remark &Remark) override; + /// The metadata serializer associated to this remark serializer. Based on the + /// container type of the current serializer, the container type of the + /// metadata serializer will change. + std::unique_ptr + metaSerializer(raw_ostream &OS, + Optional ExternalFilename = None) override; +}; + +/// Serializer of metadata for bitstream remarks. +struct BitstreamMetaSerializer : public MetaSerializer { + /// This class can be used with [1] a pre-constructed + /// BitstreamSerializerHelper, or with [2] one that is owned by the meta + /// serializer. In case of [1], we need to be able to store a reference to the + /// object, while in case of [2] we need to store the whole object. + Optional TmpHelper; + /// The actual helper, that can point to \p TmpHelper or to an external helper + /// object. + BitstreamSerializerHelper *Helper = nullptr; + + Optional StrTab; + Optional ExternalFilename; + + /// Create a new meta serializer based on \p ContainerType. + BitstreamMetaSerializer(raw_ostream &OS, + BitstreamRemarkContainerType ContainerType, + Optional StrTab = None, + Optional ExternalFilename = None) + : MetaSerializer(OS), TmpHelper(None), Helper(nullptr), StrTab(StrTab), + ExternalFilename(ExternalFilename) { + TmpHelper.emplace(ContainerType); + Helper = &*TmpHelper; + } + + /// Create a new meta serializer based on a previously built \p Helper. + BitstreamMetaSerializer(raw_ostream &OS, BitstreamSerializerHelper &Helper, + Optional StrTab = None, + Optional ExternalFilename = None) + : MetaSerializer(OS), TmpHelper(None), Helper(&Helper), StrTab(StrTab), + ExternalFilename(ExternalFilename) {} + + void emit() override; +}; + +} // end namespace remarks +} // end namespace llvm + +#endif /* LLVM_REMARKS_BITSTREAM_REMARK_SERIALIZER_H */ Index: llvm/trunk/include/llvm/Remarks/Remark.h =================================================================== --- llvm/trunk/include/llvm/Remarks/Remark.h +++ llvm/trunk/include/llvm/Remarks/Remark.h @@ -23,7 +23,8 @@ namespace llvm { namespace remarks { -constexpr uint64_t Version = 0; +/// The current version of the remark entry. +constexpr uint64_t CurrentRemarkVersion = 0; /// The debug location used to track a remark back to the source file. struct RemarkLocation { @@ -58,7 +59,8 @@ AnalysisFPCommute, AnalysisAliasing, Failure, - LastTypeValue = Failure + First = Unknown, + Last = Failure }; /// A remark type used for both emission and parsing. Index: llvm/trunk/include/llvm/Remarks/RemarkFormat.h =================================================================== --- llvm/trunk/include/llvm/Remarks/RemarkFormat.h +++ llvm/trunk/include/llvm/Remarks/RemarkFormat.h @@ -22,7 +22,7 @@ constexpr StringRef Magic("REMARKS", 7); /// The format used for serializing/deserializing remarks. -enum class Format { Unknown, YAML, YAMLStrTab }; +enum class Format { Unknown, YAML, YAMLStrTab, Bitstream }; /// Parse and validate a string for the remark format. Expected parseFormat(StringRef FormatStr); Index: llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp =================================================================== --- llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp +++ llvm/trunk/lib/Bitcode/Reader/BitcodeAnalyzer.cpp @@ -434,6 +434,13 @@ return std::move(Err); if (Signature[2] == 'A' && Signature[3] == 'G') return ClangSerializedDiagnosticsBitstream; + } else if (Signature[0] == 'R' && Signature[1] == 'M') { + if (Error Err = tryRead(Signature[2], 8)) + return std::move(Err); + if (Error Err = tryRead(Signature[3], 8)) + return std::move(Err); + if (Signature[2] == 'R' && Signature[3] == 'K') + return LLVMBitstreamRemarks; } else { if (Error Err = tryRead(Signature[2], 4)) return std::move(Err); @@ -627,6 +634,9 @@ case ClangSerializedDiagnosticsBitstream: O.OS << "Clang Serialized Diagnostics\n"; break; + case LLVMBitstreamRemarks: + O.OS << "LLVM Remarks\n"; + break; } O.OS << " # Toplevel Blocks: " << NumTopBlocks << "\n"; O.OS << "\n"; Index: llvm/trunk/lib/IR/RemarkStreamer.cpp =================================================================== --- llvm/trunk/lib/IR/RemarkStreamer.cpp +++ llvm/trunk/lib/IR/RemarkStreamer.cpp @@ -15,6 +15,7 @@ #include "llvm/IR/DiagnosticInfo.h" #include "llvm/IR/Function.h" #include "llvm/IR/GlobalValue.h" +#include "llvm/Remarks/BitstreamRemarkSerializer.h" #include "llvm/Remarks/RemarkFormat.h" #include "llvm/Remarks/RemarkSerializer.h" Index: llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp =================================================================== --- llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp +++ llvm/trunk/lib/Remarks/BitstreamRemarkSerializer.cpp @@ -0,0 +1,383 @@ +//===- BitstreamRemarkSerializer.cpp --------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +// +// This file provides the implementation of the LLVM bitstream remark serializer +// using LLVM's bitstream writer. +// +//===----------------------------------------------------------------------===// + +#include "llvm/Remarks/BitstreamRemarkSerializer.h" + +using namespace llvm; +using namespace llvm::remarks; + +BitstreamSerializerHelper::BitstreamSerializerHelper( + BitstreamRemarkContainerType ContainerType) + : Encoded(), R(), Bitstream(Encoded), ContainerType(ContainerType) {} + +static void push(SmallVectorImpl &R, StringRef Str) { + for (const char C : Str) + R.push_back(C); +} + +static void setRecordName(unsigned RecordID, BitstreamWriter &Bitstream, + SmallVectorImpl &R, StringRef Str) { + R.clear(); + R.push_back(RecordID); + push(R, Str); + Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETRECORDNAME, R); +} + +static void initBlock(unsigned BlockID, BitstreamWriter &Bitstream, + SmallVectorImpl &R, StringRef Str) { + R.clear(); + R.push_back(BlockID); + Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_SETBID, R); + + R.clear(); + push(R, Str); + Bitstream.EmitRecord(bitc::BLOCKINFO_CODE_BLOCKNAME, R); +} + +void BitstreamSerializerHelper::setupMetaBlockInfo() { + // Setup the metadata block. + initBlock(META_BLOCK_ID, Bitstream, R, MetaBlockName); + + // The container information. + setRecordName(RECORD_META_CONTAINER_INFO, Bitstream, R, + MetaContainerInfoName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_META_CONTAINER_INFO)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version. + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2)); // Type. + RecordMetaContainerInfoAbbrevID = + Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev); +} + +void BitstreamSerializerHelper::setupMetaRemarkVersion() { + setRecordName(RECORD_META_REMARK_VERSION, Bitstream, R, + MetaRemarkVersionName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_META_REMARK_VERSION)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Version. + RecordMetaRemarkVersionAbbrevID = + Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev); +} + +void BitstreamSerializerHelper::emitMetaRemarkVersion(uint64_t RemarkVersion) { + // The remark version is emitted only if we emit remarks. + R.clear(); + R.push_back(RECORD_META_REMARK_VERSION); + R.push_back(RemarkVersion); + Bitstream.EmitRecordWithAbbrev(RecordMetaRemarkVersionAbbrevID, R); +} + +void BitstreamSerializerHelper::setupMetaStrTab() { + setRecordName(RECORD_META_STRTAB, Bitstream, R, MetaStrTabName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_META_STRTAB)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Raw table. + RecordMetaStrTabAbbrevID = + Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev); +} + +void BitstreamSerializerHelper::emitMetaStrTab(const StringTable &StrTab) { + // The string table is not emitted if we emit remarks separately. + R.clear(); + R.push_back(RECORD_META_STRTAB); + + // Serialize to a blob. + std::string Buf; + raw_string_ostream OS(Buf); + StrTab.serialize(OS); + StringRef Blob = OS.str(); + Bitstream.EmitRecordWithBlob(RecordMetaStrTabAbbrevID, R, Blob); +} + +void BitstreamSerializerHelper::setupMetaExternalFile() { + setRecordName(RECORD_META_EXTERNAL_FILE, Bitstream, R, MetaExternalFileName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_META_EXTERNAL_FILE)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Filename. + RecordMetaExternalFileAbbrevID = + Bitstream.EmitBlockInfoAbbrev(META_BLOCK_ID, Abbrev); +} + +void BitstreamSerializerHelper::emitMetaExternalFile(StringRef Filename) { + // The external file is emitted only if we emit the separate metadata. + R.clear(); + R.push_back(RECORD_META_EXTERNAL_FILE); + Bitstream.EmitRecordWithBlob(RecordMetaExternalFileAbbrevID, R, Filename); +} + +void BitstreamSerializerHelper::setupRemarkBlockInfo() { + // Setup the remark block. + initBlock(REMARK_BLOCK_ID, Bitstream, R, RemarkBlockName); + + // The header of a remark. + { + setRecordName(RECORD_REMARK_HEADER, Bitstream, R, RemarkHeaderName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HEADER)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Type + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6)); // Remark Name + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6)); // Pass name + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Function name + RecordRemarkHeaderAbbrevID = + Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev); + } + + // The location of a remark. + { + setRecordName(RECORD_REMARK_DEBUG_LOC, Bitstream, R, RemarkDebugLocName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_DEBUG_LOC)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // File + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column + RecordRemarkDebugLocAbbrevID = + Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev); + } + + // The hotness of a remark. + { + setRecordName(RECORD_REMARK_HOTNESS, Bitstream, R, RemarkHotnessName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_HOTNESS)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Hotness + RecordRemarkHotnessAbbrevID = + Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev); + } + + // An argument entry with a debug location attached. + { + setRecordName(RECORD_REMARK_ARG_WITH_DEBUGLOC, Bitstream, R, + RemarkArgWithDebugLocName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITH_DEBUGLOC)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Key + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Value + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // File + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Line + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 32)); // Column + RecordRemarkArgWithDebugLocAbbrevID = + Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev); + } + + // An argument entry with no debug location attached. + { + setRecordName(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, Bitstream, R, + RemarkArgWithoutDebugLocName); + + auto Abbrev = std::make_shared(); + Abbrev->Add(BitCodeAbbrevOp(RECORD_REMARK_ARG_WITHOUT_DEBUGLOC)); + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Key + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 7)); // Value + RecordRemarkArgWithoutDebugLocAbbrevID = + Bitstream.EmitBlockInfoAbbrev(REMARK_BLOCK_ID, Abbrev); + } +} + +void BitstreamSerializerHelper::setupBlockInfo() { + // Emit magic number. + for (const char C : ContainerMagic) + Bitstream.Emit(static_cast(C), 8); + + Bitstream.EnterBlockInfoBlock(); + + // Setup the main metadata. Depending on the container type, we'll setup the + // required records next. + setupMetaBlockInfo(); + + switch (ContainerType) { + case BitstreamRemarkContainerType::SeparateRemarksMeta: + // Needs a string table that the separate remark file is using. + setupMetaStrTab(); + // Needs to know where the external remarks file is. + setupMetaExternalFile(); + break; + case BitstreamRemarkContainerType::SeparateRemarksFile: + // Contains remarks: emit the version. + setupMetaRemarkVersion(); + // Contains remarks: emit the remark abbrevs. + setupRemarkBlockInfo(); + break; + case BitstreamRemarkContainerType::Standalone: + // Contains remarks: emit the version. + setupMetaRemarkVersion(); + // Needs a string table. + setupMetaStrTab(); + // Contains remarks: emit the remark abbrevs. + setupRemarkBlockInfo(); + break; + } + + Bitstream.ExitBlock(); +} + +void BitstreamSerializerHelper::emitMetaBlock( + uint64_t ContainerVersion, Optional RemarkVersion, + Optional StrTab, Optional Filename) { + // Emit the meta block + Bitstream.EnterSubblock(META_BLOCK_ID, 3); + + // The container version and type. + R.clear(); + R.push_back(RECORD_META_CONTAINER_INFO); + R.push_back(ContainerVersion); + R.push_back(static_cast(ContainerType)); + Bitstream.EmitRecordWithAbbrev(RecordMetaContainerInfoAbbrevID, R); + + switch (ContainerType) { + case BitstreamRemarkContainerType::SeparateRemarksMeta: + assert(StrTab != None && *StrTab != nullptr); + emitMetaStrTab(**StrTab); + assert(Filename != None); + emitMetaExternalFile(*Filename); + break; + case BitstreamRemarkContainerType::SeparateRemarksFile: + assert(RemarkVersion != None); + emitMetaRemarkVersion(*RemarkVersion); + break; + case BitstreamRemarkContainerType::Standalone: + assert(RemarkVersion != None); + emitMetaRemarkVersion(*RemarkVersion); + assert(StrTab != None && *StrTab != nullptr); + emitMetaStrTab(**StrTab); + break; + } + + Bitstream.ExitBlock(); +} + +void BitstreamSerializerHelper::emitRemarkBlock(const Remark &Remark, + StringTable &StrTab) { + Bitstream.EnterSubblock(REMARK_BLOCK_ID, 4); + + R.clear(); + R.push_back(RECORD_REMARK_HEADER); + R.push_back(static_cast(Remark.RemarkType)); + R.push_back(StrTab.add(Remark.RemarkName).first); + R.push_back(StrTab.add(Remark.PassName).first); + R.push_back(StrTab.add(Remark.FunctionName).first); + Bitstream.EmitRecordWithAbbrev(RecordRemarkHeaderAbbrevID, R); + + if (const Optional &Loc = Remark.Loc) { + R.clear(); + R.push_back(RECORD_REMARK_DEBUG_LOC); + R.push_back(StrTab.add(Loc->SourceFilePath).first); + R.push_back(Loc->SourceLine); + R.push_back(Loc->SourceColumn); + Bitstream.EmitRecordWithAbbrev(RecordRemarkDebugLocAbbrevID, R); + } + + if (Optional Hotness = Remark.Hotness) { + R.clear(); + R.push_back(RECORD_REMARK_HOTNESS); + R.push_back(*Hotness); + Bitstream.EmitRecordWithAbbrev(RecordRemarkHotnessAbbrevID, R); + } + + for (const Argument &Arg : Remark.Args) { + R.clear(); + unsigned Key = StrTab.add(Arg.Key).first; + unsigned Val = StrTab.add(Arg.Val).first; + bool HasDebugLoc = Arg.Loc != None; + R.push_back(HasDebugLoc ? RECORD_REMARK_ARG_WITH_DEBUGLOC + : RECORD_REMARK_ARG_WITHOUT_DEBUGLOC); + R.push_back(Key); + R.push_back(Val); + if (HasDebugLoc) { + R.push_back(StrTab.add(Arg.Loc->SourceFilePath).first); + R.push_back(Arg.Loc->SourceLine); + R.push_back(Arg.Loc->SourceColumn); + } + Bitstream.EmitRecordWithAbbrev(HasDebugLoc + ? RecordRemarkArgWithDebugLocAbbrevID + : RecordRemarkArgWithoutDebugLocAbbrevID, + R); + } + Bitstream.ExitBlock(); +} + +void BitstreamSerializerHelper::flushToStream(raw_ostream &OS) { + OS.write(Encoded.data(), Encoded.size()); + Encoded.clear(); +} + +StringRef BitstreamSerializerHelper::getBuffer() { + return StringRef(Encoded.data(), Encoded.size()); +} + +BitstreamSerializer::BitstreamSerializer(raw_ostream &OS, SerializerMode Mode) + : RemarkSerializer(OS, Mode), + Helper(BitstreamRemarkContainerType::SeparateRemarksFile) { + assert(Mode == SerializerMode::Separate && + "For SerializerMode::Standalone, a pre-filled string table needs to " + "be provided."); + // We always use a string table with bitstream. + StrTab.emplace(); +} + +BitstreamSerializer::BitstreamSerializer(raw_ostream &OS, SerializerMode Mode, + StringTable StrTabIn) + : RemarkSerializer(OS, Mode), + Helper(Mode == SerializerMode::Separate + ? BitstreamRemarkContainerType::SeparateRemarksFile + : BitstreamRemarkContainerType::Standalone) { + StrTab = std::move(StrTabIn); +} + +void BitstreamSerializer::emit(const Remark &Remark) { + if (!DidSetUp) { + // Emit the metadata that is embedded in the remark file. + // If we're in standalone mode, serialize the string table as well. + bool IsStandalone = + Helper.ContainerType == BitstreamRemarkContainerType::Standalone; + BitstreamMetaSerializer MetaSerializer( + OS, Helper, + IsStandalone ? &*StrTab : Optional(None)); + MetaSerializer.emit(); + DidSetUp = true; + } + + assert(DidSetUp && + "The Block info block and the meta block were not emitted yet."); + Helper.emitRemarkBlock(Remark, *StrTab); + + Helper.flushToStream(OS); +} + +std::unique_ptr +BitstreamSerializer::metaSerializer(raw_ostream &OS, + Optional ExternalFilename) { + assert(Helper.ContainerType != + BitstreamRemarkContainerType::SeparateRemarksMeta); + bool IsStandalone = + Helper.ContainerType == BitstreamRemarkContainerType::Standalone; + return llvm::make_unique( + OS, + IsStandalone ? BitstreamRemarkContainerType::Standalone + : BitstreamRemarkContainerType::SeparateRemarksMeta, + &*StrTab, ExternalFilename); +} + +void BitstreamMetaSerializer::emit() { + Helper->setupBlockInfo(); + Helper->emitMetaBlock(CurrentContainerVersion, CurrentRemarkVersion, StrTab, + ExternalFilename); + Helper->flushToStream(OS); +} Index: llvm/trunk/lib/Remarks/CMakeLists.txt =================================================================== --- llvm/trunk/lib/Remarks/CMakeLists.txt +++ llvm/trunk/lib/Remarks/CMakeLists.txt @@ -1,4 +1,5 @@ add_llvm_library(LLVMRemarks + BitstreamRemarkSerializer.cpp Remark.cpp RemarkFormat.cpp RemarkParser.cpp Index: llvm/trunk/lib/Remarks/RemarkFormat.cpp =================================================================== --- llvm/trunk/lib/Remarks/RemarkFormat.cpp +++ llvm/trunk/lib/Remarks/RemarkFormat.cpp @@ -19,7 +19,8 @@ Expected llvm::remarks::parseFormat(StringRef FormatStr) { auto Result = StringSwitch(FormatStr) .Cases("", "yaml", Format::YAML) - .Cases("", "yaml-strtab", Format::YAMLStrTab) + .Case("yaml-strtab", Format::YAMLStrTab) + .Case("bitstream", Format::Bitstream) .Default(Format::Unknown); if (Result == Format::Unknown) Index: llvm/trunk/lib/Remarks/RemarkParser.cpp =================================================================== --- llvm/trunk/lib/Remarks/RemarkParser.cpp +++ llvm/trunk/lib/Remarks/RemarkParser.cpp @@ -56,6 +56,9 @@ return createStringError( std::make_error_code(std::errc::invalid_argument), "The YAML with string table format requires a parsed string table."); + case Format::Bitstream: + return createStringError(std::make_error_code(std::errc::invalid_argument), + "Parsing bitstream remarks is not supported."); case Format::Unknown: return createStringError(std::make_error_code(std::errc::invalid_argument), "Unknown remark parser format."); @@ -73,6 +76,9 @@ "table. Use yaml-strtab instead."); case Format::YAMLStrTab: return llvm::make_unique(Buf, std::move(StrTab)); + case Format::Bitstream: + return createStringError(std::make_error_code(std::errc::invalid_argument), + "Parsing bitstream remarks is not supported."); case Format::Unknown: return createStringError(std::make_error_code(std::errc::invalid_argument), "Unknown remark parser format."); @@ -89,6 +95,9 @@ case Format::YAML: case Format::YAMLStrTab: return createYAMLParserFromMeta(Buf, std::move(StrTab)); + case Format::Bitstream: + return createStringError(std::make_error_code(std::errc::invalid_argument), + "Parsing bitstream remarks is not supported."); case Format::Unknown: return createStringError(std::make_error_code(std::errc::invalid_argument), "Unknown remark parser format."); Index: llvm/trunk/lib/Remarks/RemarkSerializer.cpp =================================================================== --- llvm/trunk/lib/Remarks/RemarkSerializer.cpp +++ llvm/trunk/lib/Remarks/RemarkSerializer.cpp @@ -11,6 +11,7 @@ //===----------------------------------------------------------------------===// #include "llvm/Remarks/RemarkSerializer.h" +#include "llvm/Remarks/BitstreamRemarkSerializer.h" #include "llvm/Remarks/YAMLRemarkSerializer.h" using namespace llvm; @@ -27,6 +28,8 @@ return llvm::make_unique(OS, Mode); case Format::YAMLStrTab: return llvm::make_unique(OS, Mode); + case Format::Bitstream: + return llvm::make_unique(OS, Mode); } llvm_unreachable("Unknown remarks::Format enum"); } @@ -45,6 +48,8 @@ case Format::YAMLStrTab: return llvm::make_unique(OS, Mode, std::move(StrTab)); + case Format::Bitstream: + return llvm::make_unique(OS, Mode, std::move(StrTab)); } llvm_unreachable("Unknown remarks::Format enum"); } Index: llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp =================================================================== --- llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp +++ llvm/trunk/lib/Remarks/YAMLRemarkParser.cpp @@ -75,11 +75,11 @@ uint64_t Version = support::endian::read( Buf.data()); - if (Version != remarks::Version) + if (Version != remarks::CurrentRemarkVersion) return createStringError(std::errc::illegal_byte_sequence, "Mismatching remark version. Got %" PRId64 ", expected %" PRId64 ".", - Version, remarks::Version); + Version, remarks::CurrentRemarkVersion); Buf = Buf.drop_front(sizeof(uint64_t)); return Version; } Index: llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp =================================================================== --- llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp +++ llvm/trunk/lib/Remarks/YAMLRemarkSerializer.cpp @@ -191,7 +191,7 @@ static void emitVersion(raw_ostream &OS) { // Emit the version number: little-endian uint64_t. std::array Version; - support::endian::write64le(Version.data(), remarks::Version); + support::endian::write64le(Version.data(), remarks::CurrentRemarkVersion); OS.write(Version.data(), Version.size()); } Index: llvm/trunk/test/Bitcode/stream-types.c =================================================================== --- llvm/trunk/test/Bitcode/stream-types.c +++ llvm/trunk/test/Bitcode/stream-types.c @@ -10,3 +10,6 @@ // RUN: not llvm-bcanalyzer -dump %s.ast.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE // RUN: not llvm-bcanalyzer -dump %s.dia.incomplete 2>&1 | FileCheck %s -check-prefix=CHECK-INCOMPLETE // CHECK-INCOMPLETE: Bitcode stream should be a multiple of 4 bytes in length + +// RUN: llvm-bcanalyzer -dump %s.opt.bitstream | FileCheck %s -check-prefix=CHECK-REMARKS +// CHECK-REMARKS: Stream type: LLVM Remarks Index: llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp =================================================================== --- llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp +++ llvm/trunk/unittests/Remarks/BitstreamRemarksFormatTest.cpp @@ -0,0 +1,48 @@ +//===- unittest/Support/BitstreamRemarksFormatTest.cpp - BitCodes tests ---===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "llvm/Remarks/BitstreamRemarkContainer.h" +#include "gtest/gtest.h" + +using namespace llvm; + +// The goal for this test is to observe test failures and carefully update the +// constants when they change. + +// This should not change over time. +TEST(BitstreamRemarksFormat, Magic) { + EXPECT_EQ(remarks::ContainerMagic, "RMRK"); +} + +// This should be updated whenever any of the tests below are modified. +TEST(BitstreamRemarksFormat, ContainerVersion) { + EXPECT_EQ(remarks::CurrentContainerVersion, 0UL); +} + +// The values of the current blocks should not change over time. +// When adding new blocks, make sure to append them to the enum. +TEST(BitstreamRemarksFormat, BlockIDs) { + EXPECT_EQ(remarks::META_BLOCK_ID, 8); + EXPECT_EQ(remarks::REMARK_BLOCK_ID, 9); +} + +// The values of the current records should not change over time. +// When adding new records, make sure to append them to the enum. +TEST(BitstreamRemarksFormat, RecordIDs) { + EXPECT_EQ(remarks::RECORD_FIRST, 1); + EXPECT_EQ(remarks::RECORD_META_CONTAINER_INFO, 1); + EXPECT_EQ(remarks::RECORD_META_REMARK_VERSION, 2); + EXPECT_EQ(remarks::RECORD_META_STRTAB, 3); + EXPECT_EQ(remarks::RECORD_META_EXTERNAL_FILE, 4); + EXPECT_EQ(remarks::RECORD_REMARK_HEADER, 5); + EXPECT_EQ(remarks::RECORD_REMARK_DEBUG_LOC, 6); + EXPECT_EQ(remarks::RECORD_REMARK_HOTNESS, 7); + EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITH_DEBUGLOC, 8); + EXPECT_EQ(remarks::RECORD_REMARK_ARG_WITHOUT_DEBUGLOC, 9); + EXPECT_EQ(remarks::RECORD_LAST, 9); +} Index: llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp =================================================================== --- llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp +++ llvm/trunk/unittests/Remarks/BitstreamRemarksSerializerTest.cpp @@ -0,0 +1,341 @@ +//===- unittest/Support/BitstreamRemarksSerializerTest.cpp ----------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "llvm/Bitcode/BitcodeAnalyzer.h" +#include "llvm/Remarks/BitstreamRemarkSerializer.h" +#include "llvm/Support/raw_ostream.h" +#include "gtest/gtest.h" +#include + +// We need to supprt Windows paths as well. In order to have paths with the same +// length, use a different path according to the platform. +#ifdef _WIN32 +#define EXTERNALFILETESTPATH "C:/externalfi" +#else +#define EXTERNALFILETESTPATH "/externalfile" +#endif + +using namespace llvm; + +static void checkAnalyze(StringRef Input, StringRef Expected) { + std::string OutputBuf; + raw_string_ostream OutputOS(OutputBuf); + BCDumpOptions O(OutputOS); + O.ShowBinaryBlobs = true; + BitcodeAnalyzer BA(Input); + EXPECT_FALSE(BA.analyze(O)); // Expect no errors. + EXPECT_EQ(OutputOS.str(), Expected); +} + +static void check(remarks::SerializerMode Mode, const remarks::Remark &R, + StringRef ExpectedR, Optional ExpectedMeta, + Optional StrTab) { + // Emit the remark. + std::string InputBuf; + raw_string_ostream InputOS(InputBuf); + Expected> MaybeSerializer = [&] { + if (StrTab) + return createRemarkSerializer(remarks::Format::Bitstream, Mode, InputOS, + std::move(*StrTab)); + else + return createRemarkSerializer(remarks::Format::Bitstream, Mode, InputOS); + }(); + EXPECT_FALSE(errorToBool(MaybeSerializer.takeError())); + std::unique_ptr Serializer = + std::move(*MaybeSerializer); + Serializer->emit(R); + + // Analyze the serialized remark. + checkAnalyze(InputOS.str(), ExpectedR); + + // Analyze the serialized metadata if it's not in standalone mode. + if (ExpectedMeta) { + std::string MetaBuf; + raw_string_ostream MetaOS(MetaBuf); + std::unique_ptr MetaSerializer = + Serializer->metaSerializer(MetaOS, StringRef(EXTERNALFILETESTPATH)); + MetaSerializer->emit(); + checkAnalyze(MetaOS.str(), *ExpectedMeta); + } +} + +static void check(const remarks::Remark &R, StringRef ExpectedR, + StringRef ExpectedMeta, + Optional StrTab = None) { + return check(remarks::SerializerMode::Separate, R, ExpectedR, ExpectedMeta, + std::move(StrTab)); +} + +static void checkStandalone(const remarks::Remark &R, StringRef ExpectedR, + Optional StrTab = None) { + return check(remarks::SerializerMode::Standalone, R, ExpectedR, + /*ExpectedMeta=*/None, std::move(StrTab)); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileNoOptionals) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileNoOptionalsSeparateStrTab) { + remarks::StringTable StrTab; + StrTab.add("function"); + StrTab.add("pass"); + StrTab.add("remark"); + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'function\\x00pass\\x00remark\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n", + std::move(StrTab)); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileDebugLoc) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Loc.emplace(); + R.Loc->SourceFilePath = "path"; + R.Loc->SourceLine = 99; + R.Loc->SourceColumn = 55; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00path\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileHotness) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Hotness.emplace(999999999); + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgNoDebugLoc) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Args.emplace_back(); + R.Args.back().Key = "key"; + R.Args.back().Val = "value"; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00key\\x00value\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileArgDebugLoc) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Args.emplace_back(); + R.Args.back().Key = "key"; + R.Args.back().Val = "value"; + R.Args.back().Loc.emplace(); + R.Args.back().Loc->SourceFilePath = "path"; + R.Args.back().Loc->SourceLine = 99; + R.Args.back().Loc->SourceColumn = 55; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00key\\x00value\\x00path\\x00'\n" + " blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, SeparateRemarkFileAll) { + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Loc.emplace(); + R.Loc->SourceFilePath = "path"; + R.Loc->SourceLine = 99; + R.Loc->SourceColumn = 55; + R.Hotness.emplace(999999999); + R.Args.emplace_back(); + R.Args.back().Key = "key"; + R.Args.back().Val = "value"; + R.Args.back().Loc.emplace(); + R.Args.back().Loc->SourceFilePath = "argpath"; + R.Args.back().Loc->SourceLine = 11; + R.Args.back().Loc->SourceColumn = 66; + check(R, + "\n" + "\n" + " \n" + " \n" + "\n" + "\n" + " \n" + " \n" + " \n" + " \n" + "\n", + "\n" + "\n" + " \n" + " blob data = " + "'remark\\x00pass\\x00function\\x00path\\x00key\\x00value\\x00argpa" + "th\\x00'\n blob data = " + "'" EXTERNALFILETESTPATH"'\n" + "\n"); +} + +TEST(BitstreamRemarkSerializer, Standalone) { + // Pre-populate the string table. + remarks::StringTable StrTab; + StrTab.add("pass"); + StrTab.add("remark"); + StrTab.add("function"); + StrTab.add("path"); + StrTab.add("key"); + StrTab.add("value"); + StrTab.add("argpath"); + remarks::Remark R; + R.RemarkType = remarks::Type::Missed; + R.PassName = "pass"; + R.RemarkName = "remark"; + R.FunctionName = "function"; + R.Loc.emplace(); + R.Loc->SourceFilePath = "path"; + R.Loc->SourceLine = 99; + R.Loc->SourceColumn = 55; + R.Hotness.emplace(999999999); + R.Args.emplace_back(); + R.Args.back().Key = "key"; + R.Args.back().Val = "value"; + R.Args.back().Loc.emplace(); + R.Args.back().Loc->SourceFilePath = "argpath"; + R.Args.back().Loc->SourceLine = 11; + R.Args.back().Loc->SourceColumn = 66; + checkStandalone( + R, + "\n" + "\n" + " \n" + " \n" + " blob data = " + "'pass\\x00remark\\x00function\\x00path\\x00key\\x00value\\x00argpath\\x0" + "0'\n" + "\n" + "\n" + " \n" + " \n" + " \n" + " \n" + "\n", + std::move(StrTab)); +} Index: llvm/trunk/unittests/Remarks/CMakeLists.txt =================================================================== --- llvm/trunk/unittests/Remarks/CMakeLists.txt +++ llvm/trunk/unittests/Remarks/CMakeLists.txt @@ -1,9 +1,12 @@ set(LLVM_LINK_COMPONENTS + BitReader Remarks Support ) add_llvm_unittest(RemarksTests + BitstreamRemarksFormatTest.cpp + BitstreamRemarksSerializerTest.cpp RemarksStrTabParsingTest.cpp YAMLRemarksParsingTest.cpp YAMLRemarksSerializerTest.cpp