[Remarks] Emit a section containing remark diagnostics metadata

[thegameg]

Summary

A section containing metadata on remark diagnostics will be emitted if the flag (-mllvm) -remarks-section is present.

For now, the metadata is:

• a magic number for remarks: "REMARKS\0"
• the version number: a little-endian uint64_t
• the absolute file path to the serialized remark diagnostics: a null-terminated string

[JDevlieghere]

This needs a test.

[thegameg]

Forgot to add it to the diff.

[ormris]

llvm/test/CodeGen/X86/remarks-section.ll

2 ↗

(On Diff #191428)

Quick drive-by nit: I noticed that this uses "/dev/null". This will cause failures on the Windows bots, since Windows doesn't provide that file.

[thegameg]

llvm/test/CodeGen/X86/remarks-section.ll

2 ↗

(On Diff #191428)

Thanks for the heads-up. What would be the solution to this? Is this any different from all the other tests using -o /dev/null or > /dev/null?

[ormris]

llvm/test/CodeGen/X86/remarks-section.ll

2 ↗

(On Diff #191428)

Yes, since you're checking for the final path. Lit will replace the string "/dev/null" with a temporary file. You could use a RUN line like this:

; RUN: llc < %s -mtriple=x86_64-linux -remarks-section -pass-remarks-output=%/t.yaml | FileCheck -DPATH=%/t.yaml %s

That will store the path of the remarks file in the Filecheck variable "PATH", which you can refer to in your CHECK lines as "[[PATH]]". The "%/t" variable is also really handy. It makes sure that the path will always use forward slashes, removing backslash weirdness.

[thegameg]

Update test to use a real path instead of /dev/null. Thanks @ormris, I didn't know lit could do all that!

[ormris]

llvm/test/CodeGen/X86/remarks-section.ll

2 ↗

(On Diff #191428)

LG!

[JDevlieghere]

LGTM, apologies about the delay!

llvm/trunk/docs/CodeGenerator.rst

 A section containing metadata on function stack sizes will be emitted when
 ``TargetLoweringObjectFile::StackSizesSection`` is not null, and
 ``TargetOptions::EmitStackSizeSection`` is set (-stack-size-section). The
 section will contain an array of pairs of function symbol values (pointer size)
 and stack sizes (unsigned LEB128). The stack size values only include the space
 allocated in the function prologue. Functions with dynamic stack allocations are
 not included.
 
+Emitting remark diagnostics in the object file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A section containing metadata on remark diagnostics will be emitted when
+-remarks-section is passed. The section contains:
+
+* a magic number: "REMARKS\0"
+* the version number: a little-endian uint64_t
+* the absolute file path to the serialized remark diagnostics: a
+  null-terminated string.
+
 VLIW Packetizer
 ---------------
 
 In a Very Long Instruction Word (VLIW) architecture, the compiler is responsible
 for mapping instructions to functional-units available on the architecture. To
 that end, the compiler creates groups of instructions called *packets* or
 *bundles*. The VLIW packetizer in LLVM is a target-independent mechanism to
 enable the packetization of machine instructions.

llvm/trunk/docs/CommandGuide/llc.rst

 .. option:: -stack-size-section
 
  Emit the .stack_sizes section which contains stack size metadata. The section
  contains an array of pairs of function symbol values (pointer size) and stack
  sizes (unsigned LEB128). The stack size values only include the space allocated
  in the function prologue. Functions with dynamic stack allocations are not
  included.
 
+.. option:: -remarks-section
+
+ Emit the .remarks (ELF) / __remarks (MachO) section which contains metadata
+ about remark diagnostics.
 
 Tuning/Configuration Options
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. option:: --print-machineinstrs
 
  Print generated machine code between compilation phases (useful for debugging).
 

llvm/trunk/include/llvm/CodeGen/AsmPrinter.h

public:
   void EmitFunctionBody();
 
   void emitCFIInstruction(const MachineInstr &MI);
 
   void emitFrameAlloc(const MachineInstr &MI);
 
   void emitStackSizeSection(const MachineFunction &MF);
 
+  void emitRemarksSection(Module &M);
+
   enum CFIMoveType { CFI_M_None, CFI_M_EH, CFI_M_Debug };
   CFIMoveType needsCFIMoves() const;
 
   /// Returns false if needsCFIMoves() == CFI_M_EH for any function
   /// in the module.
   bool needsOnlyDebugCFIMoves() const { return isCFIMoveForDebugging; }
 
   bool needsSEHMoves();

llvm/trunk/include/llvm/MC/MCObjectFileInfo.h

protected:
   MCSection *TLSBSSSection; // Defaults to ".tbss".
 
   /// StackMap section.
   MCSection *StackMapSection;
 
   /// FaultMap section.
   MCSection *FaultMapSection;
 
+  /// Remarks section.
+  MCSection *RemarksSection;
+
   /// EH frame section.
   ///
   /// It is initialized on demand so it can be overwritten (with uniquing).
   MCSection *EHFrameSection;
 
   /// Section containing metadata on function stack sizes.
   MCSection *StackSizesSection;
   mutable DenseMap<const MCSymbol *, unsigned> StackSizesUniquing;
public:
   }
 
   MCSection *getTLSExtraDataSection() const { return TLSExtraDataSection; }
   const MCSection *getTLSDataSection() const { return TLSDataSection; }
   MCSection *getTLSBSSSection() const { return TLSBSSSection; }
 
   MCSection *getStackMapSection() const { return StackMapSection; }
   MCSection *getFaultMapSection() const { return FaultMapSection; }
+  MCSection *getRemarksSection() const { return RemarksSection; }
 
   MCSection *getStackSizesSection(const MCSection &TextSec) const;
 
   // ELF specific sections.
   MCSection *getDataRelROSection() const { return DataRelROSection; }
   const MCSection *getMergeableConst4Section() const {
     return MergeableConst4Section;
   }

llvm/trunk/include/llvm/Remarks/Remark.h

 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/CBindingWrapping.h"
 #include <string>
 
 namespace llvm {
 namespace remarks {
 
+constexpr uint64_t Version = 0;
+constexpr StringRef Magic("REMARKS", 7);
+
 /// The debug location used to track a remark back to the source file.
 struct RemarkLocation {
   /// Absolute path of the source file corresponding to this remark.
   StringRef SourceFilePath;
   unsigned SourceLine;
   unsigned SourceColumn;
 };
 

llvm/trunk/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

 #include "llvm/IR/GlobalObject.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Mangler.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Operator.h"
+#include "llvm/IR/RemarkStreamer.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCCodePadder.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCDirectives.h"
 #include "llvm/MC/MCDwarf.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCSection.h"
 #include "llvm/MC/MCSectionCOFF.h"
 #include "llvm/MC/MCSectionELF.h"
 #include "llvm/MC/MCSectionMachO.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/MCSymbolELF.h"
 #include "llvm/MC/MCTargetOptions.h"
 #include "llvm/MC/MCValue.h"
 #include "llvm/MC/SectionKind.h"
 #include "llvm/Pass.h"
+#include "llvm/Remarks/Remark.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/TargetRegistry.h"

 static const char *const CFGuardName = "Control Flow Guard";
 static const char *const CFGuardDescription = "Control Flow Guard Tables";
 static const char *const CodeViewLineTablesGroupName = "linetables";
 static const char *const CodeViewLineTablesGroupDescription =
   "CodeView Line Tables";
 
 STATISTIC(EmittedInsts, "Number of machine instrs printed");
 
+static cl::opt<bool> EnableRemarksSection(
+    "remarks-section",
+    cl::desc("Emit a section containing remark diagnostics metadata"),
+    cl::init(false));
+
 char AsmPrinter::ID = 0;
 
 using gcp_map_type = DenseMap<GCStrategy *, std::unique_ptr<GCMetadataPrinter>>;
 
 static gcp_map_type &getGCMap(void *&P) {
   if (!P)
     P = new gcp_map_type();
   return *(gcp_map_type*)P;
if (MAI->hasDotTypeDotSizeDirective() && GA->getValueType()->isSized() &&
         (!BaseObject || BaseObject->hasPrivateLinkage())) {
       const DataLayout &DL = M.getDataLayout();
       uint64_t Size = DL.getTypeAllocSize(GA->getValueType());
       OutStreamer->emitELFSize(Name, MCConstantExpr::create(Size, OutContext));
     }
   }
 }
 
+void AsmPrinter::emitRemarksSection(Module &M) {
+  RemarkStreamer *RS = M.getContext().getRemarkStreamer();
+  if (!RS)
+    return;
+
+  // Switch to the right section: .remarks/__remarks.
+  MCSection *RemarksSection =
+      OutContext.getObjectFileInfo()->getRemarksSection();
+  OutStreamer->SwitchSection(RemarksSection);
+
+  // Emit the magic number.
+  OutStreamer->EmitBytes(remarks::Magic);
+  // Explicitly emit a '\0'.
+  OutStreamer->EmitIntValue(/*Value=*/0, /*Size=*/1);
+
+  // Emit the version number: little-endian uint64_t.
+  // The version number is located at the offset 0x0 in the section.
+  std::array<char, 8> Version;
+  support::endian::write64le(Version.data(), remarks::Version);
+  OutStreamer->EmitBinaryData(StringRef(Version.data(), Version.size()));
+
+  // Emit the null-terminated absolute path to the remark file.
+  // The path is located at the offset 0x4 in the section.
+  StringRef FilenameRef = RS->getFilename();
+  SmallString<128> Filename = FilenameRef;
+  sys::fs::make_absolute(Filename);
+  assert(!Filename.empty() && "The filename can't be empty.");
+  OutStreamer->EmitBytes(Filename);
+  // Explicitly emit a '\0'.
+  OutStreamer->EmitIntValue(/*Value=*/0, /*Size=*/1);
+}
+
 bool AsmPrinter::doFinalization(Module &M) {
   // Set the MachineFunction to nullptr so that we can catch attempted
   // accesses to MF specific features at the module level and so that
   // we can conditionalize accesses based on whether or not it is nullptr.
   MF = nullptr;
 
   // Gather all GOT equivalent globals in the module. We really need two
   // passes over the globals: one to compute and another to avoid its emission
for (const Function &F : M) {
     GlobalValue::VisibilityTypes V = F.getVisibility();
     if (V == GlobalValue::DefaultVisibility)
       continue;
 
     MCSymbol *Name = getSymbol(&F);
     EmitVisibility(Name, V, false);
   }
 
+  // Emit the remarks section contents.
+  // FIXME: Figure out when is the safest time to emit this section. It should
+  // not come after debug info.
+  if (EnableRemarksSection)
+    emitRemarksSection(M);
+
   const TargetLoweringObjectFile &TLOF = getObjFileLowering();
 
   TLOF.emitModuleMetadata(*OutStreamer, M);
 
   if (TM.getTargetTriple().isOSBinFormatELF()) {
     MachineModuleInfoELF &MMIELF = MMI->getObjFileInfo<MachineModuleInfoELF>();
 
     // Output stubs for external and common global variables.

llvm/trunk/lib/MC/MCObjectFileInfo.cpp

DwarfTUIndexSection =
       Ctx->getMachOSection("__DWARF", "__debug_tu_index", MachO::S_ATTR_DEBUG,
                            SectionKind::getMetadata());
   StackMapSection = Ctx->getMachOSection("__LLVM_STACKMAPS", "__llvm_stackmaps",
                                          0, SectionKind::getMetadata());
 
   FaultMapSection = Ctx->getMachOSection("__LLVM_FAULTMAPS", "__llvm_faultmaps",
                                          0, SectionKind::getMetadata());
 
+  RemarksSection = Ctx->getMachOSection(
+      "__LLVM", "__remarks", MachO::S_ATTR_DEBUG, SectionKind::getMetadata());
+
   TLSExtraDataSection = TLSTLVSection;
 }
 
 void MCObjectFileInfo::initELFMCObjectFileInfo(const Triple &T, bool Large) {
   switch (T.getArch()) {
   case Triple::mips:
   case Triple::mipsel:
   case Triple::mips64:
void MCObjectFileInfo::initELFMCObjectFileInfo(const Triple &T, bool Large) {
 
   FaultMapSection =
       Ctx->getELFSection(".llvm_faultmaps", ELF::SHT_PROGBITS, ELF::SHF_ALLOC);
 
   EHFrameSection =
       Ctx->getELFSection(".eh_frame", EHSectionType, EHSectionFlags);
 
   StackSizesSection = Ctx->getELFSection(".stack_sizes", ELF::SHT_PROGBITS, 0);
+
+  RemarksSection =
+      Ctx->getELFSection(".remarks", ELF::SHT_PROGBITS, ELF::SHF_EXCLUDE);
 }
 
 void MCObjectFileInfo::initCOFFMCObjectFileInfo(const Triple &T) {
   EHFrameSection =
       Ctx->getCOFFSection(".eh_frame", COFF::IMAGE_SCN_CNT_INITIALIZED_DATA |
                                            COFF::IMAGE_SCN_MEM_READ,
                           SectionKind::getData());
 

llvm/trunk/test/CodeGen/X86/remarks-section.ll

+; RUN: llc < %s -mtriple=x86_64-linux -remarks-section -pass-remarks-output=%/t.yaml | FileCheck -DPATH=%/t.yaml %s
+; RUN: llc < %s -mtriple=x86_64-darwin -remarks-section -pass-remarks-output=%/t.yaml | FileCheck --check-prefix=CHECK-DARWIN -DPATH=%/t.yaml %s
+
+; CHECK-LABEL: func1:
+
+; CHECK: .section .remarks,"e",@progbits
+; The magic number:
+; CHECK-NEXT: .ascii "REMARKS"
+; Null-terminator:
+; CHECK-NEXT: .byte 0
+; The version:
+; CHECK-NEXT: .byte 0x00, 0x00, 0x00, 0x00
+; CHECK-NEXT: .byte 0x00, 0x00, 0x00, 0x00
+; The remark file path:
+; CHECK-NEXT: .ascii "[[PATH]]"
+; Null-terminator:
+; CHECK-NEXT: .byte 0
+
+; CHECK-DARWIN: .section __LLVM,__remarks,regular,debug
+; The magic number:
+; CHECK-DARWIN-NEXT: .ascii "REMARKS"
+; Null-terminator:
+; CHECK-DARWIN-NEXT: .byte 0
+; The version:
+; CHECK-DARWIN-NEXT: .byte 0x00, 0x00, 0x00, 0x00
+; CHECK-DARWIN-NEXT: .byte 0x00, 0x00, 0x00, 0x00
+; The remark file path:
+; CHECK-DARWIN-NEXT: .ascii "[[PATH]]"
+; Null-terminator:
+; CHECK-DARWIN-NEXT: .byte 0
+define void @func1() {
+  ret void
+}