[llvm-readobj] Allow printing of the watermark note section proposed in D66426

[chrisjackson]

Summary

This revision separates the changes to readobj that enable the watermark note section to be dumped. See D66426.

readobj can also compute the watermark hash. This computed value can then be checked against the value dumped from the proposed .note.llvm.watermark section.

[chrisjackson]

Added --compute-watermark, additional tests, use of library from update to D66426.

[chrisjackson]

Removed library test accidentally added in last update.

[jhenderson]

I've not looked at the code here yet. I'll do that once there's a consensus on D66426.

llvm/test/tools/llvm-readobj/elf-watermark.test

1 ↗

(On Diff #232417)

Assembly tests should have a '.s' extension. Also, since this is using X86_64 assembly, it needs a REQUIRES: x86. Finally, please rebase this change (there have been significant changes to the test directory structure recently).

6 ↗

(On Diff #232417)

LLD can't be used in an llvm-readobj test I'm afraid (it is not required to build check-llvm). You'll need a different way to setup this test somehow, or possibly this should be moved into the LLD testing if no sensible way can be come up with. One option might be to pre-calculate the watermark for a given set of content and use yaml2obj to generate the file with that watermark in it.

7 ↗

(On Diff #232417)

-check-prefix -> --check-prefix (also below)

10 ↗

(On Diff #232417)

Minor nit: add extra spacing to make the value being checked line up with the following lines.

llvm/test/tools/llvm-readobj/note-llvm.test

2

This test's name is note-llvm.test, but the test is specifically about the watermark note?

4–5

This part feels like it's more generically "notes can be dumped after --strip-sections has been used". Why is the watermark here special?

7

General consensus is to use -o rather than shell redirection in new tests.

Also --docnum.

42

Nit: too many blank lines

44

Same as above.

77

Same as above.

The comment related to this section should be moved to here.

115–116

The spacing looks all off here. Either remove excessive spacing so that it all lines up, or add --strict-whitespace to show that you're testing the latter.

[chrisjackson]

This revision will soon be abandoned along with D66426, unless there are any further comments.

llvm/include/llvm/BinaryFormat/ELF.h

enum : unsigned {
   NT_FILE = 0x46494c45,
   NT_PRXFPREG = 0x46e62b7f,
   NT_SIGINFO = 0x53494749,
 };
 
 // LLVM-specific notes.
 enum {
   NT_LLVM_HWASAN_GLOBALS = 3,
+  NT_LLVM_WATERMARK = 4,
 };
 
 // GNU note types
 enum {
   NT_GNU_ABI_TAG = 1,
   NT_GNU_HWCAP = 2,
   NT_GNU_BUILD_ID = 3,
   NT_GNU_GOLD_VERSION = 4,

llvm/test/tools/llvm-readobj/note-llvm.test

+## Test tools are able to dump the LLVM Watermark note section. Also test the
+## output when the type field is incorrect.
+
+# RUN: yaml2obj -docnum=1 %s > %t1.o
+# RUN: llvm-readobj --notes %t1.o | FileCheck %s --check-prefix=LLVM
+# RUN: llvm-readelf --notes %t1.o | FileCheck %s --check-prefix=GNU
+
+# LLVM:      Notes [
+# LLVM-NEXT:   NoteSection {
+# LLVM-NEXT:     Offset: 0x40
+# LLVM-NEXT:     Size: 0x20
+# LLVM-NEXT:     Note {
+# LLVM-NEXT:       Owner: LLVM
+# LLVM-NEXT:       Data size: 0xC
+# LLVM-NEXT:       Type: NT_LLVM_WATERMARK (Loadable sections watermark)
+# LLVM-NEXT:       Version: 01000000
+# LLVM-NEXT:       Watermark: 45e69f81a671eaac
+# LLVM-NEXT:     }
+# LLVM-NEXT:   }
+# LLVM-NEXT: ]
+
+# GNU:      Displaying notes found at file offset 0x00000040 with length 0x00000020:
+# GNU-NEXT: Owner                Data size    Description
+# GNU-NEXT: LLVM                 0x0000000c       NT_LLVM_WATERMARK (Loadable sections watermark)
+# GNU-NEXT:    LLVM Watermark: 45e69f81a671eaac Version: 0x01000000
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .note.llvm.watermark
+    Type:         SHT_NOTE
+    AddressAlign: 0x0000000000000004
+    Size:         32
+    Flags:        [ SHF_ALLOC ]
+    Content:      050000000c000000040000004c4c564d000000000100000045e69f81a671eaac
+
+
+# RUN: yaml2obj -docnum=2 %s > %t2.o
+# RUN: llvm-readobj --notes %t2.o | FileCheck %s --check-prefix=LLVM_UNKNOWNTYPE
+# RUN: llvm-readelf --notes %t2.o | FileCheck %s --check-prefix=GNU_UNKNOWNTYPE
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .note.llvm.watermark
+    Type:         SHT_NOTE
+    AddressAlign: 0x0000000000000004
+    Size:         32
+    Flags:        [ SHF_ALLOC ]
+    Content: 050000000c000000ff0000004c4c564d000000000100000045e69f81a671eaac
+
+# LLVM_UNKNOWNTYPE:      Notes [
+# LLVM_UNKNOWNTYPE-NEXT:  NoteSection {
+# LLVM_UNKNOWNTYPE-NEXT:    Offset: 0x40
+# LLVM_UNKNOWNTYPE-NEXT:    Size: 0x20
+# LLVM_UNKNOWNTYPE-NEXT:    Note {
+# LLVM_UNKNOWNTYPE-NEXT:      Owner: LLVM
+# LLVM_UNKNOWNTYPE-NEXT:      Data size: 0xC
+# LLVM_UNKNOWNTYPE-NEXT:      Type: Unknown note type (0x000000ff)
+# LLVM_UNKNOWNTYPE-NEXT:     }
+# LLVM_UNKNOWNTYPE-NEXT:   }
+# LLVM_UNKNOWNTYPE-NEXT: ]
+
+# GNU_UNKNOWNTYPE:      Displaying notes found at file offset 0x00000040 with length 0x00000020:
+# GNU_UNKNOWNTYPE-NEXT: Owner                Data size        Description
+# GNU_UNKNOWNTYPE-NEXT: LLVM                 0x0000000c       Unknown note type (0x000000ff)

llvm/tools/llvm-readobj/ELFDumper.cpp

if (Note.ID == NT)
       return std::string(Note.Name);
 
   std::string string;
   raw_string_ostream OS(string);
   OS << format("Unknown note type (0x%08x)", NT);
   return OS.str();
 }
 
+static std::string getLLVMNoteTypeName(const uint32_t NT) {
+  if (NT == NT_LLVM_WATERMARK)
+    return std::string("NT_LLVM_WATERMARK (Loadable sections watermark)");
+
+  std::string string;
+  raw_string_ostream OS(string);
+  OS << format("Unknown note type (0x%08x)", NT);
+  return OS.str();
+}
+
 static std::string getFreeBSDNoteTypeName(const uint32_t NT) {
   static const struct {
     uint32_t ID;
     const char *Name;
   } Notes[] = {
       {ELF::NT_FREEBSD_THRMISC, "NT_THRMISC (thrmisc structure)"},
       {ELF::NT_FREEBSD_PROCSTAT_PROC, "NT_PROCSTAT_PROC (proc data)"},
       {ELF::NT_FREEBSD_PROCSTAT_FILES, "NT_PROCSTAT_FILES (files data)"},
case ELF::NT_AMDGPU_METADATA: {
     raw_string_ostream StrOS(HSAMetadataString);
     MsgPackDoc.toYAML(StrOS);
 
     return {"AMDGPU Metadata", StrOS.str()};
   }
   }
 }
 
+template <typename ELFT>
+static void printLLVMWatermarkLLVMStyle(uint32_t NoteType,
+                                        ArrayRef<uint8_t> Desc,
+                                        ScopedPrinter &W) {
+  if (NoteType != NT_LLVM_WATERMARK)
+    return;
+
+  const size_t HashSize = 8;
+  const size_t VersionSize = 4;
+  auto Version =
+      StringRef(reinterpret_cast<const char *>(Desc.data()), VersionSize);
+  auto Hash = StringRef(
+      reinterpret_cast<const char *>(Desc.data() + VersionSize), HashSize);
+  W.printString("Version", toHex(Version, true));
+  W.printString("Watermark", toHex(Hash, true));
+}
+
+template <typename ELFT>
+static void printLLVMWatermark(raw_ostream &OS, uint32_t NoteType,
+                               ArrayRef<uint8_t> Desc) {
+
+  if (NoteType != NT_LLVM_WATERMARK)
+    return;
+
+  const size_t HashSize = 8;
+  const size_t VersionSize = 4;
+  auto Hash = StringRef(
+      reinterpret_cast<const char *>(Desc.data() + VersionSize), HashSize);
+  auto Version= StringRef(reinterpret_cast<const char *>(Desc.data()), VersionSize);
+  OS << "    LLVM Watermark: " << toHex(Hash, true)
+     << "    Version: 0x" << toHex(Version);
+}
+
 struct CoreFileMapping {
   uint64_t Start, End, Offset;
   StringRef Filename;
 };
 
 struct CoreNote {
   uint64_t PageSize;
   std::vector<CoreFileMapping> Mappings;
auto ProcessNote = [&](const Elf_Note &Note) {
     if (Name == "GNU") {
       OS << getGNUNoteTypeName(Type) << '\n';
     } else if (Name == "FreeBSD") {
       OS << getFreeBSDNoteTypeName(Type) << '\n';
     } else if (Name == "AMD") {
       OS << getAMDNoteTypeName(Type) << '\n';
     } else if (Name == "AMDGPU") {
       OS << getAMDGPUNoteTypeName(Type) << '\n';
+    } else if (Name == "LLVM") {
+      OS << getLLVMNoteTypeName(Type) << '\n';
     } else {
       StringRef NoteType = Obj->getHeader()->e_type == ELF::ET_CORE
                                ? getCoreNoteTypeName(Type)
                                : getGenericNoteTypeName(Type);
       if (!NoteType.empty())
         OS << NoteType << '\n';
       else
         OS << "Unknown note type: (" << format_hex(Type, 10) << ")\n";
if (Name == "GNU") {
                                     ELFT::TargetEndianness == support::little,
                                     sizeof(Elf_Addr));
         Expected<CoreNote> Note = readCoreNote(DescExtractor);
         if (Note)
           printCoreNote<ELFT>(OS, *Note);
         else
           reportWarning(Note.takeError(), this->FileName);
       }
+    } else if (Name == "LLVM") {
+      printLLVMWatermark<ELFT>(OS, Type, Descriptor);
     } else if (!Descriptor.empty()) {
       OS << "   description data:";
       for (uint8_t B : Descriptor)
         OS << " " << format("%02x", B);
       OS << '\n';
     }
   };
 

 }
 
 template <class ELFT>
 void LLVMStyle<ELFT>::printDynamicSymbols(const ELFO *Obj) {
   ListScope Group(W, "DynamicSymbols");
   this->dumper()->printSymbolsHelper(true);
 }
 
-template <class ELFT> void LLVMStyle<ELFT>::printDynamic(const ELFFile<ELFT> *Obj) {
+template <class ELFT>
+void LLVMStyle<ELFT>::printDynamic(const ELFFile<ELFT> *Obj) {
   Elf_Dyn_Range Table = this->dumper()->dynamic_table();
   if (Table.empty())
     return;
 
   raw_ostream &OS = W.getOStream();
   W.startLine() << "DynamicSection [ (" << Table.size() << " entries)\n";
 
   bool Is64 = ELFT::Is64Bits;
auto ProcessNote = [&](const Elf_Note &Note) {
     if (Name == "GNU") {
       W.printString("Type", getGNUNoteTypeName(Type));
     } else if (Name == "FreeBSD") {
       W.printString("Type", getFreeBSDNoteTypeName(Type));
     } else if (Name == "AMD") {
       W.printString("Type", getAMDNoteTypeName(Type));
     } else if (Name == "AMDGPU") {
       W.printString("Type", getAMDGPUNoteTypeName(Type));
+    } else if (Name == "LLVM") {
+      W.printString("Type", getLLVMNoteTypeName(Type));
     } else {
       StringRef NoteType = Obj->getHeader()->e_type == ELF::ET_CORE
                                ? getCoreNoteTypeName(Type)
                                : getGenericNoteTypeName(Type);
       if (!NoteType.empty())
         W.printString("Type", NoteType);
       else
         W.printString("Type",
if (Name == "GNU") {
                                     ELFT::TargetEndianness == support::little,
                                     sizeof(Elf_Addr));
         Expected<CoreNote> Note = readCoreNote(DescExtractor);
         if (Note)
           printCoreNoteLLVMStyle(*Note, W);
         else
           reportWarning(Note.takeError(), this->FileName);
       }
+    } else if (Name == "LLVM") {
+      printLLVMWatermarkLLVMStyle<ELFT>(Type, Descriptor, W);
     } else if (!Descriptor.empty()) {
       W.printBinaryBlock("Description data", Descriptor);
     }
   };
 
   ArrayRef<Elf_Shdr> Sections = unwrapOrError(this->FileName, Obj->sections());
   if (Obj->getHeader()->e_type != ELF::ET_CORE && !Sections.empty()) {
     for (const auto &S : Sections) {
void LLVMStyle<ELFT>::printELFLinkerOptions(const ELFFile<ELFT> *Obj) {
   ListScope L(W, "LinkerOptions");
 
   for (const Elf_Shdr &Shdr : unwrapOrError(this->FileName, Obj->sections())) {
     if (Shdr.sh_type != ELF::SHT_LLVM_LINKER_OPTIONS)
       continue;
 
     ArrayRef<uint8_t> Contents =
         unwrapOrError(this->FileName, Obj->getSectionContents(&Shdr));
-    for (const uint8_t *P = Contents.begin(), *E = Contents.end(); P < E; ) {
+    for (const uint8_t *P = Contents.begin(), *E = Contents.end(); P < E;) {
       StringRef Key = StringRef(reinterpret_cast<const char *>(P));
       StringRef Value =
           StringRef(reinterpret_cast<const char *>(P) + Key.size() + 1);
 
       W.printString(Key, Value);
 
       P = P + Key.size() + Value.size() + 2;
     }