[Disassembler][llvm-readobj] ELF note dumper abstraction

[tpr]

Summary

This commit introduces an abstraction to allow knowledge of how to dump
an ELF note record into the target that it is specific to.

llvm-readobj's ELF dumper uses the new abstraction, although no target
yet implements it. A subsequent change will move dumping of the
AMDGPU-specific note records out of llvm-readobj and into
lib/Target/AMDGPU.

Change-Id: Ib42c861bfda0ad38f0ceb2f61658215326c8fcaf

[tpr]

See D52822 for AMDGPU implementing ELFNoteDumper.

See also D52823 for MCELFNoteDisassembler, another new abstraction in TargetRegistry.

[compnerd]

include/llvm/Object/ELFNoteDumper.h

28

Why not use = default?

32

We can use the destructor as the key function rather than emitting the data globally.

tools/llvm-readobj/ELFDumper.cpp

3811

I'm not sure if I really like this approach. We already have the name that identifies who should interpret the note. There is nothing that prevents the use of the aarch64-unknown-windows-msvc-elf to use the "AMD" vendor namespace to add a note. However, I don't think that any target registered will have that triple. How would that work?

[tpr]

tools/llvm-readobj/ELFDumper.cpp

3811

This code just asks each registered (i.e. compiled in) target if it understands the note, and ignores the triples. So if the aarch64 target is compiled in and it decides to dump an AMD note type, it can.

We have the vendor name from the note, but that does not help identify the target.

Are you objecting to the whole point of this patch, to be able to delegate target-specific notes into targets and move the knowledge of them out of generic code in lib/BinaryFormat?

[compnerd]

tools/llvm-readobj/ELFDumper.cpp

3811

Yeah, thinking a bit more about it, I think that I'm not sure if the target-delegation is useful. However, moving the note processing code out of readobj I think is a great idea.

I'm trying to understand what the benefits are of the additional machinery to dispatch to a target (particularly when the notes can be used by a target that doesn't normally get regular consideration like ELF on Windows). But, moving the logic into a different location seems like something we should do.

[tpr]

tools/llvm-readobj/ELFDumper.cpp

3811

The note type I'm particularly interested in is NT_AMD_AMDGPU_PAL_METADATA, and its forthcoming replacement NT_AMD_AMDGPU_PAL_METADATA_MSGPACK. You will see that readobj currently dumps that by calling AMDGPU::PALMD::toString() in lib/Support/AMDGPUMetadata.cpp (in a slightly ad-hoc way that could be tidied).

What I am planning to do is encapsulate that metadata in a class, to include reading, writing, assembling, disassembling, dumping and code generating. That class could go in lib/Support, where the other AMDGPU metadata stuff is currently. But it will be a substantial chunk of code, and it is specific to the AMDGPU target, so I think it would be better in lib/Target/AMDGPU. To do that, I need the target dispatch thing just to be able to get to it from readobj. Then the code is present only if you enable the AMDGPU target at build time, and it is not left cluttering up lib/Support.

D52823 has the analogous abstraction for disassembling the note into a TargetStreamer.

[compnerd]

tools/llvm-readobj/ELFDumper.cpp

3811

What do you think of putting it in lib/Object/ELF? The notes are ELF specific, and, although there may be more code, I think keeping it target agnostic is better.

[tpr]

tools/llvm-readobj/ELFDumper.cpp

3811

I don't see the argument for keeping the knowledge of these target-specific ELF notes somewhere target agnostic.

Have you looked at D52823? That adds another interface to target registry, this time for disassembling these target-specific ELF notes. That really does need to be in the target, because the disassembly code will interact with the target-specific TargetStreamer. If I go ahead and do that, but keep the ELF note dumping somewhere target agnostic, then the logic will be split between two places, which doesn't seem right.

[compnerd]

tools/llvm-readobj/ELFDumper.cpp

3811

Disassembly for a specific target will require the target of course. But, I don't understand why the note is target specific. The notes are completely agnostic to what target the output is meant for. I would say that the split that you are describing with the note reading and emitting assembly for makes sense (one simply reads the target agnostic data, one emits target dependent data). Am I still not understanding something about what you are trying to accomplish?

[tpr]

This particular family of notes is target specific because it contains information on how the driver should program the hardware in order to be able to run the shaders in this graphics pipeline. So I think that justifies treating the notes as target specific.

[tpr]

Ping.

It would be good to have comments from other reviewers re the whole idea of having a TargetRegistry interface to allow readobj-style dumping of a target-specific note type to be inside the target.

include/llvm/Object/ELFNoteDumper.h

+//===- llvm/Object/ELFNoteDumper.h - ELFNoteDumper class --------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the declaration of the ELFNoteDumper class, which is used
+// to dump the contents of a note from the ELF .note section in readable form.
+// ELFNoteDumper is subclassed by targets, and you create an ELFNoteDumper via
+// the TargetRegistry.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_OBJECT_ELFNOTEDUMPER_H
+#define LLVM_OBJECT_ELFNOTEDUMPER_H
+
+#include "llvm/ADT/StringRef.h"
+
+namespace llvm {
+
+class Twine;
+
+class ELFNoteDumper {
+public:
+  ELFNoteDumper() {}
+
+  ELFNoteDumper(const ELFNoteDumper &) = delete;
+  ELFNoteDumper &operator=(const ELFNoteDumper &) = delete;
+  virtual ~ELFNoteDumper() {}
+
+  // Dump the contents of one note from the ELF .note section in readable
+  // format. This returns false if the implementation does not understand the
+  // note.
+  // The .note descriptor (i.e. contents) is passed as an StringRef of bytes as
+  // found in the ELF file, together with an indication of the endianness of
+  // the ELF file.
+  virtual bool dumpNote(unsigned Type, StringRef Name, StringRef Desc,
+                        support::endianness TargetEndianness, raw_ostream &OS) {
+    return false;
+  }
+};
+
+} // end namespace llvm
+
+#endif // LLVM_OBJECT_ELFNOTEDUMPER_H

include/llvm/Support/TargetRegistry.h

 #include <cstddef>
 #include <iterator>
 #include <memory>
 #include <string>
 
 namespace llvm {
 
 class AsmPrinter;
+class ELFNoteDumper;
 class MCAsmBackend;
 class MCAsmInfo;
 class MCAsmParser;
 class MCCodeEmitter;
 class MCContext;
 class MCDisassembler;
 class MCInstPrinter;
 class MCInstrAnalysis;
public:
   using ObjectTargetStreamerCtorTy = MCTargetStreamer *(*)(
       MCStreamer &S, const MCSubtargetInfo &STI);
   using MCRelocationInfoCtorTy = MCRelocationInfo *(*)(const Triple &TT,
                                                        MCContext &Ctx);
   using MCSymbolizerCtorTy = MCSymbolizer *(*)(
       const Triple &TT, LLVMOpInfoCallback GetOpInfo,
       LLVMSymbolLookupCallback SymbolLookUp, void *DisInfo, MCContext *Ctx,
       std::unique_ptr<MCRelocationInfo> &&RelInfo);
+  using ELFNoteDumperCtorTy = ELFNoteDumper *(*)(const Target &T);
 
 private:
   /// Next - The next registered target in the linked list, maintained by the
   /// TargetRegistry.
   Target *Next;
 
   /// The target function for checking if an architecture is supported.
   ArchMatchFnTy ArchMatchFn;
private:
   /// MCRelocationInfoCtorFn - Construction function for this target's
   /// MCRelocationInfo, if registered (default = llvm::createMCRelocationInfo)
   MCRelocationInfoCtorTy MCRelocationInfoCtorFn = nullptr;
 
   /// MCSymbolizerCtorFn - Construction function for this target's
   /// MCSymbolizer, if registered (default = llvm::createMCSymbolizer)
   MCSymbolizerCtorTy MCSymbolizerCtorFn = nullptr;
 
+  /// ELFNoteDumperCtorFn - Construction function for this target's
+  /// ELF note dumper, if registered.
+  ELFNoteDumperCtorTy ELFNoteDumperCtorFn = nullptr;
+
 public:
   Target() = default;
 
   /// @name Target Information
   /// @{
 
   // getNext - Return the next registered target.
   const Target *getNext() const { return Next; }
createMCSymbolizer(StringRef TT, LLVMOpInfoCallback GetOpInfo,
                      MCContext *Ctx,
                      std::unique_ptr<MCRelocationInfo> &&RelInfo) const {
     MCSymbolizerCtorTy Fn =
         MCSymbolizerCtorFn ? MCSymbolizerCtorFn : llvm::createMCSymbolizer;
     return Fn(Triple(TT), GetOpInfo, SymbolLookUp, DisInfo, Ctx,
               std::move(RelInfo));
   }
 
+  /// createELFNoteDumper - Create a target specific ELFNoteDumper
+  ELFNoteDumper *createELFNoteDumper() const {
+    if (!ELFNoteDumperCtorFn)
+      return nullptr;
+    return ELFNoteDumperCtorFn(*this);
+  }
+
   /// @}
 };
 
 /// TargetRegistry - Generic interface to target specific features.
 struct TargetRegistry {
   // FIXME: Make this a namespace, probably just move all the Register*
   // functions into Target (currently they all just set members on the Target
   // anyway, and Target friends this class so those functions can...
struct TargetRegistry {
   /// this is done by initializing all targets at program startup.
   ///
   /// @param T - The target being registered.
   /// @param Fn - A function to construct an MCSymbolizer for the target.
   static void RegisterMCSymbolizer(Target &T, Target::MCSymbolizerCtorTy Fn) {
     T.MCSymbolizerCtorFn = Fn;
   }
 
+  /// RegisterELFNoteDumper - Register an ELFNoteDumper
+  /// implementation for the given target.
+  ///
+  /// Clients are responsible for ensuring that registration doesn't occur
+  /// while another thread is attempting to access the registry. Typically
+  /// this is done by initializing all targets at program startup.
+  ///
+  /// @param T - The target being registered.
+  /// @param Fn - A function to construct an ELFNoteDumper for the target.
+  static void RegisterELFNoteDumper(Target &T, Target::ELFNoteDumperCtorTy Fn) {
+    T.ELFNoteDumperCtorFn = Fn;
+  }
+
   /// @}
 };
 
 //===--------------------------------------------------------------------===//
 
 /// RegisterTarget - Helper template for registering a target, for use in the
 /// target's initialization function. Usage:
 ///

tools/llvm-readobj/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
+  AllTargetsDisassemblers
   DebugInfoCodeView
   DebugInfoDWARF
   Object
   BinaryFormat
   Support
   DebugInfoCodeView
   DebugInfoMSF
   DebugInfoPDB

tools/llvm-readobj/ELFDumper.cpp

 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/Object/ELF.h"
+#include "llvm/Object/ELFNoteDumper.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Object/ELFTypes.h"
 #include "llvm/Object/Error.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Object/StackMapParser.h"
 #include "llvm/Support/AMDGPUMetadata.h"
 #include "llvm/Support/ARMAttributeParser.h"
 #include "llvm/Support/ARMBuildAttributes.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/MipsABIFlags.h"
 #include "llvm/Support/ScopedPrinter.h"
+#include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cinttypes>
 #include <cstddef>
 #include <cstdint>
 #include <cstdlib>
 #include <iterator>
 #include <memory>
auto PrintHeader = [&](const typename ELFT::Off Offset,
                          const typename ELFT::Addr Size) {
     OS << "Displaying notes found at file offset " << format_hex(Offset, 10)
        << " with length " << format_hex(Size, 10) << ":\n"
        << "  Owner                 Data size\tDescription\n";
   };
 
   auto ProcessNote = [&](const Elf_Note &Note) {
     StringRef Name = Note.getName();
+    // Elf_Note::getDesc erroneously returns an ArrayRef<Elf_Word> where the
+    // size is set to the number of bytes, not the number of Elf_Words. But
+    // fixing size to be the number of Elf_Words wouldn't work either, because
+    // n_descsz in the note header does not have to be a multiple of 4, so we
+    // would lose information in the case that the note actually contains
+    // bytes.  The code here copes with what Elf_Note::getDesc returns.
     ArrayRef<Elf_Word> Descriptor = Note.getDesc();
     Elf_Word Type = Note.getType();
 
     OS << "  " << Name << std::string(22 - Name.size(), ' ')
        << format_hex(Descriptor.size(), 10) << '\t';
 
     if (Name == "GNU") {
       OS << getGNUNoteTypeName(Type) << '\n';
       printGNUNote<ELFT>(OS, Type, Descriptor, Descriptor.size());
     } else if (Name == "FreeBSD") {
       OS << getFreeBSDNoteTypeName(Type) << '\n';
     } else if (Name == "AMD") {
       OS << getAMDGPUNoteTypeName(Type) << '\n';
       printAMDGPUNote<ELFT>(OS, Type, Descriptor, Descriptor.size());
     } else {
+      // See if any compiled-in target knows how to dump this note record. A
+      // target's ELFNoteDumper takes the descriptor as a StringRef of bytes
+      // from the ELF file, and leaves it to the implementation to sort out
+      // endianness.
+      bool Dumped = false;
+      StringRef Bytes((const char *)Descriptor.data(), Descriptor.size());
+      for (auto &T : TargetRegistry::targets()) {
+        auto Dumper = T.createELFNoteDumper();
+        if (Dumper) {
+          Dumped = Dumper->dumpNote(Type, Name, Bytes,
+                                    ELFT::TargetEndianness, OS);
+          if (Dumped)
+            break;
+        }
+      }
+      if (!Dumped)
         OS << "Unknown note type: (" << format_hex(Type, 10) << ')';
     }
     OS << '\n';
   };
 
   if (IsCore) {
     for (const auto &P : unwrapOrError(Obj->program_headers())) {
       if (P.p_type != PT_NOTE)
         continue;

tools/llvm-readobj/llvm-readobj.cpp

 #include "llvm/Support/DataTypes.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FormatVariadic.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TargetSelect.h"
 
 using namespace llvm;
 using namespace llvm::object;
 
 namespace opts {
   cl::list<std::string> InputFilenames(cl::Positional,
     cl::desc("<input object files>"),
     cl::ZeroOrMore);
static void dumpInput(StringRef File) {
   else if (WindowsResource *WinRes = dyn_cast<WindowsResource>(&Binary))
     dumpWindowsResourceFile(WinRes);
   else
     reportError(File, readobj_error::unrecognized_file_format);
 }
 
 int main(int argc, const char *argv[]) {
   InitLLVM X(argc, argv);
+  // For target-specific ELF .note dumping:
+  llvm::InitializeAllTargetInfos();
+  llvm::InitializeAllDisassemblers();
 
   // Register the target printer for --version.
   cl::AddExtraVersionPrinter(TargetRegistry::printRegisteredTargetsForVersion);
 
   // Make some commonly used short options visibile in -help.
   opts::DynamicTableShort.setHiddenFlag(cl::NotHidden);
   opts::FileHeadersShort.setHiddenFlag(cl::NotHidden);
   opts::ProgramHeadersShort.setHiddenFlag(cl::NotHidden);