[llvm][dwarf][rfc][donotcommit] Enable print of ranges addresses from .debug_info.dwo

[ayermolo]

Summary

For split dwarf some of the sections remain in the main binary. For DWARF4 it's
.debug_ranges, .debug_addr. For DWARF5 it's .debug_addr. When using
llvm-dwarfdump on .dwo/.dwp files this results in not being able to see what ranges
and addresses for DW_AT_low_pc are used in DIEs, and output having "Error: " in it.

I added a new option --main-binary=<binary> that will create a link in
DWARFContext between DWO context and main binary. This allows tool to display
addresses for DW_AT_ranges and DW_AT_low_pc.

Example (DWARF5):

DW_TAG_inlined_subroutine
        DW_AT_abstract_origin (0x00000b21 "flush_RL")
        DW_AT_ranges  (indexed (0x0) rangelist = 0x00000054
          [0x0000000000403fe2, 0x0000000000403ff3)
          [0x0000000000403ff6, 0x0000000000403ffe))

DW_TAG_subprogram
        DW_AT_low_pc  (0x0000000000403940)

[ayermolo]

Will add tests if the approach is acceptable.

[ayermolo]

rebase

[ayermolo]

rebase

[ayermolo]

rebase

[ayermolo]

update from upstream branch

[dblaikie]

Might be easier to go the other way, and when dumping the main binary, also dump associated dwo/dwp files? (Perhaps under a flag, or by default, not sure)

[clayborg]

In D159374#4635446, @dblaikie wrote:

Might be easier to go the other way, and when dumping the main binary, also dump associated dwo/dwp files? (Perhaps under a flag, or by default, not sure)

I have a patch that does this that I am going to submit soon. If you dump the main executable, if it can find the .dwo/.dwp file(s), it will dump them. I added a "--dump-dwo" option to control if we dump the .dwo file after each skeleton compile unit. I also made many options work like looking up addresses and names.

[ayermolo]

In D159374#4635446, @dblaikie wrote:

Might be easier to go the other way, and when dumping the main binary, also dump associated dwo/dwp files? (Perhaps under a flag, or by default, not sure)

From usability perspective I think this is an easier way.
--debug-info=<address> just works when dumping out .dwo or DWP file.
When dumping debug_info from random .dwo/.o files no need to copy paste the DWO ID hash to output sepcific ones.
It just works.

[clayborg]

In D159374#4635457, @ayermolo wrote:

In D159374#4635446, @dblaikie wrote:

Might be easier to go the other way, and when dumping the main binary, also dump associated dwo/dwp files? (Perhaps under a flag, or by default, not sure)

From usability perspective I think this is an easier way.
--debug-info=<address> just works when dumping out .dwo or DWP file.
When dumping debug_info from random .dwo/.o files no need to copy paste the DWO ID hash to output sepcific ones.
It just works.

Yeah, that is something I need to add to my patch:

--dwo-id <id> --debug-info-dwo <offset>

You would need to specify the DWO ID and a "--debug-info-dwo" to dump a DIE if the main executable is specified... Both workflows seem valid IMHO

[ayermolo]

Also might be easier to explain to an "average" user "Just run llvm-dwarfdump --main-binary=<path_to_main_binary> --debug-info <path_to_dwo>" Vs Dump this DWO, look up DWOID (explain where it is in DWARF4 vs 5), then run llvm-dwarfdump again now with main binary with these flags.

[ayermolo]

OK trying new workflow:
https://github.com/llvm/llvm-project/pull/65516

[ayermolo]

@dblaikie ping. :) Do you want to continue review here or in github?

[dblaikie]

Don't mind either way - but I'm out on vacation for another week or so, so will be a little bit before I get back to this. Probably GitHub I guess?

llvm/include/llvm/DebugInfo/DWARF/DWARFContext.h

class DWARFContext : public DIContext {
   /// Read compile units from the debug_info.dwo section (if necessary)
   /// and type units from the debug_types.dwo section (if necessary)
   /// and store them in DWOUnits.
   /// If \p Lazy is true, set up to parse but don't actually parse them.
   enum { EagerParse = false, LazyParse = true };
   void parseDWOUnits(bool Lazy = false);
 
   std::unique_ptr<const DWARFObject> DObj;
+  /// Can be optionally set by tools that work with .dwo/.dwp files to reference
+  /// main binary debug information. Usefull for accessing .debug_ranges and
+  /// .debug_addr section.
+  std::unique_ptr<const DWARFObject> MainBinaryDObj = nullptr;
+  /// DWARFContext for main binary.
+  std::unique_ptr<DWARFContext> MainBinaryDICtx = nullptr;
 
   /// Helper enum to distinguish between macro[.dwo] and macinfo[.dwo]
   /// section.
   enum MacroSecType {
     MacinfoSection,
     MacinfoDwoSection,
     MacroSection,
     MacroDwoSection
DWARFContext(std::unique_ptr<const DWARFObject> DObj,
                    WithColor::defaultWarningHandler);
   ~DWARFContext() override;
 
   DWARFContext(DWARFContext &) = delete;
   DWARFContext &operator=(DWARFContext &) = delete;
 
   const DWARFObject &getDWARFObj() const { return *DObj; }
 
+  const DWARFObject *getMainBinaryDWARFObj() const {
+    return MainBinaryDObj.get();
+  }
+  const DWARFContext *getDWARFContextMainBinary() const {
+    return MainBinaryDICtx.get();
+  }
+
   static bool classof(const DIContext *DICtx) {
     return DICtx->getKind() == CK_DWARF;
   }
 
   /// Dump a textual representation to \p OS. If any \p DumpOffsets are present,
   /// dump only the record at the specified offset.
   void dump(raw_ostream &OS, DIDumpOptions DumpOpts,
             std::array<std::optional<uint64_t>, DIDT_ID_Count> DumpOffsets);
public:
   /// Returns whether CU/TU should be populated manually. TU Index populated
   /// manually only for DWARF5.
   bool getParseCUTUIndexManually() const { return ParseCUTUIndexManually; }
 
   /// Sets whether CU/TU should be populated manually. TU Index populated
   /// manually only for DWARF5.
   void setParseCUTUIndexManually(bool PCUTU) { ParseCUTUIndexManually = PCUTU; }
 
+  /// Sets the object corresponding to the main binary to which the .dwo/.dwp
+  /// file belongs.
+  void setMainBinaryObjAndCreateContext(
+      const object::ObjectFile &Obj,
+      ProcessDebugRelocations RelocAction = ProcessDebugRelocations::Process,
+      const LoadedObjectInfo *L = nullptr,
+      std::function<void(Error)> RecoverableErrorHandler =
+          WithColor::defaultErrorHandler,
+      std::function<void(Error)> WarningHandler =
+          WithColor::defaultWarningHandler);
+
 private:
   /// Parse a macro[.dwo] or macinfo[.dwo] section.
   std::unique_ptr<DWARFDebugMacro>
   parseMacroOrMacinfo(MacroSecType SectionType);
 
   void addLocalsForDie(DWARFCompileUnit *CU, DWARFDie Subprogram, DWARFDie Die,
                        std::vector<DILocal> &Result);
 };
 
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_DWARF_DWARFCONTEXT_H

llvm/lib/DebugInfo/DWARF/DWARFContext.cpp

 #include "llvm/Object/Decompressor.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Object/RelocationResolver.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/DataExtractor.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/Format.h"
-#include "llvm/Support/LEB128.h"
 #include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/LEB128.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cstdint>
 #include <deque>
 #include <map>
+#include <optional>
 #include <string>
+#include <unordered_map>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 using namespace dwarf;
 using namespace object;
 
 #define DEBUG_TYPE "dwarf"
void DWARFContext::dump(
   // Dump individual sections.
   if (shouldDump(Explicit, ".debug_abbrev", DIDT_ID_DebugAbbrev,
                  DObj->getAbbrevSection()))
     getDebugAbbrev()->dump(OS);
   if (shouldDump(ExplicitDWO, ".debug_abbrev.dwo", DIDT_ID_DebugAbbrev,
                  DObj->getAbbrevDWOSection()))
     getDebugAbbrevDWO()->dump(OS);
 
+  std::unordered_map<uint64_t, DWARFUnit *> DwoIDToDUMap;
+  auto SetSections = [&](DWARFUnit &U) -> void {
+    std::optional<uint64_t> DWOID = U.getDWOId();
+    if (U.isDWOUnit() && DWOID) {
+      auto MapIter = DwoIDToDUMap.find(*DWOID);
+      if (MapIter != DwoIDToDUMap.end()) {
+        DWARFDie UnitDie = MapIter->second->getUnitDIE();
+        std::optional<uint64_t> RangeBase = UnitDie.getRangesBaseAttribute();
+        if (U.getVersion() < 5 && RangeBase)
+          U.setRangesSection(&MainBinaryDObj->getRangesSection(), *RangeBase);
+        if (std::optional<uint64_t> AddrBase =
+                MapIter->second->getAddrOffsetSectionBase())
+          U.setAddrOffsetSection(&MainBinaryDObj->getAddrSection(), *AddrBase);
+      }
+    }
+  };
   auto dumpDebugInfo = [&](const char *Name, unit_iterator_range Units) {
     OS << '\n' << Name << " contents:\n";
     if (auto DumpOffset = DumpOffsets[DIDT_ID_DebugInfo])
-      for (const auto &U : Units)
+      for (const auto &U : Units) {
+        SetSections(*U);
         U->getDIEForOffset(*DumpOffset)
             .dump(OS, 0, DumpOpts.noImplicitRecursion());
+      }
     else
-      for (const auto &U : Units)
+      for (const auto &U : Units) {
+        SetSections(*U);
         U->dump(OS, DumpOpts);
+      }
   };
   if ((DumpType & DIDT_DebugInfo)) {
     if (Explicit || getNumCompileUnits())
       dumpDebugInfo(".debug_info", info_section_units());
-    if (ExplicitDWO || getNumDWOCompileUnits())
+    if (ExplicitDWO || getNumDWOCompileUnits()) {
+      if (MainBinaryDObj) {
+        for (const auto &U : MainBinaryDICtx->compile_units())
+          if (std::optional<uint64_t> DWOID = U->getDWOId())
+            DwoIDToDUMap.insert({*DWOID, U.get()});
+      }
       dumpDebugInfo(".debug_info.dwo", dwo_info_section_units());
     }
+  }
 
   auto dumpDebugType = [&](const char *Name, unit_iterator_range Units) {
     OS << '\n' << Name << " contents:\n";
     for (const auto &U : Units)
       if (auto DumpOffset = DumpOffsets[DIDT_ID_DebugTypes])
         U->getDIEForOffset(*DumpOffset)
             .dump(OS, 0, DumpOpts.noImplicitRecursion());
       else
uint8_t DWARFContext::getCUAddrSize() {
   // In theory, different compile units may have different address byte
   // sizes, but for simplicity we just use the address byte size of the
   // first compile unit. In practice the address size field is repeated across
   // various DWARF headers (at least in version 5) to make it easier to dump
   // them independently, not to enable varying the address size.
   auto CUs = compile_units();
   return CUs.empty() ? 0 : (*CUs.begin())->getAddressByteSize();
 }
+
+void DWARFContext::setMainBinaryObjAndCreateContext(
+    const object::ObjectFile &Obj, ProcessDebugRelocations RelocAction,
+    const LoadedObjectInfo *L,
+    std::function<void(Error)> RecoverableErrorHandler,
+    std::function<void(Error)> WarningHandler) {
+  MainBinaryDObj = std::make_unique<DWARFObjInMemory>(
+      Obj, L, RecoverableErrorHandler, WarningHandler, RelocAction);
+  MainBinaryDICtx =
+      DWARFContext::create(Obj, DWARFContext::ProcessDebugRelocations::Process,
+                           nullptr, "", RecoverableErrorHandler);
+}

llvm/lib/DebugInfo/DWARF/DWARFUnit.cpp

bool DWARFUnitHeader::applyIndexEntry(const DWARFUnitIndex::Entry *Entry) {
   AbbrOffset = AbbrEntry->getOffset();
   return true;
 }
 
 Error DWARFUnit::extractRangeList(uint64_t RangeListOffset,
                                   DWARFDebugRangeList &RangeList) const {
   // Require that compile unit is extracted.
   assert(!DieArray.empty());
-  DWARFDataExtractor RangesData(Context.getDWARFObj(), *RangeSection,
-                                IsLittleEndian, getAddressByteSize());
+  DWARFDataExtractor RangesData(isDWOUnit() && Context.getMainBinaryDWARFObj()
+                                    ? *Context.getMainBinaryDWARFObj()
+                                    : Context.getDWARFObj(),
+                                *RangeSection, IsLittleEndian,
+                                getAddressByteSize());
   uint64_t ActualRangeListOffset = RangeSectionBase + RangeListOffset;
   return RangeList.extract(RangesData, &ActualRangeListOffset);
 }
 
 void DWARFUnit::clear() {
   Abbrevs = nullptr;
   BaseAddr.reset();
   RangeSectionBase = 0;

llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp

static opt<OPTION> Dump##ENUM_NAME(CMDLINE_NAME,                             \
                                      cat(SectionCategory));
 #include "llvm/BinaryFormat/Dwarf.def"
 #undef HANDLE_DWARF_SECTION
 
 // The aliased DumpDebugFrame is created by the Dwarf.def x-macro just above.
 static alias DumpDebugFrameAlias("eh-frame", desc("Alias for --debug-frame"),
                                  NotHidden, cat(SectionCategory),
                                  aliasopt(DumpDebugFrame));
+static cl::opt<std::string>
+    MainBinary("main-binary",
+               desc("Specifies the main binary for cases when .dwo/.dwp file "
+                    "is processed."),
+               cl::init(""), cat(DwarfDumpCategory));
 static list<std::string>
     ArchFilters("arch",
                 desc("Dump debug information for the specified CPU "
                      "architecture only. Architectures may be specified by "
                      "name or by number. This option can be specified "
                      "multiple times, once for each desired architecture."),
                 cat(DwarfDumpCategory));
 static opt<bool>
static bool handleArchive(StringRef Filename, Archive &Arch,
   error(Filename, std::move(Err));
 
   return Result;
 }
 
 static bool handleBuffer(StringRef Filename, MemoryBufferRef Buffer,
                          HandlerFn HandleObj, raw_ostream &OS) {
   Expected<std::unique_ptr<Binary>> BinOrErr = object::createBinary(Buffer);
+  std::unique_ptr<MemoryBuffer> MainBuffer = nullptr;
+  ErrorOr<std::unique_ptr<MemoryBuffer>> MainBuffOrErr = nullptr;
+  std::unique_ptr<Binary> MainBin = nullptr;
   error(Filename, BinOrErr.takeError());
 
   bool Result = true;
   auto RecoverableErrorHandler = [&](Error E) {
     Result = false;
     WithColor::defaultErrorHandler(std::move(E));
   };
   if (auto *Obj = dyn_cast<ObjectFile>(BinOrErr->get())) {
     if (filterArch(*Obj)) {
       std::unique_ptr<DWARFContext> DICtx = DWARFContext::create(
           *Obj, DWARFContext::ProcessDebugRelocations::Process, nullptr, "",
           RecoverableErrorHandler);
       DICtx->setParseCUTUIndexManually(ManuallyGenerateUnitIndex);
+      if (!MainBinary.empty()) {
+        MainBuffOrErr = MemoryBuffer::getFileOrSTDIN(MainBinary);
+        error(MainBinary, MainBuffOrErr.getError());
+        MainBuffer = std::move(MainBuffOrErr.get());
+        Expected<std::unique_ptr<Binary>> MainBinOrErr =
+            object::createBinary(*MainBuffer);
+        error(MainBinary, MainBinOrErr.takeError());
+        MainBin = std::move(MainBinOrErr.get());
+        if (auto *Obj = dyn_cast<ObjectFile>(MainBin.get()))
+          DICtx->setMainBinaryObjAndCreateContext(
+              *Obj, DWARFContext::ProcessDebugRelocations::Process, nullptr,
+              RecoverableErrorHandler);
+      }
       if (!HandleObj(*Obj, *DICtx, Filename, OS))
         Result = false;
     }
   } else if (auto *Fat = dyn_cast<MachOUniversalBinary>(BinOrErr->get()))
     for (auto &ObjForArch : Fat->objects()) {
       std::string ObjName =
           (Filename + "(" + ObjForArch.getArchFlagName() + ")").str();
       if (auto MachOOrErr = ObjForArch.getAsObjectFile()) {