This is an archive of the discontinued LLVM Phabricator instance.

Make a DwarfGen class that can generate DWARF for unittests and add units tests to test DWARF API.
AbandonedPublic

Authored by clayborg on Nov 18 2016, 5:24 PM.

Details

Summary

The only tests we have for the DWARF parser are the tests that use llvm-dwarfdump and expect output from textual dumps.

More DWARF parser modification are coming in the next few weeks and I wanted to add tests that can verify that we can encode and decode all form types, as well as test some other basic DWARF APIs where we ask DIE objects for their children and siblings.

DwarfGenerator.cpp was added in the lib/CodeGen directory. This file contains the code necessary to easily create DWARF for tests:

DwarfGen DG;
Triple Triple("x86_64--");
// Initialize the DWARF generator
bool success = DG.init(Triple, Version);
// Make a compile unit
DwarfGenCU &CU = DG.addCompileUnit();
// Get the compile unit DIE
DwarfGenDIE CUDie = CU.getUnitDIE();
 
CUDie.addAttribute(DW_AT_name, DW_FORM_strp, "/tmp/main.c");
CUDie.addAttribute(DW_AT_language, DW_FORM_data2, DW_LANG_C);
 
DwarfGenDIE SubprogramDie = CUDie.addChild(DW_TAG_subprogram);
SubprogramDie.addAttribute(DW_AT_name, DW_FORM_strp, "main");
SubprogramDie.addAttribute(DW_AT_low_pc, DW_FORM_addr, 0x1000U);
SubprogramDie.addAttribute(DW_AT_high_pc, DW_FORM_addr, 0x2000U);
 
DwarfGenDIE IntDie = CUDie.addChild(DW_TAG_base_type);
IntDie.addAttribute(DW_AT_name, DW_FORM_strp, "int");
IntDie.addAttribute(DW_AT_encoding, DW_FORM_data1, DW_ATE_signed);
IntDie.addAttribute(DW_AT_byte_size, DW_FORM_data1, 4);

DwarfGenDIE ArgcDie = SubprogramDie.addChild(DW_TAG_formal_parameter);
ArgcDie.addAttribute(DW_AT_name, DW_FORM_strp, "argc");
ArgcDie.addAttribute(DW_AT_type, DW_FORM_ref4, IntDie);

StringRef FileBytes = DG.generate();
MemoryBufferRef FileBuffer(FileBytes, "dwarf");
auto Obj = object::ObjectFile::createObjectFile(FileBuffer);
if (Obj) {
  DWARFContextInMemory DwarfContext(*Obj.get());
  uint32_t NumCUs = DwarfContext.getNumCompileUnits();
  ...
}

This code is backed by the AsmPrinter code that emits DWARF for the actual compiler. There are places in the AsmPrinter where it was expecting to always have an instance of llvm::DwarfDebug in order to emit certain things, but both the DwarfGen class and the the DwarfLinker both use AsmPrinters to generate DWARF that doesn't used the DwarfDebug class. Code was modified in the AsmPrinter to allow users to specify the DWARF version manually so that DW_FORM_ref_addr and a few other attribute can now be emitted correctly.

Diff Detail

Event Timeline

clayborg updated this revision to Diff 78610.Nov 18 2016, 5:24 PM
clayborg retitled this revision from to Make a DwarfGen class that can generate DWARF for unittests and add units tests to test DWARF API..
clayborg updated this object.

Forgot to remove on #include that was left in AsmPrinter.h

include/llvm/CodeGen/AsmPrinter.h
21–22

This doesn't need to be here. I will remove it in next patch.

Anyone around to review this?

dblaikie edited edge metadata.Nov 22 2016, 9:08 AM

I'm curious/would like to better understand what motivated the changes to to existing code (in lib/CodeGen/AsmPrinter, DwarfLinker, etc) - given that llvm-dsymutil was already using these APIs to generate DWARF without needing these changes? (what's different about the unit test scenario compared to the llvm-dsymutil scenario)

I'm curious/would like to better understand what motivated the changes to to existing code (in lib/CodeGen/AsmPrinter, DwarfLinker, etc) - given that llvm-dsymutil was already using these APIs to generate DWARF without needing these changes? (what's different about the unit test scenario compared to the llvm-dsymutil scenario)

llvm-dsymutil just avoids using the AsmPrinter for anything that would cause it to assert and die due to lack of having DwarfDebug. In the function DwarfLinker::DIECloner::cloneDieReferenceAttribute() we see:

if (AttrSpec.Form == dwarf::DW_FORM_ref_addr ||
    (Unit.hasODR() && isODRAttribute(AttrSpec.Attr))) {
  // We cannot currently rely on a DIEEntry to emit ref_addr
  // references, because the implementation calls back to DwarfDebug
  // to find the unit offset. (We don't have a DwarfDebug)
  // FIXME: we should be able to design DIEEntry reliance on
  // DwarfDebug away.
  uint64_t Attr;
  if (Ref < InputDIE.getOffset()) {
    // We must have already cloned that DIE.
    uint32_t NewRefOffset =
        RefUnit->getStartOffset() + NewRefDie->getOffset();
    Attr = NewRefOffset;
    Die.addValue(DIEAlloc, dwarf::Attribute(AttrSpec.Attr),
                 dwarf::DW_FORM_ref_addr, DIEInteger(Attr));
  } else {
    // A forward reference. Note and fixup later.
    Attr = 0xBADDEF;
    Unit.noteForwardReference(
        NewRefDie, RefUnit, Ctxt,
        Die.addValue(DIEAlloc, dwarf::Attribute(AttrSpec.Attr),
                     dwarf::DW_FORM_ref_addr, DIEInteger(Attr)));
  }
  return U.getRefAddrByteSize();
}

They basically manually iterate through all DIEs before they attempt to let the AsmPrinter emit then and change any DIEValue objects that refer to a DIEEntry into a DIEInteger. Not sure how they size the DIEInteger correctly when emitting it though. I don't think they need to take into account DWARF version 2 since they don't deal with that. but I haven't looked too closely at the DwarfLinker to know for sure.

By adding the DWARF version we can just use all of the code in DIE.h/.cpp as is without any changes.

We actually don't need to modify the DwarfLinker, but I was planning on letting Fred Riss know that he can remove the code that works around a DIEEntry reference with DW_FORM_ref_addr after this was in. I didn't want to change the dwarf linker further to remove his work around without talking to him.

Actually the DIEInteger class was grabbing the Dwarf version from the OutStreamer:

if (AP->OutStreamer->getContext().getDwarfVersion() == 2)

Not sure why other places were using the DwarfDebug in AsmPrinter. We currently have two places that have the DWARF version. I can change to get the DWARF version from there and might be able to get rid of some of the AsmPrinter changes.

clayborg updated this revision to Diff 78901.Nov 22 2016, 11:28 AM
clayborg edited edge metadata.

Cleaned up how DWARF versions are stored. DwarfDebug was storing its own copy of the DWARF version and the out streamer also contained a version. Centralized the DWARF version code to live in the streamer. This allows AsmPrinter clients (like many classes in DIE.cpp) to not have to rely on the AsmPrinter having a DwarfDebug class just to get the DWARF version.

David, let me know what you think about this approach. I removed a redundant DWARF version that was being stored in DwarfDebug and it cleans things up nicely.

I think it would be easier to follow if it were split up a bit more (but perhaps others who can review (Adrian et al) don't mind so much) - seems like there's at least 3 bits in here: move fields out of DwarfDebug (some of that's just refactoring, as you did in the last update), generalize support and use it in DwarfLinker to remove the special case, then add the unit test wrapper on top of the now generalized API.

I'll do 2 changes, the refactoring of DwarfDebug ivars with a new patch and the DWARF generator with this patch. I will talk with Fred and work on a fix with him when I get back from break. There won't need to be any DWARF linker changes once the version is moved.

clayborg abandoned this revision.Dec 1 2016, 5:28 PM

I will submit a new patch that is sufficiently different and don't want previous code comments to interfere.