This is an archive of the discontinued LLVM Phabricator instance.

[dsymutil] Implement DWARF5 accelerator tables.
ClosedPublic

Authored by JDevlieghere on Jul 10 2018, 8:34 AM.

Details

Summary

This patch add support for emitting DWARF5 accelerator tables
(.debug_names) from dsymutil. Just as with the Apple style accelerator
tables, it's possible to update existing dSYMs. This patch includes a
test that show how you can convert back and forth between the two types.

Diff Detail

Event Timeline

  • Add default option that uses accelerator table kind from input.

Are you worried about the case where the dSYM bundle contains CUs with different dwarf versions?

ObjC code in DWARF 4 compile units will not work correctly because of the emit-objc-methods-in-classes issue. Assuming this is a corner case which you don't-want-to-support-much-but-it-would-still-be-good-if-it-works, a better solution would be to just not put these kinds of units into the index. This way, the debugger will be slower as it will have to index them manually, but it will at least be correct.

But if you say this is good enough for you, then I'm fine with that too...

Update logic for picking the default accelerator table kind:

If we've only seen either Apple or DWARF accelerator tables, we pick the
respective one. If we've seen neither or both, the choice is determined
based on the lower DWARF version encountered.

labath accepted this revision.Jul 17 2018, 5:43 AM

Update logic for picking the default accelerator table kind:

If we've only seen either Apple or DWARF accelerator tables, we pick the
respective one. If we've seen neither or both, the choice is determined
based on the lower DWARF version encountered.

The new behavior makes more sense.

Apart from the out of date comment, this looks good to me (though I can't say I know much about how dsymutil works internally).

llvm/tools/dsymutil/DwarfLinker.h
169–171

I don't know why you deleted the other clear statement, but in any case, the comment is now out of date.

This revision is now accepted and ready to land.Jul 17 2018, 5:43 AM
  • Update outdated comment
JDevlieghere marked an inline comment as done.Jul 17 2018, 6:51 AM
JDevlieghere added inline comments.
llvm/tools/dsymutil/DwarfLinker.h
169–171

We can't clear the CUs yet as we need them during emission of the debug_names section to map DIEs to CUs.

JDevlieghere marked an inline comment as done.Jul 17 2018, 8:41 AM

@friss Do you want to have a look before I land this?

friss added a comment.Jul 17 2018, 9:13 AM

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

friss added a comment.Jul 17 2018, 9:16 AM

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

Looks like I should have updated the review before going through the code... Let me take another look

friss added inline comments.Jul 17 2018, 9:40 AM
llvm/tools/dsymutil/DwarfLinker.cpp
2094

Why would we need to update the dwarf versions, but not the Accelerator type here?

2322–2328

This new iteration is needed because of the registerModuleReference call bellow, right? I'm not sure this is sufficient as I think the registerModuleReference can pull in new debug information from dependencies of the referenced modules.

2330–2344

I'd rather we are even more conservative. I'd prefer if we chose Dwarf5 tables only if all the input is Dwarf5 and none of it has Apple tables.

llvm/tools/dsymutil/DwarfLinker.h
169–171

What kind of memory does the CU hold? Do we see an increase in overall memory consumption?

Address code review feedback @friss.

The problem determining the accelerator kind in the default case is that registerModuleReference potentially imports new compile units, which might affect the dwarf version and accelerator table kind. If this is the case, we clone all compile units which means adding entries to the accelerator table. This is problematic because the next such module might affect the decision of which kind of accelerator table to use. I think there's several solutions to this problem:

  1. Ignore this and don't let referenced modules affect the decision.
  2. Don't clone the CUs at that point would remove the need to populate the tables. This would require restructuring quite a bit of code, and also means keeping unnecessary stuff in memory longer. (e.g. it's CU list, DwarfContext, etc)
  3. Have a generic data structure that we can later convert in either an Apple or Dwarf accelerator table. This would require copying info from one struct to another, and we'd need a way to differentiate between what goes where. For the Apple case information will end up in separate tables, while for DWARF there's only one. We also designed the accelerator tables in such a way that they were configurable in their type, by going through another type we'd essentially nullify that effort.
  4. Populate both tables as long as we're in limbo about the kind. This means we're duplicating information in memory, but only for modules. As soon as we know the kind we clear the other tables and reclaim the memory.

In this patch I've implemented (4) because it was relatively straightforward and only affects CUs imported by modules.

JDevlieghere marked 5 inline comments as done.EditedJul 17 2018, 11:32 AM

maybeUpdateAccelKind() will make a final call about the kind of accelerator tables on the first object it encounters which has accelerator tables. This seems pretty arbitrary. I'd prefer if we considered Apple table the default unless all files are Dwarf5 and have Dwarf5 accelerator tables.
Other than that, the implementation looks good. I'd like to know how the LLDB test suite behaves if you force it to use the new accelerator tables.

Failing Tests (2):

lldb-Suite :: python_api/class_members/TestSBTypeClassMembers.py
lldb-Suite :: tools/lldb-server/TestAppleSimulatorOSType.py

The python_api is expected, as it also failed before my change in clang to emit the members as children of the DIE (which is missing in this run, therefore making the failure expected). The last failure was unrelated, as all categories failed.

llvm/tools/dsymutil/DwarfLinker.cpp
2322–2328

Correct. The issue is that loadClangModule calls cloneAllCompileUnits which in turn wants to emitAcceleratorEntriesForUnit. Hence at that point we need to have what type of accelerator table we are going to emit, otherwise we might change the decision after the fact. Please see my latest update of the patch for all the details.

2330–2344

This is now the case in the latest revision of this patch. As soon as we see an Apple accelerator table we update the default to that.

llvm/tools/dsymutil/DwarfLinker.h
169–171

Instead of keeping the whole CU we now only tack the necessary info in EmittedCU (a pointer and an unsigned).

JDevlieghere marked 3 inline comments as done.
  • Ignore modules for now as discussed offline.
  • Document this decision.
friss accepted this revision.Jul 25 2018, 9:54 AM

This LGTM

This revision was automatically updated to reflect the committed changes.