This is an archive of the discontinued LLVM Phabricator instance.

DWARF: Introduce DWARFTypeUnit class
ClosedPublic

Authored by labath on May 16 2019, 6:53 AM.

Details

Summary

This patch introduces the DWARFTypeUnit class, and teaches lldb to parse
type units out of both the debug_types section (DWARF v4), and from the
regular debug_info section (DWARF v5).

The most important piece of functionality - resolving DW_AT_signatures
to connect type forward declarations to their definitions - is not
implemented here, but even without that, a lot of functionality becomes
available. I've added tests for the commands that start to work after
this patch.

The changes in this patch were greatly inspired by D61505, which in turn took
over changes from D32167.

Diff Detail

Repository
rLLDB LLDB

Event Timeline

labath created this revision.May 16 2019, 6:53 AM

On Fedora 29 x86_64 I get:

FAIL: LLDB :: SymbolFile/DWARF/debug-types-basic.test (123 of 1636)
******************** TEST 'LLDB :: SymbolFile/DWARF/debug-types-basic.test' FAILED ********************
Script:
--
: 'RUN: at line 3';   /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/clang --driver-mode=g++ -pthread -target x86_64-pc-linux /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/Inputs/debug-types-basic.cpp    -g -fdebug-types-section -c -o /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-basic.test.tmp.o
: 'RUN: at line 5';   /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/ld.lld /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-basic.test.tmp.o -o /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-basic.test.tmp
: 'RUN: at line 6';   /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/lldb --no-lldbinit -S /home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-basic.test.tmp -s /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test -o exit | /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/FileCheck /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test
--
Exit Code: 1

Command Output (stderr):
--
ld.lld: warning: cannot find entry symbol _start; defaulting to 0x201000
warning: (x86_64) /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-basic.test.tmp lldb doesn’t support .debug_types debug info
error: use of undeclared identifier 'E'
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test:10:10: error: CHECK: expected string not found in input
# CHECK: struct A {
         ^
<stdin>:11:1: note: scanning from here
no type was found matching 'A'
^
<stdin>:11:11: note: possible intended match here
no type was found matching 'A'
          ^
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test:19:10: error: CHECK: expected string not found in input
# CHECK: enum E {
         ^
<stdin>:13:1: note: scanning from here
no type was found matching 'E'
^
<stdin>:14:15: note: possible intended match here
(lldb) type lookup EC
              ^
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test:27:10: error: CHECK: expected string not found in input
# CHECK: enum class EC {
         ^
<stdin>:15:1: note: scanning from here
no type was found matching 'EC'
^
<stdin>:15:24: note: possible intended match here
no type was found matching 'EC'
                       ^
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-basic.test:38:16: error: CHECK-LABEL: expected string not found in input
# CHECK-LABEL: print (EC) 1
               ^
<stdin>:17:1: note: scanning from here
(lldb) exit
^
<stdin>:17:8: note: possible intended match here
(lldb) exit
       ^

--

********************
FAIL: LLDB :: SymbolFile/DWARF/debug-types-expressions.test (144 of 1636)
******************** TEST 'LLDB :: SymbolFile/DWARF/debug-types-expressions.test' FAILED ********************
Script:
--
: 'RUN: at line 3';   /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/clang --driver-mode=g++ -pthread /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/Inputs/debug-types-expressions.cpp    -g -fdebug-types-section -o /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-expressions.test.tmp
: 'RUN: at line 5';   /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/lldb --no-lldbinit -S /home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-expressions.test.tmp -s /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-expressions.test -o exit | /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/FileCheck /home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-expressions.test
--
Exit Code: 1

Command Output (stderr):
--
warning: (x86_64) /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/SymbolFile/DWARF/Output/debug-types-expressions.test.tmp lldb doesn’t support .debug_types debug info
error: Multiple internal symbols found for 'a'
id = {0x00000e8f}, range = [0x00001555552e2348-0x00001555552e2350), name="a"
id = {0x000003ee}, range = [0x00001555552a1c78-0x00001555552a1c80), name="a"
id = {0x00000b5d}, range = [0x00001555552e2348-0x00001555552e2350), name="a"
id = {0x00000d17}, range = [0x00001555552e2348-0x00001555552e2350), name="a"

error: use of undeclared identifier 'a'
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-expressions.test:14:10: error: CHECK: expected string not found in input
# CHECK: (B *) a =
         ^
<stdin>:24:1: note: scanning from here
(lldb) print a->f()
^
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/SymbolFile/DWARF/debug-types-expressions.test:21:16: error: CHECK-LABEL: expected string not found in input
# CHECK-LABEL: print ns::A()
               ^
<stdin>:25:1: note: scanning from here
(lldb) exit
^
<stdin>:25:8: note: possible intended match here
(lldb) exit
       ^

--

********************
Testing Time: 88.82s
********************
Failing Tests (2):
    LLDB :: SymbolFile/DWARF/debug-types-basic.test
    LLDB :: SymbolFile/DWARF/debug-types-expressions.test

  Expected Passes    : 1587
  Expected Failures  : 1
  Unsupported Tests  : 46
  Unexpected Failures: 2

On Fedora 29 x86_64 I get:
...

Ah, sorry I forgot to include the part which removes the run-away-screaming-if-you-see-debug_types check. I had that in a separate commit and I didn't squash. I'll fix that in a second...

labath updated this revision to Diff 199831.May 16 2019, 8:00 AM
  • don't bail out if you see the debug_types section
  • rename a variable in the test to work around the old "Multiple internal symbols found for 'a'" problem
labath updated this revision to Diff 200230.May 20 2019, 3:01 AM
  • rebase on top of the DWARFUnitHeader patch
labath retitled this revision from DWARF: Parse type units from the debug_types section to DWARF: Introduce DWARFTypeUnit class.May 20 2019, 3:01 AM
labath edited the summary of this revision. (Show Details)

Replying here as this is the relevant patch.

Are type units still disabled with the kill switch we had in? Or is this on top of the .debug_types patch? We have an internal .debug_types patch we have been using on an older LLVM/Clang/LLDB and we had some major performance issues regarding line tables so I am not sure where we would need to put these fixes (in this patch or in the .debug_types specific patch. This patch seems to enable parsing .debug_types already. The main issue we ran into were:

  • only search actual compile unit line tables for breakpoints as many .debug_type units will point to existing line tables from real compile units
  • don't add address ranges to type unit DWARFUnits from the line tables
  • re-work how support files are parsed by sharing the line tables within a SymbolFileDWARF. .debug_types has type units that reuse line tables from real compile units (thousands of times) and we ended up seeing the same line table being parsed over and over and over. Another way to fix this is to no vend a lldb_private::CompileUnit for any units that aren't really compile units (DW_UT_type and DW_UT_split_type).

So if this patch is enabling all this, we will need to fix this patch to avoid all of the above mentioned issues. Let me know your thoughts.

Thanks for the heads up. This is a pretty good to-do list. I was aware of some of these things already (like the address ranges thing), but in the spirit of the incremental development policy, I deliberately tried to keep this patch small. I already have a patch for to stop adding address ranges for type units, but I didn't upload it yet, as I am still working on the test (that's the reason I side-stepped into dwarf5 a bit, because I believe it will make creating the test easier). Another thing, which this patch does not enable yet is the resolution of DW_AT_signatures, so with this patch alone you will be able to look up types, but not link them to variables. I also put that in a separate patch, because it was possible to do so. One more big feature, which is completely broken right now, is the combination of dwo+type units. For that I don't have a patch yet, and making that work is probably going to take a bit longer.

For these reason I don't think it's reasonable to put all of these things into one big patch, and have a fully functional and optimised implementation from the start -- having things working 100% immediately is very unlikely and a large patch increases the likelyhood of breaking unrelated things. I think we should start with this patch. The address ranges patch and DW_AT_signature thingy, can come pretty much immediately after that. These are the only things (that I am aware of) which impact correctness, so we could have an implementation ready for general audience to try out as early as the end of this week. After that the road splits up a bit, and we can proceed in several directions (e.g. I'm hoping that DWO support can be mostly independent of the line table optimizations). I might even be able to get some help with that, so that we increase our velocity here, once the initial building blocks are in place.

As for the "kill switch", I am open to keeping it in, but make it conditional on some setting or environment variable, until the implementation is more stable. However, I also don't think it's really necessary, as I believe we will have something sufficiently functional very quickly. (and even a slightly wonky experience is better than just refusing to debug a binary outright, imo)

clayborg accepted this revision.May 20 2019, 9:44 AM

Ok, as long as we get things in soon, some functionality is better than none.

This revision is now accepted and ready to land.May 20 2019, 9:44 AM
aprantl added inline comments.May 20 2019, 10:00 AM
lit/SymbolFile/DWARF/debug-types-expressions.test
29

[This is more a meta-point and I don't care very strongly about it]
For a test of the DWARF parser, is it necessary to actually attach to a running process, or could we statically inspect the types of, e.g., global variables with target var or something?

labath marked an inline comment as done.May 21 2019, 12:29 AM

Thanks. We can always re-add the check if things take longer than expected.

lit/SymbolFile/DWARF/debug-types-expressions.test
29

Yes and no. :)

I believe a lot of things can be tested without a running process, and I've tried to make use of that as much as possible. However, it is my understanding (though it's still a fairly fuzzy one) that target/frame variable and the expression evaluator use the debug info differently.

That's why I've created two tests here. The "basic" one does not require a running process and can run anywhere. I think of this as the first line of defense. Then, if anything slips past that, there's the "expression" test, which will try to use the debug info from the expression evaluator. That one requires us to be able to generate type units for the target platform (i.e., elf only at the moment).

This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptMay 22 2019, 2:08 AM
Herald added a subscriber: teemperor. · View Herald Transcript