This is an archive of the discontinued LLVM Phabricator instance.

[WIP][POC][DebugInfo] Support for DW_AT_start_scope for scoped variables
AbandonedPublic

Authored by SouraVX on Jul 31 2020, 3:43 AM.

Details

Summary

This patch is POC for correctly capturing scope information for
reduced scope variables.

For more context please refer to llvm-dev discussion:
http://lists.llvm.org/pipermail/llvm-dev/2020-April/140962.html

Intent of this patch is to convey overall design/functionality,
seek feedback and converge.
There was an unfortunate delay between the patch and discussion,
Overall design was also discussed in above link(later).

Impact on debug-info statistics(collected from compiling trunk-clang
before and after this patch):

Before this patch:

llvm-dwarfdump --show-section-sizes bin/clang
file: bin/clang
----------------------------------------------------
SECTION        SIZE (b)
-------------  ---------
.debug_info    900357322 (58.29%)
.debug_loc        868784 (0.06%)
.debug_ranges   33411008 (2.16%)
.debug_abbrev    4227906 (0.27%)
.debug_line    117115289 (7.58%)
.debug_str     182932094 (11.84%)

 Total Size: 1238912403  (80.21%)
 Total File Size: 1544622040
----------------------------------------------------

After this patch:

llvm-dwarfdump --show-section-sizes bin/clang
----------------------------------------------------
file: bin/clang
----------------------------------------------------
SECTION        SIZE (b)
-------------  ---------
.debug_info    902111430 (57.62%)
.debug_loc      19768599 (1.26%)
.debug_ranges   33412752 (2.13%)
.debug_abbrev    4305427 (0.28%)
.debug_line    117266710 (7.49%)
.debug_str     182933982 (11.69%)

 Total Size: 1259798900  (80.47%)
 Total File Size: 1565509704
----------------------------------------------------

Diff Detail

Event Timeline

SouraVX created this revision.Jul 31 2020, 3:43 AM
Herald added a project: Restricted Project. · View Herald Transcript
SouraVX requested review of this revision.Jul 31 2020, 3:43 AM

Please ignore these test cases dwarfdump-dataLocationVar.ll and dwarfdump-dataLocationExp.ll. These were generated with flang at the time when it was having bug of emitting dummy LexicalBlock. I've an approved PR in flang for this, once merged I'll correct these test cases(separately).

Orlando added inline comments.Jul 31 2020, 4:04 AM
llvm/test/DebugInfo/X86/DW_AT_start_scope.ll
6

Hi, I was just skimming the patch and noticed this.

CHECK; -> CHECK:

Also, since this test is only checking for a single variable, you could use llvm-dwarfdump -name Local - here and possibly drop this CHECK line?

SouraVX added inline comments.Jul 31 2020, 6:26 AM
llvm/test/DebugInfo/X86/DW_AT_start_scope.ll
6

Thanks for pointing this out! I'll fix this up when I revise.
BTW, please share feedback on this(overall) patch :)

https://github.com/google/bloaty might be a clearer way to compare the section sizes before/after. Though also comparing variable statistics (llvm-dwarfdump --statistics) might be helpful (though I'm guessing --statistics doesn't understand start-scope yet).

I assume in the previous discussion we discussed the complexities of what start_scope means for a variable without a contiguous address range? What was the answer to that? (if a variable/its enclosing scope has multiple discontiguous address ranges, what's the meaning of the start scope? Which ranges are/aren't in scope?)

I'll probably punt most of this review to @aprantl & co who have more context for variable location tracking.

From the [0], I've understood that this should affect debug build only (-g + -O0) ?

[0] http://lists.llvm.org/pipermail/llvm-dev/2020-April/140962.html

llvm/test/DebugInfo/X86/DW_AT_start_scope.ll
10

This should also be ';;'.

67

Usually we don't need these attributes.

101

Probably you can attach all these to the same di-location (with the same scope).

According to the --show-section-sizes output, the .debug_loc has been increased?

According to the --show-section-sizes output, the .debug_loc has been increased?

Thanks for reviewing this! Yea that's one of the drawback(Added FIXME for the same). Since I've used dbg.value intrinsic for representing this variable(all reduced scope variables), this is resulting in a location list entry. We should avoid this somehow, this is because, In our local GDB implementation DW_AT_start_scope is the only thing necessary for capturing the scope info correctly.
BTW, as per @dblaikie comment, do llvm-dwarfdump needs to be aware of DW_AT_start_scope ? How does it calculate size ? Getting the raw section size(that should capture DW_AT_start_scope too) ? Or some other mehachnism ?

According to the --show-section-sizes output, the .debug_loc has been increased?

Thanks for reviewing this! Yea that's one of the drawback(Added FIXME for the same).

When you say "this causes loc list emission even at -O0" could you be more precise? Does the debug_info actually use a location list somewhere? If so, where, if not - does debug_loc contain an unused location list? Does it contain no location lists, but gets emitted (though empty) when it'd otherwise not be emitted at all?

Since I've used dbg.value intrinsic for representing this variable(all reduced scope variables), this is resulting in a location list entry. We should avoid this somehow, this is because, In our local GDB implementation DW_AT_start_scope is the only thing necessary for capturing the scope info correctly.
BTW, as per @dblaikie comment, do llvm-dwarfdump needs to be aware of DW_AT_start_scope ?

My complaint wasn't so much about llvm-dwarfdump --show-section-sizes' output - it's fine for what it is, and it doesn't need to be aware of DW_AT_strat_scope - all it's doing is dumping the size of the sections, without any knowledge of what those sections contain (at least I hope that's how it's implemented - I usually use llvm-objdump -h | grep debug_ for that task, personally). I just meant something like 'bloaty' can do a clearer job of comparing two ELF files for relative sizes - whereas looking at the two dumps of show-section-sizes isn't as easy to visually compare the numbers.

According to the --show-section-sizes output, the .debug_loc has been increased?

Thanks for reviewing this! Yea that's one of the drawback(Added FIXME for the same).

When you say "this causes loc list emission even at -O0" could you be more precise? Does the debug_info actually use a location list somewhere? If so, where, if not - does debug_loc contain an unused location list? Does it contain no location lists, but gets emitted (though empty) when it'd otherwise not be emitted at all?

+1 for this question(s).

Since I've used dbg.value intrinsic for representing this variable(all reduced scope variables), this is resulting in a location list entry. We should avoid this somehow, this is because, In our local GDB implementation DW_AT_start_scope is the only thing necessary for capturing the scope info correctly.
BTW, as per @dblaikie comment, do llvm-dwarfdump needs to be aware of DW_AT_start_scope ?

My complaint wasn't so much about llvm-dwarfdump --show-section-sizes' output - it's fine for what it is, and it doesn't need to be aware of DW_AT_strat_scope - all it's doing is dumping the size of the sections, without any knowledge of what those sections contain (at least I hope that's how it's implemented - I usually use llvm-objdump -h | grep debug_ for that task, personally). I just meant something like 'bloaty' can do a clearer job of comparing two ELF files for relative sizes - whereas looking at the two dumps of show-section-sizes isn't as easy to visually compare the numbers.

Yes, the --show-section-sizes just looks for sections sizes (debug sections: .debug*, _debug*, _zdebug*, _gdb* etc.). There should be an option for comparing two object files, I think there is a TODO for that. :)

The DW_AT_start_scope might be interesting when calculating location coverage stats (and I guess that is what @dblaikie has thought) with llvm-dwarfdump --statistics or llvm-locstats (these create the location coverage buckets).

Sorry for the confusion/churn. This is the resultant DWARF(for the variable of interest) for this test case.

0x0000007a:       DW_TAG_variable
                    **DW_AT_location      (0x00000000:
                       [0x0000000000401178, 0x00000000004011c1): DW_OP_breg6 RBP-24)** // This is what causing increased in location list(even at "-O0 -g")
                    DW_AT_name  ("Local")
                    DW_AT_decl_file     ("/Scope.c")
                    DW_AT_decl_line     (7)
                    DW_AT_type  (0x0000008f "int")
                    DW_AT_start_scope   (0x00000017)

The DW_AT_start_scope might be interesting when calculating location coverage stats (and I guess that is what @dblaikie has thought) with llvm-dwarfdump --statistics or llvm-locstats (these create the location coverage buckets).

Right. More follow-up/after this work than anything.

Sorry for the confusion/churn. This is the resultant DWARF(for the variable of interest) for this test case.

0x0000007a:       DW_TAG_variable
                    **DW_AT_location      (0x00000000:
                       [0x0000000000401178, 0x00000000004011c1): DW_OP_breg6 RBP-24)** // This is what causing increased in location list(even at "-O0 -g")
                    DW_AT_name  ("Local")
                    DW_AT_decl_file     ("/Scope.c")
                    DW_AT_decl_line     (7)
                    DW_AT_type  (0x0000008f "int")
                    DW_AT_start_scope   (0x00000017)

How does the start scope compare to the location range of the variable, and the scope range of the enclosing scope?

In any case - this seems a bit backwards - one of the benefits of start_scope is that we should be able to use singular variable locations in more places because the scope will already be appropriately truncated by the start_scope value.

Also, general question: Why are you implementing this feature? Do you have a debugger that can/will consume this? I hesitate to add features (especially ones with a lot of surface area/additional complexity) that don't have a consumer/customer use case readily available.

Also - the spec describes start_scope as being able to contain a range list - for cases where the scope of the variable name is discontiguous. This seems relevant/useful/necessary as well (& would further help reduce the need for location lists on variables if the variable is live for its actual scope - though overall I've more concerns about the scope_start feature creating more expensive IR and DWARF than saving us anything, just that maybe the costs can be mitigated somewhat) & if the feature is important, maybe should be implemented more generally to handle discontiguous scopes.

How does the start scope compare to the location range of the variable, and the scope range of the enclosing scope?

DW_AT_start_scope value in contiguous scope case is offset from start of the Lex Block(i.e low_pc of enclosing scope).

In any case - this seems a bit backwards - one of the benefits of start_scope is that we should be able to use singular variable locations in more places because the scope will already be appropriately truncated by the start_scope value.

Indeed, that's the primary problem we want to overcome.

Also, general question: Why are you implementing this feature? Do you have a debugger that can/will consume this? I hesitate to add features (especially ones with a lot of surface area/additional complexity) that don't have a consumer/customer use case readily available.

The problem is real, reported here also https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93844, here pointer is also involved(try with clang). We have a downstream GDB that solves these sort of problems with the help of DW_AT_start_scope. Again up-streaming that is also chicken-egg problem, since no producer(that I'm aware) of produces DW_AT_start_scope. And yes (In our downstream GDB) location-list is not needed, start_scope is just enough to show the variable value correctly)

Also - the spec describes start_scope as being able to contain a range list - for cases where the scope of the variable name is discontiguous. This seems relevant/useful/necessary as well (& would further help reduce the need for location lists on variables if the variable is live for its actual scope - though overall I've more concerns about the scope_start feature creating more expensive IR and DWARF than saving us anything, just that maybe the costs can be mitigated somewhat) & if the feature is important, maybe should be implemented more generally to handle discontiguous scopes.

I started to handle/tackle contiguous scope case first and generalize subsequently. But overall it seems like debug.value based representation and capturing info has landed us here. Introducing a new intrinsic(dedicated) *MAY* help to solve the problem, but that will also cost a lot. I'm still looking/exploring the best way to do this.

djtodoro added inline comments.Aug 6 2020, 12:06 AM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

we can avoid creating .debug_loc list by introducing new DbgEntity, e.g. called DbgScopedVariable which is handled in a special way (by always creating a single location for it)? Or to set isSafeForSingleLocation (from DwarfDebug::buildLocationList) to true if ScopeBeginSym != nullptr?

SouraVX added inline comments.Aug 6 2020, 9:18 AM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

Thank you for your inputs!, I think I've been to this route(or not). Let me give it one more shot.

dblaikie added inline comments.Aug 6 2020, 3:21 PM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

Not sure introducing a new kind of DbgEntity is going to be the right general solution - I'd think the right general solution could only be achieved by introducing a new kind of scope - and having every variable in its own DIVariableScope (like DILexicalScope/DiLexicalScopeFile) - with the slight problem that DIVariableScopes can overlap... which would be problematic, since then a location could be in more than one scope.

I guess they could all nest, because C++/scoped languages do create explicitly nested scopes, almost... I think there are some quirks around variables declared in conditionals, but might work out OK?

djtodoro added inline comments.Aug 7 2020, 1:34 AM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

I agree that this would address this issue locally and may be just a mask/workaround for this implementation.
That is neat idea, and I think we should try to generalize it that way. A "disadvantage" may be the fact we'd need to carry that metadata throughout pipeline (IR && MIR).

dblaikie added inline comments.Aug 7 2020, 12:10 PM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

I agree that this would address this issue locally and may be just a mask/workaround for this implementation.

The first "this" here is referring to your DbgEntity suggestion?

That is neat idea, and I think we should try to generalize it that way. A "disadvantage" may be the fact we'd need to carry that metadata throughout pipeline (IR && MIR).

Yeah, I don't think it's cheap (Either to implement, or in terms of metadata overhead) to implement - which is one of the reasons I'm fairly hesitant about the feature in general. But I do think if it's going to be implemented, I'm not sure there's really an "incremental" approach that improves the fidelity of scope_start in steps starting from this patch - I think we do have to look at what it'd take to have fairly full fidelity - and I know of no other way than having DI scopes for each variable - and I think even then I'm worried that there are some cases where they're not strictly nested inside each other, which would be extra difficult to represent.

djtodoro added inline comments.Aug 7 2020, 12:23 PM
llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
1817–1818

I agree that this would address this issue locally and may be just a mask/workaround for this implementation.

The first "this" here is referring to your DbgEntity suggestion?

Yes, sorry for the confusion.

That is neat idea, and I think we should try to generalize it that way. A "disadvantage" may be the fact we'd need to carry that metadata throughout pipeline (IR && MIR).

Yeah, I don't think it's cheap (Either to implement, or in terms of metadata overhead) to implement - which is one of the reasons I'm fairly hesitant about the feature in general. But I do think if it's going to be implemented, I'm not sure there's really an "incremental" approach that improves the fidelity of scope_start in steps starting from this patch - I think we do have to look at what it'd take to have fairly full fidelity - and I know of no other way than having DI scopes for each variable - and I think even then I'm worried that there are some cases where they're not strictly nested inside each other, which would be extra difficult to represent.

SouraVX abandoned this revision.Mar 3 2021, 8:12 PM

code is pretty stale here, primary problem still persists. Abandoning for now.