This is an archive of the discontinued LLVM Phabricator instance.

DebugInfo: Add metadata support for disabling DWARF pub sections
ClosedPublic

Authored by dblaikie on Aug 2 2018, 7:18 PM.

Details

Summary

In cases where the debugger load time is a worthwhile tradeoff (or less
costly - such as loading from a DWP instead of a variety of DWOs
(possibly over a high-latency/distributed filesystem)) against object
file size, it can be reasonable to disable pubnames and corresponding
gdb-index creation in the linker.

A backend-flag version of this was implemented for NVPTX in
D44385/r327994 - which was fine for NVPTX which wouldn't mix-and-match
CUs. Now that it's going to be a user-facing option (likely powered by
"-gno-pubnames", the same as GCC) it should be encoded in the
DICompileUnit so it can vary per-CU.

After this, likely the NVPTX support should be migrated to the metadata
& the previous flag implementation should be removed.

(I'm open to ideas on the naming here - whether the field should be called
"pubnames" or "pubnameKind" (to match "emissionKind") & the names of the
parameters (though currently it's a normal enum not an enum class, so the names
are uncscoped within DICompileUnit - so obvious names like "None" cause naming
collissions - hence the slightly awkward "Omitted", which, admittedly, still
isn't great))

Diff Detail

Repository
rL LLVM

Event Timeline

dblaikie created this revision.Aug 2 2018, 7:18 PM

Regarding the naming: I like PubnameKind because of the aforementioned consistency (we also have AccelTableKind. Can't we use an enum class to prevent collisions?

Regarding the naming: I like PubnameKind because of the aforementioned consistency (we also have AccelTableKind. Can't we use an enum class to prevent collisions?

I submitted this before looking at the diff, I guess the reason is that you want to convert to unsigned.

include/llvm/IR/DebugInfoMetadata.h
1172 ↗(On Diff #158889)

Maybe NoPubnames like NoDebug?

How does this interact with v5 .debug_names?

Regarding the naming: I like PubnameKind because of the aforementioned consistency (we also have AccelTableKind).

Fair enough - happy to go with that. (though AccelTableKind isn't as interesting an example - since it doesn't appear in the IR, unlike pubnames and emissionKind - which is really where I'm more interested in the consistency of the name (I don't so much mind if the type itself is called PubnamesKind, but about whether the IR field should be called "pubnames: " or "pubnamesKind: ", etc (currently it's a boolean, "gnuPubnames: true" for example)))

Can't we use an enum class to prevent collisions?

Yep, though that'd be a bit inconsistent with the other enums that don't use that. But I'm OK with that.

Actually - potentially we could merge the accelerator table options into this too, to promote them to an IR feature as well. But it looks like they aren't really supported on a per-CU basis, but are across the whole compilation only. (read the spec of debug_names - I see, best to emit the widest entry possible (ie: bad idea to emit one table per CU if we're emitting multiple CUs in a single object file (LTO) - and no need not to emit it if it's been requested, I suppose - though I guess a user might reasonably request no accelerator table for /some/ CUs they expect not to debug much/are OK with the debugger indexing slowly later on if they do)). But Apple's names don't support multiple CUs at all, so they can't be unified there anyway.

OK, so maybe renaming this to something generic like "nameTable" {on, off, default} on a per-CU basis. If accelerator tables are in-use, then filter the names that go into it on the basis of this property (well, maybe Apple doesn't want to support that option with Apple tables? There doesn't seem to be any notion that the Apple accelerator table could be incomplete - CUs don't have a flag saying "I'm covered by an accelerator table" and the tables don't have an entry saying "these CUs are covered here").

@aprantl - reckon it'd be too much of a foot-gun to have a -gno-pubnames (or perhaps we could use a more general name in addition to this) that'd probably produce a rather bad (names not found) experience when combined with Apple accelerator tables.

How does this interact with v5 .debug_names?

Currently has no effect, but see above ^ - I reckon it probably should. Maybe we can generalize the name of the property? "nameTable: " {None, Default (v4 or v5 based on DWARF version), GNU (only makes sense for v4 though - ignored if you end up building v5? seems reasonable to me)}

labath added a subscriber: labath.Aug 8 2018, 1:49 AM

Actually - potentially we could merge the accelerator table options into this too, to promote them to an IR feature as well. But it looks like they aren't really supported on a per-CU basis, but are across the whole compilation only. (read the spec of debug_names - I see, best to emit the widest entry possible (ie: bad idea to emit one table per CU if we're emitting multiple CUs in a single object file (LTO) - and no need not to emit it if it's been requested, I suppose - though I guess a user might reasonably request no accelerator table for /some/ CUs they expect not to debug much/are OK with the debugger indexing slowly later on if they do)). But Apple's names don't support multiple CUs at all, so they can't be unified there anyway.

I got lost a bit when trying to parse this, but I just wanted to confirm that debug_names tables are capable indexing only a subset of CUs in the LTO case (multiple CUs in one object file), and even without emitting one table per each indexed CU -- you would just list the CUs that you index in the single uber-table. (Not that I have an idea why would anyone want to do that). You are correct in saying that apple tables do not allow this.

dblaikie updated this revision to Diff 160165.Aug 10 2018, 12:06 PM
  • Rename pubnames: field to nameTableKind:
  • Support disabling debug_names with the nameTable property of DICompileUnit

Actually - potentially we could merge the accelerator table options into this too, to promote them to an IR feature as well. But it looks like they aren't really supported on a per-CU basis, but are across the whole compilation only. (read the spec of debug_names - I see, best to emit the widest entry possible (ie: bad idea to emit one table per CU if we're emitting multiple CUs in a single object file (LTO) - and no need not to emit it if it's been requested, I suppose - though I guess a user might reasonably request no accelerator table for /some/ CUs they expect not to debug much/are OK with the debugger indexing slowly later on if they do)). But Apple's names don't support multiple CUs at all, so they can't be unified there anyway.

I got lost a bit when trying to parse this, but I just wanted to confirm that debug_names tables are capable indexing only a subset of CUs in the LTO case (multiple CUs in one object file), and even without emitting one table per each indexed CU -- you would just list the CUs that you index in the single uber-table. (Not that I have an idea why would anyone want to do that). You are correct in saying that apple tables do not allow this.

Yeah, it was a bit rambly - thanks for that. I've implemented it so that 'nameTable: None' disables debug_names for that CU (but doesn't modify the behavior for the Apple names tables).

How does this interact with v5 .debug_names?

Originally the patch had no effect on debug_names - I've now renamed the property to "nameTable" and "None" causes debug_names not to be emitted for that CU (test case shows that this works correctly for LTO situations where some CUs disable nameTable and some do not - or if they all disable nameTable then no debug_names section is produced at all).

(I'm going to write a follow-up patch to disable pubnames in DWARFv5 & clang patches with some flags to enable/disable debug_names/pubnames/gnu_pubnames from the clang driver)

Regarding the naming: I like PubnameKind because of the aforementioned consistency (we also have AccelTableKind).

How's the change now - using "NameTableKind" since this now applies to debug_names as well.

Can't we use an enum class to prevent collisions?

It's doable - just needed a few casts in places & fully qualify things in a few other places - so I've used "None, Default, GNU" as the enumerator names now.

aprantl added inline comments.Aug 15 2018, 10:06 AM
lib/Bitcode/Reader/MetadataLoader.cpp
1393 ↗(On Diff #160165)

Can you add a bitcode upgrade test?

Currently has no effect, but see above ^ - I reckon it probably should. Maybe we can generalize the name of the property? "nameTable: " {None, Default (v4 or v5 based on DWARF version), GNU (only makes sense for v4 though - ignored if you end up building v5? seems reasonable to me)}

Is there a test covering the DWARF5 + GNU CU flag combination?

Currently has no effect, but see above ^ - I reckon it probably should. Maybe we can generalize the name of the property? "nameTable: " {None, Default (v4 or v5 based on DWARF version), GNU (only makes sense for v4 though - ignored if you end up building v5? seems reasonable to me)}

Is there a test covering the DWARF5 + GNU CU flag combination?

Nah - for now they are orthogonal (pubnames (GNU or otherwise) are emitted wherever they're emitted, regardless of whether DWARF5 is used) so a test doesn't seem appropriate.

But, yeah, I think in a follow-up patch making DWARF5 disable any form of pubnames would be appropriate (given the amount of work needed for a consumer to support DWARF5, it doesn't seem unreasonable to say "hey, also you've got to support the new name table if you want a name table" (if the hash table is too complex for the consumer - they can ignore that part and use the rest of teh name table still as something like pubnames - a flat list - if I recall the format))

lib/Bitcode/Reader/MetadataLoader.cpp
1393 ↗(On Diff #160165)

I could - but I'm not sure it would test anything. The format has in some sense remained the same - this field is still zero/one before/after the change, but now new values are also valid.

Reckon it's still worth testing? What do you reckon that testing would cover?

Can you add a bitcode upgrade test?

I could - but I'm not sure it would test anything. The format has in some sense remained the same - this field is still zero/one before/after the change, but now new values are also valid.
Reckon it's still worth testing? What do you reckon that testing would cover?

It would be sufficient to find an existing .bc file with a DICompileUnit in it and add a CHECK: that tests that the new flag comes out as default(=0) (which probably means it won't be printed at all, but that's fine).

Can you add a bitcode upgrade test?

I could - but I'm not sure it would test anything. The format has in some sense remained the same - this field is still zero/one before/after the change, but now new values are also valid.
Reckon it's still worth testing? What do you reckon that testing would cover?

It would be sufficient to find an existing .bc file with a DICompileUnit in it and add a CHECK: that tests that the new flag comes out as default(=0) (which probably means it won't be printed at all, but that's fine).

Still not sure I follow - the flag isn't new in the bitcode format. In the old format it could have the values {0, 1} and in the new format it can have the values {0, 1, 2}.

So an old .ll file with "gnuPubnames: false" or "gnuPubnames: true" against an old llc would produce the same bitcode as a new .ll with "nameTable: Default" and "nameTable: GNU". So checking in a bitcode file doesn't seem like it would test anything new/different, to me?

Sorry, my mistake. I saw that you added a new field in the IR assembler and sloppily assumed that there is also a new bitcode record for it, but there isn't, so the whole point is moot.

Sorry, my mistake. I saw that you added a new field in the IR assembler and sloppily assumed that there is also a new bitcode record for it, but there isn't, so the whole point is moot.

Oh, no worries - yeah, it is certainly a bit subtle. Appreciate you taking the time/care to be sure.

Patch seem good all-up now?

(one or two patches to follow - flags in Clang to enable/disable pubnames, along with default behavior to disable pubnames in DWARF5)

Looks good from my point of view.

This revision was not accepted when it landed; it landed in state Needs Review.Aug 16 2018, 2:30 PM
This revision was automatically updated to reflect the committed changes.
labath added inline comments.Aug 24 2018, 6:20 AM
llvm/trunk/lib/CodeGen/AsmPrinter/AccelTable.cpp
558–564

I am sorry for not noticing this sooner, but I have a feeling this will break the CU references in the debug_names section in case of CUs with mixed pubnames flags (which I guess only could happen during LTO).

This code was (perhaps lazily) assuming that compile unit "unique IDs" form a continuous sequence from 0 to CUs.size()-1. This is relied upon by the lambda below which simply calls getUniqueID to get the value of the DW_IDX_compile_unit attribute. Now that the unique IDs and the CU lists in the name index don't match, I guess we'll have to create a separate lookup table for that.

(The assert here was meant to catch the case where the UniqueIDs stop being continuous, but I did not anticipate that we would start filtering them ourselves.)

578

This lambda here.