Page MenuHomePhabricator

[Driver] Add DWARF64 flag: -gdwarf64
Needs ReviewPublic

Authored by ayermolo on Oct 30 2020, 3:26 PM.

Details

Reviewers
llvm-commits
Summary

@ikudrin enabled support for dwarf64 in D87011. Adding a clang flag so it can be used through that compilation pass.

Diff Detail

Unit TestsFailed

TimeTest
390 mslinux > HWAddressSanitizer-x86_64.TestCases::sizes.cpp
Script: -- : 'RUN: at line 3'; /mnt/disks/ssd0/agent/llvm-project/build/./bin/clang --driver-mode=g++ -m64 -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -mcmodel=large -mllvm -hwasan-globals -mllvm -hwasan-use-short-granules -mllvm -hwasan-instrument-landing-pads=0 -mllvm -hwasan-instrument-personality-functions /mnt/disks/ssd0/agent/llvm-project/compiler-rt/test/hwasan/TestCases/sizes.cpp -nostdlib++ -lstdc++ -o /mnt/disks/ssd0/agent/llvm-project/build/projects/compiler-rt/test/hwasan/X86_64/TestCases/Output/sizes.cpp.tmp

Event Timeline

ayermolo created this revision.Oct 30 2020, 3:26 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 30 2020, 3:26 PM
ayermolo requested review of this revision.Oct 30 2020, 3:26 PM
ayermolo updated this revision to Diff 302326.Nov 2 2020, 9:57 AM
ayermolo updated this revision to Diff 303209.Thu, Nov 5, 11:44 AM
ayermolo updated this revision to Diff 303303.Thu, Nov 5, 5:21 PM
  1. The patch needs tests to check the added functionality.
  2. DWARF64 can be generated only for a limited number of targets. There should be diagnostics for invalid switch combinations to prevent misuse. @MaskRay mentioned that in the patch for llc, D87011#2254749, but that makes a lot more sense for user-level tools.
ayermolo updated this revision to Diff 304000.Mon, Nov 9, 3:27 PM
  1. The patch needs tests to check the added functionality.
  2. DWARF64 can be generated only for a limited number of targets. There should be diagnostics for invalid switch combinations to prevent misuse. @MaskRay mentioned that in the patch for llc, D87011#2254749, but that makes a lot more sense for user-level tools.

Thanks, added some checks and clang tests.

The failure seems unrelated to my changes. Trying locally it fails with latest upstream even without my changes.

@jansvoboda11 Can you take a look? The failing test is unrelated it seems. Fails locally even without my change.

Hi @ayermolo, do you think this might be triggered by D82756? (my only upstream patch ATM)

clang/test/Driver/debug-options.c
377–390

Probably extra character.

This change covers non-LTO cases. For LTO, I think we would need to pass it from driver to LTO. Something like this: tools::addLTOOptions -> lld -> lto::Config (Config->TargetOptions->MCTargetOptions) ->LTO Backend.

Hi @ayermolo, do you think this might be triggered by D82756? (my only upstream patch ATM)

Oh, no. Sorry I wasn't clear. I was just looking for people to review this patch, and saw you recently reviewed another patch.

This change covers non-LTO cases. For LTO, I think we would need to pass it from driver to LTO. Something like this: tools::addLTOOptions -> lld -> lto::Config (Config->TargetOptions->MCTargetOptions) ->LTO Backend.

Good point. Maybe part of a different diff, since lld needs to be modified also.

dblaikie added inline comments.
clang/include/clang/Basic/CodeGenOptions.def
35

Is there any precedent to draw from for this flag name? (Does GCC support DWARF64? Does it support it under this flag name or some other? (similarly with other gcc-like compilers (Intel's? Whoever else... )))

It looks like lld/test/COFF/lto-new-pass-manager.ll.obj was added to the patch by accident and should be removed.

clang/include/clang/Basic/CodeGenOptions.def
35

It looks like we are pioneering in that area. To me, the proposed name looks consonant with other debug-related switches.

clang/lib/Driver/ToolChains/Clang.cpp
4028

We also should check that the output format is ELF.

It looks like lld/test/COFF/lto-new-pass-manager.ll.obj was added to the patch by accident and should be removed.

It's weird. The file gets automatically added whenever I commit the files I have modified locally.

ayermolo updated this revision to Diff 306186.Wed, Nov 18, 12:32 PM
ayermolo updated this revision to Diff 306187.Wed, Nov 18, 12:35 PM
ayermolo marked 2 inline comments as done.Wed, Nov 18, 12:38 PM
ayermolo added inline comments.
clang/include/clang/Basic/CodeGenOptions.def
35

I didn't see any dwarf64 flags in gcc:
https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html

I tried to follow clang convention for other dwarf flags.

MaskRay added inline comments.Wed, Nov 18, 1:43 PM
clang/lib/Driver/ToolChains/Clang.cpp
4018

if (const Arg *A = Args.getLastArg(options::OPT_gdwarf64))

4020

The variable is only used once. Inline it

Adding DWARF64 clang flag

It is important to mention the exact option name: Add -gdwarf-64 to enable 64-bit DWARF format

ayermolo updated this revision to Diff 306223.Wed, Nov 18, 2:14 PM
ayermolo retitled this revision from Adding DWARF64 clang flag to Adding DWARF64 clang flag: -gdwarf64.
ayermolo added inline comments.
clang/lib/Driver/ToolChains/Clang.cpp
4020

Both A, and RawTriple are used more than once.

dblaikie added inline comments.Wed, Nov 18, 2:30 PM
clang/include/clang/Driver/Options.td
2145–2146

Does this actually enable debug info emission (the -gdwarf-N versions do cause debug info to be emitted, and set the dwarf version - but, say -ggnu-pubnames does not cause debug info to be emitted if it isn't otherwise requested). Experience has shown it's probably better to have flags like this not enable debug info emission - so they can be used orthogonally (eg: if a project is large, it can add -gdwarf64 to its flags permanently, even though maybe only some build modes, etc, enable debug info (with -g))

If this doesn't enable debug info emission, the phrasing of the help text might be worth changing somewhat (I wonder how we document -ggnu-pubnames, for instance/comparison/inspiration)

clang/lib/Driver/ToolChains/Clang.cpp
4022–4023

perhaps we could phrase this more generally (so it's true even when DWARFv6 comes along - and doesn't get unwieldy listing all the DWARF versions).

"DWARFv3 or later" ?

4028

Is DWARF64 not supported on MachO, for instance? (I'd have DWARF64 would be pretty usable on any platform that supported DWARF in general - certainly some system tools that are DWARF aware (including actual debuggers) might not be up to the task of using it - but if we can correctly generate it in the object files, it seems OK to accept the request (useful for folks testing out/implementing DWARF64 consumers on those platforms))

clang/test/Driver/debug-options.c
390

Typo in "architecture"

ayermolo added inline comments.Wed, Nov 18, 4:40 PM
clang/include/clang/Driver/Options.td
2145–2146

This enables DWARF64 if emission is enabled.

From DebugInfo it seems like it's only Elf Format:

bool Dwarf64 = Asm->TM.Options.MCOptions.Dwarf64 &&

DwarfVersion >= 3 &&   // DWARF64 was introduced in DWARFv3.
TT.isArch64Bit() &&    // DWARF64 requires 64-bit relocations.
TT.isOSBinFormatELF(); // Support only ELF for now.
ayermolo added inline comments.Wed, Nov 18, 4:49 PM
clang/include/clang/Driver/Options.td
2145–2146

This enables DWARF64 if emission is enabled.

clang/lib/Driver/ToolChains/Clang.cpp
4028

From DebugInfo it seems like it's only Elf Format:

bool Dwarf64 = Asm->TM.Options.MCOptions.Dwarf64 &&
                 DwarfVersion >= 3 &&   // DWARF64 was introduced in DWARFv3.
                 TT.isArch64Bit() &&    // DWARF64 requires 64-bit relocations.
                TT.isOSBinFormatELF(); // Support only ELF for now.
dblaikie added inline comments.Wed, Nov 18, 4:51 PM
clang/include/clang/Driver/Options.td
2145–2146

This enables DWARF64 if emission is enabled.

Then perhaps the Help Text phrasing should distinguish this behavior from the behavior of -gdwarf-N which not only sets the version, but enables emission. (perhaps the phrasing of flags that don't enable emission, like "-ggnu-pubnames" could provide inspiration for phrasing the help text for this new flag)

From DebugInfo it seems like it's only Elf Format:

I'm curious as to why it's ELF only, though - could you explain what the motivation is there?

(random aside: I still worry about this precedent for -g* flags that sometimes enable debug info and set a feature, and sometimes they only set the feature and don't enable debug info - it's a bit of a confusing mess and I personally rather the -fdebug* flags set a feature but don't enable emission and -g* flags enable emission, but there's a lack of clear agreement and there's examples on both sides, so not sure there's much to be done about that here)

ayermolo updated this revision to Diff 306263.Wed, Nov 18, 5:00 PM
ikudrin added inline comments.Wed, Nov 18, 9:46 PM
clang/include/clang/Driver/Options.td
2145–2146

From DebugInfo it seems like it's only Elf Format:

I'm curious as to why it's ELF only, though - could you explain what the motivation is there?

Our customers need DWARF64 for ELF and I do not have much experience with other platforms, so I cannot provide adequate support there. Some platform-specific tools, like dsymutil, have to be fixed to work with DWARF64, but I am personally a bit reluctant to do that because I cannot check the result in practice. If there will be developers who are willing to support DWARF64 on these platforms, they will be able to unlock it and use all the experience we collect with ELF, but for now, I prefer to focus on one particular platform with practical applications.

ayermolo updated this revision to Diff 306477.Thu, Nov 19, 10:48 AM
ayermolo marked 3 inline comments as done.
dblaikie added inline comments.Thu, Nov 19, 10:49 AM
clang/lib/Driver/ToolChains/Clang.cpp
4023
clang/test/Driver/debug-options.c
389
ayermolo updated this revision to Diff 306482.Thu, Nov 19, 11:07 AM
ayermolo marked an inline comment as done.
dblaikie added inline comments.Thu, Nov 19, 1:15 PM
clang/include/clang/Basic/CodeGenOptions.def
35

Huh - tried making really big binaries or anything (or checking the GCC source) to see if it does it implicitly under some conditions?
Hmm - looks like this maybe came up at the Linux Plumbers Conference & the suggested flag was -fdwarf64/32: https://linuxplumbersconf.org/event/7/contributions/746/attachments/578/1018/DWARF5-64.pdf (this avoids the "does g imply debug info" and avoids the subtle distinction between "-gdwarf64 and -gdwarf-N" the presence of the '-' changing the meaning of the number quite significantly). Though hardly authoritative
https://linuxplumbersconf.org/event/7/sessions/90/attachments/583/1201/dwarf-bof-notes-aug24-lpc-2020.txt - seems some other options were (are?) under consideration too. Might be worth touching base with the folks involved in those discussions to see where they're at with regard to naming/support?

(they also touch on the "all units must agree" issue - so not sure if the same folks involved in those discussions have also been included in the discussions around debug info 32/64 sorting as another approach that may avoid the "all units must agree" constraint (I assume that's the reason they had that constraint))

MaskRay retitled this revision from Adding DWARF64 clang flag: -gdwarf64 to [Driver] Add DWARF64 flag: -gdwarf64.Thu, Nov 19, 2:05 PM
ayermolo added inline comments.Thu, Nov 19, 3:23 PM
clang/include/clang/Basic/CodeGenOptions.def
35

In the DWARFV5-64 pdf it says 64 bit support has no patches and is after DWARF5. Although it's not clear if they are talking about DWARF64 support for V5 or in general.

I have not hacked our build system to use gcc for builds that can overflow debug_info. I scanned through gcc code and was only able to find references to dwarf 64 in go library, and in dwarf2out.c. In latter it relies on DWARF_OFFSET_SIZE macro.

I don't quite understand the "all [CU] units must agree" part either. From DWARF perspective we are free to match on CU level DWARF32/64, and consumer are free not to do anything beyond that. So if overflow occurs, will so be it. What we are trying to do in linker with sorting is being "nice" to the users, and kind of going beyond what spec requires.

Sounds like no conclusion was reached on their side, but only one of them -gdwarf64 follows naming convention of other debug flags.

  • -fdwarf64/-fdwarf32
    • or -gdwarf32 or -gdwarf64
    • or -gdbdwarf=32/64
dblaikie added inline comments.Thu, Nov 19, 4:22 PM
clang/include/clang/Basic/CodeGenOptions.def
35

only one of them -gdwarf64 follows naming convention of other debug flags.

There are many debug flags that don't use the '-g' prefix. (-fdebug-types-section comes to mind, but I think - this was discussed in depth earlier this year with regards to the -gsplit-dwarf flag, for instance: https://www.mail-archive.com/gcc@gcc.gnu.org/msg92495.html - though at least the DWARF64 flag doesn't have the legacy that -gsplit-dwarf has that complicates things further there)

ayermolo added inline comments.Thu, Nov 19, 5:22 PM
clang/include/clang/Basic/CodeGenOptions.def
35

Ah, thanks for the context. My takeaway it's a mess. :)
Personally I find it more confusing that there are debug options that start with -f and -g, rather then that some -g enable debug output. When I look at documentation I just want to have see all the related options grouped in one area/one prefix, but that's just how my brain works.
That being said I don't have particular strong opinion about naming convention of this flag. Judging from that conversation, maybe there is some preference for -f, but mainly it was a big push against changing an option after it was introduced and proliferated.

dblaikie added inline comments.Thu, Nov 19, 6:38 PM
clang/include/clang/Basic/CodeGenOptions.def
35

Yep, bit of a mess - hence the concern about making it messier/trying to drive in that general direction.

Could you reach out to the gcc mailing list, link the thread I linked, CC myself (& anyone else from this review who seems interested) and any of the folks you might be able to identify from the Plumbers conference or other context that seems to have an interest in this & ask what they think the flag should be?

I ask because so far as I can tell from prior experience, GCC is less likely to adopt Clang's convention out of compatibility, so it's more on us to try to pick something with some buy-in from their side of things lest we end up with divergent flags or semantics.

ayermolo added inline comments.Fri, Nov 20, 12:24 PM
clang/include/clang/Basic/CodeGenOptions.def
35

Made a post: https://gcc.gnu.org/pipermail/gcc/2020-November/234290.html
CCed people in this review + Mark Wielaard.