This is an archive of the discontinued LLVM Phabricator instance.

[DebugInfo] Add the -dwarf64 switch to llc and other internal tools (4/19).
ClosedPublic

Authored by ikudrin on Sep 2 2020, 6:22 AM.

Details

Summary

The patch adds a switch to enable emitting debug info in the 64-bit DWARF format. Most emitter for sections will be updated in the subsequent patches, whereas for .debug_line and .debug_frame the emitters are in the MC library, which is already updated.

For now, the switch is enabled only for 64-bit ELF targets.

Diff Detail

Event Timeline

ikudrin created this revision.Sep 2 2020, 6:22 AM
ikudrin requested review of this revision.Sep 2 2020, 6:22 AM
dblaikie accepted this revision.Sep 2 2020, 10:11 AM

Looks good - might use a bit more testing if some of the prior unit-tested patches end up rolled into this one (if this is the first test that makes the codepaths in those patches "live"/testable)

Worth testing that the length field or other parts of the line table were written correctly per the format? Or is that already/going to be tested elsewhere? (if this is the first patch that makes debug_line able to be written in DWARF64, then I'd expect it to be testing any parts of the line table that are different/noteworthy in DWARF64 - possibly using assembly testing, but if llvm-dwarfdump's DWARF64 parsing support is well tested independently, then I guess it's good to just use it here too, like for any other tests)

This revision is now accepted and ready to land.Sep 2 2020, 10:11 AM

Worth testing that the length field or other parts of the line table were written correctly per the format? Or is that already/going to be tested elsewhere? (if this is the first patch that makes debug_line able to be written in DWARF64, then I'd expect it to be testing any parts of the line table that are different/noteworthy in DWARF64 - possibly using assembly testing, but if llvm-dwarfdump's DWARF64 parsing support is well tested independently, then I guess it's good to just use it here too, like for any other tests)

The code that emits these tables is already tested with llvm/test/MC/ELF/gen-dwarf64.s. llvm-dwarfdump was updated in advance and ready to parse DWARF64 debug info, so we can trust it in the tests and do not need to overcomplicate them.

ikudrin edited the summary of this revision. (Show Details)Sep 2 2020, 11:18 PM
MaskRay added a subscriber: MaskRay.Sep 3 2020, 8:53 AM
MaskRay added inline comments.
llvm/test/DebugInfo/X86/dwarf64-support.ll
7

If -dwarf-version=2 -dwarf64 does not make sense, shouldn't the combo be errored to prevent misuse?

ikudrin added inline comments.Sep 3 2020, 9:20 AM
llvm/test/DebugInfo/X86/dwarf64-support.ll
7

I am not sure where to add that check and reporting. It looks like for internal tools erroneous combinations are just ignored. For example, for NVPTX, setting the DWARF version is silently ignored (see lines 372-374 in DwarfDebug.cpp). Thus, my change just follows the crowd.

MaskRay added inline comments.Sep 3 2020, 11:39 AM
llvm/test/DebugInfo/X86/dwarf64-support.ll
7

I guess NVPTX does so for quick MVP prototype ("[DEBUG] Initial adaptation of NVPTX target for debug info emission."). Downgrading DWARF version this way allows them to use -gdwarf-* quickly with their auxiliary target triples ("-triple" "x86_64-unknown-linux-gnu" "-aux-triple" "nvptx64-nvidia-cuda")

For clang -gdwarf-4, I think it probably does not hurt when an auxiliary target triple does support DWARF v4 and downgrades to v2. For the testing tool llc, we probably should emit better diagnostic to remind the user.

dblaikie added inline comments.Sep 3 2020, 12:00 PM
llvm/test/DebugInfo/X86/dwarf64-support.ll
7

Eh, there are a fair few flags I think at the llc level we don't bother to provide error messages about. One that comes to mind would be type units - the flag silently does nothing on non-ELF targets, for instance. Enabling debug-macro does nothing when using v4 Split DWARF.

Might be nice, but I wouldn't say it's necessary - llc's a tool for us to test LLVM with & some flags have no effect in certain circumstances. (heck, even at the clang driver level that's true - lots of flags are just no-ops in certain situations, rather than providing error messages about their incompatibility)

This revision was landed with ongoing or failed builds.Sep 14 2020, 10:24 PM
This revision was automatically updated to reflect the committed changes.

@ikudrin To clarify this will emit R_X86_64_64 bit relocations for .debug_info on 64 bit platform, correct?

@ikudrin To clarify this will emit R_X86_64_64 bit relocations for .debug_info on 64 bit platform, correct?

@ayermolo The patch series added support for the 64-bit DWARF format. You are right that many relocations in .debug_* will change from R_X86_64_32 to R_X86_64_64 with the option.

@ikudrin To clarify this will emit R_X86_64_64 bit relocations for .debug_info on 64 bit platform, correct?

Right, but only if producing 64-bit DWARF info is directly requested. Without specifying -dwarf64, nothing is changed, and the debugging info will be generated in the DWARF32 format.

Awesome, thanks.
I was trying to pass in -dwarf64 through our build system, but was still seeing 32 bit relocations. Will dig further on my end.
Thanks for working on this.

The switch is implemented only internally in LLVM. There is still some work to be done to enable producing 64-bit debugging info in clang, but I strayed a bit for another task. Hope to come back later this year.

ayermolo added a comment.EditedNov 4 2020, 10:16 AM

@ikudrin What else is left on clang side? I added a diff for passing a flag from clang to be, need to see what's up with failures, anything else that needs to be done?
Testing it locally I was able to generate binary with DWARF64, that llvm-dwarf is able to parse it. Looks like gdb and lldb support varies.

To generate 64-bit debugging info, there should be enough to pass the switch through CLANG, right. Apart from that, we will probably need some compatibility checks so that using the switch in unsupported cases prints out diagnostics. There are also some improvements on the LLD side which are better to be done to support extremely large debugging information. Not all our tools fully support DWARF64 yet, etc.

@ikudrin Can you elaborate on LLD changes. I recently started to look in to it. Reason I am interested in this, internally we are a looking in to using DWARF64, so I have bandwidth, and incentive, to help with it implementation and adoption.

I suppose that it would be helpful to arrange debugging information sections so that DWARF64 comes after DWARF32, otherwise, some 32-bit relocations in the 32-bit info could not be resolved. But that idea might be a bit controversial because usually debugging information is expected to have the same order as the sections it refers to.

@ikudrin Can you elaborate on LLD changes. I recently started to look in to it. Reason I am interested in this, internally we are a looking in to using DWARF64, so I have bandwidth, and incentive, to help with it implementation and adoption.

I wouldn't expect LLD to need to do anything specific for DWARF64 support. It should generally speaking be treating the sections as opaque, in my opinion, and treat them no differently to other sections. If a user is mixing DWARF32 and DWARF64, then I'd say it's on their heads if relocations can't reach (just the same as it is if they're using DWARF32 but really need DWARF64). Any interactions LLD does have with the contents of the DWARF sections should be controlled via the DebugInfo library, and therefore if that library works for DWARF64, LLD doesn't need any special handling.

and adoption.

We don't want to encourage unnecessary adoption of DWARF64 as it bloats debug data sizes unnecessarily in cases when the total debug data size is less than the DWARF32 limitations. DWARF64 should always be opt-in in my opinion, for the general purpose user (there might be some large code-bases out there where enabling it by default in their build system makes sense, but they're currently the exception, not the rule).

I wouldn't expect LLD to need to do anything specific for DWARF64 support. It should generally speaking be treating the sections as opaque, in my opinion, and treat them no differently to other sections. If a user is mixing DWARF32 and DWARF64, then I'd say it's on their heads if relocations can't reach (just the same as it is if they're using DWARF32 but really need DWARF64). Any interactions LLD does have with the contents of the DWARF sections should be controlled via the DebugInfo library, and therefore if that library works for DWARF64, LLD doesn't need any special handling.

That would be true if there would be no third-party libraries a project might depend on. These libraries would have 32-bit debug info, which is the recommended option from the DWARF standard, as far as I can remember. And, in general, these libraries would be added to the link after the user's code, to satisfy dependencies.

I wouldn't expect LLD to need to do anything specific for DWARF64 support. It should generally speaking be treating the sections as opaque, in my opinion, and treat them no differently to other sections. If a user is mixing DWARF32 and DWARF64, then I'd say it's on their heads if relocations can't reach (just the same as it is if they're using DWARF32 but really need DWARF64). Any interactions LLD does have with the contents of the DWARF sections should be controlled via the DebugInfo library, and therefore if that library works for DWARF64, LLD doesn't need any special handling.

That would be true if there would be no third-party libraries a project might depend on. These libraries would have 32-bit debug info, which is the recommended option from the DWARF standard, as far as I can remember. And, in general, these libraries would be added to the link after the user's code, to satisfy dependencies.

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

Exactly. Not to mention, I think for users that actually worry about 4Gig limit they have pretty complex build system that will need to be modified to get build order right. Probably doable, but looking at overall compilation pipeline, is it really the best approach? Within lld we don't have to parse entire debug section, just read few bytes in each CU to determine if it's 32 or 64 bit.
Yes theoretically it is possible that there are just so many third party libraries that they will over flow 4gig by themselves, but I think common case is they will be under 4 gigs.

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

Exactly. Not to mention, I think for users that actually worry about 4Gig limit they have pretty complex build system that will need to be modified to get build order right. Probably doable, but looking at overall compilation pipeline, is it really the best approach? Within lld we don't have to parse entire debug section, just read few bytes in each CU to determine if it's 32 or 64 bit.
Yes theoretically it is possible that there are just so many third party libraries that they will over flow 4gig by themselves, but I think common case is they will be under 4 gigs.

FWIW, this is probably a big enough discussion to deserve it's own review, probably even it's own llvm-dev thread. My personal take would be: Unless there's a specific user who needs this, probably not worth building it. If you personally have a need or support users who need it, that swings the discussion a fair bit into "what's the best way we can help these users".

wenlei added a subscriber: wenlei.Nov 9 2020, 10:38 AM

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

Exactly. Not to mention, I think for users that actually worry about 4Gig limit they have pretty complex build system that will need to be modified to get build order right. Probably doable, but looking at overall compilation pipeline, is it really the best approach? Within lld we don't have to parse entire debug section, just read few bytes in each CU to determine if it's 32 or 64 bit.
Yes theoretically it is possible that there are just so many third party libraries that they will over flow 4gig by themselves, but I think common case is they will be under 4 gigs.

FWIW, this is probably a big enough discussion to deserve it's own review, probably even it's own llvm-dev thread. My personal take would be: Unless there's a specific user who needs this, probably not worth building it. If you personally have a need or support users who need it, that swings the discussion a fair bit into "what's the best way we can help these users".

+1 for what @ayermolo and @ikudrin said. To me, using -u to force order is not only unrealistic for large code base, but also a bit hacky (-u has implication not intended for this use case). As for potential users, we're considering adopting DWARF64 for some large internal workloads which go over DWARF32 size limit, with some libraries still built with DWARF32 linked in.

I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

Even if eventually most will move to DWARF64, it will take a long time considering all the libraries out there. Having good support for mix use would make DWARF64 adoption and transition much more feasible.

Fair point on moving the discussion to llvm-dev, and happy to learn alternative ways for good support for mix use.

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

Exactly. Not to mention, I think for users that actually worry about 4Gig limit they have pretty complex build system that will need to be modified to get build order right. Probably doable, but looking at overall compilation pipeline, is it really the best approach? Within lld we don't have to parse entire debug section, just read few bytes in each CU to determine if it's 32 or 64 bit.
Yes theoretically it is possible that there are just so many third party libraries that they will over flow 4gig by themselves, but I think common case is they will be under 4 gigs.

FWIW, this is probably a big enough discussion to deserve it's own review, probably even it's own llvm-dev thread. My personal take would be: Unless there's a specific user who needs this, probably not worth building it. If you personally have a need or support users who need it, that swings the discussion a fair bit into "what's the best way we can help these users".

At least in LLD, it's not quite as simple as being added after the user's code: if a library appears on the link line it will be included in the output order as soon as it is determined it is needed. Thus if you have have three modules 1.o, 2.o, and 3.o, with 3.o in an archive 3.a and 1.o requiring 3.o, you end up with an output order of 1.o 3.o 2.o if the input order was 1.o 3.a 2.o or 3.a 1.o 2.o or an output order of 1.o 2.o 3.o if the input order was 1.o 2.o 3.a. In fact, with use of the --undefined linker switch, you can even force 3.o to appear first.

I accept using --undefined or rearranging the command-line order is less than ideal, but I'm really not convinced LLD should have any place in parsing the DWARF to determine output order. Furthermore, it's not even a reliable solution - if the objects built with DWARF32 (potentially all of which might have come from libraries) are large enough, no amount of reordering will fix the behaviour. I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

I'd guess that for a large-scale project the recommendation to use -u would be unrealistic. We are talking about projects where debugging information in a single section can easily go beyond the 4GiB limit; it is impossible for the developer to adjust the command line manually.

By the way, from a semantic point of view, I don't think it matters if the DWARF is in a different order to the data it represents - I'm just concerned about the maintenance and performance burden of having to parse the DWARF to achieve this reordering.

There is no need to parse the debug info sections. Reading only the first 4 bytes of .debug_info is enough to assess the format (there might be input files with format intermixing, but we can ignore them in the sack of simplicity). And we do not need any automatic sorting if the size of an output section is less than 4GiB.

Exactly. Not to mention, I think for users that actually worry about 4Gig limit they have pretty complex build system that will need to be modified to get build order right. Probably doable, but looking at overall compilation pipeline, is it really the best approach? Within lld we don't have to parse entire debug section, just read few bytes in each CU to determine if it's 32 or 64 bit.
Yes theoretically it is possible that there are just so many third party libraries that they will over flow 4gig by themselves, but I think common case is they will be under 4 gigs.

FWIW, this is probably a big enough discussion to deserve it's own review, probably even it's own llvm-dev thread. My personal take would be: Unless there's a specific user who needs this, probably not worth building it. If you personally have a need or support users who need it, that swings the discussion a fair bit into "what's the best way we can help these users".

+1 for what @ayermolo and @ikudrin said. To me, using -u to force order is not only unrealistic for large code base, but also a bit hacky (-u has implication not intended for this use case). As for potential users, we're considering adopting DWARF64 for some large internal workloads which go over DWARF32 size limit, with some libraries still built with DWARF32 linked in.

I think users who need DWARF64 in their libraries are just going to have to request DWARF64 versions of the libraries, if the --undefined and reordering command line options are insufficient.

Even if eventually most will move to DWARF64, it will take a long time considering all the libraries out there. Having good support for mix use would make DWARF64 adoption and transition much more feasible.

Fair point on moving the discussion to llvm-dev, and happy to learn alternative ways for good support for mix use.

I'll post later today on llvm-dev to open for broader discussion.