This is an archive of the discontinued LLVM Phabricator instance.

[MSP430] Get the DWARF pointer size from MCAsmInfo instead of DataLayout.
ClosedPublic

Authored by kuilpd on Apr 11 2023, 12:58 PM.

Details

Summary

This change will allow to put code pointers in DWARF info fields that are larger than actual pointer size, e.g. 16-bit pointers into 32-bit fields.

The need for this came up while creating support for MSP430 in LLDB. MSP430-GCC already generates DWARF info with 32-bit fields, so this change is necessary for LLDB to maintain compatibility with both GCC and LLVM binaries. Moreover, right now in LLDB there is no support for having DWARF pointer size different from ELF header type, e.g. 16-bit DWARF info within ELF32, and it seems there is no such thing as ELF16.

Since other mainline targets are made to have the same pointer size in both MCAsmInfo and DataLayout, there is no need to change anything there.

Diff Detail

Event Timeline

kuilpd created this revision.Apr 11 2023, 12:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 11 2023, 12:58 PM
Herald added a subscriber: hiraditya. · View Herald Transcript
kuilpd requested review of this revision.Apr 11 2023, 12:58 PM
asl added a comment.Apr 11 2023, 2:15 PM

The change looks good from MSP430 perspective. We will need it in any case (or something similar) for MSP430X case as there we're having 16-bit platform with 20-bit pointers. Apparently it turned out that there is no such thing as "ELF16" and msp430-gcc already generates ELF32 for MSP430 target.

kuilpd added a reviewer: Michael137.

What's the difference between these two things? (like, why are they different for MSP430? I guess the data layout represents how big a pointer is in a struct, but what does the "code pointer size" represent/why is it the right answer?)

Equally/also, this seems like an lldb bug that could be fixed there, though arguably we would want to emit it the same as GCC for GDB compatibility anyway (if GDB can't cope with the 2 byte pointers already - have you tested that?).

Also there's a few codepaths that include assertions about pointer sizes being 4 or 8 bytes, so they'd presumably have resulted in crashes for MSP430 before? So sounds like those codepaths are untested, and perhaps should be tested? (could you add that test coverage?) & maybe it'd be good to unify all the callers so this is only done in one place instead of 5?

asl added a subscriber: probinson.Apr 18 2023, 2:32 PM

@dblaikie as far as I can see, there are several interconnected issues here.

First of all, ELF could be 32-bit or 64-bit (EI_CLASS field in ELF header), there are no 16-bit ELF files defined around and I do not think it's a good idea to invent something new here. Next, LLDB indeed assumes that pointer size coincides with ELF class and I would not consider it as a bug :) And it seems there is no way to specify that inside ELF file that ELF32 file could contain 16-bit pointers.

We are seeing a precedent of msp430-gcc to generate 32-bit pointers inside ELF32 file for MSP430 architecture and eventually we need to be able to mix-and-match linking with such objects as they could be in e.g. CRT, etc.

So, it seems we need a way to distinguish between "platform address size" that it used for code generation, optimizations, etc. and "address size used for DWARF emission" (both for data and code!). Currently LLVM assumes that they are the same and the patch allows a platform to override it. Even more, MSP430X extention uses 20-bit pointers, so we'd end with something similar here one way or another.

It seems currently they things a bit messy in how we're emitting DWARF data (see https://discourse.llvm.org/t/what-is-a-codepointersize-and-how-does-it-relate-to-dwarf/64825 for example)

If you're overseeing better solution of the problem, then it would be good to hear how :)

I'm also tagging @probinson for some insights

@dblaikie as far as I can see, there are several interconnected issues here.

First of all, ELF could be 32-bit or 64-bit (EI_CLASS field in ELF header), there are no 16-bit ELF files defined around and I do not think it's a good idea to invent something new here.

Agreed.

Next, LLDB indeed assumes that pointer size coincides with ELF class and I would not consider it as a bug :)

I would - DWARF has a field for pointer size & it doesn't have to be/isn't tied to the ELF class.

And it seems there is no way to specify that inside ELF file that ELF32 file could contain 16-bit pointers.

There is at the DWARF level.

But I guess you mean for the actual code in an ELF32 file you have to have 32 bit pointers for relocations to be applied, etc? So what does it mean for for the program to have 16 bit pointers but be using 32 bit relocations?

We are seeing a precedent of msp430-gcc to generate 32-bit pointers inside ELF32 file for MSP430 architecture and eventually we need to be able to mix-and-match linking with such objects as they could be in e.g. CRT, etc.

The ability to mix and match doesn't seem like "we have to do what GCC does" - it might still mean that there can/should be fixes to consumers. But equally I'm generally a fan of "do whatever GCC does" when it comes to DWARF, since it's far too broad to say "consumers should handle anything the DWARF spec allows".

So, it seems we need a way to distinguish between "platform address size" that it used for code generation, optimizations, etc. and "address size used for DWARF emission" (both for data and code!).

I still don't know if we should/do need to distinguish those things, but maybe we do. I'm still not sure how the 16 bit pointers work (disregarding DWARF for a minute) with a 32 bit ELF file...

Currently LLVM assumes that they are the same and the patch allows a platform to override it. Even more, MSP430X extention uses 20-bit pointers, so we'd end with something similar here one way or another.

It seems currently they things a bit messy in how we're emitting DWARF data (see https://discourse.llvm.org/t/what-is-a-codepointersize-and-how-does-it-relate-to-dwarf/64825 for example)

Ah, yeah, that thread discusses some of the directions I'd have in mind.

If this is a regression, it'd be good to have context for where these things were changed to use DataLayout (a link to the commit(s)) and an explanation of why changing them back won't harm whatever motivated them to be changed to use DataLayout in the first place.

Then, yes, as mentioned in that thread, adding a single entry point for these things would be important (just fixing the 5 sites directly seems like a bad idea if these should all be kept in sync and there are easy ways to mistakenly use the wrong value as has been the case in the past, perhaps).

But I'd like to understand the problem better too/first - it doesn't seem right to me that DWARF would need a different size from the system/target itself. (& no, I don't especially know what the "codepointer" size is as distinct from the address size either)

Next, LLDB indeed assumes that pointer size coincides with ELF class and I would not consider it as a bug :)

I would - DWARF has a field for pointer size & it doesn't have to be/isn't tied to the ELF class.

And it seems there is no way to specify that inside ELF file that ELF32 file could contain 16-bit pointers.

There is at the DWARF level.

DWARF has a pointer size field, but ELF doesn't. When parsing ELF header, LLDB just assumes the pointer size to be the same as ELF type, 32 or 64 bit, no other options.
https://github.com/llvm/llvm-project/blob/main/lldb/source/Plugins/ObjectFile/ELF/ELFHeader.cpp#L120
16-bit pointers can be put in ELF32, but LLDB fails to read them while trying to extract them as 32-bit values. Wouldn't call it a bug per se, it's just how it's made, seeing as right now only ELF32 and ELF64 exist.
LLDB does use the DWARF's pointer size when extracting DWARF information, but if DWARF says it has 16-bit pointers when LLDB already assumed it's 32-bit because of ELF32, it breaks things. Can't remember which exact things, but a lot of debug info stops working. Sometimes LLDB uses ELF pointer width, sometimes DWARF, everything is interleaved there as it assumes the sizes in both are the same.
And if there is no DWARF information in the file, there seems to be no way of specifying in an ELF32 header that it has 16-bit pointers.

But I guess you mean for the actual code in an ELF32 file you have to have 32 bit pointers for relocations to be applied, etc? So what does it mean for for the program to have 16 bit pointers but be using 32 bit relocations?

I'm not sure, but I haven't encountered any problems. Pointers are just treated as 32-bit wide in ELF/DWARF information only, but once all the information has been correctly loaded by LLDB, it uses the pointer size that is specified for the target architecture.
https://github.com/llvm/llvm-project/blob/main/lldb/source/Utility/ArchSpec.cpp#L158

So, it seems we need a way to distinguish between "platform address size" that it used for code generation, optimizations, etc. and "address size used for DWARF emission" (both for data and code!).

I still don't know if we should/do need to distinguish those things, but maybe we do. I'm still not sure how the 16 bit pointers work (disregarding DWARF for a minute) with a 32 bit ELF file...

Right now as I understand the pointer size in DataLayout is used for code generation, and CodePointerSize in MCAsmInfo if used for DWARF purposes, namely in AsmPrinter, so there is already a distinction. But since for all other targets these values seem to be the same it might have been used interchangeably, so this patch suggest to actually use CodePointerSize in DWARF-related information.

If this is a regression, it'd be good to have context for where these things were changed to use DataLayout (a link to the commit(s)) and an explanation of why changing them back won't harm whatever motivated them to be changed to use DataLayout in the first place.

From what I've found, these were initially written using DataLayout, because CodePointerSize didn't exist yet. But when it was introduced, most DWARF-related code was rewritten using it, but I assume that not in all places.
For example, DwarfDebug.cpp used to use DataLayout as well:
https://github.com/llvm/llvm-project/blob/release/4.x/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp#L1694
but it got changed to CodePointerSize here and in all other places where DataLayout was used:
https://github.com/llvm/llvm-project/blob/release/5.x/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp#L1747
So I think the changes this patch suggests just fix what was overlooked. It just never broke any code because the values are the same for other targets.
This might be the commit that changed things, just not all of them:
https://github.com/llvm/llvm-project/commit/dc77b2e960b6b6d4622c89ee8d324848ac6a0609

Then, yes, as mentioned in that thread, adding a single entry point for these things would be important (just fixing the 5 sites directly seems like a bad idea if these should all be kept in sync and there are easy ways to mistakenly use the wrong value as has been the case in the past, perhaps).

If the patch just fixes what was overlooked, then changing directly is the way to go. But maybe CodePointerSize should be renamed to DWARFCodePointerSize or something else to avoid confusion from now on.

But I'd like to understand the problem better too/first - it doesn't seem right to me that DWARF would need a different size from the system/target itself. (& no, I don't especially know what the "codepointer" size is as distinct from the address size either)

The need is to simplify implementation of 16-bit targets for use in LLDB.
This way:

  1. There is no need to change how LLDB handles ELF headers
  2. Allows compatibility with GCC binaries (for MSP430, specifically)
  3. Doesn't change anything for other targets

If the patch just fixes what was overlooked, then changing directly is the way to go.

The hope/desire would be to make it harder to overlook, by not having the DWARF code reach out to these various config places & just have to "know" which one was the right one - but have it handled once in DwarfDebug and then only consult that everywhere else.

Could you look up the original commits that overlooked this and include references to them here? It'd help provide context for this change for both the review and commit/history later on. (explain how we got here, demonstrate these were overlooked/the changes proposed are consistent with that previous work, etc)

kuilpd updated this revision to Diff 516192.Apr 23 2023, 11:16 AM

Shortened access to MCAsmInfo.

kuilpd added a comment.EditedApr 23 2023, 11:18 AM

If the patch just fixes what was overlooked, then changing directly is the way to go.

The hope/desire would be to make it harder to overlook, by not having the DWARF code reach out to these various config places & just have to "know" which one was the right one - but have it handled once in DwarfDebug and then only consult that everywhere else.

Could you look up the original commits that overlooked this and include references to them here? It'd help provide context for this change for both the review and commit/history later on. (explain how we got here, demonstrate these were overlooked/the changes proposed are consistent with that previous work, etc)

As far as I could find, these changes were initially suggested in D11969 (for different reasons, though) but were abandoned, then reintroduced in D30879.
However, not everything was reintroduced, like for example line 137 in AsmPrinterDwarf.cpp from D11969.

Regarding single entry point, I made it look a bit better (just like in D30879). On top of that I could make a getCodePointerSize() in AsmPrinter.h which would take the pointer size from MCAsmInfo, right next to getPointerSize() function which takes the pointer size from DataLayout. But at this point there is already an entry point, it's just in MCAsmInfo.

dblaikie accepted this revision.May 1 2023, 5:59 PM

Not sure these elements are sufficiently well documented/specified to make this super reliable in terms of the chances this property will regress in the future - but at least eliminating all the calls as you've done here gives a better chance that someone'll copy from an existing use case rather than introducing a novel one.

llvm/lib/Target/MSP430/MCTargetDesc/MSP430MCAsmInfo.cpp
19–24

I wouldn't justify this from a "what LLDB assumes" - again, we could "just" fix LLDB if that's the problem.

23–24

It's not really compatibility with GCC binaries though - the DWARF could be different between the two compilers and that could be supported by consumers.

Maybe the simplest wording would be something like "Since this is what GCC does, we'll do it to since it's likely a better/well tested path for DWARF consumers like GDB and LLDB, rather than something that might be novel though possibly more technically correct"?

This revision is now accepted and ready to land.May 1 2023, 5:59 PM
kuilpd updated this revision to Diff 518824.May 2 2023, 12:12 PM

Reworded the explanation commentary.

This revision was landed with ongoing or failed builds.May 4 2023, 12:38 PM
This revision was automatically updated to reflect the committed changes.