This is an archive of the discontinued LLVM Phabricator instance.

[lldb] Improve/fix base address selection in location lists
ClosedPublic

Authored by labath on Nov 21 2019, 4:41 AM.

Details

Summary

Lldb support base address selection entries in location lists was broken
for a long time. This wasn't noticed until llvm started producing these
kinds of entries more frequently with r374600.

In r374769, I made a quick patch which added sufficient support for them
to get the test suite to pass. However, I did not fully understand how
this code operates, and so the fix was not complete. Specifically, what
was lacking was the ability to handle modules which were not loaded at
their preferred load address (for instance, due to ASLR).

Now that I better understand how this code works, I've come to the
conclusion that the current setup does not provide enough information
to correctly process these entries. In the current setup the location
lists were parameterized by two addresses:

  • the distance of the function start from the start of the compile unit. The purpose of this was to make the location ranges relative to the start of the function.
  • the actual address where the function was loaded at. With this the function-start-relative ranges can be translated to actual memory locations.

The reason for the two values, instead of just one (the load bias) is (I
think) MachO, where the debug info in the object files will appear to be
relative to the address zero, but the actual code it refers to
can be moved and reordered by the linker. This means that the location
lists need to be "linked" to reflect the locations in the actual linked
file.

These two bits of information were enough to correctly process location
lists which do not contain base address selection entries (and so all
entries are relative to the CU base). However, they don't work with
them because, in theory two base address can be completely unrelated (as
can happen for instace with hot/cold function splitting, where the
linker can reorder the two pars arbitrarily).

To fix that, I split the first parameter into two:

  • the compile unit base address
  • the function start address, as is known in the object file

The new algorithm becomes:

  • the location lists are processed as they were meant to be processed. The CU base address is used as the initial base address value. Base address selection entries can set a new base.
  • the difference between the "file" and "load" function start addresses is used to compute the load bias. This value is added to the final ranges to get the actual memory location.

This algorithm is correct for non-MachO debug info, as there the
location lists correctly describe the code in the final executable, and
the dynamic linker can just move the entire module, not pieces of it. It
will also be correct for MachO if the static linker preserves relative
positions of the various parts of the location lists -- I don't know
whether it actually does that, but judging by the lack of base address
selection support in dsymutil and lldb, this isn't something that has
come up in the past.

I add a test case which simulates the ASLR scenario and demonstrates
that base address selection entries now work correctly here.

Event Timeline

labath created this revision.Nov 21 2019, 4:41 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 21 2019, 4:41 AM

So I am worried by seeing any mention of "load address" and "load bias" in this patch. All information that is extracted from the DWARF should be solely in terms if "file addresses" IMHO, unless I am not understanding something from the new location lists. For mach-o DWARF in .o files, the "file address" will be a file address from the .o file (which is fine), but it will need to be linked into an executable file address before being handed out. We fix up addresses like this for any DW_OP_addr opcodes in location expressions. So typically we would have a virtual SymbolFileDWARF method that SymbolFileDWARFDebugMap would override and this would use the default SymbolFileDWARF::GetLocationList(...) function, and then link up the location list or DWARFExpression prior to handing it out. Is there something I am not understanding here?

So I am worried by seeing any mention of "load address" and "load bias" in this patch. All information that is extracted from the DWARF should be solely in terms if "file addresses" IMHO,

This is still true. The information for which goes into the construction of the DWARFExpression is based on information from the object file alone -- this is the CU base (file) address, and function base (file) address. Essentially, the only difference here is that previously we were passing the difference between these two values, whereas this patch passes the two values separately.

The "load" address goes in only when the expression is evaluated (and so it can be different for each invocation, etc.). This part is not changed at all, though I have renamed the value to better reflect how it is now used in the patch.

Let's say we have a CU whose base address is 0x1000, and a function which starts at 0x1100, and we also have a location list like: (0x142, 0x147) => DW_OP_whatever. Let's say that the module is loaded in memory such that the function now starts at 0x8100.
What happened previously that we would create the DWARFExpression with the "slide" of 0x100. Then, when we wanted to get an actual location we'd pass in 0x8100 as the "load address", and the code would compute
(0x8100 - 0x100 + 0x142, 0x8100 - 0x100 + 0x147) = (0x8142, 0x8147) to get the actual location list range.
This worked fine if you didn't have base address selection entries in the location list. Once you have those, you have a problem -- you know that you need to change the base address somehow, but it's not possible to compute how. If we pass in the "file" address of the function start (0x1100), we can compute the load bias (0x8100 - 0x1100 = 0x7000), and add that to any address coming out of the location list.

For mach-o DWARF in .o files, the "file address" will be a file address from the .o file (which is fine), but it will need to be linked into an executable file address before being handed out. We fix up addresses like this for any DW_OP_addr opcodes in location expressions. So typically we would have a virtual SymbolFileDWARF method that SymbolFileDWARFDebugMap would override and this would use the default SymbolFileDWARF::GetLocationList(...) function, and then link up the location list or DWARFExpression prior to handing it out. Is there something I am not understanding here?

The process you're describing is indeed used elsewhere (line tables for instance). It's not possible to use it here, because the location lists are parsed very late -- basically until you actually start to evaluate the expression, the location list is just a blob of bytes, so the relocation needs to happen at a later stage. I'm not sure why something similar wasn't done here too (I suspect it has something to do with DWARFExpression living separately from the rest of the DWARF plugin code), but changing that here seems complicated (particularly as I don't know all the details of mach-o linker stuff).

So I am worried by seeing any mention of "load address" and "load bias" in this patch. All information that is extracted from the DWARF should be solely in terms if "file addresses" IMHO,

This is still true. The information for which goes into the construction of the DWARFExpression is based on information from the object file alone -- this is the CU base (file) address, and function base (file) address. Essentially, the only difference here is that previously we were passing the difference between these two values, whereas this patch passes the two values separately.

Gotcha.

The "load" address goes in only when the expression is evaluated (and so it can be different for each invocation, etc.). This part is not changed at all, though I have renamed the value to better reflect how it is now used in the patch.

Let's say we have a CU whose base address is 0x1000, and a function which starts at 0x1100, and we also have a location list like: (0x142, 0x147) => DW_OP_whatever. Let's say that the module is loaded in memory such that the function now starts at 0x8100.
What happened previously that we would create the DWARFExpression with the "slide" of 0x100. Then, when we wanted to get an actual location we'd pass in 0x8100 as the "load address", and the code would compute
(0x8100 - 0x100 + 0x142, 0x8100 - 0x100 + 0x147) = (0x8142, 0x8147) to get the actual location list range.
This worked fine if you didn't have base address selection entries in the location list. Once you have those, you have a problem -- you know that you need to change the base address somehow, but it's not possible to compute how. If we pass in the "file" address of the function start (0x1100), we can compute the load bias (0x8100 - 0x1100 = 0x7000), and add that to any address coming out of the location list.

Thanks for explaining.

For mach-o DWARF in .o files, the "file address" will be a file address from the .o file (which is fine), but it will need to be linked into an executable file address before being handed out. We fix up addresses like this for any DW_OP_addr opcodes in location expressions. So typically we would have a virtual SymbolFileDWARF method that SymbolFileDWARFDebugMap would override and this would use the default SymbolFileDWARF::GetLocationList(...) function, and then link up the location list or DWARFExpression prior to handing it out. Is there something I am not understanding here?

The process you're describing is indeed used elsewhere (line tables for instance). It's not possible to use it here, because the location lists are parsed very late -- basically until you actually start to evaluate the expression, the location list is just a blob of bytes, so the relocation needs to happen at a later stage. I'm not sure why something similar wasn't done here too (I suspect it has something to do with DWARFExpression living separately from the rest of the DWARF plugin code), but changing that here seems complicated (particularly as I don't know all the details of mach-o linker stuff).

Sounds good.

lldb/include/lldb/Expression/DWARFExpression.h
88–90

Is this a "file address"? Maybe clarify here in the comment, or rename variable appropriately?

141–150

Maybe change name to "cu_file_addr"?

144

Maybe change name to "func_file_addr"?

148

Rename from "SetLocationListSlide" to "SetLocationListAddresses"? Slide doesn't seem relevant anymore

163

Maybe change name to "func_file_addr"?

225–226

Maybe change name to "func_file_addr"?

260

Maybe change name to "func_file_addr"?

labath marked 4 inline comments as done.Nov 26 2019, 4:14 AM
labath added inline comments.
lldb/include/lldb/Expression/DWARFExpression.h
88–90

No, this is an actual load address. The file address is only used in SetLocationListSlide/SetLocationListAddresses which then stores it in the object itself. All other functions take a load address, since that is intentionally not stored anywhere.

(Technically this doesn't have to be a load address -- it can be whatever the type of address you want. If you pass in a file address here (like some dumping code does), then it will still work, just the addresses coming out of the location list will be file addresses...)

labath updated this revision to Diff 231041.Nov 26 2019, 4:14 AM

Rename to {cu,func}_{file,load}_addr as appropriate.

labath added a comment.Dec 3 2019, 4:05 AM

Greg, what do you make of this version?

This revision was not accepted when it landed; it landed in state Needs Review.Dec 9 2019, 4:49 AM
This revision was automatically updated to reflect the committed changes.

Sorry for the delay, lgtm