This is an archive of the discontinued LLVM Phabricator instance.

[objcopy] Fix order of Mach-O LINKEDIT pieces during layout
ClosedPublic

Authored by drodriguez on Sep 15 2022, 1:58 PM.

Details

Summary

The exports trie and the chained fixups where in the opposite order, and
function starts happenned before them, instead of after them.

Restore the correct order and rewrite the code to make it easier to move
around in the future if needed by reusing the Offset variable and
keeping both the StartOf... and the size of each piece together.

This was found out while trying to use the system strip in a binary
already stripped by LLVM and receiving errors around chained fixups when
we enabled those in the linker.

Diff Detail

Event Timeline

drodriguez created this revision.Sep 15 2022, 1:58 PM
Herald added a project: Restricted Project. · View Herald Transcript
drodriguez requested review of this revision.Sep 15 2022, 1:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptSep 15 2022, 1:58 PM

Added reviewers who have been more involved in Mach-O side of objcopy.

Is there a sensible test case we could have for this?

The change LGTM, but a test case would be good.

Is there any documentation we can reference or explanation we can add as to why LINKEDIT needs to be in this order? I assume LLD lays things out the same way; do they have any reference for that?

kyulee added a subscriber: kyulee.Sep 16 2022, 12:31 PM

+1 to what @smeenai said

llvm/lib/ObjCopy/MachO/MachOLayoutBuilder.cpp
246
270

just in case (didn't have a minute to check myself) - is uint32_t used here (and on line 246) by design ?
(arithmetic operations with Offset are done on uint64_t anyway, uint32_t will be zero-extended to uint64_t)

drodriguez marked 2 inline comments as done.

Follow code style. Add a test. Needs D134250.

llvm/lib/ObjCopy/MachO/MachOLayoutBuilder.cpp
270

Maybe because linkedit_data_command.datasize is a uint32_t?

I have not changed this one, since I think it is trying to match the underlying format. I have changed the parameter of the lambda, since it seems to take its value from a lot of size(), which seems to be size_t, not uint32_t.

alexander-shaposhnikov added subscribers: keith, thakis.

see a small nit above, otherwise - looks good to me.
p.s. https://reviews.llvm.org/D119671 seems to be relevant, don't know the current status of that effort though.
It would be good to review/land D134250 or D119671, alternatively i think it's reasonable to add a TODO (that we should add a test once ObjectYAML is ready) and commit this fix now.
cc: @thakis , @keith

llvm/lib/ObjCopy/MachO/MachOLayoutBuilder.cpp
245

nit: size -> Size

p.s. I'd probably rename linkEditUpdateOffset -> updateOffset

This revision is now accepted and ready to land.Sep 19 2022, 10:04 PM

More code style

see a small nit above, otherwise - looks good to me.
p.s. https://reviews.llvm.org/D119671 seems to be relevant, don't know the current status of that effort though.
It would be good to review/land D134250 or D119671, alternatively i think it's reasonable to add a TODO (that we should add a test once ObjectYAML is ready) and commit this fix now.
cc: @thakis , @keith

Yes. It seems @keith and me filled the same gap. My version includes the "DataInCode" part which I wanted to test as many pieces as I could in the linkedit order.

About not waiting for D134250 (or equivalent), the test will have to be reduced a lot (and it is already missing some pieces about bindings that I don't know how to generate properly). Unless I end up in a review blockage, I will keep the dependency for the moment.

Correctly handle DyldExportsTrie

jhenderson added inline comments.Sep 21 2022, 12:06 AM
llvm/test/tools/llvm-objcopy/MachO/linkedit-order.test
2 ↗(On Diff #461738)

layout -> laid out

Rebase on top of changes from D134569, D134571 and D134250

drodriguez marked 2 inline comments as done.Sep 23 2022, 4:09 PM
jhenderson added inline comments.Sep 26 2022, 12:05 AM
llvm/test/tools/llvm-objcopy/MachO/linkedit-order.test
31 ↗(On Diff #462607)

FWIW, I don't think it would be too difficult to get FileCheck to support multiplication. If memory serves me rightly, it would just require adding another binop support, so could easily follow the existing addition support.

Use FileCheck mul() for multiplying. Improve test by checking the output several times. Add tests for both chained fixups and non-chained fixups.

drodriguez marked an inline comment as done.Sep 26 2022, 5:30 PM
drodriguez added inline comments.
llvm/test/tools/llvm-objcopy/MachO/linkedit-order.test
31 ↗(On Diff #462607)

I think the operator * might not implemented because of the precedence rules. I looked a little bit into it, but I wasn't sure that my implementation was going to be good. But thanks for the clue because I saw that there was some function-like alternatives like mul and div that allows doing the multiplication and also allows me to not have to be careful with the alignment of the last piece of linkedit (I can code the alignment in the test).

drodriguez marked an inline comment as done.Nov 4 2022, 7:13 PM

gentle ping. @drodriguez , @smeenai

There’s one diff in the stack that’s stuck. I pinged a couple of weeks ago but it did not advance a lot.

I hoped not to have to rework these last two without that, but it might be the only way. I will see what it can be done, but I am out for the weekend, so nothing until next week.

drodriguez updated this revision to Diff 474401.Nov 9 2022, 5:58 PM

Rewrite without depending on D134571

This revision was landed with ongoing or failed builds.Nov 11 2022, 12:07 PM
This revision was automatically updated to reflect the committed changes.