This is an archive of the discontinued LLVM Phabricator instance.

[llvm-objdump] Complete -chained_fixups support
ClosedPublic

Authored by BertalanD on Aug 16 2022, 11:07 AM.

Details

Summary

This commit adds definitions for the dyld_chained_import* structs.
The imports array is now printed with llvm-otool -chained_fixups. This
completes this option's implementation.

A slight difference from cctools otool is that we don't yet dump the
raw bytes of the imports entries.

When Apple's effort to upstream their chained fixups code continues,
we'll replace this code with the then-upstreamed code. But we need
something in the meantime for testing ld64.lld's chained fixups code.

Diff Detail

Event Timeline

BertalanD created this revision.Aug 16 2022, 11:07 AM
BertalanD requested review of this revision.Aug 16 2022, 11:07 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 16 2022, 11:07 AM
thakis accepted this revision.Aug 17 2022, 3:38 PM

Very nice!

llvm/include/llvm/BinaryFormat/MachO.h
2131

It'd be nice to use bit_cast from llvm/ADT/bit.h here, which will ensure that Raw is the same size as dyld_chained_import. Here this is fairly easy I think:

auto Raw = bit_cast<uint32_t>(C);
sys::swapByteOrder(Raw);
C = bit_cast<dyld_chained_import>(Raw);

Below it's a bit more tricky, but I think this might work (all untested though):

auto Raw = bit_cast<std::array<uint32_t, 2>>(C);
sys::swapByteOrder(Raw[0]);
sys::swapByteOrder(Raw[1]);
C = bit_cast<(dyld_chained_import_addend>(Raw);

(and analogously for the 64-bit version)

llvm/lib/Object/MachOObjectFile.cpp
4935

nit: no else after return

This function looks weird! It almost does if (V == a) return a a bunch of times followed by return V, and the special cases look pretty pointless at first. (I understand they're needed to sign-extend the 3 special values to int, while the input is 8 or 16 bits.) It feels like there should be a nicer way to write this.

Maybe

if (Value == static_cast<T>(MachO::BIND_SPECIAL_DYLIB_MAIN_EXECUTABLE) ||
    Value == static_cast<T>(MachO::BIND_SPECIAL_DYLIB_FLAT_LOOKUP) ||
     third case)
  return SignExtend32<sizeof(T) * 8>(Value);
return Value;

Not sure if that's actually better, but it makes the difference a bit more explicit maybe.

Up to you :)

4969

Isn't sizeof(dyld_chained_import) clearer here? (likewise in the other two cases)

5005

This has GetEncodedOrdinal with upper-case G while the function above has it with lower-case g. Can this build? Maybe you uploaded a half-committed patch?

5028

nit: should this return a malformedError instead? it's data-dependent, so it's not truly unreachable. (but up to you)

llvm/tools/llvm-objdump/MachODump.cpp
1295

It makes sense to me to wait with this until the upstreaming has happened to see how to best implement this then.

This revision is now accepted and ready to land.Aug 17 2022, 3:38 PM

(All comments optional and up to you, in case that wasn't clear.)

BertalanD added inline comments.Aug 18 2022, 1:34 AM
llvm/include/llvm/BinaryFormat/MachO.h
2131

I'm not convinced that swapping the bytes is enough for endianness conversion. https://www.naic.edu/~phil/notes/bitfieldStorage.html suggests that the bits within the bytes would also need to be reversed. I'm going to update this diff to use bit masks and shifts like D132036.

llvm/lib/Object/MachOObjectFile.cpp
4935

I didn't know about SignExtend32, thank you for mentioning it. Your version does indeed look nicer than what I wrote.

5028

I'll add an error message to where we're computing ImportSize, and then this branch will be truly unreachable.

BertalanD updated this revision to Diff 453605.Aug 18 2022, 3:14 AM
BertalanD updated this revision to Diff 453607.Aug 18 2022, 3:16 AM
thakis added inline comments.Aug 19 2022, 7:07 AM
llvm/include/llvm/BinaryFormat/MachO.h
2131

Huh, TIL, I suppose.

What I used to believe up to now: There are two separate things: in-memory byte order, and order in which bitfields get assigned to their underlying words.

The c99 standard (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf), 6.7.2.1.10: says about the latter "The order of allocation of bit-fields within a unit (high-order to low-order or low-order to high-order) is implementation-defined."

So I thought that the order bitfields are assigned is implementation-defined, but independent of byte order.

But sysv abi (https://www.uclibc.org/docs/psABI-x86_64.pdf) paragraph 3.1 says: "bit-fields are allocated from right to left". So I thought, in practice, this settles things: Bit fields are assigned right-to-left (in a register, say), and then they assume system endianness when written to memory, conceptually.

(In theory, gcc has all kinds of options here: https://gcc.gnu.org/onlinedocs/gccint/Storage-Layout.html BITS_BIG_ENDIAN, PCC_BITFIELD_TYPE_MATTERS, TARGET_MS_BITFIELD_LAYOUT_P, …)

But https://godbolt.org/z/6sqo7nEd4 shows that this clearly isn't true. (Also, what's with the 65535 in the big-endian ppc64 output there? That looks completely wrong?)

And CGRecordLayoutBuilder.cpp has:

void CGRecordLowering::setBitFieldInfo(
...
  // Reverse the bit offsets for big endian machines. Because we represent
  // a bitfield as a single large integer load, we can imagine the bits
  // counting from the most-significant-bit instead of the
  // least-significant-bit.
  if (DataLayout.isBigEndian())
    Info.Offset = Info.StorageSize - (Info.Offset + Info.Size);

And gcc seems to match clang's behavior (see godbolt link), so that seems very intentional.

Does anyone reading this know what specifies this?

…aha, the ppc sysv abi (http://refspecs.linux-foundation.org/elf/elfspec_ppc.pdf) does say: " Bit-fields are allocated from right to left (least to most significant) on Little-Endian implementations and from left to right (most to least significant) on Big-Endian implementations."

I wish I hadn't learned that!

llvm/lib/Object/MachOObjectFile.cpp
4935

(FWIW, I didn't know about SignExtend32 either while I wrote the "looks weird" paragraph. I thought about how this could be written differently, then ran rg -i signextend llvm to see how other code handles this and then found SignExtend32 in there and figured it matches well enough.)

5004

The in-register view (i.e. not in-memory, we can ignore the byte-swapping of big-endian vs little-endian here) of

struct dyld_chained_import {
  uint32_t lib_ordinal : 8;
  uint32_t weak_import : 1;
  uint32_t name_offset : 23;
};

is:

  • In little-endian (bits assigned right-to-left):
    • most significant 23 bits are name_offset
    • then 1 bit weak_import
    • then 8 bit lib_ordinal in the lowest 8 bits
  • In big-endian (bits assigned left-to-right):
    • 8 bit lib_ordinal in the highest 8 bits
    • then 1 bit weak_import
    • least significant 23 bits are name_offset

So I think even with the manual masking, this currently gets things wrong.

Also, it's weird that we don't use the dyld_chained_import structs at all. Strange for greppability, using something like http://llvm-cs.pcc.me.uk/ for cross-references, etc.

What do you think about adding back the Swap<> functions and manually swapping the bitfields around too after swapping the bytes? That assumes sysv abi, but that'll work almost everywhere (can't think of a place where it wouldn't? Maybe windows big endian? Didn't try that, but I wouldn't be surprised if clang-cl didn't work great there either), and if someone needs this to work on some obscure system, they can worry about it then?

thakis added inline comments.Aug 19 2022, 7:10 AM
llvm/lib/Object/MachOObjectFile.cpp
5004

Alternatively, something like https://github.com/freebsd/freebsd-src/blob/master/sys/netinet/ip.h#L51-L59 might also be an option.

We chatted a bit offline. Rough summary

  • https://github.com/freebsd/freebsd-src/blob/master/sys/netinet/ip.h#L51-L59 works if the on-disk format is the same for BE and LE, but since $(xcrun -show-sdk-path)/usr/include/mach-o/fixup-chains.h doesn't have any ifdefs, that's likely not the case here
  • …but all chained fixup apple platforms are little endian, so it's really unknowable
  • the current code does work on all hosts for when the on-disk data is little-endian
  • it'd be nice to do the swapping anyways, since that puts all the endianness handling in a single place
  • Bitfield swapping is different for little-endian-file-on-big-endian-host and the other way round, and the swapStruct() wrappers in BinaryFormat/MachO.h don't know which direction they're swapping in, so there's have to be a dedicated wrapper for each struct. Example for dyld_chained_import:
// Little-endian file running on big-endian host:
Raw = ((Raw & 0xff) << 24) | ((Raw & 0x100) << 15) || (Raw >> 9);

// Big-endian file running on little-endian host:
Raw = (Raw >> 24) | ((Raw & 0x800000) >> 15) || ((Raw & 0x7fffff << 9);
  • Since there's a bunch of those structs, it'd be nice to make a general helper for this. This seems to roughly work:
#include <stdio.h>

template<typename T, unsigned W>
T bitfield_swap_le(T t) {
  static_assert(W == 0, "");
  return 0;
}

template<typename T, unsigned W, unsigned First, int... Sizes>
T bitfield_swap_le(T t) {
  // Peel off rightmost field, add it on left, recurse.
  return ((t & ((1u << First) - 1)) << (W - First)) |
         bitfield_swap_le<T, W - First, Sizes...>(t >> First);
}

template<typename T, unsigned W>
T bitfield_swap_be(T t) {
  static_assert(W == 0, "");
  return 0;
}

template<typename T, unsigned W, unsigned First, int... Sizes>
T bitfield_swap_be(T t) {
  // Peel off leftmost field, add it on right, recurse.
  return ((t >> (W - First)) |
         (bitfield_swap_be<T, W - First, Sizes...>(t & ((1u << (W - First)) - 1))) << First);
}

int main() {
  {
    unsigned i = bitfield_swap_le<unsigned, 32, 8, 1, 23>(0xff);
    unsigned j = bitfield_swap_le<unsigned, 32, 8, 1, 23>(0x100);
    unsigned k = bitfield_swap_le<unsigned, 32, 8, 1, 23>(0xffff'fe00);
    printf("%08x %08x %08x\n", i, j, k);
  }
  {
    unsigned i = bitfield_swap_be<unsigned, 32, 8, 1, 23>(0xff00'0000);
    unsigned j = bitfield_swap_be<unsigned, 32, 8, 1, 23>(0x0080'0000);
    unsigned k = bitfield_swap_be<unsigned, 32, 8, 1, 23>(0x007f'ffff);
    printf("%08x %08x %08x\n", i, j, k);
  }
}

% clang bitfield_swap.cc -std=c++17
% ./a.out                          
ff000000 00800000 007fffff
000000ff 00000100 fffffe00
  • Then there could be a bitfield_swap<> member template that calls either of those based on file and host bitness, and that'd make it convenient to write a swapStruct() wrapper for each type with a bitfield
  • However, all likely host systems are little-endian too, so it also seems fine to just go back to the original code, add a comment, mark the test REQUIRES: host-byteorder-little-endian and do the rest later (or have someone who actually needs it do it)
BertalanD updated this revision to Diff 454844.Aug 23 2022, 7:57 AM

Added some extra checks. Addends are now printed as signed integers, as in Apple otool.

thakis accepted this revision.Aug 24 2022, 9:14 AM

Please land :)

llvm/lib/Object/MachOObjectFile.cpp
4996

(As discussed elsewhere, I think this is the worst of the approaches discussed above, but it's also not terribly important, so fine.)

This revision was automatically updated to reflect the committed changes.