This is an archive of the discontinued LLVM Phabricator instance.

[PPC64] V2 abi: Emit plt call stubs to the text section rather then the plt section.
ClosedPublic

Authored by sfertile on Apr 27 2018, 12:11 PM.

Details

Summary

In https://reviews.llvm.org/D45642 I was made aware that the .plt section is typically used for lazy symbol resolution stubs. Unfortunately we are writing the plt call stubs to that section for the PPC64 target. This patch implements plt call stubs as thunks. This enables them to be written to the proper section (.text rather then .plt) and frees up the .plt section for the lazy symbol resolvers.

Calls on PPC64 that potentially transfer control to a different linkage-unit need to be followed by a 'toc restore'. Previously we could determine a call needed a toc-restore by the R_PPC_PLT_OPD expr type. When redirecting to the thunks though we map that expr back to the non-plt expr (R_PPC_OPD). I've extended symbols with an extra field that records if a call to said symbol needs to be followed by a toc restore so we can differentiate R_PPC_OPD calls that need a toc-restore vs those that do not.

Diff Detail

Repository
rLLD LLVM Linker

Event Timeline

sfertile created this revision.Apr 27 2018, 12:11 PM
sfertile updated this revision to Diff 144759.May 1 2018, 11:23 AM

Rebased to top of trunk.

ruiu added inline comments.May 3 2018, 10:18 AM
ELF/Arch/PPC64.cpp
293

nit: I'd combine it to return Type == R_PPC64_REL24 && S.isInPlt();

ELF/Thunks.cpp
196

Could you expand the comment to explain what PPC64 PLT call stub is and why we need it?

511–512

Coding style.

562–563

if (Type == R_PPC64_REL24) return ... is perhaps better.

572–576

While you are here, could you remove else from these lines? We normally don't write else after return.

sfertile updated this revision to Diff 145249.May 4 2018, 12:19 PM

Rebased to pick up Zaaras patch that removed V1 abi support and addressed review comments.

sfertile marked 4 inline comments as done.May 4 2018, 12:21 PM

Hi Rui, thanks for the review. I think I've addressed all your comments.

ruiu accepted this revision.May 4 2018, 5:31 PM

LGTM

ELF/Thunks.cpp
197–203

Thank you for writing this!

502

Please remove the cast if it doesn't cause your compiler to emit a warning.

This revision is now accepted and ready to land.May 4 2018, 5:31 PM
grimar added a subscriber: grimar.May 5 2018, 1:56 AM
grimar added inline comments.
ELF/Thunks.cpp
579

btw, you do not need to return after llvm_unreachable

This revision was automatically updated to reflect the committed changes.
sfertile marked 2 inline comments as done.May 6 2018, 12:20 PM
sfertile added inline comments.
ELF/Thunks.cpp
197–203

Happy to :)

579

Thanks, I've updated this in the commit.

MaskRay added inline comments.
lld/trunk/ELF/InputSection.cpp
746 ↗(On Diff #145407)

If the file was compiled with gcc -Bsymbolic -fPIC, there may not be a nop after bl.
If the symbol also requires a PLT (its NeedsTocRestore will be true), this check will fail.

sfertile marked an inline comment as done.Aug 30 2018, 10:03 AM
sfertile added inline comments.
lld/trunk/ELF/InputSection.cpp
746 ↗(On Diff #145407)

Hi @MaskRay are you using -Bsymbolic in the linker options as well?

MaskRay added inline comments.Aug 30 2018, 9:30 PM
lld/trunk/ELF/InputSection.cpp
746 ↗(On Diff #145407)

I get a better example.

  1. Download https://packages.debian.org/jessie/ppc64el/libstdc++-4.9-dev/download
  2. Unpack with 7z then tar xf data.tar
  3. ar x ./usr/lib/gcc/powerpc64le-linux-gnu/4.9/libstdc++.a locale.o

llvm-objdump -d -r locale.o

_ZNKSt6locale2id5_M_idEv:
......
      b8:       01 00 00 48     bl .+0
                00000000000000b8:  R_PPC64_REL24        _ZNKSt6locale2id5_M_idEv
      bc:       01 00 23 39     addi 9, 3, 1     //////////// there is no nop here

If the symbol _ZNKSt6locale2id5_M_idEv has NeedsTocRestore set, this will fail to link, right?

I have checked newer versions, e.g. 5.5 6.4 6.8 they don't have such weird bl instruction. I don't know if this was a bug when producing locale.o

sfertile added inline comments.Aug 31 2018, 8:42 AM
lld/trunk/ELF/InputSection.cpp
746 ↗(On Diff #145407)

Awesome, I'll pull this down and have a closer look.

If the symbol _ZNKSt6locale2id5_M_idEv has NeedsTocRestore set, this will fail to link, right?

Yes it will, but if _ZNKSt6locale2id5_M_idEv is interposable then the missing nop is indeed a bug. For the error to be wrong I think we would need to see a situation where the target of the calll isn't in the plt but NeedsTocRestore was set.

What I observed in testing out various scenarios yesterday was that gold will emit a fatal error when there is no nop after a call to a function which doesn't have a definition in the shared object we are linking. If there is a definition there is no error even if the target of the call is interposable. In the situation where there is no nop but the target of the call is interposable the resulting shared-object is subtly broken. It will work fine as long as the dynamic loader resolves the target of the call to the definition local to the shared-object, but if it picks a definition from another shared-object or the executable then we can end up returning to this module with the wrong address in r2.

sfertile added inline comments.Aug 31 2018, 9:01 AM
lld/trunk/ELF/InputSection.cpp
746 ↗(On Diff #145407)

good_main.c:

int test(int);

int main(void) {
  return test(44);
}

bad_main.c:

int test(int);

int printf(const char*, ...);

void print(void) {
  printf("%s %d\n", __FILE__, __LINE__);
}

int main(void) {
  return test(44);
}

Here is a small test where we can assemble a.s and link it into a shared-object with gold without error, and it runs fine when used with good_main.c, but if used with bad_main.c we segfault on the lwa instruction following the call to print.

A bit more info here since the fact the call to _ZNKSt6locale2id5_M_idEv is a recursive call makes the situation a bit more muddy. I tested this with an older gcc version (5.4.0) and a newer version (7.3.1 from advanced toolchain 11). The newer compiler will emit a nop after a recursive call if the function is interposable, while the older compiler skips the nop. The linkers from the older toolchain (gold and bfd from binutils 2.26) both use plt stubs for the recursive call, and will rewrite the following nop to a toc-restore if the nop is there. I think lld behavior to emit a fatal error when trying to link _ZNKSt6locale2id5_M_idEv into a shared object is the correct one.

MaskRay added a comment.EditedSep 4 2018, 1:53 PM

A bit more info here since the fact the call to _ZNKSt6locale2id5_M_idEv is a recursive call makes the situation a bit more muddy. I tested this with an older gcc version (5.4.0) and a newer version (7.3.1 from advanced toolchain 11). The newer compiler will emit a nop after a recursive call if the function is interposable, while the older compiler skips the nop. The linkers from the older toolchain (gold and bfd from binutils 2.26) both use plt stubs for the recursive call, and will rewrite the following nop to a toc-restore if the nop is there. I think lld behavior to emit a fatal error when trying to link _ZNKSt6locale2id5_M_idEv into a shared object is the correct one.

The related TOC rewriting code in gold:

https://sourceware.org/git/?p=binutils-gdb.git;a=blob;hb=4e57b456393946cb4f90131b78e162cdec903c8a;f=gold/powerpc.cc#l9054

9069

	      // g++ as of 20130507 emits self-calls without a
	      // following nop.  This is arguably wrong since we have
	      // conflicting information.  On the one hand a global
	      // symbol and on the other a local call sequence, but
	      // don't error for this special case.
	      // It isn't possible to cheaply verify we have exactly
	      // such a call.  Allow all calls to the same section.

I need to learn more about the stuff...

Thanks for digging up that comment, its nice to know.

I need to learn more about the stuff...

Each module has its own value for the toc-pointer (I understand module is an overloaded term in compilers, here I am using it to mean the executable is a module and each shared object is a module). Anytime we potentially cross module boundaries the toc-pointer will need to be saved and later restored. From the static linkers perspective, any time the target of a call needs to be resolved by the dynamic linker the call should be followed by a toc-restore.