[PPC64] global dynamic to initial exec relaxation

[sfertile]

Summary

Patch adds support for relaxing the global-dynamic tls sequence to initial-exec tls sequence.

the relaxation performs the following transformation:
addis r3, r2, x@got@tlsgd@ha --> addis r3, r2, x@got@tprel@ha
addi r3, r3, x@got@tlsgd@l --> ld r3, x@got@tprel@l(r3)
bl __tls_get_addr(x@tlsgd) --> nop
nop --> add r3, r3, r13

and instead of emitting a DTPMOD64/DTPREL64 pair for x, we emit a single R_PPC64_TPREL64.

[ruiu]

Can you give me a pointer to the spec describing these relaxations?

ELF/Arch/PPC64.cpp

411

Add a blank line

413

nit:? remove {}

ELF/InputSection.cpp

783

Remove an extra space character at beginning of the sentence.

786

Wrong indentation.

ELF/Relocations.h

70

Did you really have to add these two new types? I'm not claiming that they are unnecessary, I'm just wondering, so could you please explain why the existing types don't work for you?

[sfertile]

In D48090#1130092, @ruiu wrote:

Can you give me a pointer to the spec describing these relaxations?

Yep, the abi has a section describing each of the tls relaxations: http://openpowerfoundation.org/wp-content/uploads/resources/leabi/content/dbdoclet.50655241_61284.html

ELF/Relocations.h

70

For R_RELAX_TLS_GD_TO_IE_GOT_OFF I don't have to add it. I could use the _ABS version and subtract the TocBase from the result in the PPC target code. Either way I have to adjust the value passed to relaxTlsGdToIe (by the TocOffset if passed the GotOffset and by TocBase if passed the syms VA) so there isn't much diffrence between them. I picked adding the new RelExpr simply for clarity since the instructions in the relaxed sequence use a got-offset and not the syms VA.

For R_TLSGD_HINT I think I can actually use R_TLSDESC_CALL instead. When I initially looked at R_TLSDESC_CALL I thought there was something it generated that was different from what I needed, but going over it again it looks like its a perfect fit.

[echristo]

In D48090#1131204, @sfertile wrote:

In D48090#1130092, @ruiu wrote:

Can you give me a pointer to the spec describing these relaxations?

Yep, the abi has a section describing each of the tls relaxations: http://openpowerfoundation.org/wp-content/uploads/resources/leabi/content/dbdoclet.50655241_61284.html

Relatedly can you expand on the comments with a reference to the part of the ABI with the associated relaxation?

-eric

[sfertile]

Updated formatting, removed the 2 new RelExprs since they aren't needed, expanded comments on the relaxation specifics.

I also fixed an issue where I had assumed the input register to the load was r3, which isn't necessary. Any register could be used for the intermediate value.

[syzaara]

ELF/Arch/PPC64.cpp

440

I think it would be more clear if we called relocateOne with the replacement relocation here.

[ruiu]

ELF/Arch/PPC64.cpp

421–425

I wonder if this comment actually explains what this function is doing. This function handles three different relocation types, but this comment seems to cover only one type.

446

Perhaps 0x7fff0000 is easier to understand than 31<<16.

452

Wrong indentation -- can you always apply clang-format?

[sfertile]

ELF/Arch/PPC64.cpp

421–425

Part 1 is the addis/addi sequence which covers the R_PPC64_GOT_TLSGD16_HA and R_PPC64_GOT_TLSGD16_LO relocations on the first 2 instructions. Even though we handle them separately and they won't nessicariily be adjacent in the source it really only makes sense to talk about the pair together.

Part 2 and 3 cover the remaining R_PPC64_TLSGD relocation which affects the last 2 instructions in the sequence.
(A call that crosses dso boundaries has to be followed by a nop on PPC which is why we can handle both instructions with a single hint relocation).

440

Now I wish I hadn't jumped the gun on abandoning the R_RELAX_TLS_GD_TO_IE_GOT_OFF RelExpr :).

I like your idea relocateOne(Loc, R_PPC64_TPREL16_HA/LO_DS, Val) makes it clear that the relocation is handled the same as a normal TPREL16. Since we are getting the syms VA though, it would have to be relocateOne(Loc, R_PPC64_TPREL16_HA/LO_DS, Val - InX::got->getVA()) which I think kills the clarity ...

I think we should pick one of the following:

1. Use a RelExpr of R_RELAX_TLS_GD_TO_IE_GOT_OFF so we get just the got offset of the symbol, then call relocateOne which handles adjusting by the toc-offset and performing the relocation.

2. Stick to the R_RELAX_TLS_GD_TO_IE_GOT_ABS RelExpr, adjust by the TocBase and relocate here (as the patch is currently doing).

I would much prefer 1, as long as everyone is ok with adding the new RelExpr.

446

I picked this format to try to convey its a 5-bit mask and shifted over to bits 16-21. When I'm reading if I see 0x1F0000 I'll typically have to stop and count the 0's to know where the mask is. Does 0x1f << 16 work for you?

452

yes, sorry about that. I've been lazy with clang-format for a while because my vim commands for it were broken. I've fixed them and will format the rest of this patch.

[sfertile]

• clang formatted patch
• Switched the relax RelExpr to one that calculates a got-offset.
• used relocateOne to make it clearer what we were relaxing to.
• updated register mask to use hex instead of decimal.

[ruiu]

LGTM

ELF/Arch/PPC64.cpp

RelExpr getRelExpr(RelType Type, const Symbol &S,
                      const uint8_t *Loc) const override;
   void writePltHeader(uint8_t *Buf) const override;
   void writePlt(uint8_t *Buf, uint64_t GotPltEntryAddr, uint64_t PltEntryAddr,
                 int32_t Index, unsigned RelOff) const override;
   void relocateOne(uint8_t *Loc, RelType Type, uint64_t Val) const override;
   void writeGotHeader(uint8_t *Buf) const override;
   bool needsThunk(RelExpr Expr, RelType Type, const InputFile *File,
                   uint64_t BranchAddr, const Symbol &S) const override;
+  RelExpr adjustRelaxExpr(RelType Type, const uint8_t *Data,
+                          RelExpr Expr) const override;
+  void relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const override;
 };
 } // namespace
 
 // Relocation masks following the #lo(value), #hi(value), #ha(value),
 // #higher(value), #highera(value), #highest(value), and #highesta(value)
 // macros defined in section 4.5.1. Relocation Types of the PPC-elf64abi
 // document.
 static uint16_t lo(uint64_t V) { return V; }
RelExpr PPC64::getRelExpr(RelType Type, const Symbol &S,
   case R_PPC64_DTPREL16_HIGHERA:
   case R_PPC64_DTPREL16_HIGHEST:
   case R_PPC64_DTPREL16_HIGHESTA:
   case R_PPC64_DTPREL16_LO:
   case R_PPC64_DTPREL16_LO_DS:
   case R_PPC64_DTPREL64:
     return R_ABS;
   case R_PPC64_TLSGD:
+    return R_TLSDESC_CALL;
   case R_PPC64_TLSLD:
   case R_PPC64_TLS:
     return R_HINT;
   default:
     return R_ABS;
   }
 }
 

 }
 
 bool PPC64::needsThunk(RelExpr Expr, RelType Type, const InputFile *File,
                        uint64_t BranchAddr, const Symbol &S) const {
   // If a function is in the plt it needs to be called through
   // a call stub.
   return Type == R_PPC64_REL24 && S.isInPlt();
 }
 
+RelExpr PPC64::adjustRelaxExpr(RelType Type, const uint8_t *Data,
+                               RelExpr Expr) const {
+  if (Expr == R_RELAX_TLS_GD_TO_IE)
+    return R_RELAX_TLS_GD_TO_IE_GOT_OFF;
+  return Expr;
+}
+
+// Reference: 3.7.4.1 of the 64-bit ELF V2 abi supplement.
+// The general dynamic code sequence for a global `x` uses 4 instructions.
+// Instruction                    Relocation                Symbol
+// addis r3, r2, x@got@tlsgd@ha   R_PPC64_GOT_TLSGD16_HA      x
+// addi  r3, r3, x@got@tlsgd@l    R_PPC64_GOT_TLSGD16_LO      x
+// bl __tls_get_addr(x@tlsgd)     R_PPC64_TLSGD               x
+//                                R_PPC64_REL24               __tls_get_addr
+// nop                            None                       None
+//
+// Relaxing to initial-exec entails:
+// 1) Convert the addis/addi pair that builds the address of the tls_index
+//    struct for 'x' to an addis/ld pair that loads an offset from a got-entry.
+// 2) Convert the call to __tls_get_addr to a nop.
+// 3) Convert the nop following the call to an add of the loaded offset to the
+//    thread pointer.
+// Since the nop must directly follow the call, the R_PPC64_TLSGD relocation is
+// used as the relaxation hint for both steps 2 and 3.
+void PPC64::relaxTlsGdToIe(uint8_t *Loc, RelType Type, uint64_t Val) const {
+  switch (Type) {
+  case R_PPC64_GOT_TLSGD16_HA:
+    // This is relaxed from addis rT, r2, sym@got@tlsgd@ha to
+    //                      addis rT, r2, sym@got@tprel@ha.
+    relocateOne(Loc, R_PPC64_GOT_TPREL16_HA, Val);
+    return;
+  case R_PPC64_GOT_TLSGD16_LO: {
+    // Relax from addi  r3, rA, sym@got@tlsgd@l to
+    //            ld r3, sym@got@tprel@l(rA)
+    uint32_t EndianOffset = Config->EKind == ELF64BEKind ? 2U : 0U;
+    uint32_t InputRegister = (read32(Loc - EndianOffset) & (0x1f << 16));
+    write32(Loc - EndianOffset, 0xE8600000 | InputRegister);
+    relocateOne(Loc, R_PPC64_GOT_TPREL16_LO_DS, Val);
+    return;
+  }
+  case R_PPC64_TLSGD:
+    write32(Loc, 0x60000000);     // bl __tls_get_addr(sym@tlsgd) --> nop
+    write32(Loc + 4, 0x7c636A14); // nop --> add r3, r3, r13
+    return;
+  default:
+    llvm_unreachable("unsupported relocation for TLS GD to IE relaxation");
+  }
+}
+
 TargetInfo *elf::getPPC64TargetInfo() {
   static PPC64 Target;
   return &Target;
 }

ELF/InputSection.cpp

static uint64_t getRelocTargetVA(const InputFile *File, RelType Type, int64_t A,
   case R_GOTREL:
     return Sym.getVA(A) - InX::Got->getVA();
   case R_GOTREL_FROM_END:
     return Sym.getVA(A) - InX::Got->getVA() - InX::Got->getSize();
   case R_GOT_FROM_END:
   case R_RELAX_TLS_GD_TO_IE_END:
     return Sym.getGotOffset() + A - InX::Got->getSize();
   case R_GOT_OFF:
+  case R_RELAX_TLS_GD_TO_IE_GOT_OFF:
     return Sym.getGotOffset() + A;
   case R_GOT_PAGE_PC:
   case R_RELAX_TLS_GD_TO_IE_PAGE_PC:
     return getAArch64Page(Sym.getGotVA() + A) - getAArch64Page(P);
   case R_GOT_PC:
   case R_RELAX_TLS_GD_TO_IE:
     return Sym.getGotVA() + A - P;
   case R_HINT:
case R_RELAX_TLS_LD_TO_LE:
       Target->relaxTlsLdToLe(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_GD_TO_LE:
     case R_RELAX_TLS_GD_TO_LE_NEG:
       Target->relaxTlsGdToLe(BufLoc, Type, TargetVA);
       break;
     case R_RELAX_TLS_GD_TO_IE:
     case R_RELAX_TLS_GD_TO_IE_ABS:
+    case R_RELAX_TLS_GD_TO_IE_GOT_OFF:
     case R_RELAX_TLS_GD_TO_IE_PAGE_PC:
     case R_RELAX_TLS_GD_TO_IE_END:
       Target->relaxTlsGdToIe(BufLoc, Type, TargetVA);
       break;
     case R_PPC_CALL:
+      // If this is a call to __tls_get_addr, it may be part of a TLS
+      // sequence that has been relaxed and turned into a nop. In this
+      // case, we don't want to handle it as a call.
+      if (read32(BufLoc) == 0x60000000) // nop
+        break;
+
       // Patch a nop (0x60000000) to a ld.
       if (Rel.Sym->NeedsTocRestore) {
         if (BufLoc + 8 > BufEnd || read32(BufLoc + 4) != 0x60000000) {
           error(getErrorLocation(BufLoc) + "call lacks nop, can't restore toc");
           break;
         }
         write32(BufLoc + 4, 0xe8410018); // ld %r2, 24(%r1)
       }

ELF/Relocations.h

enum RelExpr {
   R_PPC_CALL,
   R_PPC_CALL_PLT,
   R_PPC_TOC,
   R_RELAX_GOT_PC,
   R_RELAX_GOT_PC_NOPIC,
   R_RELAX_TLS_GD_TO_IE,
   R_RELAX_TLS_GD_TO_IE_ABS,
   R_RELAX_TLS_GD_TO_IE_END,
+  R_RELAX_TLS_GD_TO_IE_GOT_OFF,
   R_RELAX_TLS_GD_TO_IE_PAGE_PC,
   R_RELAX_TLS_GD_TO_LE,
   R_RELAX_TLS_GD_TO_LE_NEG,
   R_RELAX_TLS_IE_TO_LE,
   R_RELAX_TLS_LD_TO_LE,
   R_SIZE,
   R_TLS,
   R_TLSDESC,

test/ELF/ppc64-gd-to-ie.s

+# REQUIRES: ppc
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64le-unknown-linux %p/Inputs/ppc64-tls.s -o %t2.o
+# RUN: ld.lld -shared %t2.o -o %t3.so
+# RUN: ld.lld  %t.o %t3.so -o %t
+# RUN: llvm-objdump --section-headers %t | FileCheck --check-prefix=CheckGot %s
+# RUN: llvm-objdump -D %t | FileCheck --check-prefix=Dis %s
+# RUN: llvm-readelf -relocations --wide %t | FileCheck --check-prefix=OutputRelocs %s
+
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=powerpc64-unknown-linux %p/Inputs/ppc64-tls.s -o %t2.o
+# RUN: ld.lld -shared %t2.o -o %t3.so
+# RUN: ld.lld  %t.o %t3.so -o %t
+# RUN: llvm-objdump --section-headers %t | FileCheck --check-prefix=CheckGot %s
+# RUN: llvm-objdump -D %t | FileCheck --check-prefix=Dis %s
+# RUN: llvm-readelf -relocations --wide %t | FileCheck --check-prefix=OutputRelocs %s
+
+        .text
+        .abiversion 2
+        .globl _start
+        .p2align        4
+        .type   _start,@function
+_start:
+.Lfunc_gep0:
+        addis 2, 12, .TOC.-.Lfunc_gep0@ha
+        addi 2, 2, .TOC.-.Lfunc_gep0@l
+.Lfunc_lep0:
+        .localentry     _start, .Lfunc_lep0-.Lfunc_gep0
+        mflr 0
+        std 0, 16(1)
+        stdu 1, -32(1)
+        addis 3, 2, a@got@tlsgd@ha
+        addi 3, 3, a@got@tlsgd@l
+        bl __tls_get_addr(a@tlsgd)
+        nop
+        lwa 3, 0(3)
+        addi 1, 1, 32
+        ld 0, 16(1)
+        mtlr 0
+        blr
+
+
+        .globl other_reg
+        .p2align        4
+        .type   other_reg,@function
+other_reg:
+.Lfunc_gep1:
+        addis 2, 12, .TOC.-.Lfunc_gep1@ha
+        addi 2, 2, .TOC.-.Lfunc_gep1@l
+.Lfunc_lep1:
+        .localentry     other_reg, .Lfunc_lep1-.Lfunc_gep1
+        mflr 0
+        std 0, 16(1)
+        stdu 1, -32(1)
+        addis 5, 2, a@got@tlsgd@ha
+        addi 3, 5, a@got@tlsgd@l
+        bl __tls_get_addr(a@tlsgd)
+        nop
+        lwa 4, 0(3)
+        addis 30, 2, b@got@tlsgd@ha
+        addi 3, 30, b@got@tlsgd@l
+        bl __tls_get_addr(b@tlsgd)
+        nop
+        lwa 3, 0(3)
+        add 3, 4, 3
+        addi 1, 1, 32
+        ld 0, 16(1)
+        mtlr 0
+        blr
+
+        .globl __tls_get_addr
+        .type __tls_get_addr,@function
+__tls_get_addr:
+
+
+# CheckGot: .got          00000018 00000000100200c0 DATA
+# .got is at 0x100200c0 so the toc-base is 100280c0.
+# `a` is at .got[1], we expect the offsets to be:
+# Ha(a) = ((0x100200c8  - 0x100280c0) + 0x8000) >> 16 = 0
+# Lo(a) = (0x100200c8  - 0x100280c0) = -32760
+
+# Dis-LABEL: _start
+# Dis:         addis 3, 2, 0
+# Dis-NEXT:    ld 3, -32760(3)
+# Dis-NEXT:    nop
+# Dis-NEXT:    add 3, 3, 13
+
+# Dis-LABEL: other_reg
+# Dis:         addis 5, 2, 0
+# Dis-NEXT:    ld 3, -32760(5)
+# Dis-NEXT:    nop
+# Dis-NEXT:    add 3, 3, 13
+# Dis:         addis 30, 2, 0
+# Dis:         ld 3, -32752(30)
+# Dis-NEXT:    nop
+# Dis-NEXT:    add 3, 3, 13
+
+# Verify that the only dynamic relocations we emit are TPREL ones rather then
+# the DTPMOD64/DTPREL64 pair for general-dynamic.
+# OutputRelocs: Relocation section '.rela.dyn' at offset 0x{{[0-9a-f]+}} contains 2 entries:
+# OutputRelocs-NEXT:    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
+# OutputRelocs-NEXT:  {{[0-9a-f]+}}    {{[0-9a-f]+}}   R_PPC64_TPREL64        {{[0-9a-f]+}} a + 0
+# OutputRelocs-NEXT:  {{[0-9a-f]+}}    {{[0-9a-f]+}}   R_PPC64_TPREL64        {{[0-9a-f]+}} b + 0