[PowerPC] Replace foul hackery with real calls to __tls_get_addr

[wschmidt]

Summary

My original support for the general dynamic and local dynamic TLS models contained some fairly obtuse hacks to generate calls to __tls_get-addr when lowering a TargetGlobalTLSAddress. Rather than generating real calls, special GET_TLS_ADDR nodes were used to wrap the calls and only reveal them at assembly time. I attempted to provide correct parameter and return values by chaining CopyToReg and CopyFromReg nodes onto the GET_TLS_ADDR nodes, but this was also not fully correct. Problems were seen with two back-to-back stores to TLS variables, where the call sequences ended up overlapping, with unhappy results. Additionally, since there weren't real calls, the proper register side effects of a call were not recorded, so clobbered values were kept live across the calls.

The proper thing to do is to lower these into calls in the first place. This is relatively straightforward; see the changes to PPCTargetLowering::LowerGlobalTLSAddress(). The changes here are standard call lowering, except that we need to track the fact that these calls will require a relocation. This is done by adding a machine operand flag of MO_TLSLD or MO_TLSGD to the TargetGlobalAddress operand that appears earlier in the sequence.

The calls to LowerCallTo() eventually find their way to LowerCall_64SVR4() or LowerCall_32SVR4(), which call FinishCall(), which calls PrepareCall(). In PrepareCall(), we detect the calls to __tls_get_addr and immediately snag the TargetGlobalTLSAddress with the annotated relocation information. This becomes an extra operand on the call following the callee, which is expected for nodes of type tlscall. We change the call opcode to CALL_TLS for this case. Back in FinishCall(), we change it again to CALL_NOP_TLS for 64-bit only, since we require a TOC-restore nop following the call for the 64-bit ABIs.

During selection, patterns in PPCInstrInfo.td and PPCInstr64Bit.td convert the CALL_TLS nodes into BL_TLS nodes, and convert the CALL_NOP_TLS nodes into BL8_NOP_TLS nodes. This replaces the code removed from PPCAsmPrinter.cpp, as the BL_TLS and BL8_NOP_TLS nodes can now be emitted normally using their patterns and the associated printTLSCall print method.

Note that this removed code includes some special handling for 32-bit SVR4 PIC (vs. pic) code, to ensure the proper relocation is placed on the __tls_get_addr symbol. I believe the same effect is produced by the code I added in PPCTargetLowering::LowerGlobalTLSAddress() that places the PPCII::MO_PLT_OR_STUB flag on the symbol. I don't know whether or not this is working, because Justin apparently didn't add any test cases for this when he put that code in place. /fingerwag

>>> Justin, can you please add a test case for this so I can verify that I haven't broken your code? <<<

Finally, as a result of these changes, all references to get-tls-addr in its various guises are no longer used, so they have been removed.

There are existing TLS tests to verify that the changes haven't messed anything up (other than the 32-bit PIC stuff). I've added one new test that verifies that the problem with the original code has been fixed.

[jhibbits]

Tests for 32-bit are already included, as part of tls-pic.ll.

[• rafael]

lib/Target/PowerPC/PPCISelLowering.cpp

3726

This patch is nice, but this part looks a bit too clever. Could you pass TGTAddr down the callchain maybe?

[hfinkel]

lib/Target/PowerPC/PPCISelLowering.cpp

782–784

Please add CALL_TLS, CALL_NOP_TLS here.

1770

This block of code seems the same as the one added above. Can you please refactor this?

[wschmidt]

In D6209#5, @jhibbits wrote:

Tests for 32-bit are already included, as part of tls-pic.ll.

Yes, I saw those. However, what I'm referring to is the use of VK_PLT in the PPCAsmPrinter::EmitInstruction() code, the purpose of which is to ensure that ELF::R_PPC_PLTREL24 is applied to the __tls_get_addr symbol rather than ELF::R_PPC_REL24. You don't have a relocation test for this. See test/MC/PowerPC/tls-gd-obj.s for an example of how to do this.

[wschmidt]

Though using llvm-mc followed by llvm-readobj isn't really sufficient here, since we need to test a longer path. Something like llc with integrated-asm producing a .o and then using llvm-readobj. I haven't tried to write a test like that.

[jhibbits]

In D6209#10, @wschmidt wrote:

In D6209#5, @jhibbits wrote:

Tests for 32-bit are already included, as part of tls-pic.ll.

Yes, I saw those. However, what I'm referring to is the use of VK_PLT in the PPCAsmPrinter::EmitInstruction() code, the purpose of which is to ensure that ELF::R_PPC_PLTREL24 is applied to the __tls_get_addr symbol rather than ELF::R_PPC_REL24. You don't have a relocation test for this. See test/MC/PowerPC/tls-gd-obj.s for an example of how to do this.

I don't follow. VK_PLT is a decorator for @PLT, and I already have tests for ensuring @PLT becomes R_PPC_PLTREL24 (ppc-reloc.s). Isn't a sufficient test for this simply to ensure that @PLT is generated when that code path is followed? Do you instead expect that the inline assembler will recognize a call to __tls_get_addr() and itself add VK_PLT? The other relocations for PLT are identical to those of powerpc64 in this case. Okay, maybe a test to generate R_PPC_GOT_TLSLD/R_PPC_GOT_TLSGD instead of R_PPC_GOT_TLSLD_LO/R_PPC_GOT_TLSGD_LO might be prudent, and I can readily add, but that's not what you requested.

[wschmidt]

I don't follow. VK_PLT is a decorator for @PLT, and I already have tests for ensuring @PLT becomes R_PPC_PLTREL24 (ppc-reloc.s). Isn't a sufficient test for this simply to ensure that @PLT is generated when that code path is followed? Do you instead expect that the inline assembler will recognize a call to __tls_get_addr() and itself add VK_PLT? The other relocations for PLT are identical to those of powerpc64 in this case. Okay, maybe a test to generate R_PPC_GOT_TLSLD/R_PPC_GOT_TLSGD instead of R_PPC_GOT_TLSLD_LO/R_PPC_GOT_TLSGD_LO might be prudent, and I can readily add, but that's not what you requested.

Normally, @PLT is how you get R_PPC_PLTREL24, but that is apparently not the case here. You added some special-case code in PPCAsmPrinter.cpp that specifically places the VK_PLT on the __tls_get_addr symbol ONLY, which is never emitted with @plt. (It's in the deleted chunks in this patch.) This appears to be a way of forcing the relocation that you want, and this path is not tested in your test cases.

From what I have been able to piece together, I believe that placing MO_PLT_OR_STUB on the TGA operand early will eventually cause VK_PLT to be used, and hence generate the correct relocation. I can't tell whether this regresses your code or not, though, because there wasn't a test.

The way I know there isn't a test is that I only discovered your code when I was inspecting the patch. I deleted your VK_PLT workaround with impunity and none of the tests failed. When I spotted that I added the MO_PLT_OR_STUB code.

lib/Target/PowerPC/PPCISelLowering.cpp

782–784

Good catch, thanks!

1770

Sure, good idea.

3726

I couldn't think of a way to pass it down directly -- I'd certainly welcome a suggestion, as I agree with your assessment. Hacking it onto the Args list seemed like a worse idea. I can't really do more with the chain argument because it's used by the call-lowering code. Thoughts?

At least the present solution is guaranteed to work because we're just looking at the sequence generated by the call-lowering, which is still in progress. I.e., no optimizations or anything can get in the way.

[jhibbits]

In D6209#13, @wschmidt wrote:

I don't follow. VK_PLT is a decorator for @PLT, and I already have tests for ensuring @PLT becomes R_PPC_PLTREL24 (ppc-reloc.s). Isn't a sufficient test for this simply to ensure that @PLT is generated when that code path is followed? Do you instead expect that the inline assembler will recognize a call to __tls_get_addr() and itself add VK_PLT? The other relocations for PLT are identical to those of powerpc64 in this case. Okay, maybe a test to generate R_PPC_GOT_TLSLD/R_PPC_GOT_TLSGD instead of R_PPC_GOT_TLSLD_LO/R_PPC_GOT_TLSGD_LO might be prudent, and I can readily add, but that's not what you requested.

Normally, @PLT is how you get R_PPC_PLTREL24, but that is apparently not the case here. You added some special-case code in PPCAsmPrinter.cpp that specifically places the VK_PLT on the __tls_get_addr symbol ONLY, which is never emitted with @plt. (It's in the deleted chunks in this patch.) This appears to be a way of forcing the relocation that you want, and this path is not tested in your test cases.

From what I have been able to piece together, I believe that placing MO_PLT_OR_STUB on the TGA operand early will eventually cause VK_PLT to be used, and hence generate the correct relocation. I can't tell whether this regresses your code or not, though, because there wasn't a test.

The way I know there isn't a test is that I only discovered your code when I was inspecting the patch. I deleted your VK_PLT workaround with impunity and none of the tests failed. When I spotted that I added the MO_PLT_OR_STUB code.

There are two reasons for this: I took the path of least code change, where you were already using VK_None, on the MCSymbolExpr, and the call was already lowered, we were constructing the branch by hand. The VK_PLT normally gets added in GetSymbolRef() in PPCMCInstLower.cpp. An alternative route, in hindsight, could have been to lower it after constructing the call, but again that's hindsight, and with your change becomes irrelevant. Without explicitly adding the VK_PLT, it was not generating the @PLT you see in tls-pic.ll (I checked). So, really, it was only a case of where in the path it was added, and in this case the flag was added late, so the kind was needed.

Since you already did a code inspection, you may have seen that in PrepareCall() PPCII::MO_PLT_OR_STUB gets added automatically, making your addition unnecessary, so I think you can drop that from the patch (if it already passes the tests, then it works).

[wschmidt]

In D6209#14, @jhibbits wrote:

In D6209#13, @wschmidt wrote:

I don't follow. VK_PLT is a decorator for @PLT, and I already have tests for ensuring @PLT becomes R_PPC_PLTREL24 (ppc-reloc.s). Isn't a sufficient test for this simply to ensure that @PLT is generated when that code path is followed? Do you instead expect that the inline assembler will recognize a call to __tls_get_addr() and itself add VK_PLT? The other relocations for PLT are identical to those of powerpc64 in this case. Okay, maybe a test to generate R_PPC_GOT_TLSLD/R_PPC_GOT_TLSGD instead of R_PPC_GOT_TLSLD_LO/R_PPC_GOT_TLSGD_LO might be prudent, and I can readily add, but that's not what you requested.

Normally, @PLT is how you get R_PPC_PLTREL24, but that is apparently not the case here. You added some special-case code in PPCAsmPrinter.cpp that specifically places the VK_PLT on the __tls_get_addr symbol ONLY, which is never emitted with @plt. (It's in the deleted chunks in this patch.) This appears to be a way of forcing the relocation that you want, and this path is not tested in your test cases.

From what I have been able to piece together, I believe that placing MO_PLT_OR_STUB on the TGA operand early will eventually cause VK_PLT to be used, and hence generate the correct relocation. I can't tell whether this regresses your code or not, though, because there wasn't a test.

The way I know there isn't a test is that I only discovered your code when I was inspecting the patch. I deleted your VK_PLT workaround with impunity and none of the tests failed. When I spotted that I added the MO_PLT_OR_STUB code.

There are two reasons for this: I took the path of least code change, where you were already using VK_None, on the MCSymbolExpr, and the call was already lowered, we were constructing the branch by hand. The VK_PLT normally gets added in GetSymbolRef() in PPCMCInstLower.cpp. An alternative route, in hindsight, could have been to lower it after constructing the call, but again that's hindsight, and with your change becomes irrelevant. Without explicitly adding the VK_PLT, it was not generating the @PLT you see in tls-pic.ll (I checked). So, really, it was only a case of where in the path it was added, and in this case the flag was added late, so the kind was needed.

Since you already did a code inspection, you may have seen that in PrepareCall() PPCII::MO_PLT_OR_STUB gets added automatically, making your addition unnecessary, so I think you can drop that from the patch (if it already passes the tests, then it works).

Mea culpa -- I missed the @PLT examples in the tls-pic.ll test. I misunderstood what was going on with the extra code there, and agree that I can remove the extra code (which makes things much cleaner, yay). Thanks for educating me, and putting up with the nonsense. ;) This stuff is so dispersed throughout the code that it's hard to keep all the threads together.

[wschmidt]

This version addresses the comments I've received, with the exception of the one from Rafael. I looked at this again and still couldn't think of a good way to pass the TGA through LowerCallTo. I'm willing to do this if anyone has a good idea, but for now this is what I've got.

[hfinkel]

LGTM.

[wschmidt]

r221703, thanks!

Again, apologies to Justin for misunderstanding his test cases.

lib/Target/PowerPC/PPC.h

enum TOF {
 
     /// These values identify relocations on immediates folded
     /// into memory operations.
     MO_DTPREL_LO = 5 << 4,
     MO_TLSLD_LO  = 6 << 4,
     MO_TOC_LO    = 7 << 4,
 
     // Symbol for VK_PPC_TLS fixup attached to an ADD instruction
-    MO_TLS       = 8 << 4
+    MO_TLS       = 8 << 4,
+
+    // Symbols for VK_PPC_TLSGD and VK_PPC_TLSLD in __tls_get_addr
+    // call sequences.
+    MO_TLSLD     = 9 << 4,
+    MO_TLSGD     = 10 << 4
   };
   } // end namespace PPCII
   
 } // end namespace llvm;
 
 #endif

lib/Target/PowerPC/PPCAsmPrinter.cpp

const MCExpr *SymGotTlsGD =
                               OutContext);
     EmitToStreamer(OutStreamer,
                    MCInstBuilder(Subtarget.isPPC64() ? PPC::ADDI8 : PPC::ADDI)
                    .addReg(MI->getOperand(0).getReg())
                    .addReg(MI->getOperand(1).getReg())
                    .addExpr(SymGotTlsGD));
     return;
   }
-  case PPC::GETtlsADDR:
-    // Transform: %X3 = GETtlsADDR %X3, <ga:@sym>
-    // Into:      BL8_NOP_TLS __tls_get_addr(sym@tlsgd)
-  case PPC::GETtlsADDR32: {
-    // Transform: %R3 = GETtlsADDR32 %R3, <ga:@sym>
-    // Into:      BL_TLS __tls_get_addr(sym@tlsgd)@PLT
-
-    StringRef Name = "__tls_get_addr";
-    MCSymbol *TlsGetAddr = OutContext.GetOrCreateSymbol(Name);
-    MCSymbolRefExpr::VariantKind Kind = MCSymbolRefExpr::VK_None;
-
-    if (!Subtarget.isPPC64() && !Subtarget.isDarwin() &&
-        TM.getRelocationModel() == Reloc::PIC_)
-      Kind = MCSymbolRefExpr::VK_PLT;
-    const MCSymbolRefExpr *TlsRef = 
-      MCSymbolRefExpr::Create(TlsGetAddr, Kind, OutContext);
-    const MachineOperand &MO = MI->getOperand(2);
-    const GlobalValue *GValue = MO.getGlobal();
-    MCSymbol *MOSymbol = getSymbol(GValue);
-    const MCExpr *SymVar =
-      MCSymbolRefExpr::Create(MOSymbol, MCSymbolRefExpr::VK_PPC_TLSGD,
-                              OutContext);
-    EmitToStreamer(OutStreamer,
-                   MCInstBuilder(Subtarget.isPPC64() ?
-                                  PPC::BL8_NOP_TLS : PPC::BL_TLS)
-                   .addExpr(TlsRef)
-                   .addExpr(SymVar));
-    return;
-  }
   case PPC::ADDIStlsldHA: {
     // Transform: %Xd = ADDIStlsldHA %X2, <ga:@sym>
     // Into:      %Xd = ADDIS8 %X2, sym@got@tlsld@ha
     assert(Subtarget.isPPC64() && "Not supported for 32-bit PowerPC");
     const MachineOperand &MO = MI->getOperand(2);
     const GlobalValue *GValue = MO.getGlobal();
     MCSymbol *MOSymbol = getSymbol(GValue);
     const MCExpr *SymGotTlsLD =
const MCExpr *SymGotTlsLD =
                               OutContext);
     EmitToStreamer(OutStreamer,
                    MCInstBuilder(Subtarget.isPPC64() ? PPC::ADDI8 : PPC::ADDI)
                    .addReg(MI->getOperand(0).getReg())
                    .addReg(MI->getOperand(1).getReg())
                    .addExpr(SymGotTlsLD));
     return;
   }
-  case PPC::GETtlsldADDR:
-    // Transform: %X3 = GETtlsldADDR %X3, <ga:@sym>
-    // Into:      BL8_NOP_TLS __tls_get_addr(sym@tlsld)
-  case PPC::GETtlsldADDR32: {
-    // Transform: %R3 = GETtlsldADDR32 %R3, <ga:@sym>
-    // Into:      BL_TLS __tls_get_addr(sym@tlsld)@PLT
-
-    StringRef Name = "__tls_get_addr";
-    MCSymbol *TlsGetAddr = OutContext.GetOrCreateSymbol(Name);
-    MCSymbolRefExpr::VariantKind Kind = MCSymbolRefExpr::VK_None;
-
-    if (!Subtarget.isPPC64() && !Subtarget.isDarwin() &&
-        TM.getRelocationModel() == Reloc::PIC_)
-      Kind = MCSymbolRefExpr::VK_PLT;
-
-    const MCSymbolRefExpr *TlsRef = 
-      MCSymbolRefExpr::Create(TlsGetAddr, Kind, OutContext);
-    const MachineOperand &MO = MI->getOperand(2);
-    const GlobalValue *GValue = MO.getGlobal();
-    MCSymbol *MOSymbol = getSymbol(GValue);
-    const MCExpr *SymVar =
-      MCSymbolRefExpr::Create(MOSymbol, MCSymbolRefExpr::VK_PPC_TLSLD,
-                              OutContext);
-    EmitToStreamer(OutStreamer,
-                   MCInstBuilder(Subtarget.isPPC64() ?
-                                  PPC::BL8_NOP_TLS : PPC::BL_TLS)
-                   .addExpr(TlsRef)
-                   .addExpr(SymVar));
-    return;
-  }
   case PPC::ADDISdtprelHA:
     // Transform: %Xd = ADDISdtprelHA %X3, <ga:@sym>
     // Into:      %Xd = ADDIS8 %X3, sym@dtprel@ha
   case PPC::ADDISdtprelHA32: {
     // Transform: %Rd = ADDISdtprelHA32 %R3, <ga:@sym>
     // Into:      %Rd = ADDIS %R3, sym@dtprel@ha
     const MachineOperand &MO = MI->getOperand(2);
     const GlobalValue *GValue = MO.getGlobal();

lib/Target/PowerPC/PPCISelLowering.h

enum NodeType {
       /// code.
       SRL, SRA, SHL,
 
       /// CALL - A direct function call.
       /// CALL_NOP is a call with the special NOP which follows 64-bit
       /// SVR4 calls.
       CALL, CALL_NOP,
 
+      /// CALL_TLS and CALL_NOP_TLS - Versions of CALL and CALL_NOP used
+      /// to access TLS variables.
+      CALL_TLS, CALL_NOP_TLS,
+
       /// CHAIN,FLAG = MTCTR(VAL, CHAIN[, INFLAG]) - Directly corresponds to a
       /// MTCTR instruction.
       MTCTR,
 
       /// CHAIN,FLAG = BCTRL(CHAIN, INFLAG) - Directly corresponds to a
       /// BCTRL instruction.
       BCTRL,
 
enum NodeType {
       /// register to sym\@got\@tlsgd\@ha.
       ADDIS_TLSGD_HA,
 
       /// G8RC = ADDI_TLSGD_L G8RReg, Symbol - For the general-dynamic TLS
       /// model, produces an ADDI8 instruction that adds G8RReg to
       /// sym\@got\@tlsgd\@l.
       ADDI_TLSGD_L,
 
-      /// G8RC = GET_TLS_ADDR %X3, Symbol - For the general-dynamic TLS
-      /// model, produces a call to __tls_get_addr(sym\@tlsgd).
-      GET_TLS_ADDR,
-
       /// G8RC = ADDIS_TLSLD_HA %X2, Symbol - For the local-dynamic TLS
       /// model, produces an ADDIS8 instruction that adds the GOT base
       /// register to sym\@got\@tlsld\@ha.
       ADDIS_TLSLD_HA,
 
       /// G8RC = ADDI_TLSLD_L G8RReg, Symbol - For the local-dynamic TLS
       /// model, produces an ADDI8 instruction that adds G8RReg to
       /// sym\@got\@tlsld\@l.
       ADDI_TLSLD_L,
 
-      /// G8RC = GET_TLSLD_ADDR %X3, Symbol - For the local-dynamic TLS
-      /// model, produces a call to __tls_get_addr(sym\@tlsld).
-      GET_TLSLD_ADDR,
-
       /// G8RC = ADDIS_DTPREL_HA %X3, Symbol, Chain - For the
       /// local-dynamic TLS model, produces an ADDIS8 instruction
       /// that adds X3 to sym\@dtprel\@ha. The Chain operand is needed
       /// to tie this in place following a copy to %X3 from the result
       /// of a GET_TLSLD_ADDR.
       ADDIS_DTPREL_HA,
 
       /// G8RC = ADDI_DTPREL_L G8RReg, Symbol - For the local-dynamic TLS
SDValue EmitTailCallLoadFPAndRetAddr(SelectionDAG & DAG,
                                          SDValue &FPOpOut,
                                          bool isDarwinABI,
                                          SDLoc dl) const;
 
     SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
+    std::pair<SDValue,SDValue> lowerTLSCall(SDValue Op, SDLoc dl,
+                                            SelectionDAG &DAG) const;
     SDValue LowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSETCC(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINIT_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerADJUST_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerVASTART(SDValue Op, SelectionDAG &DAG,
                          const PPCSubtarget &Subtarget) const;

lib/Target/PowerPC/PPCISelLowering.cpp

const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   case PPCISD::LOAD:            return "PPCISD::LOAD";
   case PPCISD::LOAD_TOC:        return "PPCISD::LOAD_TOC";
   case PPCISD::DYNALLOC:        return "PPCISD::DYNALLOC";
   case PPCISD::GlobalBaseReg:   return "PPCISD::GlobalBaseReg";
   case PPCISD::SRL:             return "PPCISD::SRL";
   case PPCISD::SRA:             return "PPCISD::SRA";
   case PPCISD::SHL:             return "PPCISD::SHL";
   case PPCISD::CALL:            return "PPCISD::CALL";
   case PPCISD::CALL_NOP:        return "PPCISD::CALL_NOP";
+  case PPCISD::CALL_TLS:        return "PPCISD::CALL_TLS";
+  case PPCISD::CALL_NOP_TLS:    return "PPCISD::CALL_NOP_TLS";
   case PPCISD::MTCTR:           return "PPCISD::MTCTR";
   case PPCISD::BCTRL:           return "PPCISD::BCTRL";
   case PPCISD::RET_FLAG:        return "PPCISD::RET_FLAG";
   case PPCISD::EH_SJLJ_SETJMP:  return "PPCISD::EH_SJLJ_SETJMP";
   case PPCISD::EH_SJLJ_LONGJMP: return "PPCISD::EH_SJLJ_LONGJMP";
   case PPCISD::MFOCRF:          return "PPCISD::MFOCRF";
   case PPCISD::VCMP:            return "PPCISD::VCMP";
   case PPCISD::VCMPo:           return "PPCISD::VCMPo";
const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   case PPCISD::LD_TOC_L:        return "PPCISD::LD_TOC_L";
   case PPCISD::ADDI_TOC_L:      return "PPCISD::ADDI_TOC_L";
   case PPCISD::PPC32_GOT:       return "PPCISD::PPC32_GOT";
   case PPCISD::ADDIS_GOT_TPREL_HA: return "PPCISD::ADDIS_GOT_TPREL_HA";
   case PPCISD::LD_GOT_TPREL_L:  return "PPCISD::LD_GOT_TPREL_L";
   case PPCISD::ADD_TLS:         return "PPCISD::ADD_TLS";
   case PPCISD::ADDIS_TLSGD_HA:  return "PPCISD::ADDIS_TLSGD_HA";
   case PPCISD::ADDI_TLSGD_L:    return "PPCISD::ADDI_TLSGD_L";
-  case PPCISD::GET_TLS_ADDR:    return "PPCISD::GET_TLS_ADDR";
   case PPCISD::ADDIS_TLSLD_HA:  return "PPCISD::ADDIS_TLSLD_HA";
   case PPCISD::ADDI_TLSLD_L:    return "PPCISD::ADDI_TLSLD_L";
-  case PPCISD::GET_TLSLD_ADDR:  return "PPCISD::GET_TLSLD_ADDR";
   case PPCISD::ADDIS_DTPREL_HA: return "PPCISD::ADDIS_DTPREL_HA";
   case PPCISD::ADDI_DTPREL_L:   return "PPCISD::ADDI_DTPREL_L";
   case PPCISD::VADD_SPLAT:      return "PPCISD::VADD_SPLAT";
   case PPCISD::SC:              return "PPCISD::SC";
   }
 }
 
 EVT PPCTargetLowering::getSetCCResultType(LLVMContext &, EVT VT) const {
SDValue PPCTargetLowering::LowerBlockAddress(SDValue Op,
 
   unsigned MOHiFlag, MOLoFlag;
   bool isPIC = GetLabelAccessInfo(DAG.getTarget(), MOHiFlag, MOLoFlag);
   SDValue TgtBAHi = DAG.getTargetBlockAddress(BA, PtrVT, 0, MOHiFlag);
   SDValue TgtBALo = DAG.getTargetBlockAddress(BA, PtrVT, 0, MOLoFlag);
   return LowerLabelRef(TgtBAHi, TgtBALo, isPIC, DAG);
 }
 
+// Generate a call to __tls_get_addr for the given GOT entry Op.
+std::pair<SDValue,SDValue>
+PPCTargetLowering::lowerTLSCall(SDValue Op, SDLoc dl,
+                                SelectionDAG &DAG) const {
+
+  Type *IntPtrTy = getDataLayout()->getIntPtrType(*DAG.getContext());
+  TargetLowering::ArgListTy Args;
+  TargetLowering::ArgListEntry Entry;
+  Entry.Node = Op;
+  Entry.Ty = IntPtrTy;
+  Args.push_back(Entry);
+
+  TargetLowering::CallLoweringInfo CLI(DAG);
+  CLI.setDebugLoc(dl).setChain(DAG.getEntryNode())
+    .setCallee(CallingConv::C, IntPtrTy,
+               DAG.getTargetExternalSymbol("__tls_get_addr", getPointerTy()),
+               std::move(Args), 0);
+
+  return LowerCallTo(CLI);
+}
+
 SDValue PPCTargetLowering::LowerGlobalTLSAddress(SDValue Op,
                                               SelectionDAG &DAG) const {
 
   // FIXME: TLS addresses currently use medium model code sequences,
   // which is the most useful form.  Eventually support for small and
   // large models could be added if users need it, at the cost of
   // additional complexity.
   GlobalAddressSDNode *GA = cast<GlobalAddressSDNode>(Op);
if (Model == TLSModel::InitialExec) {
     } else
       GOTPtr = DAG.getNode(PPCISD::PPC32_GOT, dl, PtrVT);
     SDValue TPOffset = DAG.getNode(PPCISD::LD_GOT_TPREL_L, dl,
                                    PtrVT, TGA, GOTPtr);
     return DAG.getNode(PPCISD::ADD_TLS, dl, PtrVT, TPOffset, TGATLS);
   }
 
   if (Model == TLSModel::GeneralDynamic) {
-    SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, 0);
+    SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0,
+                                             PPCII::MO_TLSGD);
     SDValue GOTPtr;
     if (is64bit) {
       SDValue GOTReg = DAG.getRegister(PPC::X2, MVT::i64);
       GOTPtr = DAG.getNode(PPCISD::ADDIS_TLSGD_HA, dl, PtrVT,
                                    GOTReg, TGA);
     } else {
       GOTPtr = DAG.getNode(PPCISD::PPC32_PICGOT, dl, PtrVT);
     }
     SDValue GOTEntry = DAG.getNode(PPCISD::ADDI_TLSGD_L, dl, PtrVT,
                                    GOTPtr, TGA);
-
-    // We need a chain node, and don't have one handy.  The underlying
-    // call has no side effects, so using the function entry node
-    // suffices.
-    SDValue Chain = DAG.getEntryNode();
-    Chain = DAG.getCopyToReg(Chain, dl,
-                             is64bit ? PPC::X3 : PPC::R3, GOTEntry);
-    SDValue ParmReg = DAG.getRegister(is64bit ? PPC::X3 : PPC::R3,
-                                      is64bit ? MVT::i64 : MVT::i32);
-    SDValue TLSAddr = DAG.getNode(PPCISD::GET_TLS_ADDR, dl,
-                                  PtrVT, ParmReg, TGA);
-    // The return value from GET_TLS_ADDR really is in X3 already, but
-    // some hacks are needed here to tie everything together.  The extra
-    // copies dissolve during subsequent transforms.
-    Chain = DAG.getCopyToReg(Chain, dl, is64bit ? PPC::X3 : PPC::R3, TLSAddr);
-    return DAG.getCopyFromReg(Chain, dl, is64bit ? PPC::X3 : PPC::R3, PtrVT);
+    std::pair<SDValue, SDValue> CallResult = lowerTLSCall(GOTEntry, dl, DAG);
+    return CallResult.first;
   }
 
   if (Model == TLSModel::LocalDynamic) {
-    SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, 0);
+    SDValue TGA = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0,
+                                             PPCII::MO_TLSLD);
     SDValue GOTPtr;
     if (is64bit) {
       SDValue GOTReg = DAG.getRegister(PPC::X2, MVT::i64);
       GOTPtr = DAG.getNode(PPCISD::ADDIS_TLSLD_HA, dl, PtrVT,
                            GOTReg, TGA);
     } else {
       GOTPtr = DAG.getNode(PPCISD::PPC32_PICGOT, dl, PtrVT);
     }
     SDValue GOTEntry = DAG.getNode(PPCISD::ADDI_TLSLD_L, dl, PtrVT,
                                    GOTPtr, TGA);
-
-    // We need a chain node, and don't have one handy.  The underlying
-    // call has no side effects, so using the function entry node
-    // suffices.
-    SDValue Chain = DAG.getEntryNode();
-    Chain = DAG.getCopyToReg(Chain, dl,
-                             is64bit ? PPC::X3 : PPC::R3, GOTEntry);
-    SDValue ParmReg = DAG.getRegister(is64bit ? PPC::X3 : PPC::R3,
-                                      is64bit ? MVT::i64 : MVT::i32);
-    SDValue TLSAddr = DAG.getNode(PPCISD::GET_TLSLD_ADDR, dl,
-                                  PtrVT, ParmReg, TGA);
-    // The return value from GET_TLSLD_ADDR really is in X3 already, but
-    // some hacks are needed here to tie everything together.  The extra
-    // copies dissolve during subsequent transforms.
-    Chain = DAG.getCopyToReg(Chain, dl, is64bit ? PPC::X3 : PPC::R3, TLSAddr);
+    std::pair<SDValue, SDValue> CallResult = lowerTLSCall(GOTEntry, dl, DAG);
+    SDValue TLSAddr = CallResult.first;
+    SDValue Chain = CallResult.second;
     SDValue DtvOffsetHi = DAG.getNode(PPCISD::ADDIS_DTPREL_HA, dl, PtrVT,
-                                      Chain, ParmReg, TGA);
+                                      Chain, TLSAddr, TGA);
     return DAG.getNode(PPCISD::ADDI_DTPREL_L, dl, PtrVT, DtvOffsetHi, TGA);
   }
 
   llvm_unreachable("Unknown TLS model!");
 }
 
 SDValue PPCTargetLowering::LowerGlobalAddress(SDValue Op,
                                               SelectionDAG &DAG) const {
   EVT PtrVT = Op.getValueType();
   GlobalAddressSDNode *GSDN = cast<GlobalAddressSDNode>(Op);
   SDLoc DL(GSDN);
   const GlobalValue *GV = GSDN->getGlobal();
 
   // 64-bit SVR4 ABI code is always position-independent.
   // The actual address of the GlobalValue is stored in the TOC.
   if (Subtarget.isSVR4ABI() && Subtarget.isPPC64()) {
     SDValue GA = DAG.getTargetGlobalAddress(GV, DL, PtrVT, GSDN->getOffset());
     return DAG.getNode(PPCISD::TOC_ENTRY, DL, MVT::i64, GA,
                        DAG.getRegister(PPC::X2, MVT::i64));
   }
 
   unsigned MOHiFlag, MOLoFlag;
if (needIndirectCall) {
     if (isTailCall)
       Ops.push_back(DAG.getRegister(isPPC64 ? PPC::CTR8 : PPC::CTR, PtrVT));
   }
 
   // If this is a direct call, pass the chain and the callee.
   if (Callee.getNode()) {
     Ops.push_back(Chain);
     Ops.push_back(Callee);
+
+    // If this is a call to __tls_get_addr, find the symbol whose address
+    // is to be taken and add it to the list.  This will be used to 
+    // generate __tls_get_addr(<sym>@tlsgd) or __tls_get_addr(<sym>@tlsld).
+    // We find the symbol by walking the chain to the CopyFromReg, walking
+    // back from the CopyFromReg to the ADDI_TLSGD_L or ADDI_TLSLD_L, and
+    // pulling the symbol from that node.
+    if (ExternalSymbolSDNode *S = dyn_cast<ExternalSymbolSDNode>(Callee))
+      if (!strcmp(S->getSymbol(), "__tls_get_addr")) {
+        assert(!needIndirectCall && "Indirect call to __tls_get_addr???");
+        SDNode *AddI = Chain.getNode()->getOperand(2).getNode();
+        SDValue TGTAddr = AddI->getOperand(1);
+        assert(TGTAddr.getNode()->getOpcode() == ISD::TargetGlobalTLSAddress &&
+               "Didn't find target global TLS address where we expected one");
+        Ops.push_back(TGTAddr);
+        CallOpc = PPCISD::CALL_TLS;
+      }
   }
   // If this is a tail call add stack pointer delta.
   if (isTailCall)
     Ops.push_back(DAG.getConstant(SPDiff, MVT::i32));
 
   // Add argument registers to the end of the list so that they are known live
   // into the call.
   for (unsigned i = 0, e = RegsToPass.size(); i != e; ++i)
if (CallOpc == PPCISD::BCTRL) {
       // from allocating it), resulting in an additional register being
       // allocated and an unnecessary move instruction being generated.
       needsTOCRestore = true;
     } else if ((CallOpc == PPCISD::CALL) &&
                (!isLocalCall(Callee) ||
                 DAG.getTarget().getRelocationModel() == Reloc::PIC_)) {
       // Otherwise insert NOP for non-local calls.
       CallOpc = PPCISD::CALL_NOP;
-    }
+    } else if (CallOpc == PPCISD::CALL_TLS)
+      // For 64-bit SVR4, TLS calls are always non-local.
+      CallOpc = PPCISD::CALL_NOP_TLS;
   }
 
   Chain = DAG.getNode(CallOpc, dl, NodeTys, Ops);
   InFlag = Chain.getValue(1);
 
   if (needsTOCRestore) {
     SDVTList VTs = DAG.getVTList(MVT::Other, MVT::Glue);
     EVT PtrVT = DAG.getTargetLoweringInfo().getPointerTy();

lib/Target/PowerPC/PPCInstr64Bit.td

 def : Pat<(PPCcall_nop (i64 tglobaladdr:$dst)),
           (BL8_NOP tglobaladdr:$dst)>;
 
 def : Pat<(PPCcall (i64 texternalsym:$dst)),
           (BL8 texternalsym:$dst)>;
 def : Pat<(PPCcall_nop (i64 texternalsym:$dst)),
           (BL8_NOP texternalsym:$dst)>;
 
+def : Pat<(PPCcall_nop_tls texternalsym:$func, tglobaltlsaddr:$sym),
+          (BL8_NOP_TLS texternalsym:$func, tglobaltlsaddr:$sym)>;
+
 // Atomic operations
 let usesCustomInserter = 1 in {
   let Defs = [CR0] in {
     def ATOMIC_LOAD_ADD_I64 : Pseudo<
       (outs g8rc:$dst), (ins memrr:$ptr, g8rc:$incr), "#ATOMIC_LOAD_ADD_I64",
       [(set i64:$dst, (atomic_load_add_64 xoaddr:$ptr, i64:$incr))]>;
     def ATOMIC_LOAD_SUB_I64 : Pseudo<
       (outs g8rc:$dst), (ins memrr:$ptr, g8rc:$incr), "#ATOMIC_LOAD_SUB_I64",
def ADDIStlsgdHA: Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                          [(set i64:$rD,
                            (PPCaddisTlsgdHA i64:$reg, tglobaltlsaddr:$disp))]>,
                   isPPC64;
 def ADDItlsgdL : Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                        "#ADDItlsgdL",
                        [(set i64:$rD,
                          (PPCaddiTlsgdL i64:$reg, tglobaltlsaddr:$disp))]>,
                  isPPC64;
-def GETtlsADDR : Pseudo<(outs g8rc:$rD), (ins g8rc:$reg, tlsgd:$sym),
-                        "#GETtlsADDR",
-                        [(set i64:$rD,
-                          (PPCgetTlsAddr i64:$reg, tglobaltlsaddr:$sym))]>,
-                 isPPC64;
 def ADDIStlsldHA: Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                          "#ADDIStlsldHA",
                          [(set i64:$rD,
                            (PPCaddisTlsldHA i64:$reg, tglobaltlsaddr:$disp))]>,
                   isPPC64;
 def ADDItlsldL : Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                        "#ADDItlsldL",
                        [(set i64:$rD,
                          (PPCaddiTlsldL i64:$reg, tglobaltlsaddr:$disp))]>,
                  isPPC64;
-def GETtlsldADDR : Pseudo<(outs g8rc:$rD), (ins g8rc:$reg, tlsgd:$sym),
-                          "#GETtlsldADDR",
-                          [(set i64:$rD,
-                            (PPCgetTlsldAddr i64:$reg, tglobaltlsaddr:$sym))]>,
-                   isPPC64;
 def ADDISdtprelHA: Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                           "#ADDISdtprelHA",
                           [(set i64:$rD,
                             (PPCaddisDtprelHA i64:$reg,
                                               tglobaltlsaddr:$disp))]>,
                    isPPC64;
 def ADDIdtprelL : Pseudo<(outs g8rc:$rD), (ins g8rc_nox0:$reg, s16imm64:$disp),
                          "#ADDIdtprelL",

lib/Target/PowerPC/PPCInstrInfo.td

 def PPCppc32GOT : SDNode<"PPCISD::PPC32_GOT", SDTIntLeaf, []>;
 
 def PPCaddisGotTprelHA : SDNode<"PPCISD::ADDIS_GOT_TPREL_HA", SDTIntBinOp>;
 def PPCldGotTprelL : SDNode<"PPCISD::LD_GOT_TPREL_L", SDTIntBinOp,
                             [SDNPMayLoad]>;
 def PPCaddTls     : SDNode<"PPCISD::ADD_TLS", SDTIntBinOp, []>;
 def PPCaddisTlsgdHA : SDNode<"PPCISD::ADDIS_TLSGD_HA", SDTIntBinOp>;
 def PPCaddiTlsgdL   : SDNode<"PPCISD::ADDI_TLSGD_L", SDTIntBinOp>;
-def PPCgetTlsAddr   : SDNode<"PPCISD::GET_TLS_ADDR", SDTIntBinOp>;
 def PPCaddisTlsldHA : SDNode<"PPCISD::ADDIS_TLSLD_HA", SDTIntBinOp>;
 def PPCaddiTlsldL   : SDNode<"PPCISD::ADDI_TLSLD_L", SDTIntBinOp>;
-def PPCgetTlsldAddr : SDNode<"PPCISD::GET_TLSLD_ADDR", SDTIntBinOp>;
 def PPCaddisDtprelHA : SDNode<"PPCISD::ADDIS_DTPREL_HA", SDTIntBinOp,
                               [SDNPHasChain]>;
 def PPCaddiDtprelL   : SDNode<"PPCISD::ADDI_DTPREL_L", SDTIntBinOp>;
 
 def PPCvperm    : SDNode<"PPCISD::VPERM", SDT_PPCvperm, []>;
 
 // These nodes represent the 32-bit PPC shifts that operate on 6-bit shift
 // amounts.  These nodes are generated by the multi-precision shift code.
 def PPCsrl        : SDNode<"PPCISD::SRL"       , SDTIntShiftOp>;
 def PPCsra        : SDNode<"PPCISD::SRA"       , SDTIntShiftOp>;
 def PPCshl        : SDNode<"PPCISD::SHL"       , SDTIntShiftOp>;
 
 // These are target-independent nodes, but have target-specific formats.
 def callseq_start : SDNode<"ISD::CALLSEQ_START", SDT_PPCCallSeqStart,
                            [SDNPHasChain, SDNPOutGlue]>;
 def callseq_end   : SDNode<"ISD::CALLSEQ_END",   SDT_PPCCallSeqEnd,
                            [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]>;
 
 def SDT_PPCCall   : SDTypeProfile<0, -1, [SDTCisInt<0>]>;
 def PPCcall  : SDNode<"PPCISD::CALL", SDT_PPCCall,
                       [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
                        SDNPVariadic]>;
+def PPCcall_tls : SDNode<"PPCISD::CALL_TLS", SDT_PPCCall,
+                         [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
+                          SDNPVariadic]>;
 def PPCcall_nop  : SDNode<"PPCISD::CALL_NOP", SDT_PPCCall,
                           [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
                            SDNPVariadic]>;
+def PPCcall_nop_tls : SDNode<"PPCISD::CALL_NOP_TLS", SDT_PPCCall,
+                             [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue,
+                              SDNPVariadic]>;
 def PPCload   : SDNode<"PPCISD::LOAD", SDTypeProfile<1, 1, []>,
                        [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]>;
 def PPCload_toc : SDNode<"PPCISD::LOAD_TOC", SDTypeProfile<0, 1, []>,
                           [SDNPHasChain, SDNPSideEffect,
                            SDNPInGlue, SDNPOutGlue]>;
 def PPCmtctr      : SDNode<"PPCISD::MTCTR", SDT_PPCCall,
                            [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]>;
 def PPCbctrl : SDNode<"PPCISD::BCTRL", SDTNone,
def : Pat<(and (rotl i32:$in, i32:$sh), maskimm32:$imm),
           (RLWNM $in, $sh, (MB maskimm32:$imm), (ME maskimm32:$imm))>;
 
 // Calls
 def : Pat<(PPCcall (i32 tglobaladdr:$dst)),
           (BL tglobaladdr:$dst)>;
 def : Pat<(PPCcall (i32 texternalsym:$dst)),
           (BL texternalsym:$dst)>;
 
+def : Pat<(PPCcall_tls texternalsym:$func, tglobaltlsaddr:$sym),
+          (BL_TLS texternalsym:$func, tglobaltlsaddr:$sym)>;
 
 def : Pat<(PPCtc_return (i32 tglobaladdr:$dst),  imm:$imm),
           (TCRETURNdi tglobaladdr:$dst, imm:$imm)>;
 
 def : Pat<(PPCtc_return (i32 texternalsym:$dst), imm:$imm),
           (TCRETURNdi texternalsym:$dst, imm:$imm)>;
 
 def : Pat<(PPCtc_return CTRRC:$dst, imm:$imm),
def LDgotTprelL32: Pseudo<(outs gprc:$rD), (ins s16imm:$disp, gprc_nor0:$reg),
                              (PPCldGotTprelL tglobaltlsaddr:$disp, i32:$reg))]>;
 def : Pat<(PPCaddTls i32:$in, tglobaltlsaddr:$g),
           (ADD4TLS $in, tglobaltlsaddr:$g)>;
 
 def ADDItlsgdL32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
                          "#ADDItlsgdL32",
                          [(set i32:$rD,
                            (PPCaddiTlsgdL i32:$reg, tglobaltlsaddr:$disp))]>;
-def GETtlsADDR32 : Pseudo<(outs gprc:$rD), (ins gprc:$reg, tlsgd32:$sym),
-                          "#GETtlsADDR32",
-                          [(set i32:$rD,
-                            (PPCgetTlsAddr i32:$reg, tglobaltlsaddr:$sym))]>;
 def ADDItlsldL32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
                           "#ADDItlsldL32",
                           [(set i32:$rD,
                             (PPCaddiTlsldL i32:$reg, tglobaltlsaddr:$disp))]>;
-def GETtlsldADDR32 : Pseudo<(outs gprc:$rD), (ins gprc:$reg, tlsgd32:$sym),
-                            "#GETtlsldADDR32",
-                            [(set i32:$rD,
-                              (PPCgetTlsldAddr i32:$reg, tglobaltlsaddr:$sym))]>;
 def ADDIdtprelL32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
                            "#ADDIdtprelL32",
                            [(set i32:$rD,
                              (PPCaddiDtprelL i32:$reg, tglobaltlsaddr:$disp))]>;
 def ADDISdtprelHA32 : Pseudo<(outs gprc:$rD), (ins gprc_nor0:$reg, s16imm:$disp),
                             "#ADDISdtprelHA32",
                             [(set i32:$rD,
                               (PPCaddisDtprelHA i32:$reg,

lib/Target/PowerPC/PPCMCInstLower.cpp

case PPCII::MO_TLSLD_LO:
       RefKind = MCSymbolRefExpr::VK_PPC_GOT_TLSLD_LO;
       break;
     case PPCII::MO_TOC_LO:
       RefKind = MCSymbolRefExpr::VK_PPC_TOC_LO;
       break;
     case PPCII::MO_TLS:
       RefKind = MCSymbolRefExpr::VK_PPC_TLS;
       break;
+    case PPCII::MO_TLSGD:
+      RefKind = MCSymbolRefExpr::VK_PPC_TLSGD;
+      break;
+    case PPCII::MO_TLSLD:
+      RefKind = MCSymbolRefExpr::VK_PPC_TLSLD;
+      break;
   }
 
   if (MO.getTargetFlags() == PPCII::MO_PLT_OR_STUB && !isDarwin)
     RefKind = MCSymbolRefExpr::VK_PLT;
 
   const MCExpr *Expr = MCSymbolRefExpr::Create(Symbol, RefKind, Ctx);
 
   if (!MO.isJTI() && MO.getOffset())

test/CodeGen/PowerPC/tls-store2.ll

+; RUN: llc -march=ppc64 -mcpu=pwr7 -O2 -relocation-model=pic < %s | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-n32:64"
+target triple = "powerpc64le-unknown-linux-gnu"
+
+; Test back-to-back stores of TLS variables to ensure call sequences no
+; longer overlap.
+
+@__once_callable = external thread_local global i8**
+@__once_call = external thread_local global void ()*
+
+define i64 @call_once(i64 %flag, i8* %ptr) {
+entry:
+  %var = alloca i8*, align 8
+  store i8* %ptr, i8** %var, align 8
+  store i8** %var, i8*** @__once_callable, align 8
+  store void ()* @__once_call_impl, void ()** @__once_call, align 8
+  ret i64 %flag
+}
+
+; CHECK-LABEL: call_once:
+; CHECK: addis 3, 2, __once_callable@got@tlsgd@ha
+; CHECK: addi 3, 3, __once_callable@got@tlsgd@l
+; CHECK: bl __tls_get_addr(__once_callable@tlsgd)
+; CHECK-NEXT: nop
+; CHECK: std {{[0-9]+}}, 0(3)
+; CHECK: addis 3, 2, __once_call@got@tlsgd@ha
+; CHECK: addi 3, 3, __once_call@got@tlsgd@l
+; CHECK: bl __tls_get_addr(__once_call@tlsgd)
+; CHECK-NEXT: nop
+; CHECK: std {{[0-9]+}}, 0(3)
+
+declare void @__once_call_impl()