[AIX][TLS] Generate optimized local-exec access code sequence using X-Form loads/stores

[amyk]

Summary

This patch is a follow up to D149722, D152669 and D153645, where a slightly more optimized code
sequence is generated for 64-bit and 32-bit local-exec accesses when optimizations are turned on.

Handling is added PPCISelDAGToDAG.cpp in order to check if any D-form loads or stores that follow
an PPCISD::ADD_TLS can be optimized to use an X-Form load or store. In this particular situation,
this allows the ADD_TLS node to be removed completely.

[amyk]

Rebase patch.

[amyk]

Update this patch with a simplified way of generating the X-Form instructions, leveraging some existing infrastructure in place.

I am planning on updating this patch once again to fix a potential issue.

[amyk]

• Update patch to produce lhax and lwax when the load is sign-extending.
• Rebase patch.

[amyk]

Rebase patch and update comments in the patch.

[amyk]

Update patch to account for 32-bit implementation in D152669.

[amyk]

• Rebase patch.
• Update patch comments.

[amyk]

• Rebase patch to account for updating D152669.
• Use the correct PPCISD node (GET_TPOINTER) in canOptimizeTLSDFormToXFormOnAIX().

[amyk]

Rebase patch and simplify relocation test checks.

[amyk]

Patch rebased.

[amyk]

• Specify why we are doing this optimization in the description.
• Check on why these TLS instructions were not previously added.

llvm/lib/Target/PowerPC/P10InstrResources.td

1299

Check these definitions in the P10InstrResources.td.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

714

• Fix this comment.
• Can this function be general? Include both of the Linux and AIX parts.

743

Instead of explicitly using PPC::X13, is there something like Subtarget.getThreadPointerRegister() that we can use?

[amyk]

Address review comments:

• Simplify canOptimizeTLSDFormToXForm()
• Add getThreadPointerRegister() into PPCSubtarget.h
• Confirmed that adding the new *TLS load/store instructions are appropriate

[amyk]

llvm/lib/Target/PowerPC/PPCSubtarget.h

279 ↗

(On Diff #533580)

Is it appropriate to add this in this patch? Or should this be separate?

[amyk]

Like in D47382, it is possible that I need to add *TLS_ versions of the instructions that I added. I plan to update this patch to account for this.

[amyk]

Group code review comments.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

754

These two checks for the offset can be combined.

762

Update this comment for GET_TPOINTER, and just mention it's produced for AIX on 32-bit mode.

773

• Implement a static helper function, to check if the node of a thread pointer acquisition node (if one of the operands is the thread pointer - either R13, or __get_tpointer).
• If conditions are satisfied, return true.
• Otherwise, return false at the end of the static helper function.

832

A ternary condition for isSExt for here and below might be better.

llvm/lib/Target/PowerPC/PPCSubtarget.h

279 ↗

(On Diff #533580)

Discussed this with the team. It is appropriate to add this here.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

14

Instead of adding -O0 lines to every test, we can add one file to show the -O0 codegen.
That way, we can remove the -O0 run lines from these existing test cases.

[amyk]

Update patch to:

• Separate out the new instruction definitions (which have been added in https://reviews.llvm.org/D153645)
• Update comments
• Add a new static helper function to check if one of the operands of an ADD_TLS node uses the thread pointer

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

754

Done.

762

Done.

773

isThreadPointerAcquisitionNode() is added to account for this.

832

Done.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

14

Done.

[lei]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

765–768

783–784

[amyk]

• Rebase patch.
• Address comments from Lei: combine two if conditions, and return with the isThreadPointerAcquisitionNode() function inside canOptimizeTLSDFormToXForm().

[lei]

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

73–76

This transformation increased reg pressure with no obvious benefits. Please see if we can bail out for such cases.

[amyk]

Update the patch to only do this optimization if all users of the ADD_TLS node are normal loads with an offset of 0.
This resolves an issue that Lei pointed out with respect to increasing the usage of registers when we have a 64-bit load/store on a 32-bit system - in such a situation, we no longer perform this optimization.

[lei]

Just a few nits, otherwise LGTM.
Thanks for addressing the issue with increasing the usage of registers when we have a 64-bit load/store on a 32-bit system.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

769

Since Base is only used on line 788, consider just inlining it there vs defining a new variable.

812

Since Base only used here, can inline and remove declaration.

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

769

Actually, Base is also used here below:

SDValue Ops[] = {ST->getValue(), Base.getOperand(0), Base.getOperand(1),
                   Chain};

812

Actually, Base is also used here below, right?

SDValue Ops[] = {Base.getOperand(0), Base.getOperand(1), Chain};

llvm/lib/Target/PowerPC/P10InstrResources.td

 
 // 6 Cycles Load operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_ANY, P10LD_Read],
       (instrs
     LBZ, LBZ8,
     LD,  LDtoc,  LDtocBA,  LDtocCPT,  LDtocJTI,  LDtocL, SPILLTOVSR_LD,
     LDBRX,
      DFLOADf32, DFLOADf64, LFD,
-    LFDX,  XFLOADf32, XFLOADf64,
     LFIWAX, LIWAX,
     LFIWZX, LIWZX,
     LHA, LHA8,
-    LHAX, LHAX8,
     LHBRX, LHBRX8,
     LHZ, LHZ8,
     LVEBX,
     LVEHX,
     LVEWX,
     LVX,
     LVXL,
     LWA, LWA_32,
-    LWAX, LWAX_32,
+    LWAX, LWAXTLS, LWAXTLS_32, LWAX_32,
     LWBRX, LWBRX8,
     LWZ, LWZ8,  LWZtoc, LWZtocL,
     LXSD,
     LXSDX,
     LXSIBZX,
     LXSIHZX,
     LXSIWAX,
     LXSIWZX,

 // 6 Cycles Load operations, 2 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_ANY, P10LD_Read, P10LD_Read],
       (instrs
     DCBT,
     DCBTST,
     ICBT,
     LBZX, LBZX8, LBZXTLS, LBZXTLS_, LBZXTLS_32,
     LDX, LDXTLS, LDXTLS_, SPILLTOVSR_LDX,
+    LFDX, LFDXTLS,  XFLOADf32, XFLOADf64,
+    LHAX, LHAX8, LHAXTLS, LHAXTLS_32,
     LHZX, LHZX8, LHZXTLS, LHZXTLS_, LHZXTLS_32,
     LWZX, LWZX8, LWZXTLS, LWZXTLS_, LWZXTLS_32,
     LXVL,
     LXVLL
 )>;
 
 // 2-way crack instructions
 // 6 Cycles Load operations, and 13 Cycles Decimal Floating Point operations, 2 input operands
def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10LD_Read],
     PLXVP, PLXVPpc
 )>;
 
 // 2-way crack instructions
 // 6 Cycles Load operations, and 4 Cycles ALU2 operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C],
       (instrs
     LFS,
-    LFSX,
     LXSSP,
     LXSSPX
 )>;
 
+// 2-way crack instructions
+// 6 Cycles Load operations, and 4 Cycles ALU2 operations, 2 input operands
+def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C],
+      (instrs
+    LFSX, LFSXTLS
+)>;
+
 // 4-way crack instructions
 // 6 Cycles Load operations, 4 Cycles ALU2 operations, 3 Cycles Simple Fixed-point (SFX) operations, and 3 Cycles ALU operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C, P10W_SX_3C, P10W_DISP_ANY, P10W_FX_3C, P10W_DISP_ANY],
       (instrs
     LFSU,
     LFSUX
 )>;
 
def : InstRW<[P10W_ST_3C, P10W_DISP_ANY, P10ST_Read, P10ST_Read],
     STBUX, STBUX8,
     SPILLTOVSR_ST, STD,
     STDBRX,
     STDU,
     STDUX,
      DFSTOREf32, DFSTOREf64, STFD,
     STFDU,
     STFDUX,
-    STFDX,
     STFIWX, STIWX,
     STFS,
     STFSU,
     STFSUX,
-    STFSX,
     STH, STH8,
     STHBRX,
     STHU, STHU8,
     STHUX, STHUX8,
     STVEBX,
     STVEHX,
     STVEWX,
     STVX,

 )>;
 
 // 3 Cycles Store operations, 3 input operands
 def : InstRW<[P10W_ST_3C, P10W_DISP_ANY, P10ST_Read, P10ST_Read, P10ST_Read],
       (instrs
     CP_COPY, CP_COPY8,
     STBX, STBX8, STBXTLS, STBXTLS_, STBXTLS_32,
     SPILLTOVSR_STX, STDX, STDXTLS, STDXTLS_,
+    STFDX, STFDXTLS,
+    STFSX, STFSXTLS,
     STHX, STHX8, STHXTLS, STHXTLS_, STHXTLS_32,
     STWX, STWX8, STWXTLS, STWXTLS_, STWXTLS_32,
     STXVL,
     STXVLL
 )>;
 
 // Single crack instructions
 // 3 Cycles Store operations, 0 input operands

llvm/lib/Target/PowerPC/P9InstrResources.td

 )>;
 
 // 4 Cycle Restricted load uses a single slice but the dispatch for the whole
 // superslice.
 def : InstRW<[P9_LS_4C, IP_AGEN_1C, DISP_3SLOTS_1C],
       (instrs
     LFIWZX,
     LFDX,
+    LFDXTLS,
     LFD
 )>;
 
 // Cracked Load Instructions.
 // Load instructions that can be done in parallel.
 def : InstRW<[P9_LS_4C, P9_LS_4C, IP_AGEN_1C, IP_AGEN_1C,
               DISP_PAIR_1C],
       (instrs

 )>;
 
 // Cracked Load instruction.
 // Requires consecutive Load and ALU pieces totaling 6 cycles. The Load and ALU
 // operations cannot be done at the same time and so their latencies are added.
 def : InstRW<[P9_LoadAndALUOp_6C, IP_EXEC_1C, IP_AGEN_1C,
               DISP_1C, DISP_1C],
       (instrs
-    (instregex "LHA(X)?(8)?$"),
+    (instregex "LHA(X)?(TLS)?(8)?(_32)?$"),
     (instregex "CP_PASTE(8)?_rec$"),
-    (instregex "LWA(X)?(_32)?$"),
+    (instregex "LWA(X)?(TLS)?(_32)?$"),
     TCHECK
 )>;
 
 // Cracked Restricted Load instruction.
 // Requires consecutive Load and ALU pieces totaling 6 cycles. The Load and ALU
 // operations cannot be done at the same time and so their latencies are added.
 // Full 6 dispatches are required as this is both cracked and restricted.
 def : InstRW<[P9_LoadAndALUOp_6C, IP_EXEC_1C, IP_AGEN_1C,

 // Requires consecutive Load (4 cycles) and ALU (3 cycles) pieces totaling 7
 // cycles. The Load and ALU operations cannot be done at the same time and so
 // their latencies are added.
 // Full 6 dispatches are required as this is a restricted instruction.
 def : InstRW<[P9_LoadAndALU2Op_7C, IP_AGEN_1C, IP_EXEC_1C,
               DISP_3SLOTS_1C, DISP_3SLOTS_1C],
       (instrs
     LFSX,
+    LFSXTLS,
     LFS
 )>;
 
 // Cracked Load instruction.
 // Requires consecutive Load and ALU pieces totaling 8 cycles. The Load and ALU
 // operations cannot be done at the same time and so their latencies are added.
 // Full 4 dispatches are required as this is a cracked instruction.
 def : InstRW<[P9_LoadAndALU2Op_8C, IP_AGEN_1C, IP_EXEC_1C, DISP_1C, DISP_1C],
def : InstRW<[P9_LoadAndPMOp_8C, IP_AGEN_1C, IP_EXECE_1C, IP_EXECO_1C,
     LXVDSX,
     LXVW4X
 )>;
 
 // Single slice Restricted store operation. The restricted operation requires
 // all three dispatches for the superslice.
 def : InstRW<[P9_LS_1C, IP_EXEC_1C, IP_AGEN_1C, DISP_3SLOTS_1C],
       (instrs
-    (instregex "STF(S|D|IWX|SX|DX)$"),
+    (instregex "STF(S|D|IWX|SX|DX|SXTLS|DXTLS)$"),
     (instregex "STXS(D|DX|SPX|IWX|IBX|IHX|SP)(v)?$"),
     (instregex "STW(8)?$"),
     (instregex "(D|X)FSTORE(f32|f64)$"),
     (instregex "ST(W|H|D)BRX$"),
     (instregex "ST(B|H|D)(8)?$"),
     (instregex "ST(B|W|H|D)(CI)?X(TLS|TLS_32)?(8)?(_)?$"),
     STIWX,
     SLBIEG,

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

bool PPCDAGToDAGISel::isRotateAndMask(SDNode *N, unsigned Mask,
   if (Mask && !(Mask & Indeterminant)) {
     SH = Shift & 31;
     // make sure the mask is still a mask (wrap arounds may not be)
     return isRunOfOnes(Mask, MB, ME);
   }
   return false;
 }
 
+// canOptimizeTLSDFormToXFormOnAIX - For TLS local-exec accesses on AIX 64-bit
+// and 32-bit mode, an ADD_TLS node is produced to add the result of loading the
+// variable offset to the thread pointer (X13 on 64-bit, or in R3 on 32-bit
+// after calling .__get_tpointer).
+// This ADD_TLS, followed by a D-Form memory operation, can be optimized to use
+// an X-Form load or store, allowing the ADD_TLS node to be removed completely.
+// This can be done as long as the memory operation's immediate offset is 0 and
+// that the thread pointer is being added to.
+static bool canOptimizeTLSDFormToXFormOnAIX(SelectionDAG *CurDAG,
+                                            SDValue Base, uint64_t Offset) {
+  const PPCSubtarget &Subtarget =
+      CurDAG->getMachineFunction().getSubtarget<PPCSubtarget>();
+  if (Subtarget.isAIXABI()) {
+    // Do not do this transformation at -O0.
+    if (CurDAG->getTarget().getOptLevel() == CodeGenOpt::None)
+      return false;
+
+    // The optimization to convert the D-Form load/store into its X-Form
+    // counterpart should only occur if the immediate offset is 0.
+    if (Offset != 0)
+      return false;
+
+    // This optimization should also only be performed if the thread pointer
+    // is being added to the variable offset (loaded from the TOC).
+    // As seen in tryTLSXForm[Load|Store], Base is an ADD_TLS node.
+
+    // The first operand in this instruction should either be:
+    //  1. A GET_TPOINTER PPCISD node. This represents a call to
+    //     .__get_tpointer to get the thread pointer, which is for 32-bit only.
+    if (Base.getOperand(0).getOpcode() == PPCISD::GET_TPOINTER)
+      return true;
+    //  2. The thread pointer, stored in X13 on 64-bit mode.
+    RegisterSDNode *AddFirstOpReg =
+      dyn_cast_or_null<RegisterSDNode>(Base.getOperand(0).getNode());
+    if (!AddFirstOpReg)
+      return false;
+    if (AddFirstOpReg->getReg() != PPC::X13)
+      return false;
+  }
+  return true;
+}
+
 bool PPCDAGToDAGISel::tryTLSXFormStore(StoreSDNode *ST) {
   SDValue Base = ST->getBasePtr();
   if (Base.getOpcode() != PPCISD::ADD_TLS)
     return false;
   SDValue Offset = ST->getOffset();
   if (!Offset.isUndef())
     return false;
   if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
     return false;
+  // Check if the local-exec sequence on AIX can be optimized.
+  if (!canOptimizeTLSDFormToXFormOnAIX(CurDAG, Base, ST->getSrcValueOffset()))
+    return false;
 
   SDLoc dl(ST);
   EVT MemVT = ST->getMemoryVT();
   EVT RegVT = ST->getValue().getValueType();
 
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
       Opcode = (RegVT == MVT::i32) ? PPC::STBXTLS_32 : PPC::STBXTLS;
       break;
     }
     case MVT::i16: {
       Opcode = (RegVT == MVT::i32) ? PPC::STHXTLS_32 : PPC::STHXTLS;
       break;
     }
     case MVT::i32: {
       Opcode = (RegVT == MVT::i32) ? PPC::STWXTLS_32 : PPC::STWXTLS;
       break;
     }
     case MVT::i64: {
       Opcode = PPC::STDXTLS;
       break;
     }
+    case MVT::f32: {
+      Opcode = PPC::STFSXTLS;
+      break;
+    }
+    case MVT::f64: {
+      Opcode = PPC::STFDXTLS;
+      break;
+    }
   }
   SDValue Chain = ST->getChain();
   SDVTList VTs = ST->getVTList();
   SDValue Ops[] = {ST->getValue(), Base.getOperand(0), Base.getOperand(1),
                    Chain};
   SDNode *MN = CurDAG->getMachineNode(Opcode, dl, VTs, Ops);
   transferMemOperands(ST, MN);
   ReplaceNode(ST, MN);
   return true;
 }
 
 bool PPCDAGToDAGISel::tryTLSXFormLoad(LoadSDNode *LD) {
   SDValue Base = LD->getBasePtr();
   if (Base.getOpcode() != PPCISD::ADD_TLS)
     return false;
   SDValue Offset = LD->getOffset();
   if (!Offset.isUndef())
     return false;
   if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
     return false;
+  // Check if the local-exec sequence on AIX can be optimized.
+  if (!canOptimizeTLSDFormToXFormOnAIX(CurDAG, Base, LD->getSrcValueOffset()))
+    return false;
 
   SDLoc dl(LD);
   EVT MemVT = LD->getMemoryVT();
   EVT RegVT = LD->getValueType(0);
+  bool isSExt = LD->getExtensionType() == ISD::SEXTLOAD;
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
       Opcode = (RegVT == MVT::i32) ? PPC::LBZXTLS_32 : PPC::LBZXTLS;
       break;
     }
     case MVT::i16: {
+      if (isSExt)
+        Opcode = (RegVT == MVT::i32) ? PPC::LHAXTLS_32 : PPC::LHAXTLS;
+      else
         Opcode = (RegVT == MVT::i32) ? PPC::LHZXTLS_32 : PPC::LHZXTLS;
       break;
     }
     case MVT::i32: {
+      if (isSExt)
+        Opcode = (RegVT == MVT::i32) ? PPC::LWAXTLS_32 : PPC::LWAXTLS;
+      else
         Opcode = (RegVT == MVT::i32) ? PPC::LWZXTLS_32 : PPC::LWZXTLS;
       break;
     }
     case MVT::i64: {
       Opcode = PPC::LDXTLS;
       break;
     }
+    case MVT::f32: {
+      Opcode = PPC::LFSXTLS;
+      break;
+    }
+    case MVT::f64: {
+      Opcode = PPC::LFDXTLS;
+      break;
+    }
   }
   SDValue Chain = LD->getChain();
   SDVTList VTs = LD->getVTList();
   SDValue Ops[] = {Base.getOperand(0), Base.getOperand(1), Chain};
   SDNode *MN = CurDAG->getMachineNode(Opcode, dl, VTs, Ops);
   transferMemOperands(LD, MN);
   ReplaceNode(LD, MN);
   return true;
if (N->getValueType(0) == MVT::i64) {
                                N0, ShiftAmt);
       CurDAG->SelectNodeTo(N, PPC::ADDZE, MVT::i32, SDValue(Op, 0),
                            SDValue(Op, 1));
       return;
     }
   }
 
   case ISD::STORE: {
-    // Change TLS initial-exec D-form stores to X-form stores.
+    // Change TLS initial-exec (or TLS local-exec on AIX) D-form stores to
+    // X-form stores.
     StoreSDNode *ST = cast<StoreSDNode>(N);
-    if (EnableTLSOpt && Subtarget->isELFv2ABI() &&
+    if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()) &&
         ST->getAddressingMode() != ISD::PRE_INC)
       if (tryTLSXFormStore(ST))
         return;
     break;
   }
   case ISD::LOAD: {
     // Handle preincrement loads.
     LoadSDNode *LD = cast<LoadSDNode>(N);
     EVT LoadedVT = LD->getMemoryVT();
 
     // Normal loads are handled by code generated from the .td file.
     if (LD->getAddressingMode() != ISD::PRE_INC) {
-      // Change TLS initial-exec D-form loads to X-form loads.
-      if (EnableTLSOpt && Subtarget->isELFv2ABI())
+      // Change TLS initial-exec (or TLS local-exec on AIX) D-form loads to
+      // X-form loads.
+      if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()))
         if (tryTLSXFormLoad(LD))
           return;
       break;
     }
 
     SDValue Offset = LD->getOffset();
     if (Offset.getOpcode() == ISD::TargetConstant ||
         Offset.getOpcode() == ISD::TargetGlobalAddress) {

llvm/lib/Target/PowerPC/PPCInstr64Bit.td

 def ADD8TLS  : XOForm_1<31, 266, 0, (outs g8rc:$RT), (ins g8rc_nox0:$RA, tlsreg:$RB),
                         "add $RT, $RA, $RB", IIC_IntSimple,
                         [(set i64:$RT, (add i64:$RA, tglobaltlsaddr:$RB))]>;
 let mayLoad = 1 in {
 def LBZXTLS : XForm_1<31,  87, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                       "lbzx $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LHZXTLS : XForm_1<31, 279, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                       "lhzx $RST, $RA, $RB", IIC_LdStLoad, []>;
+def LHAXTLS : XForm_1<31, 343, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                      "lhax $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LWZXTLS : XForm_1<31,  23, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                       "lwzx $RST, $RA, $RB", IIC_LdStLoad, []>;
+def LWAXTLS : XForm_1<31, 341, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                      "lwax $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LDXTLS  : XForm_1<31,  21, (outs g8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                       "ldx $RST, $RA, $RB", IIC_LdStLD, []>, isPPC64;
 def LBZXTLS_32 : XForm_1<31,  87, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lbzx $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LHZXTLS_32 : XForm_1<31, 279, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lhzx $RST, $RA, $RB", IIC_LdStLoad, []>;
+def LHAXTLS_32 : XForm_1<31, 343, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                         "lhax $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LWZXTLS_32 : XForm_1<31,  23, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lwzx $RST, $RA, $RB", IIC_LdStLoad, []>;
+def LWAXTLS_32 : XForm_1<31, 341, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                         "lwax $RST, $RA, $RB", IIC_LdStLoad, []>;
 
 }
+let mayLoad = 1, Predicates = [HasFPU] in {
+def LFSXTLS : XForm_25<31, 535, (outs f4rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "lfsx $RST, $RA, $RB", IIC_LdStLFD, []>;
+def LFDXTLS : XForm_25<31, 599, (outs f8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "lfdx $RST, $RA, $RB", IIC_LdStLFD, []>;
+}
 
 let mayStore = 1 in {
 def STBXTLS : XForm_8<31, 215, (outs), (ins g8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                       "stbx $RST, $RA, $RB", IIC_LdStStore, []>,
                       PPC970_DGroup_Cracked;
 def STHXTLS : XForm_8<31, 407, (outs), (ins g8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                       "sthx $RST, $RA, $RB", IIC_LdStStore, []>,
                       PPC970_DGroup_Cracked;

 def STHXTLS_32 : XForm_8<31, 407, (outs), (ins gprc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                          "sthx $RST, $RA, $RB", IIC_LdStStore, []>,
                          PPC970_DGroup_Cracked;
 def STWXTLS_32 : XForm_8<31, 151, (outs), (ins gprc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                          "stwx $RST, $RA, $RB", IIC_LdStStore, []>,
                          PPC970_DGroup_Cracked;
 
 }
+let mayStore = 1, Predicates = [HasFPU] in {
+def STFSXTLS : XForm_8<31, 663, (outs), (ins f4rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "stfsx $RST, $RA, $RB", IIC_LdStSTFD, []>,
+                       PPC970_DGroup_Cracked;
+def STFDXTLS : XForm_8<31, 727, (outs), (ins f8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "stfdx $RST, $RA, $RB", IIC_LdStSTFD, []>,
+                       PPC970_DGroup_Cracked;
+}
 
 let isCommutable = 1 in
 defm ADDC8 : XOForm_1rc<31, 10, 0, (outs g8rc:$RT), (ins g8rc:$RA, g8rc:$RB),
                         "addc", "$RT, $RA, $RB", IIC_IntGeneral,
                         [(set i64:$RT, (addc i64:$RA, i64:$RB))]>,
                         PPC970_DGroup_Cracked;
 
 let Defs = [CARRY] in

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global double 0x4021947AE147AE14, align 8
 @VarInit = global double 8.787000e+01, align 8
 @IThreadLocalVarUninit = internal thread_local(localexec) global double 0.000000e+00, align 8
 @IThreadLocalVarInit = internal thread_local(localexec) global double 5.870000e+00, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global double 0.000000e+00, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(double noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(double noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define double @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfd f1, 0(r3)
+; SMALL32-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfd f1, 0(r3)
+; LARGE32-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-float.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global float 0x401D333340000000, align 4
 @VarInit = global float 0x4021666660000000, align 4
 @IThreadLocalVarUninit = internal thread_local(localexec) global float 0.000000e+00, align 4
 @IThreadLocalVarInit = internal thread_local(localexec) global float 0x4018CCCCC0000000, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global float 0.000000e+00, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(float noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(float noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define float @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    fadds f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    fadds f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    fadds f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    fadds f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    fadds f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    fadds f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f0, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lfs f1, 0(r3)
+; SMALL32-O0-NEXT:    fadds f1, f0, f1
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f0, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lfs f1, 0(r3)
+; LARGE32-O0-NEXT:    fadds f1, f0, f1
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-int.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global i32 1, align 4
 @VarInit = global i32 87, align 4
 @IThreadLocalVarUninit = internal thread_local(localexec) global i32 0, align 4
 @IThreadLocalVarInit = internal thread_local(localexec) global i32 1, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global i32 0, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C0@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    lwz r5, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r5
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r5, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r5
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C1@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    lwz r5, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r5
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r5, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r5
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C2@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    lwz r5, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r5
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r5, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r5
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C3@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    lwz r5, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r5
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r5, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r5
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define signext i32 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r4, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r4, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r4, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r4, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r4, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r4, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r4, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r4, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global i64 1, align 8
 @VarInit = global i64 87, align 8
 @IThreadLocalVarUninit = internal thread_local(localexec) global i64 0, align 8
 @IThreadLocalVarInit = internal thread_local(localexec) global i64 1, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global i64 0, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r5, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r6
-; SMALL32-NEXT:    stw r4, 4(r3)
-; SMALL32-NEXT:    stw r5, 0(r3)
+; SMALL32-NEXT:    add r7, r3, r6
+; SMALL32-NEXT:    stwx r5, r3, r6
+; SMALL32-NEXT:    stw r4, 4(r7)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r5, r3
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r6, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r6
-; LARGE32-NEXT:    stw r4, 4(r3)
-; LARGE32-NEXT:    stw r5, 0(r3)
+; LARGE32-NEXT:    add r7, r3, r6
+; LARGE32-NEXT:    stwx r5, r3, r6
+; LARGE32-NEXT:    stw r4, 4(r7)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C0@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    mr r5, r4
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    # kill: def $r4 killed $r5
+; SMALL32-O0-NEXT:    lwz r6, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r6
+; SMALL32-O0-NEXT:    stw r5, 4(r4)
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    mr r5, r4
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    # kill: def $r4 killed $r5
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r6, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r6
+; LARGE32-O0-NEXT:    stw r5, 4(r4)
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    mr r5, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r6
-; SMALL32-NEXT:    stw r4, 4(r3)
-; SMALL32-NEXT:    stw r5, 0(r3)
+; SMALL32-NEXT:    add r7, r3, r6
+; SMALL32-NEXT:    stwx r5, r3, r6
+; SMALL32-NEXT:    stw r4, 4(r7)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r5, r3
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r6, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r6
-; LARGE32-NEXT:    stw r4, 4(r3)
-; LARGE32-NEXT:    stw r5, 0(r3)
+; LARGE32-NEXT:    add r7, r3, r6
+; LARGE32-NEXT:    stwx r5, r3, r6
+; LARGE32-NEXT:    stw r4, 4(r7)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C1@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    mr r5, r4
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    # kill: def $r4 killed $r5
+; SMALL32-O0-NEXT:    lwz r6, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r6
+; SMALL32-O0-NEXT:    stw r5, 4(r4)
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    mr r5, r4
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    # kill: def $r4 killed $r5
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r6, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r6
+; LARGE32-O0-NEXT:    stw r5, 4(r4)
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r5, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r6
-; SMALL32-NEXT:    stw r4, 4(r3)
-; SMALL32-NEXT:    stw r5, 0(r3)
+; SMALL32-NEXT:    add r7, r3, r6
+; SMALL32-NEXT:    stwx r5, r3, r6
+; SMALL32-NEXT:    stw r4, 4(r7)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r5, r3
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r6, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r6
-; LARGE32-NEXT:    stw r4, 4(r3)
-; LARGE32-NEXT:    stw r5, 0(r3)
+; LARGE32-NEXT:    add r7, r3, r6
+; LARGE32-NEXT:    stwx r5, r3, r6
+; LARGE32-NEXT:    stw r4, 4(r7)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C2@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    mr r5, r4
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    # kill: def $r4 killed $r5
+; SMALL32-O0-NEXT:    lwz r6, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r6
+; SMALL32-O0-NEXT:    stw r5, 4(r4)
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    mr r5, r4
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    # kill: def $r4 killed $r5
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r6, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r6
+; LARGE32-O0-NEXT:    stw r5, 4(r4)
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    mr r5, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r6
-; SMALL32-NEXT:    stw r4, 4(r3)
-; SMALL32-NEXT:    stw r5, 0(r3)
+; SMALL32-NEXT:    add r7, r3, r6
+; SMALL32-NEXT:    stwx r5, r3, r6
+; SMALL32-NEXT:    stw r4, 4(r7)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r5, r3
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r6, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r6
-; LARGE32-NEXT:    stw r4, 4(r3)
-; LARGE32-NEXT:    stw r5, 0(r3)
+; LARGE32-NEXT:    add r7, r3, r6
+; LARGE32-NEXT:    stwx r5, r3, r6
+; LARGE32-NEXT:    stw r4, 4(r7)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C3@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    mr r5, r4
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    # kill: def $r4 killed $r5
+; SMALL32-O0-NEXT:    lwz r6, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r6
+; SMALL32-O0-NEXT:    stw r5, 4(r4)
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    mr r5, r4
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    # kill: def $r4 killed $r5
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r6, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r6
+; LARGE32-O0-NEXT:    stw r5, 4(r4)
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define i64 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r4, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r4)
-; SMALL32-NEXT:    lwz r4, 4(r4)
+; SMALL32-NEXT:    add r5, r3, r4
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r4, 4(r5)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r4, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r4)
-; LARGE32-NEXT:    lwz r4, 4(r4)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r4, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r4)
+; SMALL32-O0-NEXT:    lwz r4, 4(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r4, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r4)
+; LARGE32-O0-NEXT:    lwz r4, 4(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r6, 4(r5)
+; SMALL32-NEXT:    add r6, r3, r4
+; SMALL32-NEXT:    lwz r7, 4(r5)
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r6, 4(r6)
 ; SMALL32-NEXT:    lwz r5, 0(r5)
-; SMALL32-NEXT:    lwz r4, 4(r3)
-; SMALL32-NEXT:    lwz r3, 0(r3)
-; SMALL32-NEXT:    addc r4, r6, r4
+; SMALL32-NEXT:    addc r4, r7, r6
 ; SMALL32-NEXT:    adde r3, r5, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r4, 4(r3)
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addis r5, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C4@l(r5)
 ; LARGE32-NEXT:    lwz r6, 4(r5)
 ; LARGE32-NEXT:    lwz r5, 0(r5)
 ; LARGE32-NEXT:    addc r4, r6, r4
 ; LARGE32-NEXT:    adde r3, r5, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r6, 4(r3)
+; SMALL32-O0-NEXT:    lwz r5, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r4, 4(r3)
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addc r4, r4, r6
+; SMALL32-O0-NEXT:    adde r3, r3, r5
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r6, 4(r3)
+; LARGE32-O0-NEXT:    lwz r5, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r4, 4(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addc r4, r4, r6
+; LARGE32-O0-NEXT:    adde r3, r3, r5
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r4, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r4)
-; SMALL32-NEXT:    lwz r4, 4(r4)
+; SMALL32-NEXT:    add r5, r3, r4
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r4, 4(r5)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r4, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r4)
-; LARGE32-NEXT:    lwz r4, 4(r4)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r4, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r4)
+; SMALL32-O0-NEXT:    lwz r4, 4(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r4, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r4)
+; LARGE32-O0-NEXT:    lwz r4, 4(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r6, 4(r5)
+; SMALL32-NEXT:    add r6, r3, r4
+; SMALL32-NEXT:    lwz r7, 4(r5)
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r6, 4(r6)
 ; SMALL32-NEXT:    lwz r5, 0(r5)
-; SMALL32-NEXT:    lwz r4, 4(r3)
-; SMALL32-NEXT:    lwz r3, 0(r3)
-; SMALL32-NEXT:    addc r4, r6, r4
+; SMALL32-NEXT:    addc r4, r7, r6
 ; SMALL32-NEXT:    adde r3, r5, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r4, 4(r3)
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addis r5, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C4@l(r5)
 ; LARGE32-NEXT:    lwz r6, 4(r5)
 ; LARGE32-NEXT:    lwz r5, 0(r5)
 ; LARGE32-NEXT:    addc r4, r6, r4
 ; LARGE32-NEXT:    adde r3, r5, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadITLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r6, 4(r3)
+; SMALL32-O0-NEXT:    lwz r5, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r4, 4(r3)
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addc r4, r4, r6
+; SMALL32-O0-NEXT:    adde r3, r3, r5
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadITLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r6, 4(r3)
+; LARGE32-O0-NEXT:    lwz r5, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r4, 4(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addc r4, r4, r6
+; LARGE32-O0-NEXT:    adde r3, r3, r5
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r4, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r4)
-; SMALL32-NEXT:    lwz r4, 4(r4)
+; SMALL32-NEXT:    add r5, r3, r4
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r4, 4(r5)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r4, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r4)
-; LARGE32-NEXT:    lwz r4, 4(r4)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r4, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r4)
+; SMALL32-O0-NEXT:    lwz r4, 4(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r4, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r4)
+; LARGE32-O0-NEXT:    lwz r4, 4(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r6, 4(r5)
+; SMALL32-NEXT:    add r6, r3, r4
+; SMALL32-NEXT:    lwz r7, 4(r5)
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r6, 4(r6)
 ; SMALL32-NEXT:    lwz r5, 0(r5)
-; SMALL32-NEXT:    lwz r4, 4(r3)
-; SMALL32-NEXT:    lwz r3, 0(r3)
-; SMALL32-NEXT:    addc r4, r6, r4
+; SMALL32-NEXT:    addc r4, r7, r6
 ; SMALL32-NEXT:    adde r3, r5, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r4, 4(r3)
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addis r5, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C4@l(r5)
 ; LARGE32-NEXT:    lwz r6, 4(r5)
 ; LARGE32-NEXT:    lwz r5, 0(r5)
 ; LARGE32-NEXT:    addc r4, r6, r4
 ; LARGE32-NEXT:    adde r3, r5, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLUninit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r6, 4(r3)
+; SMALL32-O0-NEXT:    lwz r5, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r4, 4(r3)
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addc r4, r4, r6
+; SMALL32-O0-NEXT:    adde r3, r3, r5
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLUninit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r6, 4(r3)
+; LARGE32-O0-NEXT:    lwz r5, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r4, 4(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addc r4, r4, r6
+; LARGE32-O0-NEXT:    adde r3, r3, r5
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r4, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r4)
-; SMALL32-NEXT:    lwz r4, 4(r4)
+; SMALL32-NEXT:    add r5, r3, r4
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r4, 4(r5)
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r4, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r4)
-; LARGE32-NEXT:    lwz r4, 4(r4)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r4, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r4)
+; SMALL32-O0-NEXT:    lwz r4, 4(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r4, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r4)
+; LARGE32-O0-NEXT:    lwz r4, 4(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r6, 4(r5)
+; SMALL32-NEXT:    add r6, r3, r4
+; SMALL32-NEXT:    lwz r7, 4(r5)
+; SMALL32-NEXT:    lwzx r3, r3, r4
+; SMALL32-NEXT:    lwz r6, 4(r6)
 ; SMALL32-NEXT:    lwz r5, 0(r5)
-; SMALL32-NEXT:    lwz r4, 4(r3)
-; SMALL32-NEXT:    lwz r3, 0(r3)
-; SMALL32-NEXT:    addc r4, r6, r4
+; SMALL32-NEXT:    addc r4, r7, r6
 ; SMALL32-NEXT:    adde r3, r5, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r4, 4(r3)
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    add r5, r3, r4
+; LARGE32-NEXT:    lwzx r3, r3, r4
+; LARGE32-NEXT:    lwz r4, 4(r5)
 ; LARGE32-NEXT:    addis r5, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C4@l(r5)
 ; LARGE32-NEXT:    lwz r6, 4(r5)
 ; LARGE32-NEXT:    lwz r5, 0(r5)
 ; LARGE32-NEXT:    addc r4, r6, r4
 ; LARGE32-NEXT:    adde r3, r5, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadTLInit2:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r6, 4(r3)
+; SMALL32-O0-NEXT:    lwz r5, 0(r3)
+; SMALL32-O0-NEXT:    lwz r3, L..C4(r2) # @VarInit
+; SMALL32-O0-NEXT:    lwz r4, 4(r3)
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addc r4, r4, r6
+; SMALL32-O0-NEXT:    adde r3, r3, r5
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadTLInit2:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r6, 4(r3)
+; LARGE32-O0-NEXT:    lwz r5, 0(r3)
+; LARGE32-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE32-O0-NEXT:    lwz r3, L..C4@l(r3)
+; LARGE32-O0-NEXT:    lwz r4, 4(r3)
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addc r4, r4, r6
+; LARGE32-O0-NEXT:    adde r3, r3, r5
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc-large.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x6
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (15)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x22
+; RELOC:       Virtual Address: 0x12
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCU (0x30)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x2A
+; RELOC:       Virtual Address: 0x1A
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x80
+; RELOC:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (23)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x88
+; RELOC:       Virtual Address: 0x70
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (21)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc-large.ll.tmp.o
 ; SYM-NEXT: Format: aix5coff64-rs6000
 ; SYM-NEXT: Arch: powerpc64
 ; SYM-NEXT: AddressSize: 64bit
 ; SYM-NEXT: Symbols [
 ; SYM:     Index: 15
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x80
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 16
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TE (0x16)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 17
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x88
+; SYM-NEXT:     Value (RelocatableAddress): 0x70
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc-large.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU (idx: 15) IThreadLocalVarUninit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL (idx: 15) IThreadLocalVarUninit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                std 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stdx 3, 13, 4
 ; DIS-NEXT:                                       blr
-; DIS:      0000000000000020 (idx: 5) .loadTLInit:
+; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 19) VarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 8(3)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 16(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 19) VarInit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ldx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000048 (idx: 7) VarInit[RW]:
-; DIS-NEXT:       48: 00 00 00 00
-; DIS-NEXT:       4c: 00 00 00 57
-; DIS:      0000000000000050 (idx: 9) storeITLUninit[DS]:
+; DIS:      0000000000000030 (idx: 7) VarInit[RW]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT:       34: 00 00 00 57
+; DIS:      0000000000000038 (idx: 9) storeITLUninit[DS]:
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       3c: 00 00 00 00
+; DIS-NEXT:       40: 00 00 00 00
+; DIS-NEXT: 0000000000000040:  R_POS        (idx: 13) TOC[TC0]
+; DIS-NEXT:       44: 00 00 00 68
+; DIS:      0000000000000050 (idx: 11) loadTLInit[DS]:
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       54: 00 00 00 00
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       54: 00 00 00 10
 ; DIS-NEXT:       58: 00 00 00 00
 ; DIS-NEXT: 0000000000000058:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 80
-; DIS:      0000000000000068 (idx: 11) loadTLInit[DS]:
+; DIS-NEXT:       5c: 00 00 00 68
+; DIS:      0000000000000068 (idx: 15) IThreadLocalVarUninit[TE]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       6c: 00 00 00 20
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
+; DIS-NEXT:       6c: 00 00 00 00
+; DIS:      0000000000000070 (idx: 17) ThreadLocalVarInit[TE]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       74: 00 00 00 80
-; DIS:      0000000000000080 (idx: 15) IThreadLocalVarUninit[TE]:
-; DIS-NEXT:       80: 00 00 00 00
-; DIS-NEXT: 0000000000000080:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
-; DIS-NEXT:       84: 00 00 00 00
-; DIS:      0000000000000088 (idx: 17) ThreadLocalVarInit[TE]:
-; DIS-NEXT:       88: 00 00 00 00
-; DIS-NEXT: 0000000000000088:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
-; DIS-NEXT:       8c: 00 00 00 00
-; DIS:      0000000000000090 (idx: 19) VarInit[TE]:
-; DIS-NEXT:       90: 00 00 00 00
-; DIS-NEXT: 0000000000000090:  R_POS        (idx: 7) VarInit[RW]
-; DIS-NEXT:       94: 00 00 00 48
+; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
+; DIS-NEXT:       74: 00 00 00 00
+; DIS:      0000000000000078 (idx: 19) VarInit[TE]:
+; DIS-NEXT:       78: 00 00 00 00
+; DIS-NEXT: 0000000000000078:  R_POS        (idx: 7) VarInit[RW]
+; DIS-NEXT:       7c: 00 00 00 30
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 21) ThreadLocalVarInit[TL]:
 ; DIS-NEXT:        0: 00 00 00 00
 ; DIS-NEXT:        4: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000008 (idx: 23) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc-large32.ll

 ; DIS-NEXT:                                       stw 0, 40(1)
 ; DIS-NEXT:                                       mr 5, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU (idx: 17) IThreadLocalVarUninit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 6, 0(3)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL (idx: 17) IThreadLocalVarUninit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA  (idx: 1)      .__get_tpointer[PR]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 3, 6
-; DIS-NEXT:                                       stw 4, 4(3)
-; DIS-NEXT:                                       stw 5, 0(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 7, 3, 6
+; DIS-NEXT:                                       stwx 5, 3, 6
+; DIS-NEXT:                                       stw 4, 4(7)
 ; DIS-NEXT:                                       addi 1, 1, 32
 ; DIS-NEXT:                                       lwz 0, 8(1)
 ; DIS-NEXT:                                       mtlr 0
 ; DIS-NEXT:                                       blr
 ; DIS:      00000040 (idx: 7) .loadTLInit:
 ; DIS-NEXT:                                       mflr 0
 ; DIS-NEXT:                                       stwu 1, -32(1)
 ; DIS-NEXT:                                       stw 0, 40(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 19) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 4, 4(3)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 19) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA  (idx: 1)      .__get_tpointer[PR]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 3, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 4, 4(3)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 3, 0(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 5, 3, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwzx 3, 3, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 4, 4(5)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 5, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 21) VarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 5, 8(5)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 21) VarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 6, 4(5)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                lwz 5, 0(5)
 ; DIS-NEXT:                                       addc 4, 6, 4
 ; DIS-NEXT:                                       adde 3, 5, 3

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x12
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (19)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOC (0x3)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x68
+; RELOC:       Virtual Address: 0x60
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (27)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x70
+; RELOC:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (25)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc.ll.tmp.o
 ; SYM-NEXT: Format: aix5coff64-rs6000
 ; SYM-NEXT: Arch: powerpc64
 ; SYM-NEXT: AddressSize: 64bit
 ; SYM-NEXT: Symbols [
 ; SYM:     Index: 17
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x68
+; SYM-NEXT:     Value (RelocatableAddress): 0x60
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 19
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x70
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 0(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 17) IThreadLocalVarUninit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stw 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stwx 3, 13, 4
 ; DIS-NEXT:                                      blr
 ; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 3, 8(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 19) ThreadLocalVarInit[TC]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 16(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 21) VarInit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwzx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               extsw 3, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000030 (idx: 9) VarInit:
-; DIS-NEXT:       30: 00 00 00 57
-; DIS:      0000000000000038 (idx: 11) storeITLUninit[DS]:
-; DIS-NEXT:       8: 00 00 00 00
-; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       3c: 00 00 00 00
-; DIS-NEXT:       40: 00 00 00 00
-; DIS-NEXT: 0000000000000040:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       44: 00 00 00 68
-; DIS:      0000000000000050 (idx: 13) loadTLInit[DS]:
+; DIS:      000000000000002c (idx: 9) VarInit:
+; DIS-NEXT:       2c: 00 00 00 57
+; DIS:      0000000000000030 (idx: 11) storeITLUninit[DS]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT: 0000000000000030:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       34: 00 00 00 00
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       3c: 00 00 00 60
+; DIS:      0000000000000048 (idx: 13) loadTLInit[DS]:
+; DIS-NEXT:       48: 00 00 00 00
+; DIS-NEXT: 0000000000000048:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       4c: 00 00 00 10
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       54: 00 00 00 10
-; DIS-NEXT:       58: 00 00 00 00
-; DIS-NEXT: 0000000000000058:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 68
-; DIS:      0000000000000068 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       54: 00 00 00 60
+; DIS:      0000000000000060 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT:       60: 00 00 00 00
+; DIS-NEXT: 0000000000000060:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
+; DIS:      0000000000000068 (idx: 19) ThreadLocalVarInit[TC]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
-; DIS:      0000000000000070 (idx: 19) ThreadLocalVarInit[TC]:
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
+; DIS:      0000000000000070 (idx: 21) VarInit[TC]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
-; DIS:      0000000000000078 (idx: 21) VarInit[TC]:
-; DIS-NEXT:       78: 00 00 00 00
-; DIS-NEXT: 0000000000000078:  R_POS        (idx: 9) VarInit
+; DIS-NEXT: 0000000000000070:  R_POS        (idx: 9) VarInit
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 25) ThreadLocalVarInit:
 ; DIS-NEXT:        0: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000004 (idx: 27) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc32.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x3A
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (21)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOC (0x3)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x40
+; RELOC:       Virtual Address: 0x44
 ; RELOC-NEXT:       Symbol: .__get_tpointer (1)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 26
 ; RELOC-NEXT:       Type: R_RBA (0x18)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x84
+; RELOC:       Virtual Address: 0x80
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (29)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 32
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x88
+; RELOC:       Virtual Address: 0x84
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (27)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 32
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc32.ll.tmp.o

 ; SYM-NEXT:       SymbolType: XTY_ER (0x0)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       StabInfoIndex: 0x0
 ; SYM-NEXT:       StabSectNum: 0x0
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 19
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x84
+; SYM-NEXT:     Value (RelocatableAddress): 0x80
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 4
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 2
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       StabInfoIndex: 0x0
 ; SYM-NEXT:       StabSectNum: 0x0
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 21
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x88
+; SYM-NEXT:     Value (RelocatableAddress): 0x84
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 22
 ; SYM-NEXT:       SectionLen: 4
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 ; DIS-NEXT:                                      mflr 0
 ; DIS-NEXT:                                      stwu 1, -32(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 5, 0(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 19) IThreadLocalVarUninit[TC]
 ; DIS-NEXT:                                      mr 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA (idx: 1)      .__get_tpointer[PR]
 ; DIS-NEXT:                                      stw 0, 40(1)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 3, 5
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stw 4, 0(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stwx 4, 3, 5
 ; DIS-NEXT:                                      addi 1, 1, 32
 ; DIS-NEXT:                                      lwz 0, 8(1)
 ; DIS-NEXT:                                      mtlr 0
 ; DIS-NEXT:                                      blr
 ; DIS:      00000030 (idx: 7) .loadTLInit:
 ; DIS-NEXT:                                      mflr 0
 ; DIS-NEXT:                                      stwu 1, -32(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 4, 4(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 21) ThreadLocalVarInit[TC]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 5, 8(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 23) VarInit[TC]
+; DIS-NEXT:                                      stw 0, 40(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA (idx: 1)      .__get_tpointer[PR]
-; DIS-NEXT:                                      stw 0, 40(1)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 3, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwzx 3, 3, 4
 ; DIS-NEXT:                                      lwz 4, 0(5)
-; DIS-NEXT:                                      lwz 3, 0(3)
 ; DIS-NEXT:                                      add 3, 4, 3
 ; DIS-NEXT:                                      addi 1, 1, 32
 ; DIS-NEXT:                                      lwz 0, 8(1)
 ; DIS-NEXT:                                      mtlr 0
 ; DIS-NEXT:                                      blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      00000068 (idx: 11) VarInit:
-; DIS-NEXT:       68: 00 00 00 57
-; DIS:      0000006c (idx: 13) storeITLUninit[DS]:
-; DIS-NEXT:       6c: 00 00 00 00
-; DIS-NEXT: 0000006c:  R_POS        (idx: 5) .storeITLUninit
-; DIS-NEXT:       70: 00 00 00 84
-; DIS-NEXT: 00000070:  R_POS        (idx: 17) TOC[TC0]
-; DIS-NEXT:       74: 00 00 00 00
-; DIS:      00000078 (idx: 15) loadTLInit[DS]:
-; DIS-NEXT:       78: 00 00 00 30
-; DIS-NEXT: 00000078:  R_POS        (idx: 7) .loadTLInit
-; DIS-NEXT:       7c: 00 00 00 84
-; DIS-NEXT: 0000007c:  R_POS        (idx: 17) TOC[TC0]
+; DIS:      00000064 (idx: 11) VarInit:
+; DIS-NEXT:       64: 00 00 00 57
+; DIS:      00000068 (idx: 13) storeITLUninit[DS]:
+; DIS-NEXT:       68: 00 00 00 00
+; DIS-NEXT: 00000068:  R_POS        (idx: 5) .storeITLUninit
+; DIS-NEXT:       6c: 00 00 00 80
+; DIS-NEXT: 0000006c:  R_POS        (idx: 17) TOC[TC0]
+; DIS-NEXT:       70: 00 00 00 00
+; DIS:      00000074 (idx: 15) loadTLInit[DS]:
+; DIS-NEXT:       74: 00 00 00 30
+; DIS-NEXT: 00000074:  R_POS        (idx: 7) .loadTLInit
+; DIS-NEXT:       78: 00 00 00 80
+; DIS-NEXT: 00000078:  R_POS        (idx: 17) TOC[TC0]
+; DIS-NEXT:       7c: 00 00 00 00
+; DIS:      00000080 (idx: 19) IThreadLocalVarUninit[TC]:
 ; DIS-NEXT:       80: 00 00 00 00
-; DIS:      00000084 (idx: 19) IThreadLocalVarUninit[TC]:
+; DIS-NEXT: 00000080:  R_TLS_LE     (idx: 29) IThreadLocalVarUninit[UL]
+; DIS:      00000084 (idx: 21) ThreadLocalVarInit[TC]:
 ; DIS-NEXT:       84: 00 00 00 00
-; DIS-NEXT: 00000084:  R_TLS_LE     (idx: 29) IThreadLocalVarUninit[UL]
-; DIS:      00000088 (idx: 21) ThreadLocalVarInit[TC]:
-; DIS-NEXT:       88: 00 00 00 00
-; DIS-NEXT: 00000088:  R_TLS_LE     (idx: 27) ThreadLocalVarInit
-; DIS:      0000008c (idx: 23) VarInit[TC]:
-; DIS-NEXT:       8c: 00 00 00 68
-; DIS-NEXT: 0000008c:  R_POS        (idx: 11) VarInit
+; DIS-NEXT: 00000084:  R_TLS_LE     (idx: 27) ThreadLocalVarInit
+; DIS:      00000088 (idx: 23) VarInit[TC]:
+; DIS-NEXT:       88: 00 00 00 64
+; DIS-NEXT: 00000088:  R_POS        (idx: 11) VarInit
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      00000000 (idx: 27) ThreadLocalVarInit:
 ; DIS-NEXT:        0: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      00000004 (idx: 29) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-vec_insert_elt.ll

 ; CHECK-64-P10:       # %bb.0: # %entry
 ; CHECK-64-P10-NEXT:    mffprd 3, 1
 ; CHECK-64-P10-NEXT:    rlwinm 4, 4, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble1:
 ; CHECK-32-P10:       # %bb.0: # %entry
-; CHECK-32-P10-NEXT:    rlwinm 3, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 4, 1, -16
+; CHECK-32-P10-NEXT:    rlwinm 3, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 4, 3
 ; CHECK-32-P10-NEXT:    lxv 34, -16(1)
 ; CHECK-32-P10-NEXT:    blr
 entry:
   %vecins = insertelement <2 x double> %a, double %b, i32 %idx1
   ret <2 x double> %vecins
 }

 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 6
 ; CHECK-64-P10-NEXT:    rlwinm 4, 5, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble2:
 ; CHECK-32-P10:       # %bb.0: # %entry
 ; CHECK-32-P10-NEXT:    lfd 0, 0(3)
-; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 6, 1, -32
+; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -32(1)
 ; CHECK-32-P10-NEXT:    rlwinm 5, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stfdx 0, 6, 4
 ; CHECK-32-P10-NEXT:    lxv 0, -32(1)
 ; CHECK-32-P10-NEXT:    plfd 1, 1(3), 0
 ; CHECK-32-P10-NEXT:    addi 3, 1, -16
 ; CHECK-32-P10-NEXT:    stxv 0, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 3, 5

 ; CHECK-64-P10-NEXT:    ldx 3, 3, 4
 ; CHECK-64-P10-NEXT:    rlwinm 4, 5, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble3:
 ; CHECK-32-P10:       # %bb.0: # %entry
 ; CHECK-32-P10-NEXT:    plfd 0, 65536(3), 0
-; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 6, 1, -32
+; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -32(1)
 ; CHECK-32-P10-NEXT:    rlwinm 5, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stfdx 0, 6, 4
 ; CHECK-32-P10-NEXT:    lxv 0, -32(1)
 ; CHECK-32-P10-NEXT:    lfd 1, 0(3)
 ; CHECK-32-P10-NEXT:    addi 3, 1, -16
 ; CHECK-32-P10-NEXT:    stxv 0, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 3, 5

llvm/test/CodeGen/PowerPC/tls-pie-xform.ll

entry:
   ret i8 %add
 }
 
 define dso_local signext i16 @test_short_one() {
 ; CHECK-LABEL: test_short_one:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    addis 3, 2, var_short@got@tprel@ha
 ; CHECK-NEXT:    ld 3, var_short@got@tprel@l(3)
-; CHECK-NEXT:    lhzx 3, 3, var_short@tls
+; CHECK-NEXT:    lhax 3, 3, var_short@tls
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i16, ptr @var_short, align 2, !tbaa !7
   ret i16 %0
 }
 
 define dso_local void @test_short_two(i32 signext %a) {
 ; CHECK-LABEL: test_short_two:
entry:
   ret i16 %add
 }
 
 define dso_local signext i32 @test_int_one() {
 ; CHECK-LABEL: test_int_one:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    addis 3, 2, var_int@got@tprel@ha
 ; CHECK-NEXT:    ld 3, var_int@got@tprel@l(3)
-; CHECK-NEXT:    lwzx 3, 3, var_int@tls
+; CHECK-NEXT:    lwax 3, 3, var_int@tls
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i32, ptr @var_int, align 4, !tbaa !9
   ret i32 %0
 }
 
 define dso_local void @test_int_two(i32 signext %a) {
 ; CHECK-LABEL: test_int_two: