[AIX][TLS] Generate optimized local-exec access code sequence using X-Form loads/stores

[amyk]

Summary

This patch is a follow up to D149722, D152669 and D153645, where a slightly more optimized code
sequence is generated for 64-bit and 32-bit local-exec accesses when optimizations are turned on.

Handling is added PPCISelDAGToDAG.cpp in order to check if any D-form loads or stores that follow
an PPCISD::ADD_TLS can be optimized to use an X-Form load or store. In this particular situation,
this allows the ADD_TLS node to be removed completely.

[amyk]

Rebase patch.

[amyk]

Update this patch with a simplified way of generating the X-Form instructions, leveraging some existing infrastructure in place.

I am planning on updating this patch once again to fix a potential issue.

[amyk]

• Update patch to produce lhax and lwax when the load is sign-extending.
• Rebase patch.

[amyk]

Rebase patch and update comments in the patch.

[amyk]

Update patch to account for 32-bit implementation in D152669.

[amyk]

• Rebase patch.
• Update patch comments.

[amyk]

• Rebase patch to account for updating D152669.
• Use the correct PPCISD node (GET_TPOINTER) in canOptimizeTLSDFormToXFormOnAIX().

[amyk]

Rebase patch and simplify relocation test checks.

[amyk]

Patch rebased.

[amyk]

• Specify why we are doing this optimization in the description.
• Check on why these TLS instructions were not previously added.

llvm/lib/Target/PowerPC/P10InstrResources.td

1299

Check these definitions in the P10InstrResources.td.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

714

• Fix this comment.
• Can this function be general? Include both of the Linux and AIX parts.

743

Instead of explicitly using PPC::X13, is there something like Subtarget.getThreadPointerRegister() that we can use?

[amyk]

Address review comments:

• Simplify canOptimizeTLSDFormToXForm()
• Add getThreadPointerRegister() into PPCSubtarget.h
• Confirmed that adding the new *TLS load/store instructions are appropriate

[amyk]

llvm/lib/Target/PowerPC/PPCSubtarget.h

279 ↗

(On Diff #533580)

Is it appropriate to add this in this patch? Or should this be separate?

[amyk]

Like in D47382, it is possible that I need to add *TLS_ versions of the instructions that I added. I plan to update this patch to account for this.

[amyk]

Group code review comments.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

740

These two checks for the offset can be combined.

748

Update this comment for GET_TPOINTER, and just mention it's produced for AIX on 32-bit mode.

759

• Implement a static helper function, to check if the node of a thread pointer acquisition node (if one of the operands is the thread pointer - either R13, or __get_tpointer).
• If conditions are satisfied, return true.
• Otherwise, return false at the end of the static helper function.

817

A ternary condition for isSExt for here and below might be better.

llvm/lib/Target/PowerPC/PPCSubtarget.h

279 ↗

(On Diff #533580)

Discussed this with the team. It is appropriate to add this here.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

8

Instead of adding -O0 lines to every test, we can add one file to show the -O0 codegen.
That way, we can remove the -O0 run lines from these existing test cases.

[amyk]

Update patch to:

• Separate out the new instruction definitions (which have been added in https://reviews.llvm.org/D153645)
• Update comments
• Add a new static helper function to check if one of the operands of an ADD_TLS node uses the thread pointer

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

740

Done.

748

Done.

759

isThreadPointerAcquisitionNode() is added to account for this.

817

Done.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

8

Done.

[lei]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

751–754

769–770

[amyk]

• Rebase patch.
• Address comments from Lei: combine two if conditions, and return with the isThreadPointerAcquisitionNode() function inside canOptimizeTLSDFormToXForm().

[lei]

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

34–50

This transformation increased reg pressure with no obvious benefits. Please see if we can bail out for such cases.

[amyk]

Update the patch to only do this optimization if all users of the ADD_TLS node are normal loads with an offset of 0.
This resolves an issue that Lei pointed out with respect to increasing the usage of registers when we have a 64-bit load/store on a 32-bit system - in such a situation, we no longer perform this optimization.

[lei]

Just a few nits, otherwise LGTM.
Thanks for addressing the issue with increasing the usage of registers when we have a 64-bit load/store on a 32-bit system.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

755

Since Base is only used on line 788, consider just inlining it there vs defining a new variable.

798

Since Base only used here, can inline and remove declaration.

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

755

Actually, Base is also used here below:

SDValue Ops[] = {ST->getValue(), Base.getOperand(0), Base.getOperand(1),
                   Chain};

798

Actually, Base is also used here below, right?

SDValue Ops[] = {Base.getOperand(0), Base.getOperand(1), Chain};

llvm/lib/Target/PowerPC/P10InstrResources.td

 
 // 6 Cycles Load operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_ANY, P10LD_Read],
       (instrs
     LBZ, LBZ8,
     LD,  LDtoc,  LDtocBA,  LDtocCPT,  LDtocJTI,  LDtocL, SPILLTOVSR_LD,
     LDBRX,
      DFLOADf32, DFLOADf64, LFD,
-    LFDX,  XFLOADf32, XFLOADf64,
     LFIWAX, LIWAX,
     LFIWZX, LIWZX,
     LHA, LHA8,
     LHAX, LHAX8,
     LHBRX, LHBRX8,
     LHZ, LHZ8,
     LVEBX,
     LVEHX,

 // 6 Cycles Load operations, 2 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_ANY, P10LD_Read, P10LD_Read],
       (instrs
     DCBT,
     DCBTST,
     ICBT,
     LBZX, LBZX8, LBZXTLS, LBZXTLS_, LBZXTLS_32,
     LDX, LDXTLS, LDXTLS_, SPILLTOVSR_LDX,
+    LFDX, LFDXTLS,  XFLOADf32, XFLOADf64,
     LHZX, LHZX8, LHZXTLS, LHZXTLS_, LHZXTLS_32,
     LWZX, LWZX8, LWZXTLS, LWZXTLS_, LWZXTLS_32,
     LXVL,
     LXVLL
 )>;
 
 // 2-way crack instructions
 // 6 Cycles Load operations, and 13 Cycles Decimal Floating Point operations, 2 input operands
def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10LD_Read],
     PLXVP, PLXVPpc
 )>;
 
 // 2-way crack instructions
 // 6 Cycles Load operations, and 4 Cycles ALU2 operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C],
       (instrs
     LFS,
-    LFSX,
     LXSSP,
     LXSSPX
 )>;
 
+// 2-way crack instructions
+// 6 Cycles Load operations, and 4 Cycles ALU2 operations, 2 input operands
+def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C],
+      (instrs
+    LFSX, LFSXTLS
+)>;
+
 // 4-way crack instructions
 // 6 Cycles Load operations, 4 Cycles ALU2 operations, 3 Cycles Simple Fixed-point (SFX) operations, and 3 Cycles ALU operations, 1 input operands
 def : InstRW<[P10W_LD_6C, P10W_DISP_PAIR, P10W_F2_4C, P10W_SX_3C, P10W_DISP_ANY, P10W_FX_3C, P10W_DISP_ANY],
       (instrs
     LFSU,
     LFSUX
 )>;
 
def : InstRW<[P10W_ST_3C, P10W_DISP_ANY, P10ST_Read, P10ST_Read],
     STBUX, STBUX8,
     SPILLTOVSR_ST, STD,
     STDBRX,
     STDU,
     STDUX,
      DFSTOREf32, DFSTOREf64, STFD,
     STFDU,
     STFDUX,
-    STFDX,
     STFIWX, STIWX,
     STFS,
     STFSU,
     STFSUX,
-    STFSX,
     STH, STH8,
     STHBRX,
     STHU, STHU8,
     STHUX, STHUX8,
     STVEBX,
     STVEHX,
     STVEWX,
     STVX,

 )>;
 
 // 3 Cycles Store operations, 3 input operands
 def : InstRW<[P10W_ST_3C, P10W_DISP_ANY, P10ST_Read, P10ST_Read, P10ST_Read],
       (instrs
     CP_COPY, CP_COPY8,
     STBX, STBX8, STBXTLS, STBXTLS_, STBXTLS_32,
     SPILLTOVSR_STX, STDX, STDXTLS, STDXTLS_,
+    STFDX, STFDXTLS,
+    STFSX, STFSXTLS,
     STHX, STHX8, STHXTLS, STHXTLS_, STHXTLS_32,
     STWX, STWX8, STWXTLS, STWXTLS_, STWXTLS_32,
     STXVL,
     STXVLL
 )>;
 
 // Single crack instructions
 // 3 Cycles Store operations, 0 input operands

llvm/lib/Target/PowerPC/P9InstrResources.td

 )>;
 
 // 4 Cycle Restricted load uses a single slice but the dispatch for the whole
 // superslice.
 def : InstRW<[P9_LS_4C, IP_AGEN_1C, DISP_3SLOTS_1C],
       (instrs
     LFIWZX,
     LFDX,
+    LFDXTLS,
     LFD
 )>;
 
 // Cracked Load Instructions.
 // Load instructions that can be done in parallel.
 def : InstRW<[P9_LS_4C, P9_LS_4C, IP_AGEN_1C, IP_AGEN_1C,
               DISP_PAIR_1C],
       (instrs

 // Requires consecutive Load (4 cycles) and ALU (3 cycles) pieces totaling 7
 // cycles. The Load and ALU operations cannot be done at the same time and so
 // their latencies are added.
 // Full 6 dispatches are required as this is a restricted instruction.
 def : InstRW<[P9_LoadAndALU2Op_7C, IP_AGEN_1C, IP_EXEC_1C,
               DISP_3SLOTS_1C, DISP_3SLOTS_1C],
       (instrs
     LFSX,
+    LFSXTLS,
     LFS
 )>;
 
 // Cracked Load instruction.
 // Requires consecutive Load and ALU pieces totaling 8 cycles. The Load and ALU
 // operations cannot be done at the same time and so their latencies are added.
 // Full 4 dispatches are required as this is a cracked instruction.
 def : InstRW<[P9_LoadAndALU2Op_8C, IP_AGEN_1C, IP_EXEC_1C, DISP_1C, DISP_1C],
def : InstRW<[P9_LoadAndPMOp_8C, IP_AGEN_1C, IP_EXECE_1C, IP_EXECO_1C,
     LXVDSX,
     LXVW4X
 )>;
 
 // Single slice Restricted store operation. The restricted operation requires
 // all three dispatches for the superslice.
 def : InstRW<[P9_LS_1C, IP_EXEC_1C, IP_AGEN_1C, DISP_3SLOTS_1C],
       (instrs
-    (instregex "STF(S|D|IWX|SX|DX)$"),
+    (instregex "STF(S|D|IWX|SX|DX|SXTLS|DXTLS)$"),
     (instregex "STXS(D|DX|SPX|IWX|IBX|IHX|SP)(v)?$"),
     (instregex "STW(8)?$"),
     (instregex "(D|X)FSTORE(f32|f64)$"),
     (instregex "ST(W|H|D)BRX$"),
     (instregex "ST(B|H|D)(8)?$"),
     (instregex "ST(B|W|H|D)(CI)?X(TLS|TLS_32)?(8)?(_)?$"),
     STIWX,
     SLBIEG,

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

bool PPCDAGToDAGISel::isRotateAndMask(SDNode *N, unsigned Mask,
   if (Mask && !(Mask & Indeterminant)) {
     SH = Shift & 31;
     // make sure the mask is still a mask (wrap arounds may not be)
     return isRunOfOnes(Mask, MB, ME);
   }
   return false;
 }
 
+// canOptimizeTLSDFormToXFormOnAIX - For TLS local-exec accesses on AIX 64-bit,
+// an ADD_TLS node is produced to add the result of loading the variable offset
+// to the thread pointer (X13). This add, followed by a D-Form memory operation,
+// can be optimized to use an X-Form load or store, allowing the ADD_TLS node to
+// be removed completely. This can be done as long as the memory operation's
+// immediate offset is 0 and that the thread pointer is being added to.
+static bool canOptimizeTLSDFormToXFormOnAIX(SelectionDAG *CurDAG,
+                                            SDValue Base, uint64_t Offset) {
+  const PPCSubtarget &Subtarget =
+      CurDAG->getMachineFunction().getSubtarget<PPCSubtarget>();
+  if (Subtarget.isAIXABI()) {
+    // As seen above, Base is ADD_TLS. The first operand in this instruction
+    // should be the thread pointer (X13), so we should not perform this
+    // optimization if we're not dealing with X13.
+    RegisterSDNode *AddFirstOpReg =
+      dyn_cast_or_null<RegisterSDNode>(Base.getOperand(0).getNode());
+    if (!AddFirstOpReg)
+      return false;
+    if (AddFirstOpReg->getReg() != PPC::X13)
+      return false;
+    // The optimization to convert the D-Form load/store into its X-Form
+    // counterpart should only occur if the immediate offset is 0.
+    if (Offset != 0)
+      return false;
+  }
+  return true;
+}
+
 bool PPCDAGToDAGISel::tryTLSXFormStore(StoreSDNode *ST) {
   SDValue Base = ST->getBasePtr();
   if (Base.getOpcode() != PPCISD::ADD_TLS)
     return false;
   SDValue Offset = ST->getOffset();
   if (!Offset.isUndef())
     return false;
   if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
     return false;
+  // Check if we can optimize the 64-bit local-exec sequence on AIX.
+  if (!canOptimizeTLSDFormToXFormOnAIX(CurDAG, Base, ST->getSrcValueOffset()))
+    return false;
 
   SDLoc dl(ST);
   EVT MemVT = ST->getMemoryVT();
   EVT RegVT = ST->getValue().getValueType();
 
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
       Opcode = (RegVT == MVT::i32) ? PPC::STBXTLS_32 : PPC::STBXTLS;
       break;
     }
     case MVT::i16: {
       Opcode = (RegVT == MVT::i32) ? PPC::STHXTLS_32 : PPC::STHXTLS;
       break;
     }
     case MVT::i32: {
       Opcode = (RegVT == MVT::i32) ? PPC::STWXTLS_32 : PPC::STWXTLS;
       break;
     }
     case MVT::i64: {
       Opcode = PPC::STDXTLS;
       break;
     }
+    case MVT::f32: {
+      Opcode = PPC::STFSXTLS;
+      break;
+    }
+    case MVT::f64: {
+      Opcode = PPC::STFDXTLS;
+      break;
+    }
   }
   SDValue Chain = ST->getChain();
   SDVTList VTs = ST->getVTList();
   SDValue Ops[] = {ST->getValue(), Base.getOperand(0), Base.getOperand(1),
                    Chain};
   SDNode *MN = CurDAG->getMachineNode(Opcode, dl, VTs, Ops);
   transferMemOperands(ST, MN);
   ReplaceNode(ST, MN);
   return true;
 }
 
 bool PPCDAGToDAGISel::tryTLSXFormLoad(LoadSDNode *LD) {
   SDValue Base = LD->getBasePtr();
   if (Base.getOpcode() != PPCISD::ADD_TLS)
     return false;
   SDValue Offset = LD->getOffset();
   if (!Offset.isUndef())
     return false;
   if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
     return false;
+  // Check if we can optimize the 64-bit local-exec sequence on AIX.
+  if (!canOptimizeTLSDFormToXFormOnAIX(CurDAG, Base, LD->getSrcValueOffset()))
+    return false;
 
   SDLoc dl(LD);
   EVT MemVT = LD->getMemoryVT();
   EVT RegVT = LD->getValueType(0);
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
       Opcode = (RegVT == MVT::i32) ? PPC::LBZXTLS_32 : PPC::LBZXTLS;
       break;
     }
     case MVT::i16: {
       Opcode = (RegVT == MVT::i32) ? PPC::LHZXTLS_32 : PPC::LHZXTLS;
       break;
     }
     case MVT::i32: {
       Opcode = (RegVT == MVT::i32) ? PPC::LWZXTLS_32 : PPC::LWZXTLS;
       break;
     }
     case MVT::i64: {
       Opcode = PPC::LDXTLS;
       break;
     }
+    case MVT::f32: {
+      Opcode = PPC::LFSXTLS;
+      break;
+    }
+    case MVT::f64: {
+      Opcode = PPC::LFDXTLS;
+      break;
+    }
   }
   SDValue Chain = LD->getChain();
   SDVTList VTs = LD->getVTList();
   SDValue Ops[] = {Base.getOperand(0), Base.getOperand(1), Chain};
   SDNode *MN = CurDAG->getMachineNode(Opcode, dl, VTs, Ops);
   transferMemOperands(LD, MN);
   ReplaceNode(LD, MN);
   return true;
if (N->getValueType(0) == MVT::i64) {
                                N0, ShiftAmt);
       CurDAG->SelectNodeTo(N, PPC::ADDZE, MVT::i32, SDValue(Op, 0),
                            SDValue(Op, 1));
       return;
     }
   }
 
   case ISD::STORE: {
-    // Change TLS initial-exec D-form stores to X-form stores.
+    // Change TLS initial-exec (or TLS local-exec on AIX) D-form stores to
+    // X-form stores.
     StoreSDNode *ST = cast<StoreSDNode>(N);
-    if (EnableTLSOpt && Subtarget->isELFv2ABI() &&
+    if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()) &&
         ST->getAddressingMode() != ISD::PRE_INC)
       if (tryTLSXFormStore(ST))
         return;
     break;
   }
   case ISD::LOAD: {
     // Handle preincrement loads.
     LoadSDNode *LD = cast<LoadSDNode>(N);
     EVT LoadedVT = LD->getMemoryVT();
 
     // Normal loads are handled by code generated from the .td file.
     if (LD->getAddressingMode() != ISD::PRE_INC) {
-      // Change TLS initial-exec D-form loads to X-form loads.
-      if (EnableTLSOpt && Subtarget->isELFv2ABI())
+      // Change TLS initial-exec (or TLS local-exec on AIX) D-form loads to
+      // X-form loads.
+      if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()))
         if (tryTLSXFormLoad(LD))
           return;
       break;
     }
 
     SDValue Offset = LD->getOffset();
     if (Offset.getOpcode() == ISD::TargetConstant ||
         Offset.getOpcode() == ISD::TargetGlobalAddress) {

llvm/lib/Target/PowerPC/PPCInstr64Bit.td

 def LBZXTLS_32 : XForm_1<31,  87, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lbzx $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LHZXTLS_32 : XForm_1<31, 279, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lhzx $RST, $RA, $RB", IIC_LdStLoad, []>;
 def LWZXTLS_32 : XForm_1<31,  23, (outs gprc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
                          "lwzx $RST, $RA, $RB", IIC_LdStLoad, []>;
 
 }
+let mayLoad = 1, Predicates = [HasFPU] in {
+def LFSXTLS : XForm_25<31, 535, (outs f4rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "lfsx $RST, $RA, $RB", IIC_LdStLFD, []>;
+def LFDXTLS : XForm_25<31, 599, (outs f8rc:$RST), (ins ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "lfdx $RST, $RA, $RB", IIC_LdStLFD, []>;
+}
 
 let mayStore = 1 in {
 def STBXTLS : XForm_8<31, 215, (outs), (ins g8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                       "stbx $RST, $RA, $RB", IIC_LdStStore, []>,
                       PPC970_DGroup_Cracked;
 def STHXTLS : XForm_8<31, 407, (outs), (ins g8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                       "sthx $RST, $RA, $RB", IIC_LdStStore, []>,
                       PPC970_DGroup_Cracked;

 def STHXTLS_32 : XForm_8<31, 407, (outs), (ins gprc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                          "sthx $RST, $RA, $RB", IIC_LdStStore, []>,
                          PPC970_DGroup_Cracked;
 def STWXTLS_32 : XForm_8<31, 151, (outs), (ins gprc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
                          "stwx $RST, $RA, $RB", IIC_LdStStore, []>,
                          PPC970_DGroup_Cracked;
 
 }
+let mayStore = 1, Predicates = [HasFPU] in {
+def STFSXTLS : XForm_8<31, 663, (outs), (ins f4rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "stfsx $RST, $RA, $RB", IIC_LdStSTFD, []>,
+                       PPC970_DGroup_Cracked;
+def STFDXTLS : XForm_8<31, 727, (outs), (ins f8rc:$RST, ptr_rc_nor0:$RA, tlsreg:$RB),
+                       "stfdx $RST, $RA, $RB", IIC_LdStSTFD, []>,
+                       PPC970_DGroup_Cracked;
+}
 
 let isCommutable = 1 in
 defm ADDC8 : XOForm_1rc<31, 10, 0, (outs g8rc:$RT), (ins g8rc:$RA, g8rc:$RB),
                         "addc", "$RT, $RA, $RB", IIC_IntGeneral,
                         [(set i64:$RT, (addc i64:$RA, i64:$RB))]>,
                         PPC970_DGroup_Cracked;
 
 let Defs = [CARRY] in

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global double 0x4021947AE147AE14, align 8
 @VarInit = global double 8.787000e+01, align 8
 @IThreadLocalVarUninit = internal thread_local(localexec) global double 0.000000e+00, align 8
 @IThreadLocalVarInit = internal thread_local(localexec) global double 5.870000e+00, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global double 0.000000e+00, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(double noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(double noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define double @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lxsdx f0, 0, r3
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    xsadddp f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lxsdx f0, 0, r3
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    xsadddp f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-float.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global float 0x401D333340000000, align 4
 @VarInit = global float 0x4021666660000000, align 4
 @IThreadLocalVarUninit = internal thread_local(localexec) global float 0.000000e+00, align 4
 @IThreadLocalVarInit = internal thread_local(localexec) global float 0x4018CCCCC0000000, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global float 0.000000e+00, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(float noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(float noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define float @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lfs f1, 0(r3)
+; SMALL64-O0-NEXT:    fadds f1, f0, f1
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lfs f1, 0(r3)
+; LARGE64-O0-NEXT:    fadds f1, f0, f1
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-int.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global i32 1, align 4
 @VarInit = global i32 87, align 4
 @IThreadLocalVarUninit = internal thread_local(localexec) global i32 0, align 4
 @IThreadLocalVarInit = internal thread_local(localexec) global i32 1, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global i32 0, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C0@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C1@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C2@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C3@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define signext i32 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwa r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwa r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    extsw r3, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    extsw r3, r3
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
 
 @ThreadLocalVarInit = thread_local(localexec) global i64 1, align 8
 @VarInit = global i64 87, align 8
 @IThreadLocalVarUninit = internal thread_local(localexec) global i64 0, align 8
 @IThreadLocalVarInit = internal thread_local(localexec) global i64 1, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global i64 0, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C0@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C1@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C2@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: storeTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C3@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define i64 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadITLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadITLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLUninit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLUninit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   ret i64 %1
 }
 
 define i64 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
+;
+; SMALL64-O0-LABEL: loadTLInit2:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r4, 0(r3)
+; SMALL64-O0-NEXT:    ld r3, L..C4(r2) # @VarInit
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    add r3, r3, r4
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadTLInit2:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r4, 0(r3)
+; LARGE64-O0-NEXT:    addis r3, L..C4@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C4@l(r3)
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    add r3, r3, r4
+; LARGE64-O0-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load i64, ptr %0, align 8
   %2 = load i64, ptr @VarInit, align 8
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc-large.ll

 ; RELOC-NEXT:       Virtual Address: 0x6
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (15)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x22
+; RELOC-NEXT:       Virtual Address: 0x12
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCU (0x30)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x26
+; RELOC-NEXT:       Virtual Address: 0x16
 ; RELOC-NEXT:       Symbol: VarInit (19)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCU (0x30)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x2A
+; RELOC-NEXT:       Virtual Address: 0x1A
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x2E
+; RELOC-NEXT:       Virtual Address: 0x1E
 ; RELOC-NEXT:       Symbol: VarInit (19)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:   }
 ; RELOC-NEXT:   Section (index: 2) .data {
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x50
+; RELOC-NEXT:       Virtual Address: 0x38
 ; RELOC-NEXT:       Symbol: .storeITLUninit (3)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x58
+; RELOC-NEXT:       Virtual Address: 0x40
 ; RELOC-NEXT:       Symbol: TOC (13)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x68
+; RELOC-NEXT:       Virtual Address: 0x50
 ; RELOC-NEXT:       Symbol: .loadTLInit (5)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x70
+; RELOC-NEXT:       Virtual Address: 0x58
 ; RELOC-NEXT:       Symbol: TOC (13)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x80
+; RELOC-NEXT:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (23)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x88
+; RELOC-NEXT:       Virtual Address: 0x70
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (21)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x90
+; RELOC-NEXT:       Virtual Address: 0x78
 ; RELOC-NEXT:       Symbol: VarInit (7)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:   }
 ; RELOC-NEXT: ]

 ; SYM-NEXT:     Name: .text
 ; SYM-NEXT:     Value (RelocatableAddress): 0x0
 ; SYM-NEXT:     Section: .text
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 2
-; SYM-NEXT:       SectionLen: 68
+; SYM-NEXT:       SectionLen: 48
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 5
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }

 ; SYM-NEXT:       SymbolType: XTY_LD (0x2)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 5
 ; SYM-NEXT:     Name: .loadTLInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x20
+; SYM-NEXT:     Value (RelocatableAddress): 0x10
 ; SYM-NEXT:     Section: .text
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 6
 ; SYM-NEXT:       ContainingCsectSymbolIndex: 1
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 0
 ; SYM-NEXT:       SymbolType: XTY_LD (0x2)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 7
 ; SYM-NEXT:     Name: VarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x48
+; SYM-NEXT:     Value (RelocatableAddress): 0x30
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 8
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_RW (0x5)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 9
 ; SYM-NEXT:     Name: storeITLUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x50
+; SYM-NEXT:     Value (RelocatableAddress): 0x38
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 10
 ; SYM-NEXT:       SectionLen: 24
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_DS (0xA)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 11
 ; SYM-NEXT:     Name: loadTLInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x68
+; SYM-NEXT:     Value (RelocatableAddress): 0x50
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 12
 ; SYM-NEXT:       SectionLen: 24
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_DS (0xA)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 13
 ; SYM-NEXT:     Name: TOC
-; SYM-NEXT:     Value (RelocatableAddress): 0x80
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 14
 ; SYM-NEXT:       SectionLen: 0
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 2
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC0 (0xF)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 15
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x80
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 16
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TE (0x16)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 17
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x88
+; SYM-NEXT:     Value (RelocatableAddress): 0x70
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TE (0x16)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 19
 ; SYM-NEXT:     Name: VarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x90
+; SYM-NEXT:     Value (RelocatableAddress): 0x78
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc-large.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU (idx: 15) IThreadLocalVarUninit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL (idx: 15) IThreadLocalVarUninit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                std 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stdx 3, 13, 4
 ; DIS-NEXT:                                       blr
-; DIS:      0000000000000020 (idx: 5) .loadTLInit:
+; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 19) VarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 8(3)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 16(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 19) VarInit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ldx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000048 (idx: 7) VarInit[RW]:
-; DIS-NEXT:       48: 00 00 00 00
-; DIS-NEXT:       4c: 00 00 00 57
-; DIS:      0000000000000050 (idx: 9) storeITLUninit[DS]:
+; DIS:      0000000000000030 (idx: 7) VarInit[RW]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT:       34: 00 00 00 57
+; DIS:      0000000000000038 (idx: 9) storeITLUninit[DS]:
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       3c: 00 00 00 00
+; DIS-NEXT:       40: 00 00 00 00
+; DIS-NEXT: 0000000000000040:  R_POS        (idx: 13) TOC[TC0]
+; DIS-NEXT:       44: 00 00 00 68
+; DIS:      0000000000000050 (idx: 11) loadTLInit[DS]:
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       54: 00 00 00 00
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       54: 00 00 00 10
 ; DIS-NEXT:       58: 00 00 00 00
 ; DIS-NEXT: 0000000000000058:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 80
-; DIS:      0000000000000068 (idx: 11) loadTLInit[DS]:
+; DIS-NEXT:       5c: 00 00 00 68
+; DIS:      0000000000000068 (idx: 15) IThreadLocalVarUninit[TE]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       6c: 00 00 00 20
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
+; DIS-NEXT:       6c: 00 00 00 00
+; DIS:      0000000000000070 (idx: 17) ThreadLocalVarInit[TE]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       74: 00 00 00 80
-; DIS:      0000000000000080 (idx: 15) IThreadLocalVarUninit[TE]:
-; DIS-NEXT:       80: 00 00 00 00
-; DIS-NEXT: 0000000000000080:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
-; DIS-NEXT:       84: 00 00 00 00
-; DIS:      0000000000000088 (idx: 17) ThreadLocalVarInit[TE]:
-; DIS-NEXT:       88: 00 00 00 00
-; DIS-NEXT: 0000000000000088:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
-; DIS-NEXT:       8c: 00 00 00 00
-; DIS:      0000000000000090 (idx: 19) VarInit[TE]:
-; DIS-NEXT:       90: 00 00 00 00
-; DIS-NEXT: 0000000000000090:  R_POS        (idx: 7) VarInit[RW]
-; DIS-NEXT:       94: 00 00 00 48
+; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
+; DIS-NEXT:       74: 00 00 00 00
+; DIS:      0000000000000078 (idx: 19) VarInit[TE]:
+; DIS-NEXT:       78: 00 00 00 00
+; DIS-NEXT: 0000000000000078:  R_POS        (idx: 7) VarInit[RW]
+; DIS-NEXT:       7c: 00 00 00 30
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 21) ThreadLocalVarInit[TL]:
 ; DIS-NEXT:        0: 00 00 00 00
 ; DIS-NEXT:        4: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000008 (idx: 23) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc.ll

 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOC (0x3)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:   }
 ; RELOC-NEXT:   Section (index: 2) .data {
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x38
+; RELOC-NEXT:       Virtual Address: 0x30
 ; RELOC-NEXT:       Symbol: .storeITLUninit (3)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x40
+; RELOC-NEXT:       Virtual Address: 0x38
 ; RELOC-NEXT:       Symbol: TOC (15)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x50
+; RELOC-NEXT:       Virtual Address: 0x48
 ; RELOC-NEXT:       Symbol: .loadTLInit (5)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x58
+; RELOC-NEXT:       Virtual Address: 0x50
 ; RELOC-NEXT:       Symbol: TOC (15)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x68
+; RELOC-NEXT:       Virtual Address: 0x60
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (27)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x70
+; RELOC-NEXT:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (25)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:     Relocation {
-; RELOC-NEXT:       Virtual Address: 0x78
+; RELOC-NEXT:       Virtual Address: 0x70
 ; RELOC-NEXT:       Symbol: VarInit (9)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_POS (0x0)
 ; RELOC-NEXT:     }
 ; RELOC-NEXT:   }
 ; RELOC-NEXT: ]

 ; SYM-NEXT:     Name: .text
 ; SYM-NEXT:     Value (RelocatableAddress): 0x0
 ; SYM-NEXT:     Section: .text
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 2
-; SYM-NEXT:       SectionLen: 48
+; SYM-NEXT:       SectionLen: 44
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 5
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }

 ; SYM-NEXT:       SymbolType: XTY_LD (0x2)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 7
 ; SYM-NEXT:     Name: .data
-; SYM-NEXT:     Value (RelocatableAddress): 0x30
+; SYM-NEXT:     Value (RelocatableAddress): 0x2C
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 8
 ; SYM-NEXT:       SectionLen: 4
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 2
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_RW (0x5)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 9
 ; SYM-NEXT:     Name: VarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x30
+; SYM-NEXT:     Value (RelocatableAddress): 0x2C
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 10
 ; SYM-NEXT:       ContainingCsectSymbolIndex: 7
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 0
 ; SYM-NEXT:       SymbolType: XTY_LD (0x2)
 ; SYM-NEXT:       StorageMappingClass: XMC_RW (0x5)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 11
 ; SYM-NEXT:     Name: storeITLUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x38
+; SYM-NEXT:     Value (RelocatableAddress): 0x30
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 12
 ; SYM-NEXT:       SectionLen: 24
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_DS (0xA)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 13
 ; SYM-NEXT:     Name: loadTLInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x50
+; SYM-NEXT:     Value (RelocatableAddress): 0x48
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_EXT (0x2)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 14
 ; SYM-NEXT:       SectionLen: 24
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_DS (0xA)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 15
 ; SYM-NEXT:     Name: TOC
-; SYM-NEXT:     Value (RelocatableAddress): 0x68
+; SYM-NEXT:     Value (RelocatableAddress): 0x60
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 16
 ; SYM-NEXT:       SectionLen: 0
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 2
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC0 (0xF)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 17
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x68
+; SYM-NEXT:     Value (RelocatableAddress): 0x60
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 19
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x70
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM-NEXT:   Symbol {
 ; SYM-NEXT:     Index: 21
 ; SYM-NEXT:     Name: VarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x78
+; SYM-NEXT:     Value (RelocatableAddress): 0x70
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 22
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 ; SYM-NEXT:   }
 ; SYM-NEXT: ]
 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 0(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 17) IThreadLocalVarUninit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stw 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stwx 3, 13, 4
 ; DIS-NEXT:                                      blr
 ; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 3, 8(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 19) ThreadLocalVarInit[TC]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 16(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 21) VarInit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwzx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               extsw 3, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000030 (idx: 9) VarInit:
-; DIS-NEXT:       30: 00 00 00 57
-; DIS:      0000000000000038 (idx: 11) storeITLUninit[DS]:
-; DIS-NEXT:       8: 00 00 00 00
-; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       3c: 00 00 00 00
-; DIS-NEXT:       40: 00 00 00 00
-; DIS-NEXT: 0000000000000040:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       44: 00 00 00 68
-; DIS:      0000000000000050 (idx: 13) loadTLInit[DS]:
+; DIS:      000000000000002c (idx: 9) VarInit:
+; DIS-NEXT:       2c: 00 00 00 57
+; DIS:      0000000000000030 (idx: 11) storeITLUninit[DS]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT: 0000000000000030:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       34: 00 00 00 00
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       3c: 00 00 00 60
+; DIS:      0000000000000048 (idx: 13) loadTLInit[DS]:
+; DIS-NEXT:       48: 00 00 00 00
+; DIS-NEXT: 0000000000000048:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       4c: 00 00 00 10
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       54: 00 00 00 10
-; DIS-NEXT:       58: 00 00 00 00
-; DIS-NEXT: 0000000000000058:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 68
-; DIS:      0000000000000068 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       54: 00 00 00 60
+; DIS:      0000000000000060 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT:       60: 00 00 00 00
+; DIS-NEXT: 0000000000000060:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
+; DIS:      0000000000000068 (idx: 19) ThreadLocalVarInit[TC]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
-; DIS:      0000000000000070 (idx: 19) ThreadLocalVarInit[TC]:
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
+; DIS:      0000000000000070 (idx: 21) VarInit[TC]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
-; DIS:      0000000000000078 (idx: 21) VarInit[TC]:
-; DIS-NEXT:       78: 00 00 00 00
-; DIS-NEXT: 0000000000000078:  R_POS        (idx: 9) VarInit
+; DIS-NEXT: 0000000000000070:  R_POS        (idx: 9) VarInit
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 25) ThreadLocalVarInit:
 ; DIS-NEXT:        0: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000004 (idx: 27) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-vec_insert_elt.ll

 ; CHECK-64-P10:       # %bb.0: # %entry
 ; CHECK-64-P10-NEXT:    mffprd 3, 1
 ; CHECK-64-P10-NEXT:    rlwinm 4, 4, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble1:
 ; CHECK-32-P10:       # %bb.0: # %entry
-; CHECK-32-P10-NEXT:    rlwinm 3, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 4, 1, -16
+; CHECK-32-P10-NEXT:    rlwinm 3, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 4, 3
 ; CHECK-32-P10-NEXT:    lxv 34, -16(1)
 ; CHECK-32-P10-NEXT:    blr
 entry:
   %vecins = insertelement <2 x double> %a, double %b, i32 %idx1
   ret <2 x double> %vecins
 }

 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 6
 ; CHECK-64-P10-NEXT:    rlwinm 4, 5, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble2:
 ; CHECK-32-P10:       # %bb.0: # %entry
 ; CHECK-32-P10-NEXT:    lfd 0, 0(3)
-; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 6, 1, -32
+; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -32(1)
 ; CHECK-32-P10-NEXT:    rlwinm 5, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stfdx 0, 6, 4
 ; CHECK-32-P10-NEXT:    lxv 0, -32(1)
 ; CHECK-32-P10-NEXT:    plfd 1, 1(3), 0
 ; CHECK-32-P10-NEXT:    addi 3, 1, -16
 ; CHECK-32-P10-NEXT:    stxv 0, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 3, 5

 ; CHECK-64-P10-NEXT:    ldx 3, 3, 4
 ; CHECK-64-P10-NEXT:    rlwinm 4, 5, 3, 0, 28
 ; CHECK-64-P10-NEXT:    vinsdlx 2, 4, 3
 ; CHECK-64-P10-NEXT:    blr
 ;
 ; CHECK-32-P10-LABEL: testDouble3:
 ; CHECK-32-P10:       # %bb.0: # %entry
 ; CHECK-32-P10-NEXT:    plfd 0, 65536(3), 0
-; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    addi 6, 1, -32
+; CHECK-32-P10-NEXT:    rlwinm 4, 4, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stxv 34, -32(1)
 ; CHECK-32-P10-NEXT:    rlwinm 5, 5, 3, 28, 28
 ; CHECK-32-P10-NEXT:    stfdx 0, 6, 4
 ; CHECK-32-P10-NEXT:    lxv 0, -32(1)
 ; CHECK-32-P10-NEXT:    lfd 1, 0(3)
 ; CHECK-32-P10-NEXT:    addi 3, 1, -16
 ; CHECK-32-P10-NEXT:    stxv 0, -16(1)
 ; CHECK-32-P10-NEXT:    stfdx 1, 3, 5