[AIX][TLS] Generate optimized local-exec access code sequence using X-Form loads/stores

[amyk]

Summary

This patch is a follow up to D149722, D152669 and D153645, where a slightly more optimized code
sequence is generated for 64-bit and 32-bit local-exec accesses when optimizations are turned on.

Handling is added PPCISelDAGToDAG.cpp in order to check if any D-form loads or stores that follow
an PPCISD::ADD_TLS can be optimized to use an X-Form load or store. In this particular situation,
this allows the ADD_TLS node to be removed completely.

[amyk]

Rebase patch.

[amyk]

Update this patch with a simplified way of generating the X-Form instructions, leveraging some existing infrastructure in place.

I am planning on updating this patch once again to fix a potential issue.

[amyk]

• Update patch to produce lhax and lwax when the load is sign-extending.
• Rebase patch.

[amyk]

Rebase patch and update comments in the patch.

[amyk]

Update patch to account for 32-bit implementation in D152669.

[amyk]

• Rebase patch.
• Update patch comments.

[amyk]

• Rebase patch to account for updating D152669.
• Use the correct PPCISD node (GET_TPOINTER) in canOptimizeTLSDFormToXFormOnAIX().

[amyk]

Rebase patch and simplify relocation test checks.

[amyk]

Patch rebased.

[amyk]

• Specify why we are doing this optimization in the description.
• Check on why these TLS instructions were not previously added.

llvm/lib/Target/PowerPC/P10InstrResources.td

1299 ↗

(On Diff #532902)

Check these definitions in the P10InstrResources.td.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

714

• Fix this comment.
• Can this function be general? Include both of the Linux and AIX parts.

743

Instead of explicitly using PPC::X13, is there something like Subtarget.getThreadPointerRegister() that we can use?

[amyk]

Address review comments:

• Simplify canOptimizeTLSDFormToXForm()
• Add getThreadPointerRegister() into PPCSubtarget.h
• Confirmed that adding the new *TLS load/store instructions are appropriate

[amyk]

llvm/lib/Target/PowerPC/PPCSubtarget.h

279

Is it appropriate to add this in this patch? Or should this be separate?

[amyk]

Like in D47382, it is possible that I need to add *TLS_ versions of the instructions that I added. I plan to update this patch to account for this.

[amyk]

Group code review comments.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

758–761

These two checks for the offset can be combined.

780

Update this comment for GET_TPOINTER, and just mention it's produced for AIX on 32-bit mode.

791

• Implement a static helper function, to check if the node of a thread pointer acquisition node (if one of the operands is the thread pointer - either R13, or __get_tpointer).
• If conditions are satisfied, return true.
• Otherwise, return false at the end of the static helper function.

853

A ternary condition for isSExt for here and below might be better.

llvm/lib/Target/PowerPC/PPCSubtarget.h

279

Discussed this with the team. It is appropriate to add this here.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

14

Instead of adding -O0 lines to every test, we can add one file to show the -O0 codegen.
That way, we can remove the -O0 run lines from these existing test cases.

[amyk]

Update patch to:

• Separate out the new instruction definitions (which have been added in https://reviews.llvm.org/D153645)
• Update comments
• Add a new static helper function to check if one of the operands of an ADD_TLS node uses the thread pointer

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

758–761

Done.

780

Done.

791

isThreadPointerAcquisitionNode() is added to account for this.

853

Done.

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

14

Done.

[lei]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

739–740

783–786

[amyk]

• Rebase patch.
• Address comments from Lei: combine two if conditions, and return with the isThreadPointerAcquisitionNode() function inside canOptimizeTLSDFormToXForm().

[lei]

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

63–64

This transformation increased reg pressure with no obvious benefits. Please see if we can bail out for such cases.

[amyk]

Update the patch to only do this optimization if all users of the ADD_TLS node are normal loads with an offset of 0.
This resolves an issue that Lei pointed out with respect to increasing the usage of registers when we have a 64-bit load/store on a 32-bit system - in such a situation, we no longer perform this optimization.

[lei]

Just a few nits, otherwise LGTM.
Thanks for addressing the issue with increasing the usage of registers when we have a 64-bit load/store on a 32-bit system.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

787

Since Base is only used on line 788, consider just inlining it there vs defining a new variable.

836

Since Base only used here, can inline and remove declaration.

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

787

Actually, Base is also used here below:

SDValue Ops[] = {ST->getValue(), Base.getOperand(0), Base.getOperand(1),
                   Chain};

836

Actually, Base is also used here below, right?

SDValue Ops[] = {Base.getOperand(0), Base.getOperand(1), Chain};

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

bool PPCDAGToDAGISel::isRotateAndMask(SDNode *N, unsigned Mask,
   if (Mask && !(Mask & Indeterminant)) {
     SH = Shift & 31;
     // make sure the mask is still a mask (wrap arounds may not be)
     return isRunOfOnes(Mask, MB, ME);
   }
   return false;
 }
 
-bool PPCDAGToDAGISel::tryTLSXFormStore(StoreSDNode *ST) {
-  SDValue Base = ST->getBasePtr();
+// isThreadPointerAcquisitionNode - Check if the operands of an ADD_TLS
+// instruction use the thread pointer.
+static bool isThreadPointerAcquisitionNode(SDValue Base, SelectionDAG *CurDAG) {
+  assert(
+      Base.getOpcode() == PPCISD::ADD_TLS &&
+      "Only expecting the ADD_TLS instruction to acquire the thread pointer!");
+  const PPCSubtarget &Subtarget =
+      CurDAG->getMachineFunction().getSubtarget<PPCSubtarget>();
+  SDValue ADDTLSOp1 = Base.getOperand(0);
+  unsigned ADDTLSOp1Opcode = ADDTLSOp1.getOpcode();
+
+  // Account for when ADD_TLS is used for the initial-exec TLS model on Linux.
+  //
+  // Although ADD_TLS does not explicitly use the thread pointer
+  // register when LD_GOT_TPREL_L is one of it's operands, the LD_GOT_TPREL_L
+  // instruction will have a relocation specifier, @got@tprel, that is used to
+  // generate a GOT entry. The linker replaces this entry with an offset for a
+  // for a thread local variable, which will be relative to the thread pointer.
+  if (ADDTLSOp1Opcode == PPCISD::LD_GOT_TPREL_L)
+    return true;
+  // When using PC-Relative instructions for initial-exec, a MAT_PCREL_ADDR
+  // node is produced instead to represent the aforementioned situation.
+  LoadSDNode *LD = dyn_cast<LoadSDNode>(ADDTLSOp1);
+  if (LD && LD->getBasePtr().getOpcode() == PPCISD::MAT_PCREL_ADDR)
+    return true;
+
+  // A GET_TPOINTER PPCISD node (only produced on AIX 32-bit mode) as an operand
+  // to ADD_TLS represents a call to .__get_tpointer to get the thread pointer,
+  // later returning it into R3.
+  if (ADDTLSOp1Opcode == PPCISD::GET_TPOINTER)
+    return true;
+
+  // The ADD_TLS note is explicitly acquiring the thread pointer (X13/R13).
+  RegisterSDNode *AddFirstOpReg =
+      dyn_cast_or_null<RegisterSDNode>(ADDTLSOp1.getNode());
+  if (AddFirstOpReg &&
+      AddFirstOpReg->getReg() == Subtarget.getThreadPointerRegister())
+      return true;
+
+  return false;
+}
+
+// canOptimizeTLSDFormToXForm - Optimize TLS accesses when an ADD_TLS
+// instruction is present. An ADD_TLS instruction, followed by a D-Form memory
+// operation, can be optimized to use an X-Form load or store, allowing the
+// ADD_TLS node to be removed completely.
+static bool canOptimizeTLSDFormToXForm(SelectionDAG *CurDAG, SDValue Base) {
+
+  // Do not do this transformation at -O0.
+  if (CurDAG->getTarget().getOptLevel() == CodeGenOpt::None)
+    return false;
+
+  // In order to perform this optimization inside tryTLSXForm[Load|Store],
+  // Base is expected to be an ADD_TLS node.
   if (Base.getOpcode() != PPCISD::ADD_TLS)
     return false;
-  SDValue Offset = ST->getOffset();
-  if (!Offset.isUndef())
+  for (auto *ADDTLSUse : Base.getNode()->uses()) {
+    // The optimization to convert the D-Form load/store into its X-Form
+    // counterpart should only occur if the source value offset of the load/
+    // store is 0. This also means that The offset should always be undefined.
+    if (LoadSDNode *LD = dyn_cast<LoadSDNode>(ADDTLSUse)) {
+      if (LD->getSrcValueOffset() != 0 || !LD->getOffset().isUndef())
+        return false;
+    } else if (StoreSDNode *ST = dyn_cast<StoreSDNode>(ADDTLSUse)) {
+      if (ST->getSrcValueOffset() != 0 || !ST->getOffset().isUndef())
         return false;
+    } else // Don't optimize if there are ADD_TLS users that aren't load/stores.
+      return false;
+  }
+
   if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
     return false;
 
+  // Does the ADD_TLS node of the load/store use the thread pointer?
+  // If the thread pointer is not used as one of the operands of ADD_TLS,
+  // then this optimization is not valid.
+  return isThreadPointerAcquisitionNode(Base, CurDAG);
+}
+
+bool PPCDAGToDAGISel::tryTLSXFormStore(StoreSDNode *ST) {
+  SDValue Base = ST->getBasePtr();
+  if (!canOptimizeTLSDFormToXForm(CurDAG, Base))
+    return false;
+
   SDLoc dl(ST);
   EVT MemVT = ST->getMemoryVT();
   EVT RegVT = ST->getValue().getValueType();
 
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
bool PPCDAGToDAGISel::tryTLSXFormStore(StoreSDNode *ST) {
   SDNode *MN = CurDAG->getMachineNode(Opcode, dl, VTs, Ops);
   transferMemOperands(ST, MN);
   ReplaceNode(ST, MN);
   return true;
 }
 
 bool PPCDAGToDAGISel::tryTLSXFormLoad(LoadSDNode *LD) {
   SDValue Base = LD->getBasePtr();
-  if (Base.getOpcode() != PPCISD::ADD_TLS)
-    return false;
-  SDValue Offset = LD->getOffset();
-  if (!Offset.isUndef())
-    return false;
-  if (Base.getOperand(1).getOpcode() == PPCISD::TLS_LOCAL_EXEC_MAT_ADDR)
+  if (!canOptimizeTLSDFormToXForm(CurDAG, Base))
     return false;
 
   SDLoc dl(LD);
   EVT MemVT = LD->getMemoryVT();
   EVT RegVT = LD->getValueType(0);
   bool isSExt = LD->getExtensionType() == ISD::SEXTLOAD;
   unsigned Opcode;
   switch (MemVT.getSimpleVT().SimpleTy) {
     default:
       return false;
     case MVT::i8: {
       Opcode = (RegVT == MVT::i32) ? PPC::LBZXTLS_32 : PPC::LBZXTLS;
       break;
     }
     case MVT::i16: {
       if (RegVT == MVT::i32)
         Opcode = isSExt ? PPC::LHAXTLS_32 : PPC::LHZXTLS_32;
       else
         Opcode = isSExt ? PPC::LHAXTLS : PPC::LHZXTLS;
       break;
     }
     case MVT::i32: {
       if (RegVT == MVT::i32)
         Opcode = isSExt ? PPC::LWAXTLS_32 : PPC::LWZXTLS_32;
       else
if (N->getValueType(0) == MVT::i64) {
                                N0, ShiftAmt);
       CurDAG->SelectNodeTo(N, PPC::ADDZE, MVT::i32, SDValue(Op, 0),
                            SDValue(Op, 1));
       return;
     }
   }
 
   case ISD::STORE: {
-    // Change TLS initial-exec D-form stores to X-form stores.
+    // Change TLS initial-exec (or TLS local-exec on AIX) D-form stores to
+    // X-form stores.
     StoreSDNode *ST = cast<StoreSDNode>(N);
-    if (EnableTLSOpt && Subtarget->isELFv2ABI() &&
+    if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()) &&
         ST->getAddressingMode() != ISD::PRE_INC)
       if (tryTLSXFormStore(ST))
         return;
     break;
   }
   case ISD::LOAD: {
     // Handle preincrement loads.
     LoadSDNode *LD = cast<LoadSDNode>(N);
     EVT LoadedVT = LD->getMemoryVT();
 
     // Normal loads are handled by code generated from the .td file.
     if (LD->getAddressingMode() != ISD::PRE_INC) {
-      // Change TLS initial-exec D-form loads to X-form loads.
-      if (EnableTLSOpt && Subtarget->isELFv2ABI())
+      // Change TLS initial-exec (or TLS local-exec on AIX) D-form loads to
+      // X-form loads.
+      if (EnableTLSOpt && (Subtarget->isELFv2ABI() || Subtarget->isAIXABI()))
         if (tryTLSXFormLoad(LD))
           return;
       break;
     }
 
     SDValue Offset = LD->getOffset();
     if (Offset.getOpcode() == ISD::TargetConstant ||
         Offset.getOpcode() == ISD::TargetGlobalAddress) {

llvm/lib/Target/PowerPC/PPCSubtarget.h

#include "PPCGenSubtargetInfo.inc"
   }
 
   MCRegister getTOCPointerRegister() const {
     assert((is64BitELFABI() || isAIXABI()) &&
            "Should only be called when the target is a TOC based ABI.");
     return IsPPC64 ? PPC::X2 : PPC::R2;
   }
 
+  MCRegister getThreadPointerRegister() const {
+    assert((is64BitELFABI() || isAIXABI()) &&
+           "Should only be called for targets with a thread pointer register.");
+    return IsPPC64 ? PPC::X13 : PPC::R13;
+  }
+
   MCRegister getStackPointerRegister() const {
     return IsPPC64 ? PPC::X1 : PPC::R1;
   }
 
   bool isXRaySupported() const override { return IsPPC64 && IsLittleEndian; }
 
   bool isPredictableSelectIsExpensive() const {
     return PredictableSelectIsExpensive;

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-O0.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
+; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
+
+@TLInt = internal thread_local(localexec) global i32 0, align 4
+@TLLongLong = internal thread_local(localexec) global i64 0, align 8
+@TLDouble = internal thread_local(localexec) global double 0.000000e+00, align 8
+@TLFloat = internal thread_local(localexec) global float 0.000000e+00, align 4
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+
+define void @storeInt(i32 noundef %x) {
+; SMALL64-O0-LABEL: storeInt:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @TLInt
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    stw r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeInt:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    addis r4, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C0@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    stw r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeInt:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    lwz r5, L..C0(r2) # target-flags(ppc-tprel) @TLInt
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r5
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeInt:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r5, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r5
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLInt)
+  store i32 %x, ptr %0, align 4
+  ret void
+}
+
+define void @storeLongLong(i64 noundef %x) {
+; SMALL64-O0-LABEL: storeLongLong:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @TLLongLong
+; SMALL64-O0-NEXT:    add r4, r13, r4
+; SMALL64-O0-NEXT:    std r3, 0(r4)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeLongLong:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r4, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r4, L..C1@l(r4)
+; LARGE64-O0-NEXT:    add r4, r13, r4
+; LARGE64-O0-NEXT:    std r3, 0(r4)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeLongLong:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    mr r5, r4
+; SMALL32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; SMALL32-O0-NEXT:    # kill: def $r4 killed $r5
+; SMALL32-O0-NEXT:    lwz r6, L..C1(r2) # target-flags(ppc-tprel) @TLLongLong
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    mr r4, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; SMALL32-O0-NEXT:    add r4, r4, r6
+; SMALL32-O0-NEXT:    stw r5, 4(r4)
+; SMALL32-O0-NEXT:    stw r3, 0(r4)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeLongLong:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    mr r5, r4
+; LARGE32-O0-NEXT:    stw r3, 28(r1) # 4-byte Folded Spill
+; LARGE32-O0-NEXT:    # kill: def $r4 killed $r5
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r6, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    mr r4, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
+; LARGE32-O0-NEXT:    add r4, r4, r6
+; LARGE32-O0-NEXT:    stw r5, 4(r4)
+; LARGE32-O0-NEXT:    stw r3, 0(r4)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLLongLong)
+  store i64 %x, ptr %0, align 8
+  ret void
+}
+
+define void @storeDouble(double noundef %x) {
+; SMALL64-O0-LABEL: storeDouble:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @TLDouble
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stxsdx f1, 0, r3
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeDouble:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stxsdx f1, 0, r3
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeDouble:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @TLDouble
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfd f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeDouble:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfd f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLDouble)
+  store double %x, ptr %0, align 8
+  ret void
+}
+
+define void @storeFloat(float noundef %x) {
+; SMALL64-O0-LABEL: storeFloat:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @TLFloat
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    stfs f1, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: storeFloat:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    stfs f1, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: storeFloat:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @TLFloat
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    stfs f1, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: storeFloat:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    stfs f1, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLFloat)
+  store float %x, ptr %0, align 4
+  ret void
+}
+
+define i32 @loadInt() {
+; SMALL64-O0-LABEL: loadInt:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @TLInt
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lwz r3, 0(r3)
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadInt:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C0@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lwz r3, 0(r3)
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadInt:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @TLInt
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 0(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadInt:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C0@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 0(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLInt)
+  %1 = load i32, ptr %0, align 4
+  ret i32 %1
+}
+
+define i32 @loadLongLong() {
+; SMALL64-O0-LABEL: loadLongLong:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @TLLongLong
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    ld r3, 0(r3)
+; SMALL64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL64-O0-NEXT:    clrldi r3, r3, 32
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadLongLong:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C1@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    ld r3, 0(r3)
+; LARGE64-O0-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; LARGE64-O0-NEXT:    clrldi r3, r3, 32
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadLongLong:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @TLLongLong
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lwz r3, 4(r3)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadLongLong:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C1@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lwz r3, 4(r3)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLLongLong)
+  %1 = load i64, ptr %0, align 8
+  %conv = trunc i64 %1 to i32
+  ret i32 %conv
+}
+
+define i32 @loadDouble() {
+; SMALL64-O0-LABEL: loadDouble:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @TLDouble
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfd f0, 0(r3)
+; SMALL64-O0-NEXT:    # kill: def $f1 killed $f0
+; SMALL64-O0-NEXT:    xscvdpsxws f0, f0
+; SMALL64-O0-NEXT:    addi r3, r1, -12
+; SMALL64-O0-NEXT:    stfiwx f0, 0, r3
+; SMALL64-O0-NEXT:    lwz r3, -12(r1)
+; SMALL64-O0-NEXT:    clrldi r3, r3, 32
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadDouble:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C2@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfd f0, 0(r3)
+; LARGE64-O0-NEXT:    # kill: def $f1 killed $f0
+; LARGE64-O0-NEXT:    xscvdpsxws f0, f0
+; LARGE64-O0-NEXT:    addi r3, r1, -12
+; LARGE64-O0-NEXT:    stfiwx f0, 0, r3
+; LARGE64-O0-NEXT:    lwz r3, -12(r1)
+; LARGE64-O0-NEXT:    clrldi r3, r3, 32
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadDouble:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @TLDouble
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfd f0, 0(r3)
+; SMALL32-O0-NEXT:    xscvdpsxws f0, f0
+; SMALL32-O0-NEXT:    addi r3, r1, 28
+; SMALL32-O0-NEXT:    stfiwx f0, 0, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadDouble:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C2@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfd f0, 0(r3)
+; LARGE32-O0-NEXT:    xscvdpsxws f0, f0
+; LARGE32-O0-NEXT:    addi r3, r1, 28
+; LARGE32-O0-NEXT:    stfiwx f0, 0, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLDouble)
+  %1 = load double, ptr %0, align 8
+  %conv = fptosi double %1 to i32
+  ret i32 %conv
+}
+
+define i32 @loadFloat() {
+; SMALL64-O0-LABEL: loadFloat:
+; SMALL64-O0:       # %bb.0: # %entry
+; SMALL64-O0-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @TLFloat
+; SMALL64-O0-NEXT:    add r3, r13, r3
+; SMALL64-O0-NEXT:    lfs f0, 0(r3)
+; SMALL64-O0-NEXT:    fctiwz f0, f0
+; SMALL64-O0-NEXT:    stfd f0, -8(r1)
+; SMALL64-O0-NEXT:    lwa r3, -4(r1)
+; SMALL64-O0-NEXT:    clrldi r3, r3, 32
+; SMALL64-O0-NEXT:    blr
+;
+; LARGE64-O0-LABEL: loadFloat:
+; LARGE64-O0:       # %bb.0: # %entry
+; LARGE64-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE64-O0-NEXT:    ld r3, L..C3@l(r3)
+; LARGE64-O0-NEXT:    add r3, r13, r3
+; LARGE64-O0-NEXT:    lfs f0, 0(r3)
+; LARGE64-O0-NEXT:    fctiwz f0, f0
+; LARGE64-O0-NEXT:    stfd f0, -8(r1)
+; LARGE64-O0-NEXT:    lwa r3, -4(r1)
+; LARGE64-O0-NEXT:    clrldi r3, r3, 32
+; LARGE64-O0-NEXT:    blr
+;
+; SMALL32-O0-LABEL: loadFloat:
+; SMALL32-O0:       # %bb.0: # %entry
+; SMALL32-O0-NEXT:    mflr r0
+; SMALL32-O0-NEXT:    stwu r1, -32(r1)
+; SMALL32-O0-NEXT:    stw r0, 40(r1)
+; SMALL32-O0-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @TLFloat
+; SMALL32-O0-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-O0-NEXT:    add r3, r3, r4
+; SMALL32-O0-NEXT:    lfs f0, 0(r3)
+; SMALL32-O0-NEXT:    xscvdpsxws f0, f0
+; SMALL32-O0-NEXT:    addi r3, r1, 28
+; SMALL32-O0-NEXT:    stfiwx f0, 0, r3
+; SMALL32-O0-NEXT:    lwz r3, 28(r1)
+; SMALL32-O0-NEXT:    addi r1, r1, 32
+; SMALL32-O0-NEXT:    lwz r0, 8(r1)
+; SMALL32-O0-NEXT:    mtlr r0
+; SMALL32-O0-NEXT:    blr
+;
+; LARGE32-O0-LABEL: loadFloat:
+; LARGE32-O0:       # %bb.0: # %entry
+; LARGE32-O0-NEXT:    mflr r0
+; LARGE32-O0-NEXT:    stwu r1, -32(r1)
+; LARGE32-O0-NEXT:    stw r0, 40(r1)
+; LARGE32-O0-NEXT:    addis r3, L..C3@u(r2)
+; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
+; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
+; LARGE32-O0-NEXT:    add r3, r3, r4
+; LARGE32-O0-NEXT:    lfs f0, 0(r3)
+; LARGE32-O0-NEXT:    xscvdpsxws f0, f0
+; LARGE32-O0-NEXT:    addi r3, r1, 28
+; LARGE32-O0-NEXT:    stfiwx f0, 0, r3
+; LARGE32-O0-NEXT:    lwz r3, 28(r1)
+; LARGE32-O0-NEXT:    addi r1, r1, 32
+; LARGE32-O0-NEXT:    lwz r0, 8(r1)
+; LARGE32-O0-NEXT:    mtlr r0
+; LARGE32-O0-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLFloat)
+  %1 = load float, ptr %0, align 4
+  %conv = fptosi float %1 to i32
+  ret i32 %conv
+}
+
+; TOC Entry Checks.
+
+; SMALL64-O0-LABEL: .toc
+; SMALL64-O0-LABEL:L..C0:
+; SMALL64-O0-NEXT:  .tc TLInt[TC],TLInt[UL]@le
+; SMALL64-O0-LABEL:L..C1:
+; SMALL64-O0-NEXT:  .tc TLLongLong[TC],TLLongLong[UL]@le
+; SMALL64-O0-LABEL:L..C2:
+; SMALL64-O0-NEXT:  .tc TLDouble[TC],TLDouble[UL]@le
+; SMALL64-O0-LABEL:L..C3:
+; SMALL64-O0-NEXT:  .tc TLFloat[TC],TLFloat[UL]@le
+
+; LARGE64-O0-LABEL: .toc
+; LARGE64-O0-LABEL:L..C0:
+; LARGE64-O0-NEXT:  .tc TLInt[TE],TLInt[UL]@le
+; LARGE64-O0-LABEL:L..C1:
+; LARGE64-O0-NEXT:  .tc TLLongLong[TE],TLLongLong[UL]@le
+; LARGE64-O0-LABEL:L..C2:
+; LARGE64-O0-NEXT:  .tc TLDouble[TE],TLDouble[UL]@le
+; LARGE64-O0-LABEL:L..C3:
+; LARGE64-O0-NEXT:  .tc TLFloat[TE],TLFloat[UL]@le
+
+; SMALL32-O0-LABEL: .toc
+; SMALL32-O0-LABEL:L..C0:
+; SMALL32-O0-NEXT:  .tc TLInt[TC],TLInt[UL]@le
+; SMALL32-O0-LABEL:L..C1:
+; SMALL32-O0-NEXT:  .tc TLLongLong[TC],TLLongLong[UL]@le
+; SMALL32-O0-LABEL:L..C2:
+; SMALL32-O0-NEXT:  .tc TLDouble[TC],TLDouble[UL]@le
+; SMALL32-O0-LABEL:L..C3:
+; SMALL32-O0-NEXT:  .tc TLFloat[TC],TLFloat[UL]@le
+
+; LARGE32-O0-LABEL: .toc
+; LARGE32-O0-LABEL:L..C0:
+; LARGE32-O0-NEXT:  .tc TLInt[TE],TLInt[UL]@le
+; LARGE32-O0-LABEL:L..C1:
+; LARGE32-O0-NEXT:  .tc TLLongLong[TE],TLLongLong[UL]@le
+; LARGE32-O0-LABEL:L..C2:
+; LARGE32-O0-NEXT:  .tc TLDouble[TE],TLDouble[UL]@le
+; LARGE32-O0-LABEL:L..C3:
+; LARGE32-O0-NEXT:  .tc TLFloat[TE],TLFloat[UL]@le
+

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-double.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32
 ; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32
 
 @ThreadLocalVarInit = thread_local(localexec) global double 0x4021947AE147AE14, align 8
 @VarInit = global double 8.787000e+01, align 8
 @IThreadLocalVarUninit = internal thread_local(localexec) global double 0.000000e+00, align 8
 @IThreadLocalVarInit = internal thread_local(localexec) global double 5.870000e+00, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global double 0.000000e+00, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(double noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(double noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(double noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfd f1, 0(r3)
+; SMALL64-NEXT:    stfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfd f1, 0(r3)
+; LARGE64-NEXT:    stfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfd f1, 0(r3)
+; SMALL32-NEXT:    stfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfd f1, 0(r3)
+; LARGE32-NEXT:    stfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define double @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarUninit)
   %1 = load double, ptr %0, align 8
   %2 = load double, ptr @VarInit, align 8
   %add = fadd double %1, %2
   ret double %add
 }
 
 define double @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfd f1, 0(r3)
+; SMALL64-NEXT:    lfdx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfd f1, 0(r3)
+; LARGE64-NEXT:    lfdx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfd f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f1, 0(r3)
+; LARGE32-NEXT:    lfdx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
   %1 = load double, ptr %0, align 8
   ret double %1
 }
 
 define double @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfdx f0, r13, r3
 ; SMALL64-NEXT:    lfd f1, 0(r4)
-; SMALL64-NEXT:    lfd f0, 0(r3)
 ; SMALL64-NEXT:    xsadddp f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfdx f0, r13, r3
 ; LARGE64-NEXT:    lfd f1, 0(r4)
-; LARGE64-NEXT:    lfd f0, 0(r3)
 ; LARGE64-NEXT:    xsadddp f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfdx f0, r3, r4
 ; SMALL32-NEXT:    lfd f1, 0(r5)
-; SMALL32-NEXT:    lfd f0, 0(r3)
 ; SMALL32-NEXT:    xsadddp f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfd f0, 0(r3)
+; LARGE32-NEXT:    lfdx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfd f1, 0(r3)
 ; LARGE32-NEXT:    xsadddp f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-float.ll

 @IThreadLocalVarInit = internal thread_local(localexec) global float 0x4018CCCCC0000000, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global float 0.000000e+00, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(float noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(float noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(float noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    stfs f1, 0(r3)
+; SMALL64-NEXT:    stfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    stfs f1, 0(r3)
+; LARGE64-NEXT:    stfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    stfs f1, 0(r3)
+; SMALL32-NEXT:    stfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    stfs f1, 0(r3)
+; LARGE32-NEXT:    stfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define float @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load float, ptr %0, align 4
   %2 = load float, ptr @VarInit, align 4
   %add = fadd float %1, %2
   ret float %add
 }
 
 define float @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lfs f1, 0(r3)
+; SMALL64-NEXT:    lfsx f1, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lfs f1, 0(r3)
+; LARGE64-NEXT:    lfsx f1, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lfs f1, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f1, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f1, 0(r3)
+; LARGE32-NEXT:    lfsx f1, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load float, ptr %0, align 4
   ret float %1
 }
 
 define float @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lfsx f0, r13, r3
 ; SMALL64-NEXT:    lfs f1, 0(r4)
-; SMALL64-NEXT:    lfs f0, 0(r3)
 ; SMALL64-NEXT:    fadds f1, f0, f1
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lfsx f0, r13, r3
 ; LARGE64-NEXT:    lfs f1, 0(r4)
-; LARGE64-NEXT:    lfs f0, 0(r3)
 ; LARGE64-NEXT:    fadds f1, f0, f1
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lfsx f0, r3, r4
 ; SMALL32-NEXT:    lfs f1, 0(r5)
-; SMALL32-NEXT:    lfs f0, 0(r3)
 ; SMALL32-NEXT:    fadds f1, f0, f1
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lfs f0, 0(r3)
+; LARGE32-NEXT:    lfsx f0, r3, r4
 ; LARGE32-NEXT:    addis r3, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r3, L..C4@l(r3)
 ; LARGE32-NEXT:    lfs f1, 0(r3)
 ; LARGE32-NEXT:    fadds f1, f0, f1
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-int.ll

 @IThreadLocalVarInit = internal thread_local(localexec) global i32 1, align 4
 @ThreadLocalVarUninit = thread_local(localexec) global i32 0, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeITLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLUninit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeTLInit(i32 noundef signext %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    stw r3, 0(r4)
+; SMALL64-NEXT:    stwx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    stw r3, 0(r4)
+; LARGE64-NEXT:    stwx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r5, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    mr r4, r3
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r5
-; SMALL32-NEXT:    stw r4, 0(r3)
+; SMALL32-NEXT:    stwx r4, r3, r5
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: storeTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r4, r3
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r5, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r5
-; LARGE32-NEXT:    stw r4, 0(r3)
+; LARGE32-NEXT:    stwx r4, r3, r5
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define signext i32 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadITLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLUninit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarUninit)
   %1 = load i32, ptr %0, align 4
   %2 = load i32, ptr @VarInit, align 4
   %add = add nsw i32 %2, %1
   ret i32 %add
 }
 
 define signext i32 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    lwa r3, 0(r3)
+; SMALL64-NEXT:    lwax r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    lwa r3, 0(r3)
+; LARGE64-NEXT:    lwax r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
-; SMALL32-NEXT:    lwz r3, 0(r3)
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define signext i32 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    lwzx r3, r13, r3
 ; SMALL64-NEXT:    lwz r4, 0(r4)
-; SMALL64-NEXT:    lwz r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    extsw r3, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    lwzx r3, r13, r3
 ; LARGE64-NEXT:    lwz r4, 0(r4)
-; LARGE64-NEXT:    lwz r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    extsw r3, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    lwz r5, L..C4(r2) # @VarInit
-; SMALL32-NEXT:    bla .__get_tpointer[PR]
 ; SMALL32-NEXT:    stw r0, 40(r1)
-; SMALL32-NEXT:    add r3, r3, r4
+; SMALL32-NEXT:    bla .__get_tpointer[PR]
+; SMALL32-NEXT:    lwzx r3, r3, r4
 ; SMALL32-NEXT:    lwz r4, 0(r5)
-; SMALL32-NEXT:    lwz r3, 0(r3)
 ; SMALL32-NEXT:    add r3, r4, r3
 ; SMALL32-NEXT:    addi r1, r1, 32
 ; SMALL32-NEXT:    lwz r0, 8(r1)
 ; SMALL32-NEXT:    mtlr r0
 ; SMALL32-NEXT:    blr
 ;
 ; LARGE32-LABEL: loadTLInit2:
 ; LARGE32:       # %bb.0: # %entry
 ; LARGE32-NEXT:    mflr r0
 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
-; LARGE32-NEXT:    add r3, r3, r4
-; LARGE32-NEXT:    lwz r3, 0(r3)
+; LARGE32-NEXT:    lwzx r3, r3, r4
 ; LARGE32-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE32-NEXT:    lwz r4, L..C4@l(r4)
 ; LARGE32-NEXT:    lwz r4, 0(r4)
 ; LARGE32-NEXT:    add r3, r4, r3
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-longlong.ll

 @IThreadLocalVarInit = internal thread_local(localexec) global i64 1, align 8
 @ThreadLocalVarUninit = thread_local(localexec) global i64 0, align 8
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeITLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C0@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r5, r3

 ; LARGE32-NEXT:    stwu r1, -32(r1)
 ; LARGE32-NEXT:    stw r0, 40(r1)
 ; LARGE32-NEXT:    mr r5, r3
 ; LARGE32-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE32-NEXT:    lwz r6, L..C0@l(r3)
 ; LARGE32-NEXT:    bla .__get_tpointer[PR]
 ; LARGE32-NEXT:    add r3, r3, r6
 ; LARGE32-NEXT:    stw r4, 4(r3)
 ; LARGE32-NEXT:    stw r5, 0(r3)
 ; LARGE32-NEXT:    addi r1, r1, 32
 ; LARGE32-NEXT:    lwz r0, 8(r1)
 ; LARGE32-NEXT:    mtlr r0
 ; LARGE32-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarUninit)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeITLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C1@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    mr r5, r3
entry:
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLUninit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C2@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    mr r5, r3
entry:
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeTLInit(i64 noundef %x) {
 ; SMALL64-LABEL: storeTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r4, r13, r4
-; SMALL64-NEXT:    std r3, 0(r4)
+; SMALL64-NEXT:    stdx r3, r13, r4
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: storeTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r4, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r4, L..C3@l(r4)
-; LARGE64-NEXT:    add r4, r13, r4
-; LARGE64-NEXT:    std r3, 0(r4)
+; LARGE64-NEXT:    stdx r3, r13, r4
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: storeTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r6, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    mr r5, r3
entry:
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define i64 @loadITLUninit() {
 ; SMALL64-LABEL: loadITLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
entry:
   ret i64 %1
 }
 
 define i64 @loadITLUninit2() {
 ; SMALL64-LABEL: loadITLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C0@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C0@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C0(r2) # target-flags(ppc-tprel) @IThreadLocalVarUninit
entry:
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadITLInit() {
 ; SMALL64-LABEL: loadITLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
entry:
   ret i64 %1
 }
 
 define i64 @loadITLInit2() {
 ; SMALL64-LABEL: loadITLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadITLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C1@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C1@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadITLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C1(r2) # target-flags(ppc-tprel) @IThreadLocalVarInit
entry:
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLUninit() {
 ; SMALL64-LABEL: loadTLUninit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
entry:
   ret i64 %1
 }
 
 define i64 @loadTLUninit2() {
 ; SMALL64-LABEL: loadTLUninit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLUninit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C2@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C2@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLUninit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C2(r2) # target-flags(ppc-tprel) @ThreadLocalVarUninit
entry:
   %add = add nsw i64 %2, %1
   ret i64 %add
 }
 
 define i64 @loadTLInit() {
 ; SMALL64-LABEL: loadTLInit:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
-; SMALL64-NEXT:    add r3, r13, r3
-; SMALL64-NEXT:    ld r3, 0(r3)
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
-; LARGE64-NEXT:    add r3, r13, r3
-; LARGE64-NEXT:    ld r3, 0(r3)
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL32-NEXT:    bla .__get_tpointer[PR]
entry:
   ret i64 %1
 }
 
 define i64 @loadTLInit2() {
 ; SMALL64-LABEL: loadTLInit2:
 ; SMALL64:       # %bb.0: # %entry
 ; SMALL64-NEXT:    ld r3, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit
 ; SMALL64-NEXT:    ld r4, L..C4(r2) # @VarInit
-; SMALL64-NEXT:    add r3, r13, r3
+; SMALL64-NEXT:    ldx r3, r13, r3
 ; SMALL64-NEXT:    ld r4, 0(r4)
-; SMALL64-NEXT:    ld r3, 0(r3)
 ; SMALL64-NEXT:    add r3, r4, r3
 ; SMALL64-NEXT:    blr
 ;
 ; LARGE64-LABEL: loadTLInit2:
 ; LARGE64:       # %bb.0: # %entry
 ; LARGE64-NEXT:    addis r3, L..C3@u(r2)
 ; LARGE64-NEXT:    addis r4, L..C4@u(r2)
 ; LARGE64-NEXT:    ld r3, L..C3@l(r3)
 ; LARGE64-NEXT:    ld r4, L..C4@l(r4)
-; LARGE64-NEXT:    add r3, r13, r3
+; LARGE64-NEXT:    ldx r3, r13, r3
 ; LARGE64-NEXT:    ld r4, 0(r4)
-; LARGE64-NEXT:    ld r3, 0(r3)
 ; LARGE64-NEXT:    add r3, r4, r3
 ; LARGE64-NEXT:    blr
 ;
 ; SMALL32-LABEL: loadTLInit2:
 ; SMALL32:       # %bb.0: # %entry
 ; SMALL32-NEXT:    mflr r0
 ; SMALL32-NEXT:    stwu r1, -32(r1)
 ; SMALL32-NEXT:    lwz r4, L..C3(r2) # target-flags(ppc-tprel) @ThreadLocalVarInit

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc-large.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x6
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (15)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x22
+; RELOC:       Virtual Address: 0x12
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCU (0x30)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x2A
+; RELOC:       Virtual Address: 0x1A
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (17)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOCL (0x31)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x80
+; RELOC:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (23)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x88
+; RELOC:       Virtual Address: 0x70
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (21)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc-large.ll.tmp.o
 ; SYM-NEXT: Format: aix5coff64-rs6000
 ; SYM-NEXT: Arch: powerpc64
 ; SYM-NEXT: AddressSize: 64bit
 ; SYM-NEXT: Symbols [
 ; SYM:     Index: 15
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x80
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 16
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TE (0x16)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 17
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x88
+; SYM-NEXT:     Value (RelocatableAddress): 0x70
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc-large.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU (idx: 15) IThreadLocalVarUninit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL (idx: 15) IThreadLocalVarUninit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                std 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stdx 3, 13, 4
 ; DIS-NEXT:                                       blr
-; DIS:      0000000000000020 (idx: 5) .loadTLInit:
+; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 4, 2, 0
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU       (idx: 19) VarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 8(3)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 17) ThreadLocalVarInit[TE]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 16(4)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL       (idx: 19) VarInit[TE]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ldx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000048 (idx: 7) VarInit[RW]:
-; DIS-NEXT:       48: 00 00 00 00
-; DIS-NEXT:       4c: 00 00 00 57
-; DIS:      0000000000000050 (idx: 9) storeITLUninit[DS]:
+; DIS:      0000000000000030 (idx: 7) VarInit[RW]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT:       34: 00 00 00 57
+; DIS:      0000000000000038 (idx: 9) storeITLUninit[DS]:
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       3c: 00 00 00 00
+; DIS-NEXT:       40: 00 00 00 00
+; DIS-NEXT: 0000000000000040:  R_POS        (idx: 13) TOC[TC0]
+; DIS-NEXT:       44: 00 00 00 68
+; DIS:      0000000000000050 (idx: 11) loadTLInit[DS]:
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       54: 00 00 00 00
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       54: 00 00 00 10
 ; DIS-NEXT:       58: 00 00 00 00
 ; DIS-NEXT: 0000000000000058:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 80
-; DIS:      0000000000000068 (idx: 11) loadTLInit[DS]:
+; DIS-NEXT:       5c: 00 00 00 68
+; DIS:      0000000000000068 (idx: 15) IThreadLocalVarUninit[TE]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       6c: 00 00 00 20
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
+; DIS-NEXT:       6c: 00 00 00 00
+; DIS:      0000000000000070 (idx: 17) ThreadLocalVarInit[TE]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_POS        (idx: 13) TOC[TC0]
-; DIS-NEXT:       74: 00 00 00 80
-; DIS:      0000000000000080 (idx: 15) IThreadLocalVarUninit[TE]:
-; DIS-NEXT:       80: 00 00 00 00
-; DIS-NEXT: 0000000000000080:  R_TLS_LE     (idx: 23) IThreadLocalVarUninit[UL]
-; DIS-NEXT:       84: 00 00 00 00
-; DIS:      0000000000000088 (idx: 17) ThreadLocalVarInit[TE]:
-; DIS-NEXT:       88: 00 00 00 00
-; DIS-NEXT: 0000000000000088:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
-; DIS-NEXT:       8c: 00 00 00 00
-; DIS:      0000000000000090 (idx: 19) VarInit[TE]:
-; DIS-NEXT:       90: 00 00 00 00
-; DIS-NEXT: 0000000000000090:  R_POS        (idx: 7) VarInit[RW]
-; DIS-NEXT:       94: 00 00 00 48
+; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 21) ThreadLocalVarInit[TL]
+; DIS-NEXT:       74: 00 00 00 00
+; DIS:      0000000000000078 (idx: 19) VarInit[TE]:
+; DIS-NEXT:       78: 00 00 00 00
+; DIS-NEXT: 0000000000000078:  R_POS        (idx: 7) VarInit[RW]
+; DIS-NEXT:       7c: 00 00 00 30
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 21) ThreadLocalVarInit[TL]:
 ; DIS-NEXT:        0: 00 00 00 00
 ; DIS-NEXT:        4: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000008 (idx: 23) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x12
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (19)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOC (0x3)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x68
+; RELOC:       Virtual Address: 0x60
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (27)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x70
+; RELOC:       Virtual Address: 0x68
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (25)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 64
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc.ll.tmp.o
 ; SYM-NEXT: Format: aix5coff64-rs6000
 ; SYM-NEXT: Arch: powerpc64
 ; SYM-NEXT: AddressSize: 64bit
 ; SYM-NEXT: Symbols [
 ; SYM:     Index: 17
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x68
+; SYM-NEXT:     Value (RelocatableAddress): 0x60
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 18
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 3
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       Auxiliary Type: AUX_CSECT (0xFB)
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 19
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x70
+; SYM-NEXT:     Value (RelocatableAddress): 0x68
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 8
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 
 ; DIS:      {{.*}}aix-tls-le-xcoff-reloc.ll.tmp.o:  file format aix5coff64-rs6000
 ; DIS:      Disassembly of section .text:
 ; DIS:      0000000000000000 (idx: 3) .storeITLUninit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 0(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 17) IThreadLocalVarUninit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 4, 13, 4
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stw 3, 0(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stwx 3, 13, 4
 ; DIS-NEXT:                                      blr
 ; DIS:      0000000000000010 (idx: 5) .loadTLInit:
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 3, 8(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 19) ThreadLocalVarInit[TC]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               ld 4, 16(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 21) VarInit[TC]
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwzx 3, 13, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 4, 0(4)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 3, 0(3)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               extsw 3, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      0000000000000030 (idx: 9) VarInit:
-; DIS-NEXT:       30: 00 00 00 57
-; DIS:      0000000000000038 (idx: 11) storeITLUninit[DS]:
-; DIS-NEXT:       8: 00 00 00 00
-; DIS-NEXT: 0000000000000038:  R_POS        (idx: 3) .storeITLUninit
-; DIS-NEXT:       3c: 00 00 00 00
-; DIS-NEXT:       40: 00 00 00 00
-; DIS-NEXT: 0000000000000040:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       44: 00 00 00 68
-; DIS:      0000000000000050 (idx: 13) loadTLInit[DS]:
+; DIS:      000000000000002c (idx: 9) VarInit:
+; DIS-NEXT:       2c: 00 00 00 57
+; DIS:      0000000000000030 (idx: 11) storeITLUninit[DS]:
+; DIS-NEXT:       30: 00 00 00 00
+; DIS-NEXT: 0000000000000030:  R_POS        (idx: 3) .storeITLUninit
+; DIS-NEXT:       34: 00 00 00 00
+; DIS-NEXT:       38: 00 00 00 00
+; DIS-NEXT: 0000000000000038:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       3c: 00 00 00 60
+; DIS:      0000000000000048 (idx: 13) loadTLInit[DS]:
+; DIS-NEXT:       48: 00 00 00 00
+; DIS-NEXT: 0000000000000048:  R_POS        (idx: 5) .loadTLInit
+; DIS-NEXT:       4c: 00 00 00 10
 ; DIS-NEXT:       50: 00 00 00 00
-; DIS-NEXT: 0000000000000050:  R_POS        (idx: 5) .loadTLInit
-; DIS-NEXT:       54: 00 00 00 10
-; DIS-NEXT:       58: 00 00 00 00
-; DIS-NEXT: 0000000000000058:  R_POS        (idx: 15) TOC[TC0]
-; DIS-NEXT:       5c: 00 00 00 68
-; DIS:      0000000000000068 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT: 0000000000000050:  R_POS        (idx: 15) TOC[TC0]
+; DIS-NEXT:       54: 00 00 00 60
+; DIS:      0000000000000060 (idx: 17) IThreadLocalVarUninit[TC]:
+; DIS-NEXT:       60: 00 00 00 00
+; DIS-NEXT: 0000000000000060:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
+; DIS:      0000000000000068 (idx: 19) ThreadLocalVarInit[TC]:
 ; DIS-NEXT:       68: 00 00 00 00
-; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 27) IThreadLocalVarUninit[UL]
-; DIS:      0000000000000070 (idx: 19) ThreadLocalVarInit[TC]:
+; DIS-NEXT: 0000000000000068:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
+; DIS:      0000000000000070 (idx: 21) VarInit[TC]:
 ; DIS-NEXT:       70: 00 00 00 00
-; DIS-NEXT: 0000000000000070:  R_TLS_LE     (idx: 25) ThreadLocalVarInit
-; DIS:      0000000000000078 (idx: 21) VarInit[TC]:
-; DIS-NEXT:       78: 00 00 00 00
-; DIS-NEXT: 0000000000000078:  R_POS        (idx: 9) VarInit
+; DIS-NEXT: 0000000000000070:  R_POS        (idx: 9) VarInit
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      0000000000000000 (idx: 25) ThreadLocalVarInit:
 ; DIS-NEXT:        0: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      0000000000000004 (idx: 27) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...
 

llvm/test/CodeGen/PowerPC/aix-tls-le-xcoff-reloc32.ll

 ; RELOC-NEXT:     }
 ; RELOC:       Virtual Address: 0x3A
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (21)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 16
 ; RELOC-NEXT:       Type: R_TOC (0x3)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x40
+; RELOC:       Virtual Address: 0x44
 ; RELOC-NEXT:       Symbol: .__get_tpointer (1)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 26
 ; RELOC-NEXT:       Type: R_RBA (0x18)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x84
+; RELOC:       Virtual Address: 0x80
 ; RELOC-NEXT:       Symbol: IThreadLocalVarUninit (29)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 32
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
-; RELOC:       Virtual Address: 0x88
+; RELOC:       Virtual Address: 0x84
 ; RELOC-NEXT:       Symbol: ThreadLocalVarInit (27)
 ; RELOC-NEXT:       IsSigned: No
 ; RELOC-NEXT:       FixupBitValue: 0
 ; RELOC-NEXT:       Length: 32
 ; RELOC-NEXT:       Type: R_TLS_LE (0x23)
 ; RELOC-NEXT:     }
 
 ; SYM:      File: {{.*}}aix-tls-le-xcoff-reloc32.ll.tmp.o

 ; SYM-NEXT:       SymbolType: XTY_ER (0x0)
 ; SYM-NEXT:       StorageMappingClass: XMC_PR (0x0)
 ; SYM-NEXT:       StabInfoIndex: 0x0
 ; SYM-NEXT:       StabSectNum: 0x0
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 19
 ; SYM-NEXT:     Name: IThreadLocalVarUninit
-; SYM-NEXT:     Value (RelocatableAddress): 0x84
+; SYM-NEXT:     Value (RelocatableAddress): 0x80
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 20
 ; SYM-NEXT:       SectionLen: 4
 ; SYM-NEXT:       ParameterHashIndex: 0x0
 ; SYM-NEXT:       TypeChkSectNum: 0x0
 ; SYM-NEXT:       SymbolAlignmentLog2: 2
 ; SYM-NEXT:       SymbolType: XTY_SD (0x1)
 ; SYM-NEXT:       StorageMappingClass: XMC_TC (0x3)
 ; SYM-NEXT:       StabInfoIndex: 0x0
 ; SYM-NEXT:       StabSectNum: 0x0
 ; SYM-NEXT:     }
 ; SYM-NEXT:   }
 ; SYM:     Index: 21
 ; SYM-NEXT:     Name: ThreadLocalVarInit
-; SYM-NEXT:     Value (RelocatableAddress): 0x88
+; SYM-NEXT:     Value (RelocatableAddress): 0x84
 ; SYM-NEXT:     Section: .data
 ; SYM-NEXT:     Type: 0x0
 ; SYM-NEXT:     StorageClass: C_HIDEXT (0x6B)
 ; SYM-NEXT:     NumberOfAuxEntries: 1
 ; SYM-NEXT:     CSECT Auxiliary Entry {
 ; SYM-NEXT:       Index: 22
 ; SYM-NEXT:       SectionLen: 4
 ; SYM-NEXT:       ParameterHashIndex: 0x0

 ; DIS-NEXT:                                      mflr 0
 ; DIS-NEXT:                                      stwu 1, -32(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 5, 0(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 19) IThreadLocalVarUninit[TC]
 ; DIS-NEXT:                                      mr 4, 3
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA (idx: 1)      .__get_tpointer[PR]
 ; DIS-NEXT:                                      stw 0, 40(1)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 3, 5
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stw 4, 0(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               stwx 4, 3, 5
 ; DIS-NEXT:                                      addi 1, 1, 32
 ; DIS-NEXT:                                      lwz 0, 8(1)
 ; DIS-NEXT:                                      mtlr 0
 ; DIS-NEXT:                                      blr
 ; DIS:      00000030 (idx: 7) .loadTLInit:
 ; DIS-NEXT:                                      mflr 0
 ; DIS-NEXT:                                      stwu 1, -32(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 4, 4(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 21) ThreadLocalVarInit[TC]
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwz 5, 8(2)
 ; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOC        (idx: 23) VarInit[TC]
+; DIS-NEXT:                                      stw 0, 40(1)
 ; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               bla 0
 ; DIS-NEXT: {{0*}}[[#ADDR]]: R_RBA (idx: 1)      .__get_tpointer[PR]
-; DIS-NEXT:                                      stw 0, 40(1)
-; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               add 3, 3, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}               lwzx 3, 3, 4
 ; DIS-NEXT:                                      lwz 4, 0(5)
-; DIS-NEXT:                                      lwz 3, 0(3)
 ; DIS-NEXT:                                      add 3, 4, 3
 ; DIS-NEXT:                                      addi 1, 1, 32
 ; DIS-NEXT:                                      lwz 0, 8(1)
 ; DIS-NEXT:                                      mtlr 0
 ; DIS-NEXT:                                      blr
 
 ; DIS:      Disassembly of section .data:
-; DIS:      00000068 (idx: 11) VarInit:
-; DIS-NEXT:       68: 00 00 00 57
-; DIS:      0000006c (idx: 13) storeITLUninit[DS]:
-; DIS-NEXT:       6c: 00 00 00 00
-; DIS-NEXT: 0000006c:  R_POS        (idx: 5) .storeITLUninit
-; DIS-NEXT:       70: 00 00 00 84
-; DIS-NEXT: 00000070:  R_POS        (idx: 17) TOC[TC0]
-; DIS-NEXT:       74: 00 00 00 00
-; DIS:      00000078 (idx: 15) loadTLInit[DS]:
-; DIS-NEXT:       78: 00 00 00 30
-; DIS-NEXT: 00000078:  R_POS        (idx: 7) .loadTLInit
-; DIS-NEXT:       7c: 00 00 00 84
-; DIS-NEXT: 0000007c:  R_POS        (idx: 17) TOC[TC0]
+; DIS:      00000064 (idx: 11) VarInit:
+; DIS-NEXT:       64: 00 00 00 57
+; DIS:      00000068 (idx: 13) storeITLUninit[DS]:
+; DIS-NEXT:       68: 00 00 00 00
+; DIS-NEXT: 00000068:  R_POS        (idx: 5) .storeITLUninit
+; DIS-NEXT:       6c: 00 00 00 80
+; DIS-NEXT: 0000006c:  R_POS        (idx: 17) TOC[TC0]
+; DIS-NEXT:       70: 00 00 00 00
+; DIS:      00000074 (idx: 15) loadTLInit[DS]:
+; DIS-NEXT:       74: 00 00 00 30
+; DIS-NEXT: 00000074:  R_POS        (idx: 7) .loadTLInit
+; DIS-NEXT:       78: 00 00 00 80
+; DIS-NEXT: 00000078:  R_POS        (idx: 17) TOC[TC0]
+; DIS-NEXT:       7c: 00 00 00 00
+; DIS:      00000080 (idx: 19) IThreadLocalVarUninit[TC]:
 ; DIS-NEXT:       80: 00 00 00 00
-; DIS:      00000084 (idx: 19) IThreadLocalVarUninit[TC]:
+; DIS-NEXT: 00000080:  R_TLS_LE     (idx: 29) IThreadLocalVarUninit[UL]
+; DIS:      00000084 (idx: 21) ThreadLocalVarInit[TC]:
 ; DIS-NEXT:       84: 00 00 00 00
-; DIS-NEXT: 00000084:  R_TLS_LE     (idx: 29) IThreadLocalVarUninit[UL]
-; DIS:      00000088 (idx: 21) ThreadLocalVarInit[TC]:
-; DIS-NEXT:       88: 00 00 00 00
-; DIS-NEXT: 00000088:  R_TLS_LE     (idx: 27) ThreadLocalVarInit
-; DIS:      0000008c (idx: 23) VarInit[TC]:
-; DIS-NEXT:       8c: 00 00 00 68
-; DIS-NEXT: 0000008c:  R_POS        (idx: 11) VarInit
+; DIS-NEXT: 00000084:  R_TLS_LE     (idx: 27) ThreadLocalVarInit
+; DIS:      00000088 (idx: 23) VarInit[TC]:
+; DIS-NEXT:       88: 00 00 00 64
+; DIS-NEXT: 00000088:  R_POS        (idx: 11) VarInit
 
 ; DIS:      Disassembly of section .tdata:
 ; DIS:      00000000 (idx: 27) ThreadLocalVarInit:
 ; DIS-NEXT:        0: 00 00 00 01
 
 ; DIS:      Disassembly of section .tbss:
 ; DIS:      00000004 (idx: 29) IThreadLocalVarUninit[UL]:
 ; DIS-NEXT: ...