[AIX][TLS] Produce a faster local-exec access sequence with -maix-small-local-exec-tls (And optimize when load/store offsets are 0)

[amyk]

Summary

This patch utilizes the -maix-small-local-exec-tls option added in
D155544 to produce a faster access sequence for the local-exec TLS
model, where loading from the TOC can be avoided.

The patch either produces an addi/la with a displacement off of r13
(the thread pointer) when the address is calculated, or it produces an
addi/la followed by a load/store when the address is calculated and
used for further accesses.

This patch also optimizes this sequence a bit more where we can remove
the addi/la when the load/store offset is 0. A follow up patch will
be posted to account for when the load/store offset is non-zero, and
currently in these situations we keep the addi/la that precedes the
load/store.

Furthermore, this access sequence is only performed for TLS variables
that are less than ~32KB in size.

[amyk]

Update check to be slightly less than 32KB to account for additional overhead.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1566–1567

Fix sentence to make statement based on target and not host.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3394

What does AIXTLSUpperDisplacement represent? It is already given a value of 32K - 8. Should the hard coded subtraction be here, should the value of AIXTLSUpperDisplacement be further adjusted instead, or is there unintentional double adjustment happening?

For reference, I encountered no issues linking the result of assembly that performs la (using the subject access pattern) for the past-the-end address of a 32K - 1 local-exec TLS variable.

3410

Typo fix: "on supported" -> "only supported"

[hubert.reinterpretcast]

Instead, an addi/la with a displacement followed by a load/store
directly off of r13 (the thread pointer) is produced.

@amyk, the patch description seems overly focused on the load/store cases (as opposed to the "pure" address calculation cases).
Also, the sentence quoted above seems to imply that r13 is used as an operand of the load/store and not the addi/la (which is probably wrong in the cases where the addi/la is kept).

I also didn't see (in my skim through the tests) any case where the address is simply calculated and not used for a further access.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

This change looks suspicious.

I am getting:

Global must be word-aligned for LD, STD, LWA!
UNREACHABLE executed at /terrannew/hstong/.Lpcoral03/llvmbld/dev/llvm-project/llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp:1523!

With:

clang --target=powerpc64-ibm-aix -maix-small-local-exec-tls -O -c -o /dev/null -xc - \
<<<$'__attribute__((__tls_model__("local-exec"))) __thread struct { int x __attribute__((__packed__)); } a;\nint f() { return a.x; }'

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

The comment is out-of-date given the number of RequiresMod4Offset checks in the function. It seems the comment is bolstering the legitimacy of outdated code. The check on dyn_cast<ConstantSDNode>(ImmOpnd) further below seems to be a better version of the code here. Perhaps the code here should just be removed?

7725

Note: Looks like better version of the code is here.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1572

Typo?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

154

Should be constexpr.

3384

The last sentence is redundant with statements made on the 32-bit path.

3392

ICE/assert:

clang: /terrannew/hstong/.Lpcoral03/llvmbld/dev/llvm-project/llvm/include/llvm/IR/DataLayout.h:669: TypeSize llvm::DataLayout::getTypeSizeInBits(Type *) const: Assertion `Ty->isSized() && "Cannot getTypeInfo() on a type that is unsized!"' failed.

with

clang --target=powerpc64-ibm-aix -c -o /dev/null -xc++ -<<<$'extern __thread struct X x [[gnu::tls_model("local-exec")]];\nvoid *f() { return &x; }'

A similar case with an array of unknown bound does not assert; however, it does generate the small TLS access pattern despite the size being unknown (and potentially huge).

3406–3407

The comment should be clear that there is a 32-bit version that is merely omitted by the implementation here.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

40

The symbol would be a label (no storage class attached in the assembly syntax) if it is an alias or if data sections is disabled. What is the comment trying to say about the form of the symbol?

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1558

Nit: Indentation.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1524–1526

Not a single test failed when I removed this. Perhaps this does not belong in this patch (but a future one)?

1538–1555

Same comment re: the cases other than ADDI8.

1562

For the purposes of this patch, maybe assert up front that we are handling the ADDI8 case.

[nemanjai]

Sorry, I forgot to submit these comments when I reviewed this.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1559

I'm not sure I follow what the purpose of this function is. It would seem that it checks whether we are lowering the AIX Small TLS instruction and change any PPCIADDI8 to PPC::LA8. However, if neither of those are true, it doesn't do anything.

The issue I have with it is that if this returns false, we break out of the switch and simply call

LowerPPCMachineInstrToMCInst(MI, TmpInst, *this);
EmitToStreamer(*OutStreamer, TmpInst);

which would have happened if we never called this function anyway. So it would seem that we don't need the additions from lines 1541 to 1564. We can simply add PPC::ADDI8 into the switch above, and change it to PPC::LA8 if both of the following conditions are met:
Subtarget->hasAIXSmallLocalExecTLS() && MO.getTargetFlags() & PPCII::MO_TPREL_FLAG

Or maybe I'm missing something.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

Right, if Base.getOperand(1) is not a constant, we need to check the alignment of the GlobalAddress.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3394

I agree. We shouldn't be doing any arithmetic here on AIXTLSUpperDisplacement. We should set it to what we want it and only perform comparisons.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

46

I suppose it doesn't matter for precedence of & over &&, but I think it is still easier to read this condition if you parenthesize this part as (MO.getTargetFlags() & PPCII::MO_TPREL_FLAG).

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

36

This should be moved down past the early return.

47

This can be TM.getSymbol like the toc-data case. No need for unnecessary differences between the two.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

32–48

I replaced the entire function to unconditionally use AP.getSymbol(GV) for MO_GlobalAddress cases. All LIT and test-suite cases pass in 3-stage bootstrap builds on ppc64le Linux.

This should make sense: Going through the previously-default path to get a symbol using just the name is a loss of information (which was found to be problematic for toc-data and small TLS). I see no reason to keep the lossy path.

I strongly suggest going with a generic common path (which becomes well-tested) versus having special-case paths.

Since the change is not actually NFC and this patch seems to be the thing that we know can test it, I guess it goes in with this patch.

Note: I removed the part that does things with the offset. I am not sure if belongs in this function. The name of the function, GetSymbolFromOperand suggests that perhaps only the symbol itself should be considered. Also, the result of this function gets passed to GetSymbolRef, which already expects to get the offset from the MachineOperand.

[shchenz]

Is there any reason that we limit fast tls access to only local-exec mode? Can initial exec mode use the fast access? We are trying to add initial exec mode support in https://reviews.llvm.org/D156292

[DiggerLin]

according to https://www.ibm.com/docs/en/aix/7.2?topic=program-using-thread-local-storage

both 32-bit and 64-bit support faster access sequence for the local-exec TLS model. Do you want to modify the summary Since the patch only support 64-bit.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3394

according to https://www.ibm.com/docs/en/aix/7.2?topic=program-using-thread-local-storage

The local-dynamic and local-exec access methods have a faster code sequence that can be used if the total size of thread-local variables is smaller than 62 KB (I think it is typo here). If the total size of the region is too large, the link-editor will patch the code by generating extra instructions, negating the benefit of using the faster code sequence.

we need to calculate sum of the total size of thread-local variables somewhere and check whether the value is less than 32K.
does the AIXTLSUpperDisplacement want to express sum of the total size of thread-local variables ? if it the calculation is not correct.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3394

There is no intent to calculate the "total size of thread-local variables" here (there is no real point in doing so at a translation-unit level anyway). The documentation you quote is fairly loose about its use of "can". It is also incorrect in stating that the benefits of the faster code sequence are necessarily negated: an additional benefit of the faster code sequence is less use of the TOC.

It is entirely possible to have a variable > 32K allocated into the area accessible with the small TLS access sequence; however, not all bytes of such a variable can be accessed directly. This does not mean that we can't generate small TLS access sequences for such a variable, but the IBM XL compiler went with the limit (and it is less adventurous for us to follow suit on this).

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

96–97

In light of https://reviews.llvm.org/D156292, I believe this should have an assertion added about the TLS model (see llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp change in the aforementioned patch).

[nemanjai]

I don't think the last set of comments were addressed yet. Requesting changes to remove it from my queue.

[amyk]

• Update comments
• Add assert for TLS model in PPCMCInstLower.cpp
• Update GetSymbolFromOperand()
• Only produce the faster local-exec sequences when the global variable is sized and non-empty (to catch cases like an extern struct or unbounded array)
• Clean up PPCAsmPrinter.cpp

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1524–1526

This is no longer needed, as Nemanja previously pointed out.

1538–1555

I'm going to remove the other cases since they are not needed.

1559

Yes, you're right. This appears not to be needed. I originally had this because the implementation was more complex than expected, but now it is more simplified so I don't need this.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3392

Thanks for catching this. I've made it so that extra cases like these just simply use the TOC-based local-exec method.

3394

I have set AIXTLSUpperDisplacement to be the value that we want.
It is meant to represent that maximum size of the TLS variable, in which we can produce the small local-exec sequence for.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

32–48

Thanks for the suggestion and for discussing your questions, and concerns with me offline regarding this change.

96–97

I have updated it to add an assert for the local-exec model. Is this what you meant?

[hubert.reinterpretcast]

Partial re-review comments.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1539–1541

The actually important thing is the target flag and the instruction opcode.

1543–1544

Restrict the check to just the target flag (moving the other "condition" to be just an assertion).

[amyk]

• Rebase patch.
• Update comment and condition, as well as adding an assertion in PPCAsmPrinter.cpp.

[hubert.reinterpretcast]

Additional partial re-review comments.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

151–153

We chose the maximum based on the behaviour of the reference implementation (not from first principles).

154

"Displacement" implies "offset", not size. Maybe AIXSmallTlsPolicySizeLimit?

3382–3394

Since the limit is not derived from first principles, phrase as a policy choice. Additionally, start off by mentioning the option (so the comments make sense in the context of the earlier comment about the TOC-based path).

3395–3399

Check HasAIXSmallLocalExecTLS first before setting up for more checks.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

96–97

Yes.

[amyk]

Update PPCISelLowering.cpp to:

• Reword comments
• Change AIXTLSUpperDisplacement to AIXSmallTlsPolicySizeLimit
• Move additional checks on the global variable only if the non-TOC based local-exec access method is enabled

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

If we are to keep the condition, this needs a comment for why HasAIXSmallLocalExecTLS is a sufficient guard to limit the behaviour to just the new AIX "small TLS" accesses. The logic would be that the option is only valid for AIX and, on AIX, a ADDI/ADDI8 node with a global address for the immediate operand would necessarily be for "small TLS" (although possibly for the local-dynamic TLS model with future developments).

However: The optimization (with the checks) is generally sound, so maybe there should be no such check at all (except to limit the scope of the current change because some tests need updating). Yes, there can be downstream code that may have issues with generating the global address as an immediate operand for the load/store instruction, but changing the downstream code seems to be the solution for those cases.

If we keep the check, there should be a TODO comment to remove the artificial limitation.

7656–7657

Aside from the checks that limit the global address (as opposed to constant) immediate operand handling to "small TLS", this code does the same thing as the code for the toc-data cases (except that which operand is the immediate operand is different between the two cases).

The code can be made common (using ImmOpnd) in the non-ReplaceFlags path at line 7717.

Note: isa followed by cast/dyn_cast is an anti-pattern.

7656–7657

The comment is already wrong because the non-ReplaceFlags path can handle non-zero load/store offsets in combination with an original constant immediate operand that is not divisible by 4. Additionally, the other cases now include the toc-data cases (which does not necessarily provide aligned addresses).

I would rather have no comment than a comment that is actively wrong. Further below, I recommend a FIXME comment.

7664

Please add a FIXME comment that this does not account for cases where the offset from the load/store causes the combined immediate operand to be sufficiently aligned of the instruction encoding.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3392

Sorry for the extra churn: with the new name, we should use <= here (and adjust the value to 32751).

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

97

asserts print the spelling of the expression, so the \n is not needed.

[amyk]

@hubert.reinterpretcast I was going to update the patch, and then I thought about your comments a bit more.
Just wanted to double check on some suggestions before I misunderstand anything and update the patch if that's OK.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

This has been accounted for in PPCISelLowering.cpp.

7656–7657

• I think there are possibilities that HasAIXSmallLocalExecTLS may not be necessary here. Just wanted to check, when you say "artificial limitation", do you mean that HasAIXSmallLocalExecTLS check?
• Are you suggesting that this else if should be removed as we have ImmOpnd on 7677, and that it goes into the the code in 7717? Or am I misunderstanding? Just figured I'd double check because as you mentioned, this is checking the operand of the ADDI, whereas I believe the below accounts for the cases where we're dealing with the offset from the load/store and combined immediate operand.
• Are you also suggesting that the initial condition should not be if (isa<GlobalAddressSDNode>(BaseOp1) but rather a dyn_cast of a GlobalAddressSDNode?

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7656–7657

• I think there are possibilities that HasAIXSmallLocalExecTLS may not be necessary here. Just wanted to check, when you say "artificial limitation", do you mean that HasAIXSmallLocalExecTLS check?

Yes. Removing it does enable the optimization in more cases (e.g., TLS on Linux) though. It may make sense to pre-commit this change first.

• Are you suggesting that this else if should be removed as we have ImmOpnd on 7677, and that it goes into the the code in 7717? Or am I misunderstanding? Just figured I'd double check because as you mentioned, this is checking the operand of the ADDI, whereas I believe the below accounts for the cases where we're dealing with the offset from the load/store and combined immediate operand.

No. This comment is about the non-else if part: That code can be removed (some adjustment would be needed to the else if part here, but there are multiple choices for how much to do) and the substantially similar code near 7649 can be moved into the code at 7717 (with usage of ImmOpnd). That way, we have one less copy of the logic.

• Are you also suggesting that the initial condition should not be if (isa<GlobalAddressSDNode>(BaseOp1) but rather a dyn_cast of a GlobalAddressSDNode?

Yes, and that is what the code at 7649 does.

7664

For the else if part here, you may want to remove it in favour of the logic at line 7723.

[amyk]

Update PPCISelDAGToDAG.cpp as per review comments.

[amyk]

llvm/test/CodeGen/PowerPC/ppc64-nonfunc-calls.ll

40

@nemanjai I think remember before that you mentioned that perhaps it is not necessary to check if the Subtarget.hasAIXSmallLocalExecTLS(); flag in PPCISelDAGToDAG.cpp - and that checking if the global variable is aligned should be sufficient.

Through the recent comments on this patch, I think it makes sense that we don’t need to check for this flag. This would mean the change that I am doing apply to Linux, as well. Furthermore, I also updated PPCISelDAGToDAG.cpp to remove a portion that seemed like it had an equivalent portion later on in the file.

After doing some functional testing, I didn’t see any issues, with the exception of one Linux test case affected by the change I am making. This test case was introduced awhile back in 87deb0b8e3f14855e6c4652993be86684c2dd429. I believe in here we’re bitcasting a TLS global and we’re treating this as a call through a function pointer. The node we’re dealing with in the test case is a GlobalAddressSDNode and it is aligned/passes my alignment check, so it ends up getting folded.

Before my patch, the TLS relocation is on the addis/addi as expected, and the subsequent loads load off of r3:

# %bb.0:                                # %entry
	mflr 0
	stdu 1, -112(1)
	addis 3, 13, tls_something@tprel@ha
	std 0, 128(1)
	addi 3, 3, tls_something@tprel@l
	std 2, 40(1)
	ld 4, 0(3)
	ld 11, 16(3)
	ld 2, 8(3)
	mtctr 4
	bctrl
	ld 2, 40(1)
	addi 1, 1, 112
	ld 0, 16(1)
	mtlr 0
	blr

And after my patch, the ld with the zero offset gets affected, and none of the others do (which makes sense because we have non-zero offsets).

# %bb.0:                                # %entry
	mflr 0
	stdu 1, -112(1)
	addis 3, 13, tls_something@tprel@ha
	std 0, 128(1)
	addi 4, 3, tls_something@tprel@l
	ld 3, tls_something@tprel@l(3)
	std 2, 40(1)
	ld 11, 16(4)
	ld 2, 8(4)
	mtctr 3
	bctrl
	ld 2, 40(1)
	addi 1, 1, 112
	ld 0, 16(1)
	mtlr 0
	blr

In any case, I figured I’d ask if you have any thoughts/opinions on this since you requested changes and also had the original thought of not checking for hasAIXSmallLocalExecTLS() in the beginning.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7708–7709

Suggested wording change.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3392

This comment has not been addressed.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

97

This comment has not been addressed.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

100

Suggested wording change.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

7664

Please add a FIXME comment that this does not account for cases where the offset from the load/store causes the combined immediate operand to be sufficiently aligned of the instruction encoding.

My comment above is incorrect. The offset from the load/store was already aligned for the instruction encoding, so the combination will only remain aligned if the offset from the addi was also a multiple of 4.

Anyhow, the check here was redundant anyway (and has been removed). The change just happens to have no functional effect.

[amyk]

Reword comments, and address Hubert's previous comments that were previously missed.

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3392

Ah, I am very sorry. I thought I had addressed it but it turns out that I did not.
I have addressed it now.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

97

Sorry, addressed now.

100

@hubert.reinterpretcast After rebasing the patch, initial-exec also uses the same flag and code path. Thus, I don't think the assert() is appropriate here anymore.

I've updated this to just a check for local-exec and just setting the RefKind in this condition, which I think should be OK since it won't get set in the initial-exec case.

diff --git a/llvm/lib/Target/PowerPC/PPCMCInstLower.cpp b/llvm/lib/Target/PowerPC/PPCMCInstLower.cpp
index 92a1311e57a8..e2bb28b397c0 100644
--- a/llvm/lib/Target/PowerPC/PPCMCInstLower.cpp
+++ b/llvm/lib/Target/PowerPC/PPCMCInstLower.cpp
@@ -96,9 +96,8 @@ static MCOperand GetSymbolRef(const MachineOperand &MO, const MCSymbol *Symbol,
   else if (MO.getTargetFlags() == PPCII::MO_TPREL_FLAG) {
     assert(MO.isGlobal() && "Only expecting a global MachineOperand here!");
     TLSModel::Model Model = TM.getTLSModel(MO.getGlobal());
-    assert(Model == TLSModel::LocalExec &&
-           "Only local-exec accesses are handled!");
-    RefKind = MCSymbolRefExpr::VK_PPC_AIX_TLSLE;
+    if (Model == TLSModel::LocalExec)
+      RefKind = MCSymbolRefExpr::VK_PPC_AIX_TLSLE;
   }

   const MachineInstr *MI = MO.getParent();

I was wondering if you had any thoughts or objections to this.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3392

No problem; thanks for addressing.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

100

Considering that this change to set the RefKind was not needed for the base local-exec code model enablement, there is no reason to expect that initial-exec code actually gets here (and if it does, we should want to inspect the result of what this function does).

In other words, this code only became necessary for local-exec when we started applying relocations on the instruction operand, and since initial-exec TLS relocations on such operands are not allowed, the assert should stay. Actually, the old wording is correct with this rationale (so let's go with the old wording).

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

3389

Condition needs update with rebase.

[amyk]

Address comment of adding an additional TLS local-exec model check into PPCISelLowering.cpp.
I have not yet addressed the comment regarding the assert in PPCMCInstLower.cpp, as this is still being discussed.

[hubert.reinterpretcast]

LGTM with request to add a comment.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

100

It seems we run this code for TOC reference cases before we replace with a reference to the TOC entry. That may not be ideal, but it works.

We can leave the code as-is (with the if check). A comment would help:
For the local-exec TLS model, we may generate the offset from the TLS base as an immediate operand (instead of using a TOC entry); set the relocation type in case the result is used for purposes other than a TOC reference. In TOC reference cases, this result is discarded.

[amyk]

Address Hubert's comment of adding an comment to PPCMCInstLower.cpp.

@nemanjai I see you've requested changes from last time. If you're able to take a look again, that would be greatly appreciated.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

100

Thanks for discussing with me offline regarding this. I've added the comment.

[hubert.reinterpretcast]

Thanks @amyk for adding the comment. Confirming LGTM.

[amyk]

Patch rebased.

[nemanjai]

LGTM. The patch is easy to follow now. Thank you for your patience.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

34

Please add a comment here as to why this method is preferred. Perhaps something like:

// Get the symbol from the global, accounting for XCOFF-specific
// intricacies (see TargetLoweringObjectFileXCOFF::getTargetSymbol).

llvm/test/CodeGen/PowerPC/ppc64-nonfunc-calls.ll

40

This seems fine to me.

llvm/lib/Target/PowerPC/MCTargetDesc/PPCXCOFFObjectWriter.cpp

case PPC::fixup_ppc_half16: {
     default:
       report_fatal_error("Unsupported modifier for half16 fixup.");
     case MCSymbolRefExpr::VK_None:
       return {XCOFF::RelocationType::R_TOC, SignAndSizeForHalf16};
     case MCSymbolRefExpr::VK_PPC_U:
       return {XCOFF::RelocationType::R_TOCU, SignAndSizeForHalf16};
     case MCSymbolRefExpr::VK_PPC_L:
       return {XCOFF::RelocationType::R_TOCL, SignAndSizeForHalf16};
+    case MCSymbolRefExpr::VK_PPC_AIX_TLSLE:
+      return {XCOFF::RelocationType::R_TLS_LE, SignAndSizeForHalf16};
     }
   } break;
   case PPC::fixup_ppc_half16ds:
   case PPC::fixup_ppc_half16dq: {
     if (IsPCRel)
       report_fatal_error("Invalid PC-relative relocation.");
     switch (Modifier) {
     default:
       llvm_unreachable("Unsupported Modifier");
     case MCSymbolRefExpr::VK_None:
       return {XCOFF::RelocationType::R_TOC, 15};
     case MCSymbolRefExpr::VK_PPC_L:
       return {XCOFF::RelocationType::R_TOCL, 15};
+    case MCSymbolRefExpr::VK_PPC_AIX_TLSLE:
+      return {XCOFF::RelocationType::R_TLS_LE, 15};
     }
   } break;
   case PPC::fixup_ppc_br24:
     // Branches are 4 byte aligned, so the 24 bits we encode in
     // the instruction actually represents a 26 bit offset.
     return {XCOFF::RelocationType::R_RBR, EncodedSignednessIndicator | 25};
   case PPC::fixup_ppc_br24abs:
     return {XCOFF::RelocationType::R_RBA, EncodedSignednessIndicator | 25};

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

case PPC::LWA: {
     // that can't be supported.
     unsigned OpNum = (MI->getOpcode() == PPC::STD) ? 2 : 1;
     const MachineOperand &MO = MI->getOperand(OpNum);
     if (MO.isGlobal()) {
       const DataLayout &DL = MO.getGlobal()->getParent()->getDataLayout();
       if (MO.getGlobal()->getPointerAlignment(DL) < 4)
         llvm_unreachable("Global must be word-aligned for LD, STD, LWA!");
     }
     // Now process the instruction normally.
     break;
   }
   case PPC::PseudoEIEIO: {
     EmitToStreamer(
         *OutStreamer,
         MCInstBuilder(PPC::ORI).addReg(PPC::X2).addReg(PPC::X2).addImm(0));
     EmitToStreamer(
         *OutStreamer,
         MCInstBuilder(PPC::ORI).addReg(PPC::X2).addReg(PPC::X2).addImm(0));
     EmitToStreamer(*OutStreamer, MCInstBuilder(PPC::EnforceIEIO));
     return;
   }
+  case PPC::ADDI8: {
+    // The faster non-TOC-based local-exec sequence is represented by `addi`
+    // with an immediate operand having the MO_TPREL_FLAG. Such an instruction
+    // does not otherwise arise.
+    const MachineOperand &MO = MI->getOperand(2);
+    if ((MO.getTargetFlags() & PPCII::MO_TPREL_FLAG) != 0) {
+      assert(
+          Subtarget->hasAIXSmallLocalExecTLS() &&
+          "addi with thread-pointer only expected with local-exec small TLS");
+      LowerPPCMachineInstrToMCInst(MI, TmpInst, *this);
+      TmpInst.setOpcode(PPC::LA8);
+      EmitToStreamer(*OutStreamer, TmpInst);
+      return;
+    }
+    break;
+  }
   }
 
   LowerPPCMachineInstrToMCInst(MI, TmpInst, *this);
   EmitToStreamer(*OutStreamer, TmpInst);
 }
 
 void PPCLinuxAsmPrinter::emitGNUAttributes(Module &M) {
   // Emit float ABI into GNU attribute
   Metadata *MD = M.getModuleFlag("float-abi");
   MDString *FloatABI = dyn_cast_or_null<MDString>(MD);
   if (!FloatABI)
     return;
   StringRef flt = FloatABI->getString();
   // TODO: Support emitting soft-fp and hard double/single attributes.
   if (flt == "doubledouble")
     OutStreamer->emitGNUAttribute(Tag_GNU_Power_ABI_FP,
                                   Val_GNU_Power_ABI_HardFloat_DP |
                                       Val_GNU_Power_ABI_LDBL_IBM128);
   else if (flt == "ieeequad")
     OutStreamer->emitGNUAttribute(Tag_GNU_Power_ABI_FP,
                                   Val_GNU_Power_ABI_HardFloat_DP |
                                       Val_GNU_Power_ABI_LDBL_IEEE128);
   else if (flt == "ieeedouble")
     OutStreamer->emitGNUAttribute(Tag_GNU_Power_ABI_FP,
                                   Val_GNU_Power_ABI_HardFloat_DP |
                                       Val_GNU_Power_ABI_LDBL_64);
 }
 

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

while (Position != CurDAG->allnodes_begin()) {
     switch (Base.getMachineOpcode()) {
     default: continue;
 
     case PPC::ADDI8:
     case PPC::ADDI:
       // In some cases (such as TLS) the relocation information
       // is already in place on the operand, so copying the operand
       // is sufficient.
       ReplaceFlags = false;
-      // For these cases, the immediate may not be divisible by 4, in
-      // which case the fold is illegal for DS-form instructions.  (The
-      // other cases provide aligned addresses and are always safe.)
-      if (RequiresMod4Offset &&
-          (!isa<ConstantSDNode>(Base.getOperand(1)) ||
-           Base.getConstantOperandVal(1) % 4 != 0))
-        continue;
       break;
     case PPC::ADDIdtprelL:
       Flags = PPCII::MO_DTPREL_LO;
       break;
     case PPC::ADDItlsldL:
       Flags = PPCII::MO_TLSLD_LO;
       break;
     case PPC::ADDItocL:
       Flags = PPCII::MO_TOC_LO;
       break;
     }
 
     SDValue ImmOpnd = Base.getOperand(1);
 
     // On PPC64, the TOC base pointer is guaranteed by the ABI only to have
     // 8-byte alignment, and so we can only use offsets less than 8 (otherwise,
if (ReplaceFlags) {
 
         SDValue HImmOpnd = HBase.getOperand(1);
         if (HImmOpnd != ImmOpnd)
           continue;
 
         UpdateHBase = true;
       }
     } else {
+      // Global addresses can be folded, but only if they are sufficiently
+      // aligned.
+      if (RequiresMod4Offset) {
+        if (GlobalAddressSDNode *GA =
+                dyn_cast<GlobalAddressSDNode>(ImmOpnd)) {
+          const GlobalValue *GV = GA->getGlobal();
+          Align Alignment = GV->getPointerAlignment(CurDAG->getDataLayout());
+          if (Alignment < 4)
+            continue;
+        }
+      }
+
       // If we're directly folding the addend from an addi instruction, then:
       //  1. In general, the offset on the memory access must be zero.
       //  2. If the addend is a constant, then it can be combined with a
       //     non-zero offset, but only if the result meets the encoding
       //     requirements.
       if (auto *C = dyn_cast<ConstantSDNode>(ImmOpnd)) {
         Offset += C->getSExtValue();
 
         if (RequiresMod4Offset && (Offset % 4) != 0)
           continue;
 
         if (!isInt<16>(Offset))
           continue;
 

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

 STATISTIC(NumDynamicAllocaProbed, "Number of dynamic stack allocation probed");
 
 static bool isNByteElemShuffleMask(ShuffleVectorSDNode *, unsigned, int);
 
 static SDValue widenVec(SelectionDAG &DAG, SDValue Vec, const SDLoc &dl);
 
 static const char AIXSSPCanaryWordName[] = "__ssp_canary_word";
 
+// A faster local-exec TLS access sequence (enabled with the
+// -maix-small-local-exec-tls option) can be produced for TLS variables;
+// consistent with the IBM XL compiler, we apply a max size of slightly under
+// 32KB.
+constexpr uint64_t AIXSmallTlsPolicySizeLimit = 32751;
+
 // FIXME: Remove this once the bug has been fixed!
 extern cl::opt<bool> ANDIGlueBug;
 
 PPCTargetLowering::PPCTargetLowering(const PPCTargetMachine &TM,
                                      const PPCSubtarget &STI)
     : TargetLowering(TM), Subtarget(STI) {
   // Initialize map that relates the PPC addressing modes to the computed flags
   // of a load/store instruction. The map is used to determine the optimal
SDValue PPCTargetLowering::LowerGlobalTLSAddressAIX(SDValue Op,
 
   if (DAG.getTarget().useEmulatedTLS())
     report_fatal_error("Emulated TLS is not yet supported on AIX");
 
   SDLoc dl(GA);
   const GlobalValue *GV = GA->getGlobal();
   EVT PtrVT = getPointerTy(DAG.getDataLayout());
   bool Is64Bit = Subtarget.isPPC64();
+  bool HasAIXSmallLocalExecTLS = Subtarget.hasAIXSmallLocalExecTLS();
   TLSModel::Model Model = getTargetMachine().getTLSModel(GV);
+  bool IsTLSLocalExecModel = Model == TLSModel::LocalExec;
 
-  if (Model == TLSModel::LocalExec || Model == TLSModel::InitialExec) {
+  if (IsTLSLocalExecModel || Model == TLSModel::InitialExec) {
     SDValue VariableOffsetTGA =
         DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, PPCII::MO_TPREL_FLAG);
     SDValue VariableOffset = getTOCEntry(DAG, dl, VariableOffsetTGA);
     SDValue TLSReg;
-    if (Is64Bit)
+    if (Is64Bit) {
       // For local-exec and initial-exec on AIX (64-bit), the sequence generated
       // involves a load of the variable offset (from the TOC), followed by an
       // add of the loaded variable offset to R13 (the thread pointer).
       // This code sequence looks like:
       //    ld reg1,var[TC](2)
       //    add reg2, reg1, r13     // r13 contains the thread pointer
       TLSReg = DAG.getRegister(PPC::X13, MVT::i64);
-    else
+
+      // With the -maix-small-local-exec-tls option, produce a faster access
+      // sequence for local-exec TLS variables where the offset from the TLS
+      // base is encoded as an immediate operand.
+      //
+      // We only utilize the faster local-exec access sequence when the TLS
+      // variable has a size within the policy limit. We treat types that are
+      // not sized or are empty as being over the policy size limit.
+      if (HasAIXSmallLocalExecTLS && IsTLSLocalExecModel) {
+        Type *GVType = GV->getValueType();
+        if (GVType->isSized() && !GVType->isEmptyTy() &&
+            GV->getParent()->getDataLayout().getTypeAllocSize(GVType) <=
+                AIXSmallTlsPolicySizeLimit)
+          return DAG.getNode(PPCISD::Lo, dl, PtrVT, VariableOffsetTGA, TLSReg);
+      }
+    } else {
       // For local-exec and initial-exec on AIX (32-bit), the sequence generated
       // involves loading the variable offset from the TOC, generating a call to
       // .__get_tpointer to get the thread pointer (which will be in R3), and
       // adding the two together:
       //    lwz reg1,var[TC](2)
       //    bla .__get_tpointer
       //    add reg2, reg1, r3
       TLSReg = DAG.getNode(PPCISD::GET_TPOINTER, dl, PtrVT);
+
+      // We do not implement the 32-bit version of the faster access sequence
+      // for local-exec that is controlled by -maix-small-local-exec-tls.
+      if (HasAIXSmallLocalExecTLS)
+        report_fatal_error("The small-local-exec TLS access sequence is "
+                           "currently only supported on AIX (64-bit mode).");
+    }
     return DAG.getNode(PPCISD::ADD_TLS, dl, PtrVT, TLSReg, VariableOffset);
   }
 
   // Only Local-Exec, Initial-Exec and General-Dynamic TLS models are currently
   // supported models. If Local- or Initial-exec are not possible or specified,
   // all GlobalTLSAddress nodes are lowered using the general-dynamic model.
   // We need to generate two TOC entries, one for the variable offset, one for
   // the region handle. The global address for the TOC entry of the region

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/Mangler.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/Target/TargetLoweringObjectFile.h"
 using namespace llvm;
 
 static MCSymbol *GetSymbolFromOperand(const MachineOperand &MO,
                                       AsmPrinter &AP) {
-  const TargetMachine &TM = AP.TM;
-  Mangler &Mang = TM.getObjFileLowering()->getMangler();
-  const DataLayout &DL = AP.getDataLayout();
-  MCContext &Ctx = AP.OutContext;
+  if (MO.isGlobal()) {
+    const GlobalValue *GV = MO.getGlobal();
+    return AP.getSymbol(GV);
+  }
 
-  SmallString<128> Name;
-  if (!MO.isGlobal()) {
-    assert(MO.isSymbol() && "Isn't a symbol reference");
-    Mangler::getNameWithPrefix(Name, MO.getSymbolName(), DL);
-  } else {
-    const GlobalValue *GV = MO.getGlobal();
-    TM.getNameWithPrefix(Name, GV, Mang);
-  }
+
+  SmallString<128> Name;
+  const DataLayout &DL = AP.getDataLayout();
 
+  MCContext &Ctx = AP.OutContext;
   MCSymbol *Sym = Ctx.getOrCreateSymbol(Name);
-
   return Sym;
 }
 
 static MCOperand GetSymbolRef(const MachineOperand &MO, const MCSymbol *Symbol,
                               AsmPrinter &Printer) {
   MCContext &Ctx = Printer.OutContext;
   MCSymbolRefExpr::VariantKind RefKind = MCSymbolRefExpr::VK_None;
 
   unsigned access = MO.getTargetFlags() & PPCII::MO_ACCESS_MASK;
 
case PPCII::MO_TOC_LO:
       break;
     case PPCII::MO_TLS:
       bool IsPCRel = (MO.getTargetFlags() & ~access) == PPCII::MO_PCREL_FLAG;
       RefKind = IsPCRel ? MCSymbolRefExpr::VK_PPC_TLS_PCREL
                         : MCSymbolRefExpr::VK_PPC_TLS;
       break;
   }
 
+  const TargetMachine &TM = Printer.TM;
+
   if (MO.getTargetFlags() == PPCII::MO_PLT)
     RefKind = MCSymbolRefExpr::VK_PLT;
   else if (MO.getTargetFlags() == PPCII::MO_PCREL_FLAG)
     RefKind = MCSymbolRefExpr::VK_PCREL;
   else if (MO.getTargetFlags() == (PPCII::MO_PCREL_FLAG | PPCII::MO_GOT_FLAG))
     RefKind = MCSymbolRefExpr::VK_PPC_GOT_PCREL;
   else if (MO.getTargetFlags() == (PPCII::MO_PCREL_FLAG | PPCII::MO_TPREL_FLAG))
     RefKind = MCSymbolRefExpr::VK_TPREL;
   else if (MO.getTargetFlags() == PPCII::MO_GOT_TLSGD_PCREL_FLAG)
     RefKind = MCSymbolRefExpr::VK_PPC_GOT_TLSGD_PCREL;
   else if (MO.getTargetFlags() == PPCII::MO_GOT_TLSLD_PCREL_FLAG)
     RefKind = MCSymbolRefExpr::VK_PPC_GOT_TLSLD_PCREL;
   else if (MO.getTargetFlags() == PPCII::MO_GOT_TPREL_PCREL_FLAG)
     RefKind = MCSymbolRefExpr::VK_PPC_GOT_TPREL_PCREL;
+  else if (MO.getTargetFlags() == PPCII::MO_TPREL_FLAG) {
+    assert(MO.isGlobal() && "Only expecting a global MachineOperand here!");
+    TLSModel::Model Model = TM.getTLSModel(MO.getGlobal());
+    // For the local-exec TLS model, we may generate the offset from the TLS
+    // base as an immediate operand (instead of using a TOC entry).
+    // Set the relocation type in case the result is used for purposes other
+    // than a TOC reference. In TOC reference cases, this result is discarded.
+    if (Model == TLSModel::LocalExec)
+      RefKind = MCSymbolRefExpr::VK_PPC_AIX_TLSLE;
+  }
 
   const MachineInstr *MI = MO.getParent();
   const MachineFunction *MF = MI->getMF();
   const Module *M = MF->getFunction().getParent();
   const PPCSubtarget *Subtarget = &(MF->getSubtarget<PPCSubtarget>());
-  const TargetMachine &TM = Printer.TM;
 
   unsigned MIOpcode = MI->getOpcode();
   assert((Subtarget->isUsingPCRelativeCalls() || MIOpcode != PPC::BL8_NOTOC) &&
          "BL8_NOTOC is only valid when using PC Relative Calls.");
   if (Subtarget->isUsingPCRelativeCalls()) {
     if (MIOpcode == PPC::TAILB || MIOpcode == PPC::TAILB8 ||
         MIOpcode == PPC::TCRETURNdi || MIOpcode == PPC::TCRETURNdi8 ||
         MIOpcode == PPC::BL8_NOTOC || MIOpcode == PPC::BL8_NOTOC_RM) {

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-char.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global i8 1, align 1
+@VarInit = local_unnamed_addr global i8 87, align 1
+@IThreadLocalVarInit = internal thread_local(localexec) global i8 1, align 1
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+@c = thread_local(localexec) global [87 x i8] zeroinitializer, align 1
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, c[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, c[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @c)
+  %arrayidx = getelementptr inbounds [87 x i8], ptr %0, i64 0, i64 1
+  ret ptr %arrayidx
+}
+
+define void @storeITLInit(i8 noundef zeroext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stb r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stb r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @IThreadLocalVarInit)
+  store i8 %x, ptr %0, align 1
+  ret void
+}
+
+define void @storeTLInit(i8 noundef zeroext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stb r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stb r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @ThreadLocalVarInit)
+  store i8 %x, ptr %0, align 1
+  ret void
+}
+
+define zeroext i8 @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @IThreadLocalVarInit)
+  %1 = load i8, ptr %0, align 1
+  ret i8 %1
+}
+
+define zeroext i8 @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    clrldi r3, r3, 56
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    clrldi r3, r3, 56
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @IThreadLocalVarInit)
+  %1 = load i8, ptr %0, align 1
+  %2 = load i8, ptr @VarInit, align 1
+  %add = add i8 %2, %1
+  ret i8 %add
+}
+
+define zeroext i8 @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @ThreadLocalVarInit)
+  %1 = load i8, ptr %0, align 1
+  ret i8 %1
+}
+
+define zeroext i8 @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    clrldi r3, r3, 56
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    clrldi r3, r3, 56
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @ThreadLocalVarInit)
+  %1 = load i8, ptr %0, align 1
+  %2 = load i8, ptr @VarInit, align 1
+  %add = add i8 %2, %1
+  ret i8 %add
+}
+
+define void @loadStore1(i8 noundef zeroext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lbz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stb r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lbz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stb r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @IThreadLocalVarInit)
+  %1 = load i8, ptr %0, align 1
+  %add = add i8 %1, 9
+  store i8 %add, ptr %0, align 1
+  ret void
+}

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-double.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global double 1.000000e+00, align 8
+@VarInit = local_unnamed_addr global double 8.700000e+01, align 8
+@IThreadLocalVarInit = internal thread_local(localexec) global double 1.000000e+00, align 8
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+@f = thread_local(localexec) global [87 x double] zeroinitializer, align 8
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, f[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 48
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, f[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 48
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @f)
+  %arrayidx = getelementptr inbounds [87 x double], ptr %0, i64 0, i64 6
+  ret ptr %arrayidx
+}
+
+define void @storeITLInit(double noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  store double %x, ptr %0, align 8
+  ret void
+}
+
+define void @storeTLInit(double noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfd f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfd f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  store double %x, ptr %0, align 8
+  ret void
+}
+
+define double @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load double, ptr %0, align 8
+  ret double %1
+}
+
+define double @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f1, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xsadddp f1, f0, f1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f1, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xsadddp f1, f0, f1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load double, ptr %0, align 8
+  %2 = load double, ptr @VarInit, align 8
+  %add = fadd double %1, %2
+  ret double %add
+}
+
+define double @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  %1 = load double, ptr %0, align 8
+  ret double %1
+}
+
+define double @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f0, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f1, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xsadddp f1, f0, f1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f0, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f1, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xsadddp f1, f0, f1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  %1 = load double, ptr %0, align 8
+  %2 = load double, ptr @VarInit, align 8
+  %add = fadd double %1, %2
+  ret double %add
+}
+
+define void @loadStore1(double noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    vspltisw v2, 1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xvcvsxwdp vs0, vs34
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    vspltisw v2, 8
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xsadddp f0, f1, f0
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xvcvsxwdp vs1, vs34
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xsadddp f0, f0, f1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfd f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    vspltisw v2, 1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xvcvsxwdp vs0, vs34
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    vspltisw v2, 8
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xsadddp f0, f1, f0
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xvcvsxwdp vs1, vs34
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xsadddp f0, f0, f1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfd f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load double, ptr %0, align 8
+  %inc = fadd double %1, 1.000000e+00
+  %add = fadd double %inc, 8.000000e+00
+  store double %add, ptr %0, align 8
+  ret void
+}

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-float.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global float 1.000000e+00, align 4
+@VarInit = local_unnamed_addr global float 8.700000e+01, align 4
+@IThreadLocalVarInit = internal thread_local(localexec) global float 1.000000e+00, align 4
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+@e = thread_local(localexec) global [87 x float] zeroinitializer, align 4
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, e[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 16
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, e[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 16
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @e)
+  %arrayidx = getelementptr inbounds [87 x float], ptr %0, i64 0, i64 4
+  ret ptr %arrayidx
+}
+
+define void @storeITLInit(float noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  store float %x, ptr %0, align 4
+  ret void
+}
+
+define void @storeTLInit(float noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfs f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfs f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  store float %x, ptr %0, align 4
+  ret void
+}
+
+define float @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load float, ptr %0, align 4
+  ret float %1
+}
+
+define float @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    fadds f1, f0, f1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    fadds f1, f0, f1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load float, ptr %0, align 4
+  %2 = load float, ptr @VarInit, align 4
+  %add = fadd float %1, %2
+  ret float %add
+}
+
+define float @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f1, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  %1 = load float, ptr %0, align 4
+  ret float %1
+}
+
+define float @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f0, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    fadds f1, f0, f1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f0, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    fadds f1, f0, f1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  %1 = load float, ptr %0, align 4
+  %2 = load float, ptr @VarInit, align 4
+  %add = fadd float %1, %2
+  ret float %add
+}
+
+define void @loadStore1(float noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    vspltisw v2, 1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xvcvsxwdp vs0, vs34
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    vspltisw v2, 8
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xvcvsxwdp vs2, vs34
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    fadds f0, f1, f0
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    fadds f0, f0, f2
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfs f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    vspltisw v2, 1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f1, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xvcvsxwdp vs0, vs34
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    vspltisw v2, 8
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xvcvsxwdp vs2, vs34
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    fadds f0, f1, f0
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    fadds f0, f0, f2
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfs f0, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load float, ptr %0, align 4
+  %inc = fadd float %1, 1.000000e+00
+  %add = fadd float %inc, 8.000000e+00
+  store float %add, ptr %0, align 4
+  ret void
+}

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-int.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global i32 1, align 4
+@VarInit = local_unnamed_addr global i32 87, align 4
+@IThreadLocalVarInit = internal thread_local(localexec) global i32 1, align 4
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+%struct.anon = type { i32 }
+@ThreadLocalStruct = thread_local(localexec) global %struct.anon zeroinitializer, align 1
+@a = thread_local(localexec) global [87 x i32] zeroinitializer, align 4
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, a[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 12
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, a[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 12
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @a)
+  %arrayidx = getelementptr inbounds [87 x i32], ptr %0, i64 0, i64 3
+  ret ptr %arrayidx
+}
+
+define signext i32 @testUnaligned() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: testUnaligned:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, ThreadLocalStruct[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwa r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: testUnaligned:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, ThreadLocalStruct[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwa r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @ThreadLocalStruct)
+  %x = getelementptr inbounds %struct.anon, ptr %0, i32 0, i32 0
+  %1 = load i32, ptr %x, align 1
+  ret i32 %1
+}
+
+define void @storeITLInit(i32 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  store i32 %x, ptr %0, align 4
+  ret void
+}
+
+define void @storeTLInit(i32 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  store i32 %x, ptr %0, align 4
+  ret void
+}
+
+define signext i32 @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwa r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwa r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load i32, ptr %0, align 4
+  ret i32 %1
+}
+
+define signext i32 @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    extsw r3, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    extsw r3, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load i32, ptr %0, align 4
+  %2 = load i32, ptr @VarInit, align 4
+  %add = add nsw i32 %2, %1
+  ret i32 %add
+}
+
+define signext i32 @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwa r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwa r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  %1 = load i32, ptr %0, align 4
+  ret i32 %1
+}
+
+define signext i32 @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    extsw r3, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    extsw r3, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @ThreadLocalVarInit)
+  %1 = load i32, ptr %0, align 4
+  %2 = load i32, ptr @VarInit, align 4
+  %add = add nsw i32 %2, %1
+  ret i32 %add
+}
+
+define void @loadStore1(i32 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @IThreadLocalVarInit)
+  %1 = load i32, ptr %0, align 4
+  %add = add nsw i32 %1, 9
+  store i32 %add, ptr %0, align 4
+  ret void
+}
+

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-largeaccess.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+; Test disassembly of object.
+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mattr=+aix-small-local-exec-tls \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -xcoff-traceback-table=false \
+; RUN:      --code-model=large -filetype=obj -o %t.o < %s
+; RUN: llvm-objdump -D -r --symbol-description %t.o | FileCheck --check-prefix=DIS %s
+
+@mySmallLocalExecTLSv1 = thread_local(localexec) global [8187 x i32] zeroinitializer, align 4
+@mySmallLocalExecTLS2 = thread_local(localexec) global [4000 x i32] zeroinitializer, align 4
+@mySmallLocalExecTLS3 = thread_local(localexec) global [4000 x i32] zeroinitializer, align 4
+@mySmallLocalExecTLS4 = thread_local(localexec) global [4000 x i32] zeroinitializer, align 4
+@mySmallLocalExecTLS5 = thread_local(localexec) global [4000 x i32] zeroinitializer, align 4
+@mySmallLocalExecTLSv2 = thread_local(localexec) global [9000 x i32] zeroinitializer, align 4
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+
+; All accesses use a "faster" local-exec sequence directly off the thread pointer.
+define signext i32 @StoreArrays1() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: StoreArrays1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, mySmallLocalExecTLSv1[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r4, 1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r5, 4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r6, mySmallLocalExecTLS2[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r7, 2
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r4, mySmallLocalExecTLSv1[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, mySmallLocalExecTLS3[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r5, 24(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r3, 3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r7, 320(r6)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, 324(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, mySmallLocalExecTLS4[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r6, mySmallLocalExecTLS5[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r7, 88
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r3, 102
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r5, 328(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r7, 332(r6)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: StoreArrays1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, mySmallLocalExecTLSv1[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r4, 1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r5, 4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r6, mySmallLocalExecTLS2[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r7, 2
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r4, mySmallLocalExecTLSv1[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, mySmallLocalExecTLS3[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r5, 24(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r3, 3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r7, 320(r6)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, 324(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, mySmallLocalExecTLS4[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r6, mySmallLocalExecTLS5[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r7, 88
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r3, 102
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r5, 328(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r7, 332(r6)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLSv1)
+  store i32 1, ptr %0, align 4
+  %arrayidx1 = getelementptr inbounds [8187 x i32], ptr %0, i64 0, i64 6
+  store i32 4, ptr %arrayidx1, align 4
+  %1 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS2)
+  %arrayidx2 = getelementptr inbounds [4000 x i32], ptr %1, i64 0, i64 80
+  store i32 2, ptr %arrayidx2, align 4
+  %2 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS3)
+  %arrayidx3 = getelementptr inbounds [4000 x i32], ptr %2, i64 0, i64 81
+  store i32 3, ptr %arrayidx3, align 4
+  %3 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS4)
+  %arrayidx4 = getelementptr inbounds [4000 x i32], ptr %3, i64 0, i64 82
+  store i32 4, ptr %arrayidx4, align 4
+  %4 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS5)
+  %arrayidx5 = getelementptr inbounds [4000 x i32], ptr %4, i64 0, i64 83
+  store i32 88, ptr %arrayidx5, align 4
+  %5 = load i32, ptr %0, align 4
+  %6 = load i32, ptr %arrayidx1, align 4
+  %7 = load i32, ptr %arrayidx2, align 4
+  %8 = load i32, ptr %arrayidx3, align 4
+  %9 = load i32, ptr %arrayidx4, align 4
+  %add = add i32 %5, 88
+  %add9 = add i32 %add, %6
+  %add11 = add i32 %add9, %7
+  %add13 = add i32 %add11, %8
+  %add15 = add i32 %add13, %9
+  ret i32 %add15
+}
+
+; Example of one access using the regular local-exec access from the TOC.
+define signext i32 @StoreArrays2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: StoreArrays2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # target-flags(ppc-tprel) @mySmallLocalExecTLSv2
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r4, 1
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r5, 4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r6, mySmallLocalExecTLS2[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r7, 2
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r13, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r4, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, mySmallLocalExecTLS3[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r5, 24(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r3, 3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r7, 320(r6)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, 324(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, mySmallLocalExecTLS4[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r6, mySmallLocalExecTLS5[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r7, 88
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    li r3, 102
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r5, 328(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r7, 332(r6)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: StoreArrays2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r4, 1
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r5, 4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r6, mySmallLocalExecTLS2[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r7, 2
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r13, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r4, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, mySmallLocalExecTLS3[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r5, 24(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r3, 3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r7, 320(r6)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, 324(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, mySmallLocalExecTLS4[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r6, mySmallLocalExecTLS5[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r7, 88
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    li r3, 102
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r5, 328(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r7, 332(r6)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLSv2)
+  store i32 1, ptr %0, align 4
+  %arrayidx1 = getelementptr inbounds [9000 x i32], ptr %0, i64 0, i64 6
+  store i32 4, ptr %arrayidx1, align 4
+  %1 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS2)
+  %arrayidx2 = getelementptr inbounds [4000 x i32], ptr %1, i64 0, i64 80
+  store i32 2, ptr %arrayidx2, align 4
+  %2 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS3)
+  %arrayidx3 = getelementptr inbounds [4000 x i32], ptr %2, i64 0, i64 81
+  store i32 3, ptr %arrayidx3, align 4
+  %3 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS4)
+  %arrayidx4 = getelementptr inbounds [4000 x i32], ptr %3, i64 0, i64 82
+  store i32 4, ptr %arrayidx4, align 4
+  %4 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @mySmallLocalExecTLS5)
+  %arrayidx5 = getelementptr inbounds [4000 x i32], ptr %4, i64 0, i64 83
+  store i32 88, ptr %arrayidx5, align 4
+  %5 = load i32, ptr %0, align 4
+  %6 = load i32, ptr %arrayidx1, align 4
+  %7 = load i32, ptr %arrayidx2, align 4
+  %8 = load i32, ptr %arrayidx3, align 4
+  %9 = load i32, ptr %arrayidx4, align 4
+  %add = add i32 %5, 88
+  %add9 = add i32 %add, %6
+  %add11 = add i32 %add9, %7
+  %add13 = add i32 %add11, %8
+  %add15 = add i32 %add13, %9
+  ret i32 %add15
+}
+
+; DIS:      {{.*}}aix-small-local-exec-tls-largeaccess.ll.tmp.o:  file format aix5coff64-rs6000
+; DIS:      Disassembly of section .text:
+; DIS:      0000000000000000 (idx: 3) .StoreArrays1:
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 3, 13, 0
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 15) mySmallLocalExecTLSv1[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 4, 1
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 5, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 6, 13, 32748
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 17) mySmallLocalExecTLS2[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 7, 2
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 4, 0(13)
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 15) mySmallLocalExecTLSv1[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 4, 13, -16788
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 19) mySmallLocalExecTLS3[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 5, 24(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 3, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 7, 320(6)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 3, 324(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 4, 13, -788
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 21) mySmallLocalExecTLS4[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 6, 13, 15212
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 23) mySmallLocalExecTLS5[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 7, 88
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 3, 102
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 5, 328(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 7, 332(6)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                blr
+
+; DIS:      0000000000000050 (idx: 5) .StoreArrays2:
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addis 3, 2, 0
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCU (idx: 13) mySmallLocalExecTLSv2[TE]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 4, 1
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 5, 4
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 6, 13, 32748
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 17) mySmallLocalExecTLS2[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 7, 2
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                ld 3, 0(3)
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TOCL (idx: 13) mySmallLocalExecTLSv2[TE]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                add 3, 13, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 4, 0(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 4, 13, -16788
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 19) mySmallLocalExecTLS3[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 5, 24(3)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 3, 3
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 7, 320(6)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 3, 324(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 4, 13, -788
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 21) mySmallLocalExecTLS4[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                addi 6, 13, 15212
+; DIS-NEXT: {{0*}}[[#ADDR + 2]]: R_TLS_LE (idx: 23) mySmallLocalExecTLS5[TL]
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 7, 88
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                li 3, 102
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 5, 328(4)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                stw 7, 332(6)
+; DIS-NEXT: [[#%x, ADDR:]]: {{.*}}                blr
+
+; DIS:      Disassembly of section .data:
+; DIS:      00000000000000a0 (idx: 7) StoreArrays1[DS]:
+; DIS-NEXT:       a0: 00 00 00 00
+; DIS-NEXT: 00000000000000a0:  R_POS  (idx: 3) .StoreArrays1
+; DIS-NEXT:       a4: 00 00 00 00
+; DIS-NEXT:       a8: 00 00 00 00
+; DIS-NEXT: 00000000000000a8:  R_POS        (idx: 11) TOC[TC0]
+; DIS-NEXT:       ac: 00 00 00 d0
+
+; DIS:      00000000000000b8 (idx: 9) StoreArrays2[DS]:
+; DIS-NEXT:       b8: 00 00 00 00
+; DIS-NEXT: 00000000000000b8:  R_POS  (idx: 5) .StoreArrays2
+; DIS-NEXT:       bc: 00 00 00 50
+; DIS-NEXT:       c0: 00 00 00 00
+; DIS-NEXT: 00000000000000c0:  R_POS        (idx: 11) TOC[TC0]
+; DIS-NEXT:       c4: 00 00 00 d0
+
+; DIS:      00000000000000d0 (idx: 13) mySmallLocalExecTLSv2[TE]:
+; DIS-NEXT:       d0: 00 00 00 00
+; DIS-NEXT: 00000000000000d0:  R_TLS_LE     (idx: 25) mySmallLocalExecTLSv2[TL]
+; DIS-NEXT:       d4: 00 01 79 ec

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-longlong.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global i64 1, align 8
+@VarInit = local_unnamed_addr global i64 87, align 8
+@IThreadLocalVarInit = internal thread_local(localexec) global i64 1, align 8
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+%struct.anon = type { i64 }
+@ThreadLocalStruct = thread_local(localexec) global %struct.anon zeroinitializer, align 1
+@d = thread_local(localexec) global [87 x i64] zeroinitializer, align 8
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, d[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, d[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @d)
+  ret ptr %0
+}
+
+define i64 @testUnaligned() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: testUnaligned:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, ThreadLocalStruct[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: testUnaligned:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, ThreadLocalStruct[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = call align 1 ptr @llvm.threadlocal.address.p0(ptr align 1 @ThreadLocalStruct)
+  %x = getelementptr inbounds %struct.anon, ptr %0, i32 0, i32 0
+  %1 = load i64, ptr %x, align 1
+  ret i64 %1
+}
+
+define void @storeITLInit(i64 noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    std r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    std r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  store i64 %x, ptr %0, align 8
+  ret void
+}
+
+define void @storeTLInit(i64 noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    std r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    std r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  store i64 %x, ptr %0, align 8
+  ret void
+}
+
+define i64 @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load i64, ptr %0, align 8
+  ret i64 %1
+}
+
+define i64 @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load i64, ptr %0, align 8
+  %2 = load i64, ptr @VarInit, align 8
+  %add = add nsw i64 %2, %1
+  ret i64 %add
+}
+
+define i64 @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  %1 = load i64, ptr %0, align 8
+  ret i64 %1
+}
+
+define i64 @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @ThreadLocalVarInit)
+  %1 = load i64, ptr %0, align 8
+  %2 = load i64, ptr @VarInit, align 8
+  %add = add nsw i64 %2, %1
+  ret i64 %add
+}
+
+define void @loadStore1(i64 noundef %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    std r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    std r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @IThreadLocalVarInit)
+  %1 = load i64, ptr %0, align 8
+  %add = add nsw i64 %1, 9
+  store i64 %add, ptr %0, align 8
+  ret void
+}

llvm/test/CodeGen/PowerPC/aix-small-local-exec-tls-short.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
+
+@ThreadLocalVarInit = thread_local(localexec) global i16 1, align 2
+@VarInit = local_unnamed_addr global i16 87, align 2
+@IThreadLocalVarInit = internal thread_local(localexec) global i16 1, align 2
+declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
+@b = thread_local(localexec) global [87 x i16] zeroinitializer, align 2
+
+define nonnull ptr @AddrTest1() local_unnamed_addr #0 {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, b[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: AddrTest1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, b[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @b)
+  %arrayidx = getelementptr inbounds [87 x i16], ptr %0, i64 0, i64 2
+  ret ptr %arrayidx
+}
+
+define void @storeITLInit(i16 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    sth r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    sth r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @IThreadLocalVarInit)
+  store i16 %x, ptr %0, align 2
+  ret void
+}
+
+define void @storeTLInit(i16 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    sth r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    sth r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @ThreadLocalVarInit)
+  store i16 %x, ptr %0, align 2
+  ret void
+}
+
+define signext i16 @loadITLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lha r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lha r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @IThreadLocalVarInit)
+  %1 = load i16, ptr %0, align 2
+  ret i16 %1
+}
+
+define signext i16 @loadITLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lhz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lhz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    extsh r3, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadITLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lhz r4, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lhz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    extsh r3, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @IThreadLocalVarInit)
+  %1 = load i16, ptr %0, align 2
+  %2 = load i16, ptr @VarInit, align 2
+  %add = add i16 %2, %1
+  ret i16 %add
+}
+
+define signext i16 @loadTLInit() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lha r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lha r3, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @ThreadLocalVarInit)
+  %1 = load i16, ptr %0, align 2
+  ret i16 %1
+}
+
+define signext i16 @loadTLInit2() {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, L..C0(r2) # @VarInit
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lhz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lhz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    extsh r3, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadTLInit2:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addis r3, L..C0@u(r2)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lhz r4, ThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, L..C0@l(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lhz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    add r3, r3, r4
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    extsh r3, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @ThreadLocalVarInit)
+  %1 = load i16, ptr %0, align 2
+  %2 = load i16, ptr @VarInit, align 2
+  %add = add i16 %2, %1
+  ret i16 %add
+}
+
+define void @loadStore1(i16 noundef signext %x) {
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lhz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    sth r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadStore1:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lhz r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r3, 9
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    sth r3, IThreadLocalVarInit[TL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
+entry:
+  %0 = tail call align 2 ptr @llvm.threadlocal.address.p0(ptr align 2 @IThreadLocalVarInit)
+  %1 = load i16, ptr %0, align 2
+  %add = add i16 %1, 9
+  store i16 %add, ptr %0, align 2
+  ret void
+}
+

llvm/test/CodeGen/PowerPC/aix-tls-le-ldst-O0.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL64-O0
 ; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE64-O0
 ; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff < %s | FileCheck %s \
 ; RUN:      --check-prefix=SMALL32-O0
 ; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
 ; RUN:      -mtriple powerpc-ibm-aix-xcoff --code-model=large < %s \
 ; RUN:      | FileCheck %s --check-prefix=LARGE32-O0
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff -mattr=+aix-small-local-exec-tls < %s \
+; RUN:      | FileCheck %s --check-prefix=SMALL-LOCAL-EXEC-SMALLCM64
+; RUN: llc  -O0 -verify-machineinstrs -mcpu=pwr7 -ppc-asm-full-reg-names \
+; RUN:      -mtriple powerpc64-ibm-aix-xcoff --code-model=large \
+; RUN:      -mattr=+aix-small-local-exec-tls < %s | FileCheck %s \
+; RUN:      --check-prefix=SMALL-LOCAL-EXEC-LARGECM64
 
 @TLInt = internal thread_local(localexec) global i32 0, align 4
 @TLLongLong = internal thread_local(localexec) global i64 0, align 8
 @TLDouble = internal thread_local(localexec) global double 0.000000e+00, align 8
 @TLFloat = internal thread_local(localexec) global float 0.000000e+00, align 4
 declare nonnull ptr @llvm.threadlocal.address.p0(ptr nonnull) #1
 
 define void @storeInt(i32 noundef %x) {

 ; LARGE32-O0-NEXT:    mr r4, r3
 ; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
 ; LARGE32-O0-NEXT:    add r4, r4, r5
 ; LARGE32-O0-NEXT:    stw r3, 0(r4)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeInt:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, TLInt[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stw r3, 0(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeInt:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, TLInt[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stw r3, 0(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLInt)
   store i32 %x, ptr %0, align 4
   ret void
 }
 
 define void @storeLongLong(i64 noundef %x) {
 ; SMALL64-O0-LABEL: storeLongLong:

 ; LARGE32-O0-NEXT:    lwz r3, 28(r1) # 4-byte Folded Reload
 ; LARGE32-O0-NEXT:    add r4, r4, r6
 ; LARGE32-O0-NEXT:    stw r5, 4(r4)
 ; LARGE32-O0-NEXT:    stw r3, 0(r4)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeLongLong:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r4, TLLongLong[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    std r3, 0(r4)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeLongLong:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r4, TLLongLong[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    std r3, 0(r4)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLLongLong)
   store i64 %x, ptr %0, align 8
   ret void
 }
 
 define void @storeDouble(double noundef %x) {
 ; SMALL64-O0-LABEL: storeDouble:

 ; LARGE32-O0-NEXT:    lwz r4, L..C2@l(r3)
 ; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
 ; LARGE32-O0-NEXT:    add r3, r3, r4
 ; LARGE32-O0-NEXT:    stfd f1, 0(r3)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeDouble:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLDouble[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stxsdx f1, 0, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeDouble:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLDouble[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stxsdx f1, 0, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLDouble)
   store double %x, ptr %0, align 8
   ret void
 }
 
 define void @storeFloat(float noundef %x) {
 ; SMALL64-O0-LABEL: storeFloat:

 ; LARGE32-O0-NEXT:    lwz r4, L..C3@l(r3)
 ; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
 ; LARGE32-O0-NEXT:    add r3, r3, r4
 ; LARGE32-O0-NEXT:    stfs f1, 0(r3)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: storeFloat:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLFloat[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: storeFloat:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLFloat[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfs f1, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLFloat)
   store float %x, ptr %0, align 4
   ret void
 }
 
 define i32 @loadInt() {
 ; SMALL64-O0-LABEL: loadInt:

 ; LARGE32-O0-NEXT:    lwz r4, L..C0@l(r3)
 ; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
 ; LARGE32-O0-NEXT:    add r3, r3, r4
 ; LARGE32-O0-NEXT:    lwz r3, 0(r3)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadInt:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLInt[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadInt:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLInt[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLInt)
   %1 = load i32, ptr %0, align 4
   ret i32 %1
 }
 
 define i32 @loadLongLong() {
 ; SMALL64-O0-LABEL: loadLongLong:

 ; LARGE32-O0-NEXT:    lwz r4, L..C1@l(r3)
 ; LARGE32-O0-NEXT:    bla .__get_tpointer[PR]
 ; LARGE32-O0-NEXT:    add r3, r3, r4
 ; LARGE32-O0-NEXT:    lwz r3, 4(r3)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadLongLong:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLLongLong[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadLongLong:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLLongLong[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    ld r3, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    # kill: def $r3 killed $r3 killed $x3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLLongLong)
   %1 = load i64, ptr %0, align 8
   %conv = trunc i64 %1 to i32
   ret i32 %conv
 }
 
 define i32 @loadDouble() {

 ; LARGE32-O0-NEXT:    xscvdpsxws f0, f0
 ; LARGE32-O0-NEXT:    addi r3, r1, 28
 ; LARGE32-O0-NEXT:    stfiwx f0, 0, r3
 ; LARGE32-O0-NEXT:    lwz r3, 28(r1)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadDouble:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLDouble[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfd f0, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    # kill: def $f1 killed $f0
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    xscvdpsxws f0, f0
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    addi r3, r1, -12
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfiwx f0, 0, r3
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwz r3, -12(r1)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadDouble:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLDouble[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfd f0, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    # kill: def $f1 killed $f0
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    xscvdpsxws f0, f0
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    addi r3, r1, -12
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfiwx f0, 0, r3
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwz r3, -12(r1)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 8 ptr @llvm.threadlocal.address.p0(ptr align 8 @TLDouble)
   %1 = load double, ptr %0, align 8
   %conv = fptosi double %1 to i32
   ret i32 %conv
 }
 
 define i32 @loadFloat() {

 ; LARGE32-O0-NEXT:    xscvdpsxws f0, f0
 ; LARGE32-O0-NEXT:    addi r3, r1, 28
 ; LARGE32-O0-NEXT:    stfiwx f0, 0, r3
 ; LARGE32-O0-NEXT:    lwz r3, 28(r1)
 ; LARGE32-O0-NEXT:    addi r1, r1, 32
 ; LARGE32-O0-NEXT:    lwz r0, 8(r1)
 ; LARGE32-O0-NEXT:    mtlr r0
 ; LARGE32-O0-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-SMALLCM64-LABEL: loadFloat:
+; SMALL-LOCAL-EXEC-SMALLCM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    la r3, TLFloat[UL]@le(r13)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lfs f0, 0(r3)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    fctiwz f0, f0
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    stfd f0, -8(r1)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    lwa r3, -4(r1)
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-SMALLCM64-NEXT:    blr
+;
+; SMALL-LOCAL-EXEC-LARGECM64-LABEL: loadFloat:
+; SMALL-LOCAL-EXEC-LARGECM64:       # %bb.0: # %entry
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    la r3, TLFloat[UL]@le(r13)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lfs f0, 0(r3)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    fctiwz f0, f0
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    stfd f0, -8(r1)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    lwa r3, -4(r1)
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    clrldi r3, r3, 32
+; SMALL-LOCAL-EXEC-LARGECM64-NEXT:    blr
 entry:
   %0 = tail call align 4 ptr @llvm.threadlocal.address.p0(ptr align 4 @TLFloat)
   %1 = load float, ptr %0, align 4
   %conv = fptosi float %1 to i32
   ret i32 %conv
 }
 
 ; TOC Entry Checks.

llvm/test/CodeGen/PowerPC/ppc64-nonfunc-calls.ll

 entry:
   tail call void @tls_something() #0
   ret void
 
 ; CHECK-LABEL: @bar
 ; CHECK-DAG: addis [[REG1:[0-9]+]], 13, tls_something@tprel@ha
 ; CHECK-DAG: std 2, 40(1)
 ; CHECK-DAG: addi [[REG3:[0-9]+]], [[REG1]], tls_something@tprel@l
-; CHECK-DAG: ld [[REG2:[0-9]+]], 0([[REG3]])
+; CHECK-DAG: ld [[REG2:[0-9]+]], tls_something@tprel@l([[REG1]])
 ; CHECK-DAG: ld 11, 16([[REG3]])
 ; CHECK-DAG: ld 2, 8([[REG3]])
 ; CHECK-DAG: mtctr [[REG2]]
 ; CHECK: bctrl
 ; CHECK: ld 2, 40(1)
 ; CHECK: blr
 }