[AIX] TOC pseudo expansion for 64bit large + 64bit small + 32bit large modes

[Xiangling_L]

Summary

On AIX, for toc-referenced globals,

1. under large code model, we generate:

[32 bit]
LWZtocL(@sym, ADDIStocHA(%r2, @sym))
[64 bit]
LDtocL(@sym, ADDIStocHA8(%x2, @sym)).

2. under small code model, we generate:

[32 bit]
LWZtoc @op1, %r2
[64 bit]
LDtoc @op1, %x2

In this patch, we provide support for peudo ops including ADDIStocHA8, ADDIStocHA, LWZtocL, LDtoc, LDtocL for AIX('LWZtoc' is supported by an earlier patch already), lowering them from MIR to assembly.

[Xiangling_L]

Rebase on latest master to pick up 'isDarwin' related patch

[sfertile]

llvm/lib/Target/PowerPC/MCTargetDesc/PPCInstPrinter.cpp

80

SRE is only used in an assert, which will introduce an unused local warning when asserts are disabled.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

516

Should we try to include 'TOC' in the name since this is specifically for mapping the TOC pseudos symbol ref operand into an MCSymbol?

736

isDarwin --> IsDarwin. (There are a couple places in EmitInstruction that will need to be updated similarly)

[Xiangling_L]

Move variable 'SRE' into assertion & rename function name to 'getMCSymbolForTOCPseudoMO'

[sfertile]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

755

Very minor nit: We can slightly simplify if you drop the `!' and reverse the true/false sides.

829

Was the ';' after 'block-address' meant to be a comma?

838

Minor nit: We can fold away the if:

bool GlobalToc = MO.isGlobal() && Subtarget->isGVIndirectSymbol(MO.getGlobal());

880

Instead of using an 'if' statement that will be empty when assertions are disabled, can we fold this condition into the assert? ie

LLVM_DEBUG(
  assert((!MO.isGlobal() || Subtarget->isGVIndirectSymbol(MO.getGlobal())) && ...

908–909

Ditto on folding away the if.

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

1–5

Please add '-verify-machine-instr` and an mcpu option to each llc invocation.

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

3

ditto.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/MCTargetDesc/PPCInstPrinter.cpp

77

Add "an" before "addis".

80

Do not use dyn_cast for its Boolean value. Use isa.

81

Same comment regarding "an".

82

Add "if it is an expression at all" to the end of the sentence.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

923

There is quite a bit of noise in this patch from these NFC changes. Please split them out.

llvm/lib/Target/PowerPC/PPCInstrInfo.td

3172 ↗

(On Diff #223652)

Why the whitespace change on this line? The surrounding defs have spaces around the colon.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

829

Add "or" before "is a block address".
Add "the" before "large code model".
Add a comma after "Otherwise".

[jasonliu]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

908

nit: MOSymbol could move down a bit to be closer to where it get used.

Also I would want to add const to this MOSymbol. But it seems that a lot of similar MOSymbols in this file do not have const with them, only add const for this one here would create inconsistency. Not sure if it's worth to have a NFC patch to add const for all the MOSymbol in this file first.

llvm/lib/Target/PowerPC/PPCInstrInfo.td

3171 ↗

(On Diff #223420)

Curious about what the effect is for adding hasSideEffects and isReMaterializable.
We already have ADDIStocHA before for the other targets, but they did not require hasSideEffects = 0, and isReMaterializable = 1.
So what is special about the AIX target that we need to add them? Or is it simply an omission before and it's actually needed for the other targets as well?
I try to remove this line here and no test case would fail. If we need those, should we add test case for it?

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

1–5

Do we want to add -verify-machineinstrs to every llc invocation?

[Xiangling_L]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

908

As Sean mentioned to me, Stefan is doing this NFC patch about MOSymbol, so I think I can put this up there: https://github.ibm.com/compiler/llvm-project/pull/672.

923

Thanks for mentioning this, will do.

[Xiangling_L]

llvm/lib/Target/PowerPC/PPCInstrInfo.td

3171 ↗

(On Diff #223420)

Some of my rational of adding these two properties are as follows, and please feel free to raise your further concerns:

1. For the instruction property hasSideEffects, it's used to indicate "does the instruction have side effects that are not captured by any operands of the instruction or other flags". And it's unset by default, the TableGen will infer its value from the instruction pattern when possible. One rationale I used to add this property is that since ADDIStocHA and ADDIStocHA8 [using under 64-bit mode] have the same instruction pattern, it should be safe to explicitly specify this hasSideEffects=0 for ADDIStocHA. And also considering ADDIStocHA itself, what it does is to load a value from TOC, it does have no side effect like implicit function call, volatile variable read etc, so I set this property to be 0.

2.For the instruction property isReMaterializable = 1, meaning this instruction has no side effects and requires no operands that aren't always available. The only allowed uses are constants and unallocatable physical registers so that the instructions result is independent of the place in the function.
Since it loads value from TOC, and TOC register is always reserved on PPC target, so I think it's safe to set it as 1. Plus, the same reason as above, I would prefer sync the behavior of ADDIStocHA and ADDIStocHA8`.

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

1–5

Thank you for reminding me of this, will do.

1–5

Yes, I will add it and cpu level for testcases as well.

[Xiangling_L]

Address 2nd round comments;
split out variable name NFC patch;
update testcases;

Notes:
A NFC patch about getMCSymbolForTOCPseudoMO will be splited out from this patch, and after that patch is landed, I will rebase this lowering patch onto it, then post following variable name NFC patch and 'hasSideEffects & isRematerializable' NFC patch.

[Xiangling_L]

Rebase on latest master after the PPCAsmPrinter cleanup NFC patch landed.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

757

Similar cases below have two more spaces of indentation.

847

See comment below re: the !.

890

I believe @sfertile made a comment about avoiding the ! in the condition in cases like these.

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

27

This does not follow. REG3 apparently holds the address of the operand for the load, so REG4 holds the address of the target of the store. We are loading the value to REG4 though, so its value will be clobbered before we get to the store.

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

40

That the ordering and interleaving of the logical operations involved differ between the various cases seem to indicate that the test is already too complicated. Please reduce the test to use a single memory operand (e.g., store a constant or return the value read).

[Xiangling_L]

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

40

@sfertile I guess your original purpose of creating this testcase is to test if load from TOC works for both load and store?

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

27

Sorry, it was my mistake, it should be [[REG5:[0-9]+]]

[Xiangling_L]

correct code format & the testcase

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

40

If that is indeed the intent, then the goal can be achieved with more tests that are simpler.

[hubert.reinterpretcast]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

928–929

Two more spaces here.

[Xiangling_L]

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

40

Thanks, I will update the testcase

[Xiangling_L]

Update testcases: split into simpler ones

[hubert.reinterpretcast]

LGTM with minor changes.

llvm/lib/Target/PowerPC/MCTargetDesc/PPCInstPrinter.cpp

82

Indentation:

assert(isa<MCSymbolRefExpr>(MI->getOperand(2).getExpr()) &&
       "The third operand of an addis instruction should be a symbol "
       "reference expression if it is an expression at all.");

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

736

The space after assert seems odd. clang-format removes this space.

768

We know IsDarwin is false here. For code readability purposes, I am okay with this line as-is because the Darwin code is supposedly going away.

823

Same comment about the space.

863

Same comment about the space.

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

333

This patch adds ADDIStocHA to two switches with a different ordering relative to ADDIStoHA8 in each switch.

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

15

This is a function entry point label, so

SMALL-LABEL: .test_load:{{$}}

is appropriate.

20

Same comment re: function entry label.

33–42

Same comment re: function entry label.

38

Same comment re: function entry label.

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

16

Same comment re: function entry label.

18

Just a note:
Currently, the front-end seems to not generate signext or zeroext into the IR for 64-bit AIX. It seems the default behaviour matches zeroext (meaning the return type is unsigned).

21

Same comment re: function entry label.

34

Same comment re: function entry label.

39

Same comment re: function entry label.

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

18

We discussed this today, and we want to explicitly add the zeroext in (or signext if that is was is intended). @Xiangling_L, please add this change on the commit as well. Thanks.

[Xiangling_L]

Minor changes & add zeroext to 64bit testcase

[hubert.reinterpretcast]

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

16

Does this pass without a colon? Check also for the cases below.

29

i32 zeroext %0

[Xiangling_L]

update the 64 bit testcase

[hubert.reinterpretcast]

LGTM. Thanks.

[Xiangling_L]

In D68341#1711822, @hubert.reinterpretcast wrote:

LGTM. Thanks.

Thank you for your reviews.

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

16

Thanks for mentioning this, it happended to pass because it matched .test_load in function descriptor. I will update it to function entry point label.

[saugustine]

This revision breaks builds with assertions disabled because IsPPC64 (introduced on line 539 of PPCAsmPrinter.cpp is unused when assertions are turned off.

[saugustine]

https://reviews.llvm.org/D69123 fixes this break.

llvm/include/llvm/MC/MCExpr.h

enum VariantKind : uint16_t {
     VK_PPC_GOT_LO,         // symbol@got@l
     VK_PPC_GOT_HI,         // symbol@got@h
     VK_PPC_GOT_HA,         // symbol@got@ha
     VK_PPC_TOCBASE,        // symbol@tocbase
     VK_PPC_TOC,            // symbol@toc
     VK_PPC_TOC_LO,         // symbol@toc@l
     VK_PPC_TOC_HI,         // symbol@toc@h
     VK_PPC_TOC_HA,         // symbol@toc@ha
+    VK_PPC_U,              // symbol@u
+    VK_PPC_L,              // symbol@l
     VK_PPC_DTPMOD,         // symbol@dtpmod
     VK_PPC_TPREL_LO,       // symbol@tprel@l
     VK_PPC_TPREL_HI,       // symbol@tprel@h
     VK_PPC_TPREL_HA,       // symbol@tprel@ha
     VK_PPC_TPREL_HIGH,     // symbol@tprel@high
     VK_PPC_TPREL_HIGHA,    // symbol@tprel@higha
     VK_PPC_TPREL_HIGHER,   // symbol@tprel@higher
     VK_PPC_TPREL_HIGHERA,  // symbol@tprel@highera

llvm/lib/MC/MCExpr.cpp

StringRef MCSymbolRefExpr::getVariantKindName(VariantKind Kind) {
   case VK_PPC_GOT_LO: return "got@l";
   case VK_PPC_GOT_HI: return "got@h";
   case VK_PPC_GOT_HA: return "got@ha";
   case VK_PPC_TOCBASE: return "tocbase";
   case VK_PPC_TOC: return "toc";
   case VK_PPC_TOC_LO: return "toc@l";
   case VK_PPC_TOC_HI: return "toc@h";
   case VK_PPC_TOC_HA: return "toc@ha";
+  case VK_PPC_U: return "u";
+  case VK_PPC_L: return "l";
   case VK_PPC_DTPMOD: return "dtpmod";
   case VK_PPC_TPREL_LO: return "tprel@l";
   case VK_PPC_TPREL_HI: return "tprel@h";
   case VK_PPC_TPREL_HA: return "tprel@ha";
   case VK_PPC_TPREL_HIGH: return "tprel@high";
   case VK_PPC_TPREL_HIGHA: return "tprel@higha";
   case VK_PPC_TPREL_HIGHER: return "tprel@higher";
   case VK_PPC_TPREL_HIGHERA: return "tprel@highera";
return StringSwitch<VariantKind>(Name.lower())
     .Case("got@h", VK_PPC_GOT_HI)
     .Case("got@ha", VK_PPC_GOT_HA)
     .Case("local", VK_PPC_LOCAL)
     .Case("tocbase", VK_PPC_TOCBASE)
     .Case("toc", VK_PPC_TOC)
     .Case("toc@l", VK_PPC_TOC_LO)
     .Case("toc@h", VK_PPC_TOC_HI)
     .Case("toc@ha", VK_PPC_TOC_HA)
+    .Case("u", VK_PPC_U)
+    .Case("l", VK_PPC_L)
     .Case("tls", VK_PPC_TLS)
     .Case("dtpmod", VK_PPC_DTPMOD)
     .Case("tprel@l", VK_PPC_TPREL_LO)
     .Case("tprel@h", VK_PPC_TPREL_HI)
     .Case("tprel@ha", VK_PPC_TPREL_HA)
     .Case("tprel@high", VK_PPC_TPREL_HIGH)
     .Case("tprel@higha", VK_PPC_TPREL_HIGHA)
     .Case("tprel@higher", VK_PPC_TPREL_HIGHER)

llvm/lib/Target/PowerPC/MCTargetDesc/PPCInstPrinter.cpp

if (RegName[0] == 'q' /* QPX */) {
     return;
   }
 
   OS << RegName;
 }
 
 void PPCInstPrinter::printInst(const MCInst *MI, raw_ostream &O,
                                StringRef Annot, const MCSubtargetInfo &STI) {
+  // Customize printing of the addis instruction on AIX. When an operand is a
+  // symbol reference, the instruction syntax is changed to look like a load
+  // operation, i.e:
+  //     Transform:  addis $rD, $rA, $src --> addis $rD, $src($rA).
+  if (TT.isOSAIX() &&
+      (MI->getOpcode() == PPC::ADDIS8 || MI->getOpcode() == PPC::ADDIS) &&
+      MI->getOperand(2).isExpr()) {
+    assert((MI->getOperand(0).isReg() && MI->getOperand(1).isReg()) &&
+           "The first and the second operand of an addis instruction"
+           " should be registers.");
+
+    assert(isa<MCSymbolRefExpr>(MI->getOperand(2).getExpr()) &&
+           "The third operand of an addis instruction should be a symbol "
+           "reference expression if it is an expression at all.");
+
+    O << "\taddis ";
+    printOperand(MI, 0, O);
+    O << ", ";
+    printOperand(MI, 2, O);
+    O << "(";
+    printOperand(MI, 1, O);
+    O << ")";
+    return;
+  }
+
   // Check for slwi/srwi mnemonics.
   if (MI->getOpcode() == PPC::RLWINM) {
     unsigned char SH = MI->getOperand(2).getImm();
     unsigned char MB = MI->getOperand(3).getImm();
     unsigned char ME = MI->getOperand(4).getImm();
     bool useSubstituteMnemonic = false;
     if (SH <= 31 && MB == 0 && ME == (31-SH)) {
       O << "\tslwi "; useSubstituteMnemonic = true;

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

void PPCAsmPrinter::EmitTlsCall(const MachineInstr *MI,
   EmitToStreamer(*OutStreamer,
                  MCInstBuilder(Subtarget->isPPC64() ?
                                PPC::BL8_NOP_TLS : PPC::BL_TLS)
                  .addExpr(TlsRef)
                  .addExpr(SymVar));
 }
 
 /// Map a machine operand for a TOC pseudo-machine instruction to its
 /// corresponding MCSymbol.
 static MCSymbol *getMCSymbolForTOCPseudoMO(const MachineOperand &MO,
                                            AsmPrinter &AP) {
   switch (MO.getType()) {
   case MachineOperand::MO_GlobalAddress:
     return AP.getSymbol(MO.getGlobal());
   case MachineOperand::MO_ConstantPoolIndex:
     return AP.GetCPISymbol(MO.getIndex());
   case MachineOperand::MO_JumpTableIndex:
     return AP.GetJTISymbol(MO.getIndex());
   case MachineOperand::MO_BlockAddress:
     return AP.GetBlockAddressSymbol(MO.getBlockAddress());
   default:
     llvm_unreachable("Unexpected operand type to get symbol.");
   }
 }
 
 /// EmitInstruction -- Print out a single PowerPC MI in Darwin syntax to
 /// the current output stream.
 ///
 void PPCAsmPrinter::EmitInstruction(const MachineInstr *MI) {
   MCInst TmpInst;
   const bool IsDarwin = TM.getTargetTriple().isOSDarwin();
+  const bool IsPPC64 = Subtarget->isPPC64();
+  const bool IsAIX = Subtarget->isAIXABI();
   const Module *M = MF->getFunction().getParent();
   PICLevel::Level PL = M->getPICLevel();
 
 #ifndef NDEBUG
   // Validate that SPE and FPU are mutually exclusive in codegen
   if (!MI->isInlineAsm()) {
     for (const MachineOperand &MO: MI->operands()) {
       if (MO.isReg()) {
case PPC::LWZtoc: {
     // Transform %rN = LWZtoc @op1, %r2
     LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
     // Change the opcode to LWZ.
     TmpInst.setOpcode(PPC::LWZ);
 
     const MachineOperand &MO = MI->getOperand(1);
     assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
-           "Unexpected operand type for LWZtoc pseudo.");
+           "Invalid operand for LWZtoc.");
 
     // Map the operand to its corresponding MCSymbol.
     const MCSymbol *const MOSymbol = getMCSymbolForTOCPseudoMO(MO, *this);
-    const bool IsAIX = TM.getTargetTriple().isOSAIX();
 
     // Create a reference to the GOT entry for the symbol. The GOT entry will be
     // synthesized later.
     if (PL == PICLevel::SmallPIC && !IsAIX) {
       const MCExpr *Exp =
         MCSymbolRefExpr::create(MOSymbol, MCSymbolRefExpr::VK_GOT,
                                 OutContext);
       TmpInst.getOperand(1) = MCOperand::createExpr(Exp);
       EmitToStreamer(*OutStreamer, TmpInst);
       return;
     }
 
-    // Otherwise use the TOC. 'TOCEntry' is a label used to reference the
+    // Otherwise, use the TOC. 'TOCEntry' is a label used to reference the
     // storage allocated in the TOC which contains the address of
     // 'MOSymbol'. Said TOC entry will be synthesized later.
     MCSymbol *TOCEntry = lookUpOrCreateTOCEntry(MOSymbol);
     const MCExpr *Exp =
         MCSymbolRefExpr::create(TOCEntry, MCSymbolRefExpr::VK_None, OutContext);
 
     // AIX uses the label directly as the lwz displacement operand for
     // references into the toc section. The displacement value will be generated
case PPC::LWZtoc: {
     TmpInst.getOperand(1) = MCOperand::createExpr(Exp);
     EmitToStreamer(*OutStreamer, TmpInst);
     return;
   }
   case PPC::LDtocJTI:
   case PPC::LDtocCPT:
   case PPC::LDtocBA:
   case PPC::LDtoc: {
+    assert(!IsDarwin && "TOC is an ELF/XCOFF construct");
+
     // Transform %x3 = LDtoc @min1, %x2
     LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
     // Change the opcode to LD.
     TmpInst.setOpcode(PPC::LD);
 
     const MachineOperand &MO = MI->getOperand(1);
     assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
            "Invalid operand!");
 
     // Map the machine operand to its corresponding MCSymbol, then map the
     // global address operand to be a reference to the TOC entry we will
     // synthesize later.
     MCSymbol *TOCEntry =
         lookUpOrCreateTOCEntry(getMCSymbolForTOCPseudoMO(MO, *this));
 
+    const MCSymbolRefExpr::VariantKind VK =
+        IsAIX ? MCSymbolRefExpr::VK_None : MCSymbolRefExpr::VK_PPC_TOC;
     const MCExpr *Exp =
-      MCSymbolRefExpr::create(TOCEntry, MCSymbolRefExpr::VK_PPC_TOC,
-                              OutContext);
+        MCSymbolRefExpr::create(TOCEntry, VK, OutContext);
     TmpInst.getOperand(1) = MCOperand::createExpr(Exp);
     EmitToStreamer(*OutStreamer, TmpInst);
     return;
   }
+  case PPC::ADDIStocHA: {
+    assert((IsAIX && !IsPPC64 && TM.getCodeModel() == CodeModel::Large) &&
+           "This pseudo should only be selected for 32-bit large code model on"
+           " AIX.");
+
+    // Transform %rd = ADDIStocHA %rA, @sym(%r2)
+    LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
+
+    // Change the opcode to ADDIS.
+    TmpInst.setOpcode(PPC::ADDIS);
+
+    const MachineOperand &MO = MI->getOperand(2);
+    assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+           "Invalid operand for ADDIStocHA.");
+
+    // Map the machine operand to its corresponding MCSymbol.
+    MCSymbol *MOSymbol = getMCSymbolForTOCPseudoMO(MO, *this);
+
+    // Always use TOC on AIX. Map the global address operand to be a reference
+    // to the TOC entry we will synthesize later. 'TOCEntry' is a label used to
+    // reference the storage allocated in the TOC which contains the address of
+    // 'MOSymbol'.
+    MCSymbol *TOCEntry = lookUpOrCreateTOCEntry(MOSymbol);
+    const MCExpr *Exp = MCSymbolRefExpr::create(TOCEntry,
+                                                MCSymbolRefExpr::VK_PPC_U,
+                                                OutContext);
+    TmpInst.getOperand(2) = MCOperand::createExpr(Exp);
+    EmitToStreamer(*OutStreamer, TmpInst);
+    return;
+  }
+  case PPC::LWZtocL: {
+    assert(IsAIX && !IsPPC64 && TM.getCodeModel() == CodeModel::Large &&
+           "This pseudo should only be selected for 32-bit large code model on"
+           " AIX.");
+
+    // Transform %rd = LWZtocL @sym, %rs.
+    LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
+    // Change the opcode to lwz.
+    TmpInst.setOpcode(PPC::LWZ);
+
+    const MachineOperand &MO = MI->getOperand(1);
+    assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
+           "Invalid operand for LWZtocL.");
+
+    // Map the machine operand to its corresponding MCSymbol.
+    MCSymbol *MOSymbol = getMCSymbolForTOCPseudoMO(MO, *this);
+
+    // Always use TOC on AIX. Map the global address operand to be a reference
+    // to the TOC entry we will synthesize later. 'TOCEntry' is a label used to
+    // reference the storage allocated in the TOC which contains the address of
+    // 'MOSymbol'.
+    MCSymbol *TOCEntry = lookUpOrCreateTOCEntry(MOSymbol);
+    const MCExpr *Exp = MCSymbolRefExpr::create(TOCEntry,
+                                                MCSymbolRefExpr::VK_PPC_L,
+                                                OutContext);
+    TmpInst.getOperand(1) = MCOperand::createExpr(Exp);
+    EmitToStreamer(*OutStreamer, TmpInst);
+    return;
+  }
   case PPC::ADDIStocHA8: {
+    assert(!IsDarwin && "TOC is an ELF/XCOFF construct");
+
     // Transform %xd = ADDIStocHA8 %x2, @sym
     LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
     // Change the opcode to ADDIS8. If the global address is the address of
     // an external symbol, is a jump table address, is a block address, or is a
     // constant pool index with large code model enabled, then generate a TOC
     // entry and reference that. Otherwise, reference the symbol directly.
     TmpInst.setOpcode(PPC::ADDIS8);
 
     const MachineOperand &MO = MI->getOperand(2);
     assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() || MO.isBlockAddress()) &&
            "Invalid operand for ADDIStocHA8!");
 
     const MCSymbol *MOSymbol = getMCSymbolForTOCPseudoMO(MO, *this);
 
     const bool GlobalToc =
         MO.isGlobal() && Subtarget->isGVIndirectSymbol(MO.getGlobal());
     if (GlobalToc || MO.isJTI() || MO.isBlockAddress() ||
         (MO.isCPI() && TM.getCodeModel() == CodeModel::Large))
       MOSymbol = lookUpOrCreateTOCEntry(MOSymbol);
 
+    const MCSymbolRefExpr::VariantKind VK =
+        IsAIX ? MCSymbolRefExpr::VK_PPC_U : MCSymbolRefExpr::VK_PPC_TOC_HA;
+
     const MCExpr *Exp =
-      MCSymbolRefExpr::create(MOSymbol, MCSymbolRefExpr::VK_PPC_TOC_HA,
-                              OutContext);
+        MCSymbolRefExpr::create(MOSymbol, VK, OutContext);
 
     if (!MO.isJTI() && MO.getOffset())
       Exp = MCBinaryExpr::createAdd(Exp,
                                     MCConstantExpr::create(MO.getOffset(),
                                                            OutContext),
                                     OutContext);
 
     TmpInst.getOperand(2) = MCOperand::createExpr(Exp);
     EmitToStreamer(*OutStreamer, TmpInst);
     return;
   }
   case PPC::LDtocL: {
+    assert(!IsDarwin && "TOC is an ELF/XCOFF construct");
+
     // Transform %xd = LDtocL @sym, %xs
     LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
     // Change the opcode to LD. If the global address is the address of
     // an external symbol, is a jump table address, is a block address, or is
     // a constant pool index with large code model enabled, then generate a
     // TOC entry and reference that. Otherwise, reference the symbol directly.
     TmpInst.setOpcode(PPC::LD);
 
     const MachineOperand &MO = MI->getOperand(1);
     assert((MO.isGlobal() || MO.isCPI() || MO.isJTI() ||
             MO.isBlockAddress()) &&
            "Invalid operand for LDtocL!");
 
     LLVM_DEBUG(assert(
         (!MO.isGlobal() || Subtarget->isGVIndirectSymbol(MO.getGlobal())) &&
         "LDtocL used on symbol that could be accessed directly is "
         "invalid. Must match ADDIStocHA8."));
 
     const MCSymbol *MOSymbol = getMCSymbolForTOCPseudoMO(MO, *this);
 
     if (!MO.isCPI() || TM.getCodeModel() == CodeModel::Large)
       MOSymbol = lookUpOrCreateTOCEntry(MOSymbol);
 
+    const MCSymbolRefExpr::VariantKind VK =
+        IsAIX ? MCSymbolRefExpr::VK_PPC_L : MCSymbolRefExpr::VK_PPC_TOC_LO;
     const MCExpr *Exp =
-      MCSymbolRefExpr::create(MOSymbol, MCSymbolRefExpr::VK_PPC_TOC_LO,
-                              OutContext);
+        MCSymbolRefExpr::create(MOSymbol, VK, OutContext);
     TmpInst.getOperand(1) = MCOperand::createExpr(Exp);
     EmitToStreamer(*OutStreamer, TmpInst);
     return;
   }
   case PPC::ADDItocL: {
     // Transform %xd = ADDItocL %xs, @sym
     LowerPPCMachineInstrToMCInst(MI, TmpInst, *this, IsDarwin);
 
     // Change the opcode to ADDI8. If the global address is external, then
     // generate a TOC entry and reference that. Otherwise, reference the
     // symbol directly.
     TmpInst.setOpcode(PPC::ADDI8);
 
     const MachineOperand &MO = MI->getOperand(2);
     assert((MO.isGlobal() || MO.isCPI()) && "Invalid operand for ADDItocL.");
 
     LLVM_DEBUG(assert(
         !(MO.isGlobal() && Subtarget->isGVIndirectSymbol(MO.getGlobal())) &&
         "Interposable definitions must use indirect access."));
 
     const MCExpr *Exp =
         MCSymbolRefExpr::create(getMCSymbolForTOCPseudoMO(MO, *this),
                                 MCSymbolRefExpr::VK_PPC_TOC_LO, OutContext);
     TmpInst.getOperand(2) = MCOperand::createExpr(Exp);
     EmitToStreamer(*OutStreamer, TmpInst);
     return;
   }
   case PPC::ADDISgotTprelHA: {
     // Transform: %xd = ADDISgotTprelHA %x2, @sym
     // Into:      %xd = ADDIS8 %x2, sym@got@tlsgd@ha
     assert(Subtarget->isPPC64() && "Not supported for 32-bit PowerPC");
     const MachineOperand &MO = MI->getOperand(2);
     const GlobalValue *GValue = MO.getGlobal();
     MCSymbol *MOSymbol = getSymbol(GValue);
     const MCExpr *SymGotTprel =
         MCSymbolRefExpr::create(MOSymbol, MCSymbolRefExpr::VK_PPC_GOT_TPREL_HA,
                                 OutContext);
     EmitToStreamer(*OutStreamer, MCInstBuilder(PPC::ADDIS8)
                                  .addReg(MI->getOperand(0).getReg())
                                  .addReg(MI->getOperand(1).getReg())
                                  .addExpr(SymGotTprel));
     return;
   }
   case PPC::LDgotTprelL:
   case PPC::LDgotTprelL32: {

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

default:
     // flag set.
     llvm_unreachable("Unknown rematerializable operation!");
     break;
   case PPC::LI:
   case PPC::LI8:
   case PPC::LIS:
   case PPC::LIS8:
   case PPC::QVGPCI:
+  case PPC::ADDIStocHA:
   case PPC::ADDIStocHA8:
   case PPC::ADDItocL:
   case PPC::LOAD_STACK_GUARD:
   case PPC::XXLXORz:
   case PPC::XXLXORspz:
   case PPC::XXLXORdpz:
   case PPC::XXLEQVOnes:
   case PPC::V_SET0B:
   case PPC::V_SET0H:
   case PPC::V_SET0:

llvm/test/CodeGen/PowerPC/lower-globaladdr32-aix-asm.ll

-; RUN: llc -mtriple powerpc-ibm-aix-xcoff \
-; RUN: -code-model=small < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mtriple powerpc-ibm-aix-xcoff \
+; RUN: -code-model=small < %s | FileCheck %s --check-prefixes=CHECK,SMALL
+
+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mtriple powerpc-ibm-aix-xcoff \
+; RUN: -code-model=large < %s | FileCheck %s --check-prefixes=CHECK,LARGE
 
-@b = common global i32 0
 @a = common global i32 0
 
-define void @test() {
-  %1 = load i32, i32* @b
-  store i32 %1, i32* @a
+define i32 @test_load() {
+entry:
+  %0 = load i32, i32* @a
+  ret i32 %0
+}
+
+; SMALL-LABEL: .test_load:{{$}}
+; SMALL: lwz [[REG1:[0-9]+]], LC0(2)
+; SMALL: lwz [[REG2:[0-9]+]], 0([[REG1]])
+; SMALL: blr
+
+; LARGE-LABEL: .test_load:{{$}}
+; LARGE: addis [[REG1:[0-9]+]], LC0@u(2)
+; LARGE: lwz [[REG2:[0-9]+]], LC0@l([[REG1]])
+; LARGE: lwz [[REG3:[0-9]+]], 0([[REG2]])
+; LARGE: blr
+
+@b = common global i32 0
+
+define void @test_store(i32 %0) {
+  store i32 %0, i32* @b
   ret void
 }
 
-; CHECK-LABEL:   test
-; CHECK-DAG:       lwz [[REG1:[0-9]+]], LC0(2)
-; CHECK-DAG:       lwz [[REG2:[0-9]+]], LC1(2)
-; CHECK-DAG:       lwz [[REG3:[0-9]+]], 0([[REG1]])
-; CHECK:           stw [[REG3]], 0([[REG2]])
-; CHECK:           blr
+; SMALL-LABEL: .test_store:{{$}}
+; SMALL: lwz [[REG1:[0-9]+]], LC1(2)
+; SMALL: stw [[REG2:[0-9]+]], 0([[REG1]])
+; SMALL: blr
+
+; LARGE-LABEL: .test_store:{{$}}
+; LARGE: addis [[REG1:[0-9]+]], LC1@u(2)
+; LARGE: lwz [[REG2:[0-9]+]], LC1@l([[REG1]])
+; LARGE: stw [[REG3:[0-9]+]], 0([[REG2]])
+; LARGE: blr
 
 ; TODO Update test when TOC-entry emission lands.
 ; CHECK-NOT: .tc

llvm/test/CodeGen/PowerPC/lower-globaladdr64-aix-asm.ll

+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mtriple powerpc64-ibm-aix-xcoff \
+; RUN: -code-model=small < %s | FileCheck %s --check-prefixes=CHECK,SMALL
+
+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mtriple powerpc64-ibm-aix-xcoff \
+; RUN: -code-model=large < %s | FileCheck %s --check-prefixes=CHECK,LARGE
+
+@a = common global i32 0
+
+define zeroext i32 @test_load() {
+entry:
+  %0 = load i32, i32* @a
+  ret i32 %0
+}
+
+; SMALL-LABEL: .test_load:{{$}}
+; SMALL: ld [[REG1:[0-9]+]], LC0(2)
+; SMALL: lwz [[REG2:[0-9]+]], 0([[REG1]])
+; SMALL: blr
+
+; LARGE-LABEL: .test_load:{{$}}
+; LARGE: addis [[REG1:[0-9]+]], LC0@u(2)
+; LARGE: ld [[REG2:[0-9]+]], LC0@l([[REG1]])
+; LARGE: lwz [[REG3:[0-9]+]], 0([[REG2]])
+; LARGE: blr
+
+@b = common global i32 0
+
+define void @test_store(i32 zeroext %0) {
+  store i32 %0, i32* @b
+  ret void
+}
+
+; SMALL-LABEL: .test_store:{{$}}
+; SMALL: ld [[REG1:[0-9]+]], LC1(2)
+; SMALL: stw [[REG2:[0-9]+]], 0([[REG1]])
+; SMALL: blr
+
+; LARGE-LABEL: .test_store:{{$}}
+; LARGE: addis [[REG1:[0-9]+]], LC1@u(2)
+; LARGE: ld [[REG2:[0-9]+]], LC1@l([[REG1]])
+; LARGE: stw [[REG3:[0-9]+]], 0([[REG2]])
+; LARGE: blr
+
+; TODO Update test when TOC-entry emission lands.
+; CHECK-NOT: .tc