[PowerPC][Future] Enable Tail Calls for PC Relative Code

[stefanp]

Summary

Tail Calls were initially disabled for PC Relative code because it was not safe to make certain assumptions about the tail calls (namely that all compiled functions no longer used the TOC pointer in R2). However, once all of the TOC pointer references have been removed it is safe to tail call everything that was tail called prior to the PC relative additions as well as a number of new cases.
For example, it is now possible to tail call indirect functions as there is no need to save and restore the TOC pointer for indirect functions if the caller is marked as may clobber R2 (st_other=1). For the same reason it is now also possible to tail call functions that are external.

[kamaub]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1549–1550

Update the comment to reflect the added condition.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

5587–5599

Please put condition inside an #ifndef NDEBUG block, there may be flags for an empty if condition when building without asserts otherwise.

[NeHuang]

Some nit comments added

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

5509

nit: would it be better to adjust the if check in this way?

if (CFlags.IsTailCall) {
  if (!(CFlags.IsIndirect && Subtarget.isUsingPCRelativeCalls())) {
    assert(((Callee.getOpcode() == ISD::Register &&
             cast<RegisterSDNode>(Callee)->getReg() == PPC::CTR) ||
            Callee.getOpcode() == ISD::TargetExternalSymbol ||
            Callee.getOpcode() == ISD::TargetGlobalAddress ||
            isa<ConstantSDNode>(Callee)) &&
           "Expecting a global address, external symbol, absolute value or "
           "register");
    assert(CallOpc == PPCISD::TC_RETURN &&
           "Unexpected call opcode for a tail call.");
  }
  DAG.getMachineFunction().getFrameInfo().setHasTailCall();
  return DAG.getNode(CallOpc, dl, MVT::Other, Ops);
}

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

88–89

nit: Can we move unsigned MIOpcode = MI->getOpcode(); down after line 97 and combined the check in line 91 with line 99?

96

nit: can we move the check before line 99 to save some check for non-pcrel mode?

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

6

good to add a general description for the test

[stefanp]

Addressed review comments.

[NeHuang]

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

7

nit: checks -> check. behaviour -> behavior

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

5580

s/for callee/for the callee

5583

s/ExtrernalSymbolSDNode/ExternalSymbolSDNode

[stefanp]

Addressed review nits.

[stefanp]

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

7

Fixed the checks but I'll leave the spelling of behaviour/behavior. :)

[nemanjai]

None of the comments on their own really require another round of review, but there are a number of them so I'd prefer to see the updates before approving them.

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

1690

Maybe some additional info in this comment along the lines of:

// The TCRETURNdi variants are direct calls. Valid targets for those are
// MO_GlobalAddress operands as well as MO_ExternalSymbol with PC-Rel
// since we can tail call external functions with PC-Rel (i.e. don't need to
// worry about different TOC pointers).

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

4796

It is interesting that we have !isa<ExternalSymbolSDNode>(Callee) when the subsequent check will immediately return false if !isFunctionGlobalAddress(Callee).
I think we should simplify this into:

// All variants of 64-bit ELF ABIs without PC-Relative addressing
// require that the caller and callee share the same TOC for
// TCO/SCO. We cannot guarantee this for indirect
// calls or calls to external functions. When PC-Relative addressing
// is used, the concept of the TOC is no longer applicable so this
// check is not required.
if (!Subtarget.isUsingPCRelativeCalls() &&
    !callsShareTOCBase(&Caller, Callee, getTargetMachine()))
  return false;

5511

With non-assert builds, this will be an empty block so we should write this differently. Also, the second assert seems perfectly valid in both cases - won't the opcode still be PPCISD::TC_RETURN even for indirect tail calls?
Perhaps we should just add another disjunction to the assert:

assert(((Callee.getOpcode() == ISD::Register &&
         cast<RegisterSDNode>(Callee)->getReg() == PPC::CTR) ||
        Callee.getOpcode() == ISD::TargetExternalSymbol ||
        Callee.getOpcode() == ISD::TargetGlobalAddress ||
        isa<ConstantSDNode>(Callee) ||
        (CFlags.IsIndirect && Subtarget.isUsingPCRelativeCalls()) &&
       "Expecting a global address, external symbol, absolute value, "
       "register or an indirect call (with PC-Rel addressing)");

5587

I think it is time to refactor this debug dump to something more meaningful. We no longer check the linkage of the callee for deciding to tail call so it seems quite meaningless to show it in the debug dump.
I think we can simplify this to something like:

LLVM_DEBUG(dbgs() << "TCO caller: " << DAG.getMachineFunction().getName()
                  << "\nTCO callee: ");
LLVM_DEBUG(Callee.dump());

And then we don't need to wrap this in an NDEBUG macro.

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

102

Please don't create conditional block that contain only asserts.
This assert can be placed before this code and just be

assert((Subtarget->isUsingPCRelativeCalls() ||
        MIOpcode != PPC::BL8_NOTOC) &&
        "BL8_NOTOC is only valid when using PC Relative Calls.");

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

3

Why are we adding -enable-ppc-quad-precision? I don't see any use of float128.

[stefanp]

Replied to comments. I'll update the patch in a couple of min.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

4796

Actually we do need both guards. When I first saw your comment I thought the same thing but then I realized (after ppc64-nonfunc-calls.ll failed) that isFunctionGlobalAddress is not the same as GlobalAddressSDNode *G = dyn_cast<GlobalAddressSDNode>(Callee); which is tested as the first thing in callsShareTOCBase.
Here is that check. This function can still return false even if Callee is a GlobalAddressSDNode.

static bool isFunctionGlobalAddress(SDValue Callee) {
  if (GlobalAddressSDNode *G = dyn_cast<GlobalAddressSDNode>(Callee)) {
    if (Callee.getOpcode() == ISD::GlobalTLSAddress ||
        Callee.getOpcode() == ISD::TargetGlobalTLSAddress)
      return false;

    return G->getGlobal()->getValueType()->isFunctionTy();
  }

  return false;
}

Take the case where we have a GlobalSDNode where the value type is not a function type. (This would be the case for indirect calls.) The isFunctionGlobalAddress would return false but dyn_cast<GlobalAddressSDNode>(Callee); would not.
I've kept both if conditions but I've flattened them so that Subtarget.isUsingPCRelativeCalls() is in both.

5587

FYI:
Yes, these dubug messages are more useful than what we had before.
However, the Callee.dump() won't always give the user useful information. For indirect calls the Callee node is sometimes a copy like this one:
C Code:

typedef int (*ptrfunc) ();
int TailCallParamFuncPtr(ptrfunc passedfunc) {
  return (*passedfunc) ();
}

Produces this debug output:

TCO caller: TailCallParamFuncPtr
TCO callee: t2: i64,ch = CopyFromReg t0, Register:i64 %0

Not super useful but much better than the nothing we would output before. :)
Also, since it's an indirect call we have no way of knowing the name of the function. This is probably the best we can do.

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

3

Because I copy-pasted these lines form another test? Sorry, I'll fix it.

[stefanp]

Addressed Review Comments.

[anil9]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

1549–1550

nit : asm so we have -> asm as we have

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

1694

nits: (i.e don't) -> (i.e we don't)
Some of the external funcions -> functions.
(like memcpy for example) -> like memcpy for example,

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

4790

nit: It seems you need to format these comments, they seem to be wrapping around even before reaching 80 chars.

[nemanjai]

Thanks for addressing my comments. The remaining nits do not at all seem like a reason to block the approval. LGTM.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

4796

Oh, I see. That's great, thanks for the explanation.

5508–5509

Nit: the message for the assert didn't change according to my comments.

5587

I think that's fairly useful. It really is a value that comes in from outside the function and we don't know anything more about it. The best we can do seems adequate to me.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

if (Subtarget->isELFv2ABI() && UsesX2OrR2 &&
     // 2) A function uses the TOC pointer R2. This function may or may not have
     //    calls. In this case st_other=[2,6] and the global and local entry
     //    points are different. Code to correctly setup the TOC pointer in R2
     //    is put between the global and local entry points. This case is
     //    covered by the if statatement above.
     // 3) A function does not use the TOC pointer R2 but does have calls.
     //    In this case st_other=1 since we do not know whether or not any
     //    of the callees clobber R2. This case is dealt with in this else if
-    //    block.
+    //    block. Tail calls are considered calls and the st_other should also
+    //    be set to 1 in that case as well.
     // 4) The function does not use the TOC pointer but R2 is used inside
     //    the function. In this case st_other=1 once again.
     // 5) This function uses inline asm. We mark R2 as reserved if the function
     //    has inline asm so we have to assume that it may be used.
-    if (MF->getFrameInfo().hasCalls() || MF->hasInlineAsm() ||
-        (!PPCFI->usesTOCBasePtr() && UsesX2OrR2)) {
+    if (MF->getFrameInfo().hasCalls() || MF->getFrameInfo().hasTailCall() ||
+        MF->hasInlineAsm() || (!PPCFI->usesTOCBasePtr() && UsesX2OrR2)) {
       PPCTargetStreamer *TS =
           static_cast<PPCTargetStreamer *>(OutStreamer->getTargetStreamer());
       if (TS)
         TS->emitLocalEntry(cast<MCSymbolELF>(CurrentFnSym),
                            MCConstantExpr::create(1, OutContext));
     }
   }
 }

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

void PPCFrameLowering::createTailCallBranchInstr(MachineBasicBlock &MBB) const {
   MachineBasicBlock::iterator MBBI = MBB.getFirstTerminator();
 
   // If we got this far a first terminator should exist.
   assert(MBBI != MBB.end() && "Failed to find the first terminator.");
 
   DebugLoc dl = MBBI->getDebugLoc();
   const PPCInstrInfo &TII = *Subtarget.getInstrInfo();
 
   // Create branch instruction for pseudo tail call return instruction
   unsigned RetOpcode = MBBI->getOpcode();
   if (RetOpcode == PPC::TCRETURNdi) {
     MBBI = MBB.getLastNonDebugInstr();
     MachineOperand &JumpTarget = MBBI->getOperand(0);
+    if (JumpTarget.isGlobal())
       BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILB)).
         addGlobalAddress(JumpTarget.getGlobal(), JumpTarget.getOffset());
+    else if (JumpTarget.isSymbol())
+      BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILB)).
+        addExternalSymbol(JumpTarget.getSymbolName());
+    else
+      llvm_unreachable("Expecting Global or External Symbol");
   } else if (RetOpcode == PPC::TCRETURNri) {
     MBBI = MBB.getLastNonDebugInstr();
     assert(MBBI->getOperand(0).isReg() && "Expecting register operand.");
     BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILBCTR));
   } else if (RetOpcode == PPC::TCRETURNai) {
     MBBI = MBB.getLastNonDebugInstr();
     MachineOperand &JumpTarget = MBBI->getOperand(0);
     BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILBA)).addImm(JumpTarget.getImm());
   } else if (RetOpcode == PPC::TCRETURNdi8) {
     MBBI = MBB.getLastNonDebugInstr();
     MachineOperand &JumpTarget = MBBI->getOperand(0);
+    if (JumpTarget.isGlobal())
       BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILB8)).
         addGlobalAddress(JumpTarget.getGlobal(), JumpTarget.getOffset());
+    else if (JumpTarget.isSymbol())
+      BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILB8)).
+        addExternalSymbol(JumpTarget.getSymbolName());
+    else
+      llvm_unreachable("Expecting Global or External Symbol");
   } else if (RetOpcode == PPC::TCRETURNri8) {
     MBBI = MBB.getLastNonDebugInstr();
     assert(MBBI->getOperand(0).isReg() && "Expecting register operand.");
     BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILBCTR8));
   } else if (RetOpcode == PPC::TCRETURNai8) {
     MBBI = MBB.getLastNonDebugInstr();
     MachineOperand &JumpTarget = MBBI->getOperand(0);
     BuildMI(MBB, MBBI, dl, TII.get(PPC::TAILBA8)).addImm(JumpTarget.getImm());

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

if (CalculateStackSlotUsed(Param.VT, Param.ArgVT, Param.Flags,
                                Subtarget.hasQPX()))
       return true;
   }
   return false;
 }
 
 static bool
 hasSameArgumentList(const Function *CallerFn, ImmutableCallSite CS) {
+  // If the call site does not have a valid instruction pointer we don't have
+  // enough information to determine if we have the same argument list.
+  // We return false just to be safe.
+  if (!CS.getInstruction())
+    return false;
+
   if (CS.arg_size() != CallerFn->arg_size())
     return false;
 
   ImmutableCallSite::arg_iterator CalleeArgIter = CS.arg_begin();
   ImmutableCallSite::arg_iterator CalleeArgEnd = CS.arg_end();
   Function::const_arg_iterator CallerArgIter = CallerFn->arg_begin();
 
   for (; CalleeArgIter != CalleeArgEnd; ++CalleeArgIter, ++CallerArgIter) {
PPCTargetLowering::IsEligibleForTailCallOptimization_64SVR4(
                                     CallingConv::ID CalleeCC,
                                     ImmutableCallSite CS,
                                     bool isVarArg,
                                     const SmallVectorImpl<ISD::OutputArg> &Outs,
                                     const SmallVectorImpl<ISD::InputArg> &Ins,
                                     SelectionDAG& DAG) const {
   bool TailCallOpt = getTargetMachine().Options.GuaranteedTailCallOpt;
 
-  // FIXME: Tail calls are currently disabled when using PC Relative addressing.
-  // The issue is that PC Relative is only partially implemented and so there
-  // is currently a mix of functions that require the TOC and functions that do
-  // not require it. If we have A calls B calls C and both A and B require the
-  // TOC and C does not and is marked as clobbering R2 then it is not safe for
-  // B to tail call C. Since we do not have the information of whether or not
-  // a funciton needs to use the TOC here in this function we need to be
-  // conservatively safe and disable all tail calls for now.
-  if (Subtarget.isUsingPCRelativeCalls()) return false;
-
   if (DisableSCO && !TailCallOpt) return false;
 
   // Variadic argument functions are not supported.
   if (isVarArg) return false;
 
   auto &Caller = DAG.getMachineFunction().getFunction();
   // Check that the calling conventions are compatible for tco.
   if (!areCallingConvEligibleForTCO_64SVR4(Caller.getCallingConv(), CalleeCC))
if (any_of(Outs, [](const ISD::OutputArg& OA) { return OA.Flags.isByVal(); }))
     return false;
 
   // If callee and caller use different calling conventions, we cannot pass
   // parameters on stack since offsets for the parameter area may be different.
   if (Caller.getCallingConv() != CalleeCC &&
       needStackSlotPassParameters(Subtarget, Outs))
     return false;
 
+  // With a full implementation of PCRelative Calls callers no longer use the
+  // TOC and so no longer need to restore the TOC pointer. In this case it
+  // does not really matter if the callee is an indirect call or if it shares
+  // the TOC base with the caller.
+  if (!Subtarget.isUsingPCRelativeCalls()) {
     // No TCO/SCO on indirect call because Caller have to restore its TOC
-  if (!isFunctionGlobalAddress(Callee) &&
-      !isa<ExternalSymbolSDNode>(Callee))
+    if (!isFunctionGlobalAddress(Callee) && !isa<ExternalSymbolSDNode>(Callee))
       return false;
-
     // If the caller and callee potentially have different TOC bases then we
     // cannot tail call since we need to restore the TOC pointer after the call.
     // ref: https://bugzilla.mozilla.org/show_bug.cgi?id=973977
     if (!callsShareTOCBase(&Caller, Callee, getTargetMachine()))
       return false;
+  }
 
   // TCO allows altering callee ABI, so we don't have to check further.
   if (CalleeCC == CallingConv::Fast && TailCallOpt)
     return true;
 
   if (DisableSCO) return false;
 
   // If callee use the same argument list that caller is using, then we can
else
     prepareIndirectCall(DAG, Callee, Glue, Chain, dl);
 
   // Build the operand list for the call instruction.
   SmallVector<SDValue, 8> Ops;
   buildCallOperands(Ops, CFlags, dl, DAG, RegsToPass, Glue, Chain, Callee,
                     SPDiff, Subtarget);
 
   // Emit tail call.
   if (CFlags.IsTailCall) {
+    // Indirect tail call when using PC Relative calls do not have the same
+    // constraints.
+    if (!CFlags.IsIndirect || !Subtarget.isUsingPCRelativeCalls()) {
       assert(((Callee.getOpcode() == ISD::Register &&
                cast<RegisterSDNode>(Callee)->getReg() == PPC::CTR) ||
               Callee.getOpcode() == ISD::TargetExternalSymbol ||
               Callee.getOpcode() == ISD::TargetGlobalAddress ||
               isa<ConstantSDNode>(Callee)) &&
              "Expecting a global address, external symbol, absolute value or "
              "register");
       assert(CallOpc == PPCISD::TC_RETURN &&
              "Unexpected call opcode for a tail call.");
+    }
     DAG.getMachineFunction().getFrameInfo().setHasTailCall();
     return DAG.getNode(CallOpc, dl, MVT::Other, Ops);
   }
 
   std::array<EVT, 2> ReturnTypes = {{MVT::Other, MVT::Glue}};
   Chain = DAG.getNode(CallOpc, dl, ReturnTypes, Ops);
   Glue = Chain.getValue(1);
 
if (isTailCall) {
     else
       isTailCall = IsEligibleForTailCallOptimization(Callee, CallConv, isVarArg,
                                                      Ins, DAG);
     if (isTailCall) {
       ++NumTailCalls;
       if (!getTargetMachine().Options.GuaranteedTailCallOpt)
         ++NumSiblingCalls;
 
-      assert(isa<GlobalAddressSDNode>(Callee) &&
+      // PC Relative calls no longer guarantee that the callee is a Global
+      // Address Node. The callee could be an indirect tail call in which
+      // case the SDValue for the callee could be a load (to load the address
+      // of a function pointer) or it may be a register copy (to move the
+      // address of the callee from a function parameter into a virtual
+      // register). It may also be an ExternalSymbolSDNode (ex memcopy).
+      assert((Subtarget.isUsingPCRelativeCalls() ||
+              isa<GlobalAddressSDNode>(Callee)) &&
              "Callee should be an llvm::Function object.");
+#ifndef NDEBUG
+      if (isa<GlobalAddressSDNode>(Callee)) {
         LLVM_DEBUG(
             const GlobalValue *GV =
                 cast<GlobalAddressSDNode>(Callee)->getGlobal();
             const unsigned Width =
                 80 - strlen("TCO caller: ") - strlen(", callee linkage: 0, 0");
             dbgs() << "TCO caller: "
                    << left_justify(DAG.getMachineFunction().getName(), Width)
                    << ", callee linkage: " << GV->getVisibility() << ", "
                    << GV->getLinkage() << "\n");
       }
+#endif // NDEBUG
+    }
   }
 
   if (!isTailCall && CS && CS.isMustTailCall())
     report_fatal_error("failed to perform tail call elimination on a call "
                        "site marked musttail");
 
   // When long calls (i.e. indirect calls) are always used, calls are always
   // made via function pointer. If we have a function name, first translate it

llvm/lib/Target/PowerPC/PPCMCInstLower.cpp

static MCOperand GetSymbolRef(const MachineOperand &MO, const MCSymbol *Symbol,
 
   if (MO.getTargetFlags() == PPCII::MO_PLT)
     RefKind = MCSymbolRefExpr::VK_PLT;
   else if (MO.getTargetFlags() == PPCII::MO_PCREL_FLAG)
     RefKind = MCSymbolRefExpr::VK_PCREL;
   else if (MO.getTargetFlags() == (PPCII::MO_PCREL_FLAG | PPCII::MO_GOT_FLAG))
     RefKind = MCSymbolRefExpr::VK_PPC_GOT_PCREL;
 
   const MachineInstr *MI = MO.getParent();
-
-  if (MI->getOpcode() == PPC::BL8_NOTOC)
-    RefKind = MCSymbolRefExpr::VK_PPC_NOTOC;
-
   const MachineFunction *MF = MI->getMF();
   const Module *M = MF->getFunction().getParent();
   const PPCSubtarget *Subtarget = &(MF->getSubtarget<PPCSubtarget>());
   const TargetMachine &TM = Printer.TM;
+
+  unsigned MIOpcode = MI->getOpcode();
+  if (Subtarget->isUsingPCRelativeCalls()) {
+    if (MIOpcode == PPC::TAILB || MIOpcode == PPC::TAILB8 ||
+        MIOpcode == PPC::TCRETURNdi || MIOpcode == PPC::TCRETURNdi8 ||
+        MIOpcode == PPC::BL8_NOTOC) {
+      RefKind = MCSymbolRefExpr::VK_PPC_NOTOC;
+    }
+  } else {
+    assert(MIOpcode != PPC::BL8_NOTOC &&
+           "BL8_NOTOC is only valid when using PC Relative Calls.");
+  }
+
   const MCExpr *Expr = MCSymbolRefExpr::create(Symbol, RefKind, Ctx);
   // If -msecure-plt -fPIC, add 32768 to symbol.
   if (Subtarget->isSecurePlt() && TM.isPositionIndependent() &&
       M->getPICLevel() == PICLevel::BigPIC &&
       MO.getTargetFlags() == PPCII::MO_PLT)
     Expr =
         MCBinaryExpr::createAdd(Expr, MCConstantExpr::create(32768, Ctx), Ctx);
 

llvm/test/CodeGen/PowerPC/pcrel-got-indirect.ll

 ; CHECK-NEXT:    blr
 entry:
   store i32 3, i32* getelementptr inbounds (%struct.Struct, %struct.Struct* @structure, i64 0, i32 2), align 4
   ret void
 }
 
 define dso_local void @ReadFuncPtr() local_unnamed_addr  {
 ; CHECK-LABEL: ReadFuncPtr:
-; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    mflr r0
-; CHECK-NEXT:    std r0, 16(r1)
-; CHECK-NEXT:    stdu r1, -32(r1)
-; CHECK-NEXT:    .cfi_def_cfa_offset 32
-; CHECK-NEXT:    .cfi_offset lr, 16
+; CHECK:         .localentry ReadFuncPtr, 1
+; CHECK-NEXT:  # %bb.0: # %entry
 ; CHECK-NEXT:    pld r3, ptrfunc@got@pcrel(0), 1
-; CHECK-NEXT:    ld r3, 0(r3)
-; CHECK-NEXT:    mtctr r3
-; CHECK-NEXT:    bctrl
-; CHECK-NEXT:    addi r1, r1, 32
-; CHECK-NEXT:    ld r0, 16(r1)
-; CHECK-NEXT:    mtlr r0
-; CHECK-NEXT:    blr
+; CHECK-NEXT:    ld r12, 0(r3)
+; CHECK-NEXT:    mtctr r12
+; CHECK-NEXT:    bctr
+; CHECK-NEXT:    #TC_RETURNr8 ctr 0
 entry:
   %0 = load void ()*, void ()** bitcast (void (...)** @ptrfunc to void ()**), align 8
   tail call void %0()
   ret void
 }
 
 define dso_local void @WriteFuncPtr() local_unnamed_addr  {
 ; CHECK-LABEL: WriteFuncPtr:

llvm/test/CodeGen/PowerPC/pcrel-tail-calls.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=future -enable-ppc-quad-precision -ppc-asm-full-reg-names \
+; RUN:   -ppc-vsr-nums-as-vr < %s | FileCheck %s
+
+; The tests check the behaviour of PC Relative tail calls. When using
+; PC Relative we are able to do more tail calling than we have done in
+; the past as we no longer need to restore the TOC pointer into R2 after
+; most calls.
+
+@Func = external local_unnamed_addr global i32 (...)*, align 8
+@FuncLocal = common dso_local local_unnamed_addr global i32 (...)* null, align 8
+
+; No calls in this function but we assign the function pointers.
+define dso_local void @AssignFuncPtr() local_unnamed_addr {
+; CHECK-LABEL: AssignFuncPtr:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    pld r3, Func@got@pcrel(0), 1
+; CHECK-NEXT:    pld r4, Function@got@pcrel(0), 1
+; CHECK-NEXT:    std r4, 0(r3)
+; CHECK-NEXT:    pstd r4, FuncLocal@PCREL(0), 1
+; CHECK-NEXT:    blr
+entry:
+  store i32 (...)* @Function, i32 (...)** @Func, align 8
+  store i32 (...)* @Function, i32 (...)** @FuncLocal, align 8
+  ret void
+}
+
+declare signext i32 @Function(...)
+
+define dso_local void @TailCallLocalFuncPtr() local_unnamed_addr {
+; CHECK-LABEL: TailCallLocalFuncPtr:
+; CHECK:         .localentry TailCallLocalFuncPtr, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    pld r12, FuncLocal@PCREL(0), 1
+; CHECK-NEXT:    mtctr r12
+; CHECK-NEXT:    bctr
+; CHECK-NEXT:    #TC_RETURNr8 ctr 0
+entry:
+  %0 = load i32 ()*, i32 ()** bitcast (i32 (...)** @FuncLocal to i32 ()**), align 8
+  %call = tail call signext i32 %0()
+  ret void
+}
+
+define dso_local void @TailCallExtrnFuncPtr() local_unnamed_addr {
+; CHECK-LABEL: TailCallExtrnFuncPtr:
+; CHECK:         .localentry TailCallExtrnFuncPtr, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    pld r3, Func@got@pcrel(0), 1
+; CHECK-NEXT:    ld r12, 0(r3)
+; CHECK-NEXT:    mtctr r12
+; CHECK-NEXT:    bctr
+; CHECK-NEXT:    #TC_RETURNr8 ctr 0
+entry:
+  %0 = load i32 ()*, i32 ()** bitcast (i32 (...)** @Func to i32 ()**), align 8
+  %call = tail call signext i32 %0()
+  ret void
+}
+
+define dso_local signext i32 @TailCallParamFuncPtr(i32 (...)* nocapture %passedfunc) local_unnamed_addr {
+; CHECK-LABEL: TailCallParamFuncPtr:
+; CHECK:         .localentry TailCallParamFuncPtr, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    mtctr r3
+; CHECK-NEXT:    mr r12, r3
+; CHECK-NEXT:    bctr
+; CHECK-NEXT:    #TC_RETURNr8 ctr 0
+entry:
+  %callee.knr.cast = bitcast i32 (...)* %passedfunc to i32 ()*
+  %call = tail call signext i32 %callee.knr.cast()
+  ret i32 %call
+}
+
+define dso_local signext i32 @NoTailIndirectCall(i32 (...)* nocapture %passedfunc, i32 signext %a) local_unnamed_addr {
+; CHECK-LABEL: NoTailIndirectCall:
+; CHECK:         .localentry NoTailIndirectCall, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    mflr r0
+; CHECK-NEXT:    .cfi_def_cfa_offset 48
+; CHECK-NEXT:    .cfi_offset lr, 16
+; CHECK-NEXT:    .cfi_offset r30, -16
+; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
+; CHECK-NEXT:    std r0, 16(r1)
+; CHECK-NEXT:    stdu r1, -48(r1)
+; CHECK-NEXT:    mtctr r3
+; CHECK-NEXT:    mr r12, r3
+; CHECK-NEXT:    mr r30, r4
+; CHECK-NEXT:    bctrl
+; CHECK-NEXT:    add r3, r3, r30
+; CHECK-NEXT:    extsw r3, r3
+; CHECK-NEXT:    addi r1, r1, 48
+; CHECK-NEXT:    ld r0, 16(r1)
+; CHECK-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; CHECK-NEXT:    mtlr r0
+; CHECK-NEXT:    blr
+entry:
+  %callee.knr.cast = bitcast i32 (...)* %passedfunc to i32 ()*
+  %call = tail call signext i32 %callee.knr.cast()
+  %add = add nsw i32 %call, %a
+  ret i32 %add
+}
+
+define dso_local signext i32 @TailCallDirect() local_unnamed_addr {
+; CHECK-LABEL: TailCallDirect:
+; CHECK:         .localentry TailCallDirect, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    b Function@notoc
+; CHECK-NEXT:    #TC_RETURNd8 Function@notoc 0
+entry:
+  %call = tail call signext i32 bitcast (i32 (...)* @Function to i32 ()*)()
+  ret i32 %call
+}
+
+define dso_local signext i32 @NoTailCallDirect(i32 signext %a) local_unnamed_addr {
+; CHECK-LABEL: NoTailCallDirect:
+; CHECK:         .localentry NoTailCallDirect, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    mflr r0
+; CHECK-NEXT:    .cfi_def_cfa_offset 48
+; CHECK-NEXT:    .cfi_offset lr, 16
+; CHECK-NEXT:    .cfi_offset r30, -16
+; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
+; CHECK-NEXT:    std r0, 16(r1)
+; CHECK-NEXT:    stdu r1, -48(r1)
+; CHECK-NEXT:    mr r30, r3
+; CHECK-NEXT:    bl Function@notoc
+; CHECK-NEXT:    add r3, r3, r30
+; CHECK-NEXT:    extsw r3, r3
+; CHECK-NEXT:    addi r1, r1, 48
+; CHECK-NEXT:    ld r0, 16(r1)
+; CHECK-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; CHECK-NEXT:    mtlr r0
+; CHECK-NEXT:    blr
+entry:
+  %call = tail call signext i32 bitcast (i32 (...)* @Function to i32 ()*)()
+  %add = add nsw i32 %call, %a
+  ret i32 %add
+}
+
+define dso_local signext i32 @TailCallDirectLocal() local_unnamed_addr {
+; CHECK-LABEL: TailCallDirectLocal:
+; CHECK:         .localentry TailCallDirectLocal, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    b LocalFunction@notoc
+; CHECK-NEXT:    #TC_RETURNd8 LocalFunction@notoc 0
+entry:
+  %call = tail call fastcc signext i32 @LocalFunction()
+  ret i32 %call
+}
+
+define dso_local signext i32 @NoTailCallDirectLocal(i32 signext %a) local_unnamed_addr {
+; CHECK-LABEL: NoTailCallDirectLocal:
+; CHECK:         .localentry NoTailCallDirectLocal, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    mflr r0
+; CHECK-NEXT:    .cfi_def_cfa_offset 48
+; CHECK-NEXT:    .cfi_offset lr, 16
+; CHECK-NEXT:    .cfi_offset r30, -16
+; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
+; CHECK-NEXT:    std r0, 16(r1)
+; CHECK-NEXT:    stdu r1, -48(r1)
+; CHECK-NEXT:    mr r30, r3
+; CHECK-NEXT:    bl LocalFunction@notoc
+; CHECK-NEXT:    add r3, r3, r30
+; CHECK-NEXT:    extsw r3, r3
+; CHECK-NEXT:    addi r1, r1, 48
+; CHECK-NEXT:    ld r0, 16(r1)
+; CHECK-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; CHECK-NEXT:    mtlr r0
+; CHECK-NEXT:    blr
+entry:
+  %call = tail call fastcc signext i32 @LocalFunction()
+  %add = add nsw i32 %call, %a
+  ret i32 %add
+}
+
+define dso_local signext i32 @TailCallAbs() local_unnamed_addr {
+; CHECK-LABEL: TailCallAbs:
+; CHECK:         .localentry TailCallAbs, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    li r3, 400
+; CHECK-NEXT:    mtctr r3
+; CHECK-NEXT:    li r12, 400
+; CHECK-NEXT:    bctr
+; CHECK-NEXT:    #TC_RETURNr8 ctr 0
+entry:
+  %call = tail call signext i32 inttoptr (i64 400 to i32 ()*)()
+  ret i32 %call
+}
+
+define dso_local signext i32 @NoTailCallAbs(i32 signext %a) local_unnamed_addr {
+; CHECK-LABEL: NoTailCallAbs:
+; CHECK:         .localentry NoTailCallAbs, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    mflr r0
+; CHECK-NEXT:    .cfi_def_cfa_offset 48
+; CHECK-NEXT:    .cfi_offset lr, 16
+; CHECK-NEXT:    .cfi_offset r30, -16
+; CHECK-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
+; CHECK-NEXT:    std r0, 16(r1)
+; CHECK-NEXT:    stdu r1, -48(r1)
+; CHECK-NEXT:    mr r30, r3
+; CHECK-NEXT:    li r3, 400
+; CHECK-NEXT:    mtctr r3
+; CHECK-NEXT:    li r12, 400
+; CHECK-NEXT:    bctrl
+; CHECK-NEXT:    add r3, r3, r30
+; CHECK-NEXT:    extsw r3, r3
+; CHECK-NEXT:    addi r1, r1, 48
+; CHECK-NEXT:    ld r0, 16(r1)
+; CHECK-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; CHECK-NEXT:    mtlr r0
+; CHECK-NEXT:    blr
+entry:
+  %call = tail call signext i32 inttoptr (i64 400 to i32 ()*)()
+  %add = add nsw i32 %call, %a
+  ret i32 %add
+}
+
+; Function Attrs: noinline
+; This function should be tail called and not inlined.
+define internal fastcc signext i32 @LocalFunction() unnamed_addr #0 {
+; CHECK-LABEL: LocalFunction:
+; CHECK:         .localentry LocalFunction, 1
+; CHECK-NEXT:  # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    li r3, 42
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    extsw r3, r3
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 asm "li $0, 42", "=&r"()
+  ret i32 %0
+}
+
+attributes #0 = { noinline }
+