[ppc64] Enable sibling call optimization on ppc64 ELFv1/ELFv2 abi

[cycheng]

Summary

This patch enable sibling call optimization on ppc64 ELFv1/ELFv2 abi, and add a couple of test cases.
This patch also passed llvm/clang bootstrap test, and spec2006 build/run/result validation.

Original issue: https://llvm.org/bugs/show_bug.cgi?id=25617

Great thanks to Tom's help, he contributed a lot to this patch.

[hfinkel]

lib/Target/PowerPC/PPCISelLowering.cpp

3858

Function names should start with a lower-case letter.

3898

Function names should start with a lower-case letter.

3929

Indent more to line up with other args.

3963

Use std::any_of (with a small lambda).

3986

Are you planning on doing this in follow-up? It seems simple enough to check. Unless there's some complication here, I'd rather you do this up-front so that we get more testing coverage on all of the rest of the logic.

4017

Variables start with a capital letter.

4020

Variables start with a capital letter.

4020

I don't think that the first live-in register in the list is guaranteed to be the first argument's register. Also, you say "vreg id"; are these virtual registers or physical registers at this point?

[cycheng]

lib/Target/PowerPC/PPCISelLowering.cpp

3986

Then I can generate second patch quickly ; P

OK, I am going to fix it in this patch.

4020

Mmmm.. I will try to find other method for getting first argument.

They are virtual registers.

[cycheng]

Fix all issues mentioned in http://reviews.llvm.org/D16315#342919

• [OK] Use std::any_of
• [OK] Source Format
• [OK] Implement Caller/Callee Argument Checking, If Callee and Caller have the same arguments, we can apply SCO on it.
• [OK] Delete struct return checking code because llvm has guaranteed it.

This patch passed llvm bootstrap test, we also used the bootstrap result to build/run/validate SPEC2006 INT benchmark, and it passed SPEC2006 INT test. The llvm version we use is "r251575" (Oct 28 2015)

But we found the latest revision of llvm (r257864) with our patch can crash in bootstrap test stage 3, we are investigating it now.

About struct return: Originally, we worried if Callee and Caller can use different struct return pointer. But llvm has check it for us (SelectionDAGBuilder::LowerCallTo), so we delete the code.

[cycheng]

Undo unnecessary changes.

We found a bug when we enable shrink-wrapping + SCO, and do llvm bootstrap testing.
After we dig into the bug, we think it's NOT SCO's issue, we have tested disable shrink-wrapping, this patch can pass bootstrap testing, spec2006 testing.

The bug is, in some cases, shrink-wrapping + SCO will cause the tail call branch disappear, we don't know why and we are investigating it.

[cycheng]

Add new test case for testing combination of sibling call optimizing and shrink wrapping.

NOTE! This patch is dependent on http://reviews.llvm.org/D17606

[kbarton]

lib/Target/PowerPC/PPCISelLowering.cpp

3916

Variable names should be camel case (i.e., remove the underscores).
http://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

3936

I'd recommend const Function *CallerFn, instead of const Function* CallerFn.
The * modifies the parameter, not the type, so it makes sense to put it closer to the parameter.

3973

The GuaranteedTailCallOpt concerns me. Is this used to guarantee functional correctness in some cases?
There are several checks below that will return false, even if TallCallOpt is true. Are we guaranteed this is always going to be correct?

If it is not for correctness, then why can it not be disabled? It overrides the DisableSCO flag on line 3971, but then is ignored later on line 4008.

4675

Please add a message here indicating why this assert fails.

test/CodeGen/PowerPC/ppc64-sibcall-shrinkwrap.ll

3

Please add run steps for powerpc64le-unknown-linux-gnu as well.

test/CodeGen/PowerPC/ppc64-sibcall.ll

3

Please add run steps for powerpc64le-unknown-linux-gnu as well.

[cycheng]

Fixed issues mentioned by Kit

lib/Target/PowerPC/PPCISelLowering.cpp

3916

Thanks!

3936

Agree!

3973

"GuaranteedTailCallOpt" is original code path.

SibCallOpt and TailCallOpt share the following condition checking:

• isVarArg
• Same Calling Convention
• by val parameters checking
• indirect call checking
• resideInSameModule checking

Specific for TailCallOpt checking:

• Calling convention should be "CallingConv::Fast"

Specific for SibCallOpt checking:

• hasSameArgumentList
• needStackSlotPassParameters

So even if TallCallOpt is true, there are still some cases that can not be TailCallOptimized.
And if TallCallOpt is true, we don't need to check SibCallOpt condition.

4675

assert(isa<GlobalAddressSDNode>(Callee) &&
       "Callee should be an llvm::Function object.");

[kbarton]

LGTM

[cycheng]

Committed r265506

lib/Target/PowerPC/PPCISelLowering.h

private:
 
     bool
     IsEligibleForTailCallOptimization(SDValue Callee,
                                       CallingConv::ID CalleeCC,
                                       bool isVarArg,
                                       const SmallVectorImpl<ISD::InputArg> &Ins,
                                       SelectionDAG& DAG) const;
 
+    bool
+    IsEligibleForTailCallOptimization_64SVR4(
+                                    SDValue Callee,
+                                    CallingConv::ID CalleeCC,
+                                    ImmutableCallSite *CS,
+                                    bool isVarArg,
+                                    const SmallVectorImpl<ISD::OutputArg> &Outs,
+                                    const SmallVectorImpl<ISD::InputArg> &Ins,
+                                    SelectionDAG& DAG) const;
+
     SDValue EmitTailCallLoadFPAndRetAddr(SelectionDAG & DAG,
                                          int SPDiff,
                                          SDValue Chain,
                                          SDValue &LROpOut,
                                          SDValue &FPOpOut,
                                          bool isDarwinABI,
                                          SDLoc dl) const;
 

lib/Target/PowerPC/PPCISelLowering.cpp

 #include "PPCISelLowering.h"
 #include "MCTargetDesc/PPCPredicates.h"
 #include "PPCCallingConv.h"
 #include "PPCMachineFunctionInfo.h"
 #include "PPCPerfectShuffle.h"
 #include "PPCTargetMachine.h"
 #include "PPCTargetObjectFile.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/CodeGen/CallingConvLower.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineLoopInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/SelectionDAG.h"
 #include "llvm/CodeGen/TargetLoweringObjectFileImpl.h"
 #include "llvm/IR/CallingConv.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/Format.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetOptions.h"
 
 using namespace llvm;
 
+#define DEBUG_TYPE "ppc-lowering"
+
 static cl::opt<bool> DisablePPCPreinc("disable-ppc-preinc",
 cl::desc("disable preincrement load/store generation on PPC"), cl::Hidden);
 
 static cl::opt<bool> DisableILPPref("disable-ppc-ilp-pref",
 cl::desc("disable setting the node scheduling preference to ILP on PPC"), cl::Hidden);
 
 static cl::opt<bool> DisablePPCUnaligned("disable-ppc-unaligned",
 cl::desc("disable unaligned load/store generation on PPC"), cl::Hidden);
 
+static cl::opt<bool> DisableSCO("disable-ppc-sco",
+cl::desc("disable sibling call optimization on ppc"), cl::Hidden);
+
+STATISTIC(NumTailCalls, "Number of tail calls");
+STATISTIC(NumSiblingCalls, "Number of sibling calls");
+
 // FIXME: Remove this once the bug has been fixed!
 extern cl::opt<bool> ANDIGlueBug;
 
 PPCTargetLowering::PPCTargetLowering(const PPCTargetMachine &TM,
                                      const PPCSubtarget &STI)
     : TargetLowering(TM), Subtarget(STI) {
   // Use _setjmp/_longjmp instead of setjmp/longjmp.
   setUseUnderscoreSetJmp(true);
static int CalculateTailCallSPDiff(SelectionDAG& DAG, bool isTailCall,
   int SPDiff = (int)CallerMinReservedArea - (int)ParamSize;
   // Remember only if the new adjustement is bigger.
   if (SPDiff < FI->getTailCallSPDelta())
     FI->setTailCallSPDelta(SPDiff);
 
   return SPDiff;
 }
 
+static bool isFunctionGlobalAddress(SDValue Callee);
+
+static bool
+resideInSameModule(SDValue Callee, Reloc::Model RelMod) {
+  // If !G, Callee can be an external symbol.
+  GlobalAddressSDNode *G = dyn_cast<GlobalAddressSDNode>(Callee);
+  if (!G) return false;
+
+  const GlobalValue *GV = G->getGlobal();
+
+  if (GV->isDeclaration()) return false;
+
+  switch(GV->getLinkage()) {
+  default: llvm_unreachable("unknow linkage type");
+  case GlobalValue::AvailableExternallyLinkage:
+  case GlobalValue::ExternalWeakLinkage:
+    return false;
+
+  // Callee with weak linkage is allowed if it has hidden or protected
+  // visibility
+  case GlobalValue::LinkOnceAnyLinkage:
+  case GlobalValue::LinkOnceODRLinkage: // e.g. c++ inline functions
+  case GlobalValue::WeakAnyLinkage:
+  case GlobalValue::WeakODRLinkage:     // e.g. c++ template instantiation
+    if (GV->hasDefaultVisibility())
+      return false;
+
+  case GlobalValue::ExternalLinkage:
+  case GlobalValue::InternalLinkage:
+  case GlobalValue::PrivateLinkage:
+    break;
+  }
+
+  // With '-fPIC', calling default visiblity function need insert 'nop' after
+  // function call, no matter that function resides in same module or not, so
+  // we treat it as in different module.
+  if (RelMod == Reloc::PIC_ && GV->hasDefaultVisibility())
+    return false;
+
+  return true;
+}
+
+static bool
+needStackSlotPassParameters(const PPCSubtarget &Subtarget,
+                            const SmallVectorImpl<ISD::OutputArg> &Outs) {
+  assert(Subtarget.isSVR4ABI() && Subtarget.isPPC64());
+
+  const unsigned PtrByteSize = 8;
+  const unsigned LinkageSize = Subtarget.getFrameLowering()->getLinkageSize();
+
+  static const MCPhysReg GPR[] = {
+    PPC::X3, PPC::X4, PPC::X5, PPC::X6,
+    PPC::X7, PPC::X8, PPC::X9, PPC::X10,
+  };
+  static const MCPhysReg VR[] = {
+    PPC::V2, PPC::V3, PPC::V4, PPC::V5, PPC::V6, PPC::V7, PPC::V8,
+    PPC::V9, PPC::V10, PPC::V11, PPC::V12, PPC::V13
+  };
+
+  const unsigned NumGPRs = array_lengthof(GPR);
+  const unsigned NumFPRs = 13;
+  const unsigned NumVRs = array_lengthof(VR);
+  const unsigned ParamAreaSize = NumGPRs * PtrByteSize;
+
+  unsigned NumBytes = LinkageSize;
+  unsigned AvailableFPRs = NumFPRs;
+  unsigned AvailableVRs = NumVRs;
+
+  for (const ISD::OutputArg& Param : Outs) {
+    if (Param.Flags.isNest()) continue;
+
+    if (CalculateStackSlotUsed(Param.VT, Param.ArgVT, Param.Flags,
+                               PtrByteSize, LinkageSize, ParamAreaSize,
+                               NumBytes, AvailableFPRs, AvailableVRs,
+                               Subtarget.hasQPX()))
+      return true;
+  }
+  return false;
+}
+
+static bool
+hasSameArgumentList(const Function *CallerFn, ImmutableCallSite *CS) {
+  if (CS->arg_size() != CallerFn->getArgumentList().size())
+    return false;
+
+  ImmutableCallSite::arg_iterator CalleeArgIter = CS->arg_begin();
+  ImmutableCallSite::arg_iterator CalleeArgEnd = CS->arg_end();
+  Function::const_arg_iterator CallerArgIter = CallerFn->arg_begin();
+
+  for (; CalleeArgIter != CalleeArgEnd; ++CalleeArgIter, ++CallerArgIter) {
+    const Value* CalleeArg = *CalleeArgIter;
+    const Value* CallerArg = &(*CallerArgIter);
+    if (CalleeArg == CallerArg)
+      continue;
+
+    // e.g. @caller([4 x i64] %a, [4 x i64] %b) {
+    //        tail call @callee([4 x i64] undef, [4 x i64] %b)
+    //      }
+    // 1st argument of callee is undef and has the same type as caller.
+    if (CalleeArg->getType() == CallerArg->getType() &&
+        isa<UndefValue>(CalleeArg))
+      continue;
+
+    return false;
+  }
+
+  return true;
+}
+
+bool
+PPCTargetLowering::IsEligibleForTailCallOptimization_64SVR4(
+                                    SDValue Callee,
+                                    CallingConv::ID CalleeCC,
+                                    ImmutableCallSite *CS,
+                                    bool isVarArg,
+                                    const SmallVectorImpl<ISD::OutputArg> &Outs,
+                                    const SmallVectorImpl<ISD::InputArg> &Ins,
+                                    SelectionDAG& DAG) const {
+  bool TailCallOpt = getTargetMachine().Options.GuaranteedTailCallOpt;
+
+  if (DisableSCO && !TailCallOpt) return false;
+
+  // Variadic argument functions are not supported.
+  if (isVarArg) return false;
+
+  MachineFunction &MF = DAG.getMachineFunction();
+  CallingConv::ID CallerCC = MF.getFunction()->getCallingConv();
+
+  // Tail or Sibling call optimization (TCO/SCO) needs callee and caller has
+  // the same calling convention
+  if (CallerCC != CalleeCC) return false;
+
+  // SCO support C calling convention
+  if (CalleeCC != CallingConv::Fast && CalleeCC != CallingConv::C)
+    return false;
+
+  // Functions containing by val parameters are not supported.
+  if (std::any_of(Ins.begin(), Ins.end(),
+                  [](const ISD::InputArg& IA) { return IA.Flags.isByVal(); }))
+    return false;
+
+  // No TCO/SCO on indirect call because Caller have to restore its TOC
+  if (!isFunctionGlobalAddress(Callee) &&
+      !isa<ExternalSymbolSDNode>(Callee))
+    return false;
+
+  // Check if Callee resides in the same module, because for now, PPC64 SVR4 ABI
+  // (ELFv1/ELFv2) doesn't allow tail calls to a symbol resides in another
+  // module.
+  // ref: https://bugzilla.mozilla.org/show_bug.cgi?id=973977
+  if (!resideInSameModule(Callee, getTargetMachine().getRelocationModel()))
+    return false;
+
+  // TCO allows altering callee ABI, so we don't have to check further.
+  if (CalleeCC == CallingConv::Fast && TailCallOpt)
+    return true;
+
+  if (DisableSCO) return false;
+
+  // If callee use the same argument list that caller is using, then we can
+  // apply SCO on this case. If it is not, then we need to check if callee needs
+  // stack for passing arguments.
+  if (!hasSameArgumentList(MF.getFunction(), CS) &&
+      needStackSlotPassParameters(Subtarget, Outs)) {
+    return false;
+  }
+
+  return true;
+}
+
 /// IsEligibleForTailCallOptimization - Check whether the call is eligible
 /// for tail call optimization. Targets which want to do tail call
 /// optimization should implement this function.
 bool
 PPCTargetLowering::IsEligibleForTailCallOptimization(SDValue Callee,
                                                      CallingConv::ID CalleeCC,
                                                      bool isVarArg,
                                       const SmallVectorImpl<ISD::InputArg> &Ins,
PPCTargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
   SDValue Chain                         = CLI.Chain;
   SDValue Callee                        = CLI.Callee;
   bool &isTailCall                      = CLI.IsTailCall;
   CallingConv::ID CallConv              = CLI.CallConv;
   bool isVarArg                         = CLI.IsVarArg;
   bool IsPatchPoint                     = CLI.IsPatchPoint;
   ImmutableCallSite *CS                 = CLI.CS;
 
-  if (isTailCall)
+  if (isTailCall) {
+    if (Subtarget.isSVR4ABI() && Subtarget.isPPC64())
+      isTailCall =
+        IsEligibleForTailCallOptimization_64SVR4(Callee, CallConv, CS,
+                                                 isVarArg, Outs, Ins, DAG);
+    else
       isTailCall = IsEligibleForTailCallOptimization(Callee, CallConv, isVarArg,
                                                      Ins, DAG);
+    if (isTailCall) {
+      ++NumTailCalls;
+      if (!getTargetMachine().Options.GuaranteedTailCallOpt)
+        ++NumSiblingCalls;
+
+      assert(isa<GlobalAddressSDNode>(Callee) &&
+             "Callee should be an llvm::Function object.");
+      DEBUG(
+        const GlobalValue *GV = cast<GlobalAddressSDNode>(Callee)->getGlobal();
+        const unsigned Width = 80 - strlen("TCO caller: ")
+                                  - strlen(", callee linkage: 0, 0");
+        dbgs() << "TCO caller: "
+               << left_justify(DAG.getMachineFunction().getName(), Width)
+               << ", callee linkage: "
+               << GV->getVisibility() << ", " << GV->getLinkage() << "\n"
+      );
+    }
+  }
 
   if (!isTailCall && CS && CS->isMustTailCall())
     report_fatal_error("failed to perform tail call elimination on a call "
                        "site marked musttail");
 
   if (Subtarget.isSVR4ABI()) {
     if (Subtarget.isPPC64())
       return LowerCall_64SVR4(Chain, Callee, CallConv, isVarArg,
PPCTargetLowering::LowerCall_64SVR4(SDValue Chain, SDValue Callee,
                                     SDLoc dl, SelectionDAG &DAG,
                                     SmallVectorImpl<SDValue> &InVals,
                                     ImmutableCallSite *CS) const {
 
   bool isELFv2ABI = Subtarget.isELFv2ABI();
   bool isLittleEndian = Subtarget.isLittleEndian();
   unsigned NumOps = Outs.size();
   bool hasNest = false;
+  bool IsSibCall = false;
 
   EVT PtrVT = DAG.getTargetLoweringInfo().getPointerTy(DAG.getDataLayout());
   unsigned PtrByteSize = 8;
 
   MachineFunction &MF = DAG.getMachineFunction();
 
+  if (isTailCall && !getTargetMachine().Options.GuaranteedTailCallOpt)
+    IsSibCall = true;
+
   // Mark this function as potentially containing a function that contains a
   // tail call. As a consequence the frame pointer will be used for dynamicalloc
   // and restoring the callers stack pointer in this functions epilog. This is
   // done because by tail calling the called function might overwrite the value
   // in this function's (MF) stack pointer stack slot 0(SP).
   if (getTargetMachine().Options.GuaranteedTailCallOpt &&
       CallConv == CallingConv::Fast)
     MF.getInfo<PPCFunctionInfo>()->setHasFastCall();
PPCTargetLowering::LowerCall_64SVR4(SDValue Chain, SDValue Callee,
   // FIXME: On ELFv2, it may be unnecessary to allocate the parameter area.
   NumBytes = std::max(NumBytes, LinkageSize + 8 * PtrByteSize);
 
   // Tail call needs the stack to be aligned.
   if (getTargetMachine().Options.GuaranteedTailCallOpt &&
       CallConv == CallingConv::Fast)
     NumBytes = EnsureStackAlignment(Subtarget.getFrameLowering(), NumBytes);
 
+  int SPDiff = 0;
+
   // Calculate by how many bytes the stack has to be adjusted in case of tail
   // call optimization.
-  int SPDiff = CalculateTailCallSPDiff(DAG, isTailCall, NumBytes);
+  if (!IsSibCall)
+    SPDiff = CalculateTailCallSPDiff(DAG, isTailCall, NumBytes);
 
   // To protect arguments on the stack from being clobbered in a tail call,
   // force all the loads to happen before doing any other lowering.
   if (isTailCall)
     Chain = DAG.getStackArgumentTokenFactor(Chain);
 
   // Adjust the stack pointer for the new arguments...
   // These operations are automatically eliminated by the prolog/epilog pass
-  Chain = DAG.getCALLSEQ_START(Chain, DAG.getIntPtrConstant(NumBytes, dl, true),
-                               dl);
+  if (!IsSibCall)
+    Chain = DAG.getCALLSEQ_START(Chain,
+                                 DAG.getIntPtrConstant(NumBytes, dl, true), dl);
   SDValue CallSeqStart = Chain;
 
   // Load the return address and frame pointer so it can be move somewhere else
   // later.
   SDValue LROp, FPOp;
   Chain = EmitTailCallLoadFPAndRetAddr(DAG, SPDiff, Chain, LROp, FPOp, true,
                                        dl);
 
PPCTargetLowering::LowerCall_64SVR4(SDValue Chain, SDValue Callee,
   // and flag operands which copy the outgoing args into the appropriate regs.
   SDValue InFlag;
   for (unsigned i = 0, e = RegsToPass.size(); i != e; ++i) {
     Chain = DAG.getCopyToReg(Chain, dl, RegsToPass[i].first,
                              RegsToPass[i].second, InFlag);
     InFlag = Chain.getValue(1);
   }
 
-  if (isTailCall)
+  if (isTailCall && !IsSibCall)
     PrepareTailCall(DAG, InFlag, Chain, dl, true, SPDiff, NumBytes, LROp,
                     FPOp, true, TailCallArguments);
 
   return FinishCall(CallConv, dl, isTailCall, isVarArg, IsPatchPoint, hasNest,
                     DAG, RegsToPass, InFlag, Chain, CallSeqStart, Callee,
                     SPDiff, NumBytes, Ins, InVals, CS);
 }
 

test/CodeGen/PowerPC/ppc64-calls.ll

 ; RUN: llc < %s -march=ppc64 -mcpu=pwr7 | FileCheck %s
 target datalayout = "E-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v128:128:128-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 define void @foo() nounwind readnone noinline {
   ret void
 }
 
 define weak void @foo_weak() nounwind {
   ret void
 }
 
 ; Calls to local function does not require the TOC restore 'nop'
 define void @test_direct() nounwind readnone {
 ; CHECK-LABEL: test_direct:
   tail call void @foo() nounwind
-; CHECK: bl foo
+; Because of tail call optimization, it can be 'b' instruction.
+; CHECK: [[BR:b[l]?]] foo
 ; CHECK-NOT: nop
   ret void
 }
 
 ; Calls to weak function requires a TOC restore 'nop' because they
 ; may be overridden in a different module.
 define void @test_weak() nounwind readnone {
 ; CHECK-LABEL: test_weak:

test/CodeGen/PowerPC/ppc64-sibcall-shrinkwrap.ll

+; RUN: llc < %s -mtriple=powerpc64-unknown-linux-gnu --enable-shrink-wrap=false | FileCheck %s -check-prefix=CHECK-SCO-ONLY
+; RUN: llc < %s -mtriple=powerpc64-unknown-linux-gnu --enable-shrink-wrap=true | FileCheck %s -check-prefix=CHECK-SCO-SHRK
+; RUN: llc < %s -mtriple=powerpc64le-unknown-linux-gnu --enable-shrink-wrap=false | FileCheck %s -check-prefix=CHECK-SCO-ONLY
+; RUN: llc < %s -mtriple=powerpc64le-unknown-linux-gnu --enable-shrink-wrap=true | FileCheck %s -check-prefix=CHECK-SCO-SHRK
+
+%"class.clang::NamedDecl" = type { i32 }
+declare void @__assert_fail();
+
+define i8 @_ZNK5clang9NamedDecl23getLinkageAndVisibilityEv(
+    %"class.clang::NamedDecl"* %this) {
+entry:
+  %tobool = icmp eq %"class.clang::NamedDecl"* %this, null
+  br i1 %tobool, label %cond.false, label %exit
+
+cond.false:
+  tail call void @__assert_fail()
+  unreachable
+
+exit:
+  %DeclKind = getelementptr inbounds
+                            %"class.clang::NamedDecl",
+                            %"class.clang::NamedDecl"* %this, i64 0, i32 0
+  %bf.load = load i32, i32* %DeclKind, align 4
+  %call.i = tail call i8 @LVComputationKind(
+    %"class.clang::NamedDecl"* %this,
+    i32 %bf.load)
+  ret i8 %call.i
+
+; CHECK-SCO-SHRK-LABEL: _ZNK5clang9NamedDecl23getLinkageAndVisibilityEv:
+; CHECK-SCO-SHRK: b LVComputationKind
+; CHECK-SCO-SHRK: #TC_RETURNd8
+; CHECK-SCO-SHRK: stdu 1, -{{[0-9]+}}(1)
+; CHECK-SCO-SHRK: bl __assert_fail
+;
+; CHECK-SCO-ONLY-LABEL: _ZNK5clang9NamedDecl23getLinkageAndVisibilityEv:
+; CHECK-SCO-ONLY: stdu 1, -{{[0-9]+}}(1)
+; CHECK-SCO-ONLY: b LVComputationKind
+; CHECK-SCO-ONLY: #TC_RETURNd8
+; CHECK-SCO-ONLY: bl __assert_fail
+}
+
+define fastcc i8 @LVComputationKind(
+    %"class.clang::NamedDecl"* %D,
+    i32 %computation) {
+  ret i8 0
+}

test/CodeGen/PowerPC/ppc64-sibcall.ll

+; RUN: llc < %s -O1 -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu | FileCheck %s -check-prefix=CHECK-SCO
+; RUN: llc < %s -O1 -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO-HASQPX
+; RUN: llc < %s -O1 -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO-HASQPX
+
+; No combination of "powerpc64le-unknown-linux-gnu" + "CHECK-SCO", because
+; only Power8 (and later) fully support LE.
+
+%S_56 = type { [13 x i32], i32 }
+%S_64 = type { [15 x i32], i32 }
+%S_32 = type { [7 x i32], i32 }
+
+; Function Attrs: noinline nounwind
+define void @callee_56_copy([7 x i64] %a, %S_56* %b) #0 { ret void }
+define void @callee_64_copy([8 x i64] %a, %S_64* %b) #0 { ret void }
+
+; Function Attrs: nounwind
+define void @caller_56_reorder_copy(%S_56* %b, [7 x i64] %a) #1 {
+  tail call void @callee_56_copy([7 x i64] %a, %S_56* %b)
+  ret void
+
+; CHECK-SCO-LABEL: caller_56_reorder_copy:
+; CHECK-SCO-NOT: stdu 1
+; CHECK-SCO: TC_RETURNd8 callee_56_copy
+}
+
+define void @caller_64_reorder_copy(%S_64* %b, [8 x i64] %a) #1 {
+  tail call void @callee_64_copy([8 x i64] %a, %S_64* %b)
+  ret void
+
+; CHECK-SCO-LABEL: caller_64_reorder_copy:
+; CHECK-SCO: bl callee_64_copy
+}
+
+define void @callee_64_64_copy([8 x i64] %a, [8 x i64] %b) #0 { ret void }
+define void @caller_64_64_copy([8 x i64] %a, [8 x i64] %b) #1 {
+  tail call void @callee_64_64_copy([8 x i64] %a, [8 x i64] %b)
+  ret void
+
+; CHECK-SCO-LABEL: caller_64_64_copy:
+; CHECK-SCO: b callee_64_64_copy
+}
+
+define void @caller_64_64_reorder_copy([8 x i64] %a, [8 x i64] %b) #1 {
+  tail call void @callee_64_64_copy([8 x i64] %b, [8 x i64] %a)
+  ret void
+
+; CHECK-SCO-LABEL: caller_64_64_reorder_copy:
+; CHECK-SCO: bl callee_64_64_copy
+}
+
+define void @caller_64_64_undef_copy([8 x i64] %a, [8 x i64] %b) #1 {
+  tail call void @callee_64_64_copy([8 x i64] %a, [8 x i64] undef)
+  ret void
+
+; CHECK-SCO-LABEL: caller_64_64_undef_copy:
+; CHECK-SCO: b callee_64_64_copy
+}
+
+define void @arg8_callee(
+  float %a, i32 signext %b, float %c, i32* %d,
+  i8 zeroext %e, float %f, i32* %g, i32 signext %h)
+{
+  ret void
+}
+
+define void @arg8_caller(float %a, i32 signext %b, i8 zeroext %c, i32* %d) {
+entry:
+  tail call void @arg8_callee(float undef, i32 signext undef, float undef,
+                              i32* %d, i8 zeroext undef, float undef,
+                              i32* undef, i32 signext undef)
+  ret void
+
+; CHECK-SCO-LABEL: arg8_caller:
+; CHECK-SCO: b arg8_callee
+}
+
+; Struct return test
+
+; Function Attrs: noinline nounwind
+define void @callee_sret_56(%S_56* noalias sret %agg.result) #0 { ret void }
+define void @callee_sret_32(%S_32* noalias sret %agg.result) #0 { ret void }
+
+; Function Attrs: nounwind
+define void @caller_do_something_sret_32(%S_32* noalias sret %agg.result) #1 {
+  %1 = alloca %S_56, align 4
+  %2 = bitcast %S_56* %1 to i8*
+  call void @callee_sret_56(%S_56* nonnull sret %1)
+  tail call void @callee_sret_32(%S_32* sret %agg.result)
+  ret void
+
+; CHECK-SCO-LABEL: caller_do_something_sret_32:
+; CHECK-SCO: stdu 1
+; CHECK-SCO: bl callee_sret_56
+; CHECK-SCO: addi 1
+; CHECK-SCO: TC_RETURNd8 callee_sret_32
+}
+
+define void @caller_local_sret_32(%S_32* %a) #1 {
+  %tmp = alloca %S_32, align 4
+  tail call void @callee_sret_32(%S_32* nonnull sret %tmp)
+  ret void
+
+; CHECK-SCO-LABEL: caller_local_sret_32:
+; CHECK-SCO: bl callee_sret_32
+}
+
+attributes #0 = { noinline nounwind  }
+attributes #1 = { nounwind }
+
+; vector <4 x i1> test
+
+define void @callee_v4i1(i8 %a, <4 x i1> %b, <4 x i1> %c) { ret void }
+define void @caller_v4i1_reorder(i8 %a, <4 x i1> %b, <4 x i1> %c) {
+  tail call void @callee_v4i1(i8 %a, <4 x i1> %c, <4 x i1> %b)
+  ret void
+
+; <4 x i1> is 32 bytes aligned, if subtarget doesn't support qpx, then we can't
+; place b, c to qpx register, so we can't do sco on caller_v4i1_reorder
+
+; CHECK-SCO-LABEL: caller_v4i1_reorder:
+; CHECK-SCO: bl callee_v4i1
+
+; CHECK-SCO-HASQPX-LABEL: caller_v4i1_reorder:
+; CHECK-SCO-HASQPX: b callee_v4i1
+}
+
+define void @f128_callee(i32* %ptr, ppc_fp128 %a, ppc_fp128 %b) { ret void }
+define void @f128_caller(i32* %ptr, ppc_fp128 %a, ppc_fp128 %b) {
+  tail call void @f128_callee(i32* %ptr, ppc_fp128 %a, ppc_fp128 %b)
+  ret void
+
+; CHECK-SCO-LABEL: f128_caller:
+; CHECK-SCO: b f128_callee
+}
+
+; weak linkage test
+%class.T = type { [2 x i8] }
+
+define weak_odr hidden void @wo_hcallee(%class.T* %this, i8* %c) { ret void }
+define void @wo_hcaller(%class.T* %this, i8* %c) {
+  tail call void @wo_hcallee(%class.T* %this, i8* %c)
+  ret void
+
+; CHECK-SCO-LABEL: wo_hcaller:
+; CHECK-SCO: b wo_hcallee
+}
+
+define weak_odr protected void @wo_pcallee(%class.T* %this, i8* %c) { ret void }
+define void @wo_pcaller(%class.T* %this, i8* %c) {
+  tail call void @wo_pcallee(%class.T* %this, i8* %c)
+  ret void
+
+; CHECK-SCO-LABEL: wo_pcaller:
+; CHECK-SCO: b wo_pcallee
+}
+
+define weak_odr void @wo_callee(%class.T* %this, i8* %c) { ret void }
+define void @wo_caller(%class.T* %this, i8* %c) {
+  tail call void @wo_callee(%class.T* %this, i8* %c)
+  ret void
+
+; CHECK-SCO-LABEL: wo_caller:
+; CHECK-SCO: bl wo_callee
+}
+
+define weak protected void @w_pcallee(i8* %ptr) { ret void }
+define void @w_pcaller(i8* %ptr) {
+  tail call void @w_pcallee(i8* %ptr)
+  ret void
+
+; CHECK-SCO-LABEL: w_pcaller:
+; CHECK-SCO: b w_pcallee
+}
+
+define weak hidden void @w_hcallee(i8* %ptr) { ret void }
+define void @w_hcaller(i8* %ptr) {
+  tail call void @w_hcallee(i8* %ptr)
+  ret void
+
+; CHECK-SCO-LABEL: w_hcaller:
+; CHECK-SCO: b w_hcallee
+}
+
+define weak void @w_callee(i8* %ptr) { ret void }
+define void @w_caller(i8* %ptr) {
+  tail call void @w_callee(i8* %ptr)
+  ret void
+
+; CHECK-SCO-LABEL: w_caller:
+; CHECK-SCO: bl w_callee
+}