[SystemZ] implement adjustInliningThreshold()

[jonpa]

Summary

This patch boosts the inlining threshold for a particular type of functions that are using an incoming argument only as a memcpy source.

[uweigand]

LGTM, thanks!

[jonpa]

Maybe add a test as well?

[jonpa]

llvm/test/CodeGen/SystemZ/inline-thresh-adjust.ll

2

This doesn't check that a particular function is actually inlined, but it can at least make sure that this type of function gets a boosted threshold, so maybe it could still be used, or?

[uweigand]

llvm/test/CodeGen/SystemZ/inline-thresh-adjust.ll

2

How stable are these particular numbers (1012, 1687)? I don't think it would be a good idea to hard-code those in a test case if they change randomly with any codegen change?

If we want to test whether the function is inlined, can't we simply verify whether the function name symbol does not exist in the output (because the function was completely eliminated)?

[jonpa]

llvm/test/CodeGen/SystemZ/inline-thresh-adjust.ll

2

They may change I guess, but the test case is only running the inliner pass and nothing else, so it would not be sensitive to any codegen change.

I think those numbers may change if the inliner heuristics are modified but I was hoping that then it would be trivial to update the test.

I could also try to use the original function and check that it gets inlined, but that may be a little tricky as the cost is adjusted per the context of the call-site. I guess I could just reduce the original imagick file.

[jonpa]

Test updated to avoid hard coded values.

[uweigand]

See inline comment. Otherwise LGTM. Thanks!

llvm/test/CodeGen/SystemZ/inline-thresh-adjust.ll

12

Maybe also allow wildcards for the cost.

llvm/lib/Target/SystemZ/SystemZTargetTransformInfo.h

public:
   explicit SystemZTTIImpl(const SystemZTargetMachine *TM, const Function &F)
       : BaseT(TM, F.getParent()->getDataLayout()), ST(TM->getSubtargetImpl(F)),
         TLI(ST->getTargetLowering()) {}
 
   /// \name Scalar TTI Implementations
   /// @{
 
   unsigned getInliningThresholdMultiplier() { return 3; }
+  unsigned adjustInliningThreshold(const CallBase *CB) const;
 
   InstructionCost getIntImmCost(const APInt &Imm, Type *Ty,
                                 TTI::TargetCostKind CostKind);
 
   InstructionCost getIntImmCostInst(unsigned Opcode, unsigned Idx,
                                     const APInt &Imm, Type *Ty,
                                     TTI::TargetCostKind CostKind,
                                     Instruction *Inst = nullptr);

llvm/lib/Target/SystemZ/SystemZTargetTransformInfo.cpp

 #define DEBUG_TYPE "systemztti"
 
 //===----------------------------------------------------------------------===//
 //
 // SystemZ cost model.
 //
 //===----------------------------------------------------------------------===//
 
+static bool isUsedAsMemCpySource(const Value *V, bool &OtherUse) {
+  bool UsedAsMemCpySource = false;
+  for (const User *U : V->users())
+    if (const Instruction *User = dyn_cast<Instruction>(U)) {
+      if (isa<BitCastInst>(User) || isa<GetElementPtrInst>(User)) {
+        UsedAsMemCpySource |= isUsedAsMemCpySource(User, OtherUse);
+        continue;
+      }
+      if (const MemCpyInst *Memcpy = dyn_cast<MemCpyInst>(User)) {
+        if (Memcpy->getOperand(1) == V && !Memcpy->isVolatile()) {
+          UsedAsMemCpySource = true;
+          continue;
+        }
+      }
+      OtherUse = true;
+    }
+  return UsedAsMemCpySource;
+}
+
+unsigned SystemZTTIImpl::adjustInliningThreshold(const CallBase *CB) const {
+  unsigned Bonus = 0;
+
+  // Increase the threshold if an incoming argument is used only as a memcpy
+  // source.
+  if (Function *Callee = CB->getCalledFunction())
+    for (Argument &Arg : Callee->args()) {
+      bool OtherUse = false;
+      if (isUsedAsMemCpySource(&Arg, OtherUse) && !OtherUse)
+        Bonus += 150;
+    }
+
+  LLVM_DEBUG(if (Bonus)
+               dbgs() << "++ SZTTI Adding inlining bonus: " << Bonus << "\n";);
+  return Bonus;
+}
+
 InstructionCost SystemZTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty,
                                               TTI::TargetCostKind CostKind) {
   assert(Ty->isIntegerTy());
 
   unsigned BitSize = Ty->getPrimitiveSizeInBits();
   // There is no cost model for constants with a bit size of 0. Return TCC_Free
   // here, so that constant hoisting will ignore this constant.
   if (BitSize == 0)

llvm/test/CodeGen/SystemZ/inline-thresh-adjust.ll

+; RUN: opt < %s -mtriple=systemz-unknown -mcpu=z15 -inline -disable-output \
+; RUN:   -debug-only=inline,systemztti 2>&1 | FileCheck %s
+; REQUIRES: asserts
+;
+; Check that the inlining threshold is incremented for a function using an
+; argument only as a memcpy source.
+
+; CHECK: Inlining calls in: root_function
+; CHECK:     Inlining {{.*}} Call:   call void @leaf_function_A(i8* %Dst)
+; CHECK:     ++ SZTTI Adding inlining bonus: 150
+; CHECK:     Inlining {{.*}} Call:   call void @leaf_function_B(i8* %Dst, i8* %Src)
+
+define void @leaf_function_A(i8* %Dst)  {
+entry:
+  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %Dst, i8* undef, i64 16, i1 false)
+  ret void
+}
+
+define void @leaf_function_B(i8* %Dst, i8* %Src)  {
+entry:
+  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %Dst, i8* %Src, i64 16, i1 false)
+  ret void
+}
+
+define void @root_function(i8* %Dst, i8* %Src) {
+entry:
+  call void @leaf_function_A(i8* %Dst)
+  call void @leaf_function_B(i8* %Dst, i8* %Src)
+  ret void
+}
+
+declare void @llvm.memcpy.p0i8.p0i8.i64(i8* noalias nocapture writeonly, i8* noalias nocapture readonly, i64, i1 immarg)