TTI: Add getCallInstrCost.

[mzolotukhin]

Summary

TTI: Add getCallInstrCost.

This is the third of 6 patches for enabling vectorization of calls.

[hfinkel]

include/llvm/CodeGen/BasicTTIImpl.h

681

That's not quite right. You still need to call getCallInstrCost with the associated vector types.

[mzolotukhin]

include/llvm/CodeGen/BasicTTIImpl.h

681

Hi Hal,

I don't think I get it. When the types are vector, we first call getCallInstrCost for corresponding scalar types and then return ScalarCalls*ScalarCost+ScalarizationCost for not vectorizable functions and ScalarCost for vectorizable ones. If the types are scalar, we just return 10. What's missing here?

Could you please elaborate your point a little?

[hfinkel]

include/llvm/CodeGen/BasicTTIImpl.h

681

The key problem here is that this logic does not belong inside the cost-model interface. What the cost model is supposed to provide is an interface that can be used for the vectorizer (and other clients) to ask the target, "Given this instruction on values of these types, please provide an estimate of the relative reciprocal throughput." The logic of how to transition from scalar instructions to vector instructions must live in the vectorizer, not inside the cost model. You're assuming too much about the actions that will be taken by the user of the interface by using the function-vectorization logic inside the most model to provide an estimate based on, not only the provided instruction and types, but also how the vectorizer will likely replace the queried function call with some other one.

We might not want to create external function declarations for each function we might wish to query as part of the interface, so we'll either need to allow for F to be nullptr, or change the interface.

What we're querying here is the cost of the function call, as provided, any logic related to vectorization needs to live inside the vectorizer, only the vectorizer knows how it will vectorize, and how to formulate any scalarization cost. The only scalarization costs that the TTI interface should know about, are those that will be incurred by the type legalization process in CodeGen -- those introduced at the IR level by TTI's users, need to be estimated by TTI's users based on how that scalarization is actually done.

[mzolotukhin]

include/llvm/CodeGen/BasicTTIImpl.h

681

Ok, I see what you mean now, thank you for the explanation!

Then my plan is to leave in TTI only trivial getCallCost, which will always return 10 (in default implementation). The current logic, which takes into account scalarization costs, will be moved to the vectorizer almost unchanged. With such changes back-ends could control the vectorizer by setting costs of Call, InsertElement and ExtractElement instructions. Does it sound good?

[mzolotukhin]

include/llvm/CodeGen/BasicTTIImpl.h

681

Actually, we use getScalarizationCost in other places in BasicTTIImple too: e.g. in getArithmeticInstrCost, getCastInstrCost, and others - do you think we need to change these places as well?

[hfinkel]

include/llvm/CodeGen/BasicTTIImpl.h

681

Yes, that sounds like the right way to do it; thanks!

The other places in BasicTTIImpl use getScalarizationCost to model what happens during type/operation legalization in CodeGen, so that's different (and should stay).

[mzolotukhin]

include/llvm/CodeGen/BasicTTIImpl.h

681

Hi Hal,

I looked at what we need to change to make it this way:

1. We need to implement getScalarizationCost outside TTI, so that it can be used in LoopVectorizer, SLPVectorizer, IndVarSimplify, CodeGenPrepare and other passes that currently might use getArithInstrCost/getCastInstrCost/etc. with a vector type.
2. We need to change all these uses with

if (Ty->isVectorTy())
  cost = getScalarizationCost(Ty) + getInstrCost(Ty)
else
  cost = getInstrCost(Ty)

3. We need to make sure the we don't actually change costs on any platform.

All of these seem to me like a pretty massive, and probably not so elegant change - we need to change a lot of places, and we'll introduce a lot of very similar code. Also, it might be true that the vectorizer might want to determine a specific way to scalarize a value, but other users of getArithInstrCost/etc. might don't care about it that much. E.g. I don't think it's worth adding this logic to CodeGenPrepare - it'll be the same as we have now in TTI anyway (and we'll need to keep it both in and outside TTI).

Did I understand what you suggest correctly at all?

[hfinkel]

include/llvm/CodeGen/BasicTTIImpl.h

681

Did I understand what you suggest correctly at all?

No, not really ;)

We need to implement getScalarizationCost outside TTI

No, we should leave it as it.

I think we have a terminology problem here. There are two types of 'scalarization': One type is that done by CodeGen's type / operator legalization framework. This is what is modeled by getScalarizationCost, and that should stay as is. getArithInstrCost/getCastInstrCost/etc. use that logic to estimate the cost of the scalarization that will be performed by legalization, and that's fine.

As we've discussed, things are a bit different for calls. Calls don't really get 'scalarized' by CodeGen in the same sense. A call is a call, and the only meaningful 'scalarization' that happens might be in breaking up its parameters for argument passing and reconstituting its return value if those types are not legal (and, even if they are, depending on the calling convention). This is different for intrinsics, but for calls to actual functions, there is no sense in which type legalization turns one call into many calls. The vectorizer, with knowledge of certain calls, might do this (because it understands their semantics), but it does this "manually" (by inserting extractelement/insertelement instructions and then different calls). This process should be modeled by the vectorizer by getting the cost of those instructions it will actually insert. This is very different from what is done for arithmetic instructions (where we let CodeGen do the legalization).

[mzolotukhin]

• Remove scalarization cost estimate (now it belongs to the vectorizer).

[hfinkel]

include/llvm/Analysis/TargetTransformInfo.h

453

Please specifically mention that F may by nullptr.

[mzolotukhin]

• Expand comments for getCallInstrCost. Mention that F might be nullptr.

[hfinkel]

include/llvm/Analysis/TargetTransformInfo.h

41

Remove this (should not needed, as noted below).

454

I don't think we need TLI here given the current strategy.

[mzolotukhin]

• Remove TLI argument from getCallInstrCost.

[hfinkel]

LGTM.

[mzolotukhin]

Thanks, committed in r232524!

include/llvm/Analysis/TargetTransformInfo.h

 
 class Function;
 class GlobalValue;
 class Loop;
 class PreservedAnalyses;
 class Type;
 class User;
 class Value;
 
 /// \brief Information about a load/store intrinsic defined by the target.
 struct MemIntrinsicInfo {
   MemIntrinsicInfo()
       : ReadMem(false), WriteMem(false), Vol(false), MatchingId(0),
         NumMemRefs(0), PtrVal(nullptr) {}
   bool ReadMem;
   bool WriteMem;
   bool Vol;
public:
   ///  ((v0+v2), (v1+v3), undef, undef)
   unsigned getReductionCost(unsigned Opcode, Type *Ty,
                             bool IsPairwiseForm) const;
 
   /// \returns The cost of Intrinsic instructions.
   unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
                                  ArrayRef<Type *> Tys) const;
 
+  /// \returns The cost of Call instructions.
+  unsigned getCallInstrCost(Function *F, Type *RetTy,
+                            ArrayRef<Type *> Tys) const;
+
   /// \returns The number of pieces into which the provided type must be
   /// split during legalization. Zero is returned when the answer is unknown.
   unsigned getNumberOfParts(Type *Tp) const;
 
   /// \returns The cost of the address computation. For most targets this can be
   /// merged into the instruction indexing mode. Some targets might want to
   /// distinguish between address computation for memory operations on vector
   /// types and scalar types. Such targets should override this function.
virtual unsigned getMemoryOpCost(unsigned Opcode, Type *Src,
                                    unsigned AddressSpace) = 0;
   virtual unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
                                          unsigned Alignment,
                                          unsigned AddressSpace) = 0;
   virtual unsigned getReductionCost(unsigned Opcode, Type *Ty,
                                     bool IsPairwiseForm) = 0;
   virtual unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
                                          ArrayRef<Type *> Tys) = 0;
+  virtual unsigned getCallInstrCost(Function *F, Type *RetTy,
+                                    ArrayRef<Type *> Tys) = 0;
   virtual unsigned getNumberOfParts(Type *Tp) = 0;
   virtual unsigned getAddressComputationCost(Type *Ty, bool IsComplex) = 0;
   virtual unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) = 0;
   virtual bool getTgtMemIntrinsic(IntrinsicInst *Inst,
                                   MemIntrinsicInfo &Info) = 0;
   virtual Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
                                                    Type *ExpectedType) = 0;
 };
public:
   unsigned getReductionCost(unsigned Opcode, Type *Ty,
                             bool IsPairwiseForm) override {
     return Impl.getReductionCost(Opcode, Ty, IsPairwiseForm);
   }
   unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
                                  ArrayRef<Type *> Tys) override {
     return Impl.getIntrinsicInstrCost(ID, RetTy, Tys);
   }
+  unsigned getCallInstrCost(Function *F, Type *RetTy,
+                            ArrayRef<Type *> Tys) override {
+    return Impl.getCallInstrCost(F, RetTy, Tys);
+  }
   unsigned getNumberOfParts(Type *Tp) override {
     return Impl.getNumberOfParts(Tp);
   }
   unsigned getAddressComputationCost(Type *Ty, bool IsComplex) override {
     return Impl.getAddressComputationCost(Ty, IsComplex);
   }
   unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) override {
     return Impl.getCostOfKeepingLiveOverCall(Tys);

include/llvm/Analysis/TargetTransformInfoImpl.h

unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
     return 1;
   }
 
   unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
                                  ArrayRef<Type *> Tys) {
     return 1;
   }
 
+  unsigned getCallInstrCost(Function *F, Type *RetTy, ArrayRef<Type *> Tys) {
+    return 1;
+  }
+
   unsigned getNumberOfParts(Type *Tp) { return 0; }
 
   unsigned getAddressComputationCost(Type *Tp, bool) { return 0; }
 
   unsigned getReductionCost(unsigned, Type *, bool) { return 1; }
 
   unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) { return 0; }
 

include/llvm/CodeGen/BasicTTIImpl.h

 #ifndef LLVM_CODEGEN_BASICTTIIMPL_H
 #define LLVM_CODEGEN_BASICTTIIMPL_H
 
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/TargetTransformInfoImpl.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Target/TargetLowering.h"
 #include "llvm/Target/TargetSubtargetInfo.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
 
 namespace llvm {
 
 extern cl::opt<unsigned> PartialUnrollingThreshold;
 
 /// \brief Base class which can be used to help build a TTI implementation.
 ///
 /// This class provides as much implementation of the TTI interface as is
if (RetTy->isVectorTy()) {
           IID, RetTy->getScalarType(), Tys);
       return 10 * Cost * Num;
     }
 
     // This is going to be turned into a library call, make it expensive.
     return 10;
   }
 
+  /// \brief Compute a cost of the given call instruction.
+  ///
+  /// Compute the cost of calling function F with return type RetTy and
+  /// argument types Tys. F might be nullptr, in this case the cost of an
+  /// arbitrary call with the specified signature will be returned.
+  /// This is used, for instance,  when we estimate call of a vector
+  /// counterpart of the given function.
+  /// \param F Called function, might be nullptr.
+  /// \param RetTy,Tys Return value and argument types.
+  /// \returns The cost of Call instruction.
+  unsigned getCallInstrCost(Function *F, Type *RetTy, ArrayRef<Type *> Tys) {
+    return 10;
+  }
+
   unsigned getNumberOfParts(Type *Tp) {
     std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Tp);
     return LT.first;
   }
 
   unsigned getAddressComputationCost(Type *Ty, bool IsComplex) { return 0; }
 
   unsigned getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwise) {
     assert(Ty->isVectorTy() && "Expect a vector type");
     unsigned NumVecElts = Ty->getVectorNumElements();
     unsigned NumReduxLevels = Log2_32(NumVecElts);
     unsigned ArithCost =
         NumReduxLevels *
         static_cast<T *>(this)->getArithmeticInstrCost(Opcode, Ty);

lib/Analysis/TargetTransformInfo.cpp

 }
 
 unsigned
 TargetTransformInfo::getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
                                            ArrayRef<Type *> Tys) const {
   return TTIImpl->getIntrinsicInstrCost(ID, RetTy, Tys);
 }
 
+unsigned TargetTransformInfo::getCallInstrCost(Function *F, Type *RetTy,
+                                               ArrayRef<Type *> Tys) const {
+  return TTIImpl->getCallInstrCost(F, RetTy, Tys);
+}
+
 unsigned TargetTransformInfo::getNumberOfParts(Type *Tp) const {
   return TTIImpl->getNumberOfParts(Tp);
 }
 
 unsigned TargetTransformInfo::getAddressComputationCost(Type *Tp,
                                                         bool IsComplex) const {
   return TTIImpl->getAddressComputationCost(Tp, IsComplex);
 }