[LV] NFCI: Do cost comparison on InstructionCost directly.

[sdesmalen]

Summary

Instead of performing the isMoreProfitable() operation on
InstructionCost::CostTy the operation is performed on InstructionCost
directly, so that it can handle the case where one of the costs is
Invalid.

This patch also changes the CostTy to be int64_t, so that the type is
wide enough to deal with multiplications with e.g. unsigned MaxTripCount.

[dmgreen]

i64 for the type sounds OK to me. (The old methods was known not to overflow because of divideCeil being a int64_t and the other values being int32. But with saturation that should be fine now too)

Does this mean that invalid costs now don't vectorize? (For an invalid vector cost)

Is it possible to add a test for that?

[sdesmalen]

In D105113#2847034, @dmgreen wrote:

i64 for the type sounds OK to me. (The old methods was known not to overflow because of divideCeil being a int64_t and the other values being int32. But with saturation that should be fine now too)

Does this mean that invalid costs now don't vectorize? (For an invalid vector cost). Is it possible to add a test for that?

Not yet, that requires a few more code-changes. The behaviour to avoid vectorization with Invalid costs is added in D105473 (with corresponding tests and some logic to print out useful opt-remarks).

This patch is intended to be NFC (I'll add NFCI to the title of the patch)

[dmgreen]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

6592

These still use unsigned. Are we happy to assume that getRegUsageForType / getTypeLegalizationCost will always produce a small, sensible value?

[sdesmalen]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

6592

Yes, I think it's safe to assume that, but I'd be happy to add an assert for it if you think that's worth it?

[sdesmalen]

Added assert to ensure the range of getRegisterUsage is sensible.

[dmgreen]

Thanks. The assert sounds good. LGTM

[sdesmalen]

Replaced std::numeric_limits<InstructionCost::CostTy>::max()
by InstructionCost::getMax()

As suggested in D105473.

llvm/include/llvm/Support/InstructionCost.h

 #include <limits>
 
 namespace llvm {
 
 class raw_ostream;
 
 class InstructionCost {
 public:
-  using CostType = int;
+  using CostType = int64_t;
 
   /// These states can currently be used to indicate whether a cost is valid or
   /// invalid. Examples of an invalid cost might be where the cost is
   /// prohibitively expensive and the user wants to prevent certain
   /// optimizations being performed. Or perhaps the cost is simply unknown
   /// because the operation makes no sense in certain circumstances. These
   /// states can be expanded in future to support other cases if necessary.
   enum CostState { Valid, Invalid };

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

if (ElementCount MinVF =
       }
     }
   }
   return MaxVF;
 }
 
 bool LoopVectorizationCostModel::isMoreProfitable(
     const VectorizationFactor &A, const VectorizationFactor &B) const {
-  InstructionCost::CostType CostA = *A.Cost.getValue();
-  InstructionCost::CostType CostB = *B.Cost.getValue();
+  InstructionCost CostA = A.Cost;
+  InstructionCost CostB = B.Cost;
 
   unsigned MaxTripCount = PSE.getSE()->getSmallConstantMaxTripCount(TheLoop);
 
   if (!A.Width.isScalable() && !B.Width.isScalable() && FoldTailByMasking &&
       MaxTripCount) {
     // If we are folding the tail and the trip count is a known (possibly small)
     // constant, the trip count will be rounded up to an integer number of
     // iterations. The total cost will be PerIterationCost*ceil(TripCount/VF),
     // which we compare directly. When not folding the tail, the total cost will
     // be PerIterationCost*floor(TC/VF) + Scalar remainder cost, and so is
     // approximated with the per-lane cost below instead of using the tripcount
     // as here.
-    int64_t RTCostA = CostA * divideCeil(MaxTripCount, A.Width.getFixedValue());
-    int64_t RTCostB = CostB * divideCeil(MaxTripCount, B.Width.getFixedValue());
+    auto RTCostA = CostA * divideCeil(MaxTripCount, A.Width.getFixedValue());
+    auto RTCostB = CostB * divideCeil(MaxTripCount, B.Width.getFixedValue());
     return RTCostA < RTCostB;
   }
 
   // When set to preferred, for now assume vscale may be larger than 1, so
   // that scalable vectorization is slightly favorable over fixed-width
   // vectorization.
   if (Hints->isScalableVectorizationPreferred())
     if (A.Width.isScalable() && !B.Width.isScalable())
LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
   DenseMap<unsigned, InstrList> TransposeEnds;
 
   // Transpose the EndPoints to a list of values that end at each index.
   for (auto &Interval : EndPoint)
     TransposeEnds[Interval.second].push_back(Interval.first);
 
   SmallPtrSet<Instruction *, 8> OpenIntervals;
   SmallVector<RegisterUsage, 8> RUs(VFs.size());
   SmallVector<SmallMapVector<unsigned, unsigned, 4>, 8> MaxUsages(VFs.size());
 
   LLVM_DEBUG(dbgs() << "LV(REG): Calculating max register usage:\n");
 
   // A lambda that gets the register usage for the given type and VF.
   const auto &TTICapture = TTI;
-  auto GetRegUsage = [&TTICapture](Type *Ty, ElementCount VF) {
+  auto GetRegUsage = [&TTICapture](Type *Ty, ElementCount VF) -> int64_t {
     if (Ty->isTokenTy() || !VectorType::isValidElementType(Ty))
       return 0;
     return *TTICapture.getRegUsageForType(VectorType::get(Ty, VF)).getValue();
   };
 
   for (unsigned int i = 0, s = IdxToInstr.size(); i < s; ++i) {
     Instruction *I = IdxToInstr[i];