[LV] NFCI: Do cost comparison on InstructionCost directly.

[sdesmalen]

Summary

Instead of performing the isMoreProfitable() operation on
InstructionCost::CostTy the operation is performed on InstructionCost
directly, so that it can handle the case where one of the costs is
Invalid.

This patch also changes the CostTy to be int64_t, so that the type is
wide enough to deal with multiplications with e.g. unsigned MaxTripCount.

[dmgreen]

i64 for the type sounds OK to me. (The old methods was known not to overflow because of divideCeil being a int64_t and the other values being int32. But with saturation that should be fine now too)

Does this mean that invalid costs now don't vectorize? (For an invalid vector cost)

Is it possible to add a test for that?

[sdesmalen]

In D105113#2847034, @dmgreen wrote:

i64 for the type sounds OK to me. (The old methods was known not to overflow because of divideCeil being a int64_t and the other values being int32. But with saturation that should be fine now too)

Does this mean that invalid costs now don't vectorize? (For an invalid vector cost). Is it possible to add a test for that?

Not yet, that requires a few more code-changes. The behaviour to avoid vectorization with Invalid costs is added in D105473 (with corresponding tests and some logic to print out useful opt-remarks).

This patch is intended to be NFC (I'll add NFCI to the title of the patch)

[dmgreen]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

6597

These still use unsigned. Are we happy to assume that getRegUsageForType / getTypeLegalizationCost will always produce a small, sensible value?

[sdesmalen]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

6597

Yes, I think it's safe to assume that, but I'd be happy to add an assert for it if you think that's worth it?

[sdesmalen]

Added assert to ensure the range of getRegisterUsage is sensible.

[dmgreen]

Thanks. The assert sounds good. LGTM

[sdesmalen]

Replaced std::numeric_limits<InstructionCost::CostTy>::max()
by InstructionCost::getMax()

As suggested in D105473.

llvm/include/llvm/Support/InstructionCost.h

 #include <limits>
 
 namespace llvm {
 
 class raw_ostream;
 
 class InstructionCost {
 public:
-  using CostType = int;
+  using CostType = int64_t;
 
   /// CostState describes the state of a cost.
   enum CostState {
     Valid,  /// < The cost value represents a valid cost, even when the
             /// cost-value is large.
     Invalid /// < Invalid indicates there is no way to represent the cost as a
             /// numeric value. This state exists to represent a possible issue,
             /// e.g. if the cost-model knows the operation cannot be expanded

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

if (ElementCount MinVF =
       }
     }
   }
   return MaxVF;
 }
 
 bool LoopVectorizationCostModel::isMoreProfitable(
     const VectorizationFactor &A, const VectorizationFactor &B) const {
-  InstructionCost::CostType CostA = *A.Cost.getValue();
-  InstructionCost::CostType CostB = *B.Cost.getValue();
+  InstructionCost CostA = A.Cost;
+  InstructionCost CostB = B.Cost;
 
   unsigned MaxTripCount = PSE.getSE()->getSmallConstantMaxTripCount(TheLoop);
 
   if (!A.Width.isScalable() && !B.Width.isScalable() && FoldTailByMasking &&
       MaxTripCount) {
     // If we are folding the tail and the trip count is a known (possibly small)
     // constant, the trip count will be rounded up to an integer number of
     // iterations. The total cost will be PerIterationCost*ceil(TripCount/VF),
     // which we compare directly. When not folding the tail, the total cost will
     // be PerIterationCost*floor(TC/VF) + Scalar remainder cost, and so is
     // approximated with the per-lane cost below instead of using the tripcount
     // as here.
-    int64_t RTCostA = CostA * divideCeil(MaxTripCount, A.Width.getFixedValue());
-    int64_t RTCostB = CostB * divideCeil(MaxTripCount, B.Width.getFixedValue());
+    auto RTCostA = CostA * divideCeil(MaxTripCount, A.Width.getFixedValue());
+    auto RTCostB = CostB * divideCeil(MaxTripCount, B.Width.getFixedValue());
     return RTCostA < RTCostB;
   }
 
   // When set to preferred, for now assume vscale may be larger than 1, so
   // that scalable vectorization is slightly favorable over fixed-width
   // vectorization.
   if (Hints->isScalableVectorizationPreferred())
     if (A.Width.isScalable() && !B.Width.isScalable())
VectorizationFactor LoopVectorizationCostModel::selectVectorizationFactor(
   const VectorizationFactor ScalarCost(ElementCount::getFixed(1), ExpectedCost);
   VectorizationFactor ChosenFactor = ScalarCost;
 
   bool ForceVectorization = Hints->getForce() == LoopVectorizeHints::FK_Enabled;
   if (ForceVectorization && VFCandidates.size() > 1) {
     // Ignore scalar width, because the user explicitly wants vectorization.
     // Initialize cost to max so that VF = 2 is, at least, chosen during cost
     // evaluation.
-    ChosenFactor.Cost = std::numeric_limits<InstructionCost::CostType>::max();
+    ChosenFactor.Cost = InstructionCost::getMax();
   }
 
   for (const auto &i : VFCandidates) {
     // The cost for scalar VF=1 is already calculated, so ignore it.
     if (i.isScalar())
       continue;
 
     // Notice that the vector loop needs to be executed less times, so
LoopVectorizationCostModel::calculateRegisterUsage(ArrayRef<ElementCount> VFs) {
   DenseMap<unsigned, InstrList> TransposeEnds;
 
   // Transpose the EndPoints to a list of values that end at each index.
   for (auto &Interval : EndPoint)
     TransposeEnds[Interval.second].push_back(Interval.first);
 
   SmallPtrSet<Instruction *, 8> OpenIntervals;
   SmallVector<RegisterUsage, 8> RUs(VFs.size());
   SmallVector<SmallMapVector<unsigned, unsigned, 4>, 8> MaxUsages(VFs.size());
 
   LLVM_DEBUG(dbgs() << "LV(REG): Calculating max register usage:\n");
 
   // A lambda that gets the register usage for the given type and VF.
   const auto &TTICapture = TTI;
-  auto GetRegUsage = [&TTICapture](Type *Ty, ElementCount VF) {
+  auto GetRegUsage = [&TTICapture](Type *Ty, ElementCount VF) -> unsigned {
     if (Ty->isTokenTy() || !VectorType::isValidElementType(Ty))
       return 0;
-    return *TTICapture.getRegUsageForType(VectorType::get(Ty, VF)).getValue();
+    InstructionCost::CostType RegUsage =
+        *TTICapture.getRegUsageForType(VectorType::get(Ty, VF)).getValue();
+    assert(RegUsage >= 0 && RegUsage <= std::numeric_limits<unsigned>::max() &&
+           "Nonsensical values for register usage.");
+    return RegUsage;
   };
 
   for (unsigned int i = 0, s = IdxToInstr.size(); i < s; ++i) {
     Instruction *I = IdxToInstr[i];
 
     // Remove all of the instructions that end at this location.
     InstrList &List = TransposeEnds[i];
     for (Instruction *ToRemove : List)