[LV] Disable epilogue vectorization for scalable VFs

[c-rhodes]

Summary

Epilogue vectorization doesn't support scalable vectorization factors
yet, disable it for now.

[bmahjour]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

5778

We can just return Result around line 5792 (please see suggestion below), then we don't need to change the check near line 9209.

5792

// FIXME: This can be fixed for scalable vectors later, because at this stage
// the LoopVectorizer will only consider vectorizing a loop with scalable
// vectors when the loop has a hint to enable vectorization for a given VF.
if (MainLoopVF.isScalable()) {
  LLVM_DEBUG(dbgs() << "LEV: Epilogue vectorization for scalable vectors not yet supported.\n");
  return Result;
}

9210

please see comment above.

[c-rhodes]

Address comments and added test.

[c-rhodes]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

5778

We can just return Result around line 5792 (please see suggestion below), then we don't need to change the check near line 9209.

That makes more sense, cheers. Added a simple test as well.

[sdesmalen]

LGTM

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

5778

I guess there is no reason why the tail can't still be vectorized with a fixed-width VF, but given the state of scalable loop-vec I don't think that is really something to worry about at this point.

[bmahjour]

LGTM

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

bool LoopVectorizationCostModel::isEpilogueVectorizationProfitable(
   if (VF.getFixedValue() >= EpilogueVectorizationMinVF)
     return true;
   return false;
 }
 
 VectorizationFactor
 LoopVectorizationCostModel::selectEpilogueVectorizationFactor(
     const ElementCount MainLoopVF, const LoopVectorizationPlanner &LVP) {
+  // FIXME: This can be fixed for scalable vectors later, because at this stage
+  // the LoopVectorizer will only consider vectorizing a loop with scalable
+  // vectors when the loop has a hint to enable vectorization for a given VF.
+  assert(!MainLoopVF.isScalable() && "scalable vectors not yet supported");
+
   VectorizationFactor Result = VectorizationFactor::Disabled();
   if (!EnableEpilogueVectorization) {
     LLVM_DEBUG(dbgs() << "LEV: Epilogue vectorization is disabled.\n";);
     return Result;
   }
 
   if (!isScalarEpilogueAllowed()) {
     LLVM_DEBUG(
         dbgs() << "LEV: Unable to vectorize epilogue because no epilogue is "
                   "allowed.\n";);
     return Result;
   }
 
   // Not really a cost consideration, but check for unsupported cases here to
   // simplify the logic.
   if (!isCandidateForEpilogueVectorization(*TheLoop, MainLoopVF)) {
     LLVM_DEBUG(
         dbgs() << "LEV: Unable to vectorize epilogue because the loop is "
                   "not a supported candidate.\n";);
     return Result;
   }
ORE->emit([&]() {
              << "interleaved loop (interleaved count: "
              << NV("InterleaveCount", IC) << ")";
     });
   } else {
     // If we decided that it is *legal* to vectorize the loop, then do it.
 
     // Consider vectorizing the epilogue too if it's profitable.
     VectorizationFactor EpilogueVF =
-      CM.selectEpilogueVectorizationFactor(VF.Width, LVP);
-    if (EpilogueVF.Width.isVector()) {
+        VF.Width.isScalable()
+            ? VectorizationFactor::Disabled()
+            : CM.selectEpilogueVectorizationFactor(VF.Width, LVP);
+    if (EpilogueVF != VectorizationFactor::Disabled() &&
+        EpilogueVF.Width.isVector()) {
 
       // The first pass vectorizes the main loop and creates a scalar epilogue
       // to be vectorized by executing the plan (potentially with a different
       // factor) again shortly afterwards.
       EpilogueLoopVectorizationInfo EPI(VF.Width.getKnownMinValue(), IC,
                                         EpilogueVF.Width.getKnownMinValue(), 1);
       EpilogueVectorizerMainLoop MainILV(L, PSE, LI, DT, TLI, TTI, AC, ORE, EPI,
                                          &LVL, &CM, BFI, PSI);