[LV] Check if value was already not uniform for previous VF.

[fhahn]

Summary

If the value was already known to not be uniform for the previous
(smaller VF), it cannot be uniform for the larger VF.

This slightly reduces compile-time, once uniformity checks are becoming
a bit more expensive due to using SCEV rewriting (D148841).

[Ayal]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

4684

This looks right.
Also holds for non-MemOpUse?
Furthermore, if value is uniform for PrevVF, suffice to only check if SCEVs of lanes 0 and VF-1 are equal.

[fhahn]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

4684

Also holds for non-MemOpUse?

Yes, but the loop that uses isUniformMemOpUse also collects pointers where only lane 0 is demanded (for widened memory ops), so checking in the loop directly isn't possible without first dis-entangling this logic.

Furthermore, if value is uniform for PrevVF, suffice to only check if SCEVs of lanes 0 and VF-1 are equal.

Hmm, I think we would have to check if lane 0 matches the lanes PrevVF..VF-1? Might be good as follow-up, as it will require adjusting the API a bit I think to provide the additional info.

[Ayal]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

4684

Also holds for non-MemOpUse?

Yes, but the loop that uses isUniformMemOpUse also collects pointers where only lane 0 is demanded (for widened memory ops), so checking in the loop directly isn't possible without first dis-entangling this logic.

Agreed, uniformness should be disentangled from first-lane-used-ness.

Furthermore, if value is uniform for PrevVF, suffice to only check if SCEVs of lanes 0 and VF-1 are equal.

Hmm, I think we would have to check if lane 0 matches the lanes PrevVF..VF-1? Might be good as follow-up, as it will require adjusting the API a bit I think to provide the additional info.

Stitching piecewise uniformity - if a value is uniform across PrevVF then its SCEVs for lanes {0,1,..,PrevVF-1} are equal to each other, and so are those of lanes/offsets {PrevVF,PrevVF+1...,VF-1}, right? If so then suffice the select and compare one sample from each for the value to be uniform across VF.

In any case these can be dealt with in separate follow-up patches.

[fhahn]

Rebase so this can be applied on current main.

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

4684

Sounds good, so should this initial patch go in as is for now?

[Ayal]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

4684

Sure, just noting more room for improving how we go about analyzing uniformity/divergence.

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

void LoopVectorizationCostModel::collectLoopUniforms(ElementCount VF) {
 
   // Start with the conditional branch. If the branch condition is an
   // instruction contained in the loop that is only used by the branch, it is
   // uniform.
   auto *Cmp = dyn_cast<Instruction>(Latch->getTerminator()->getOperand(0));
   if (Cmp && TheLoop->contains(Cmp) && Cmp->hasOneUse())
     addToWorklistIfAllowed(Cmp);
 
+  auto PrevVF = VF.divideCoefficientBy(2);
   // Return true if all lanes perform the same memory operation, and we can
   // thus chose to execute only one.
   auto isUniformMemOpUse = [&](Instruction *I) {
+    // If the value was already known to not be uniform for the previous
+    // (smaller VF), it cannot be uniform for the larger VF.
+    if (PrevVF.isVector()) {
+      auto Iter = Uniforms.find(PrevVF);
+      if (Iter != Uniforms.end() && !Iter->second.contains(I))
+        return false;
+    }
     if (!Legal->isUniformMemOp(*I))
       return false;
     if (isa<LoadInst>(I))
       // Loading the same address always produces the same result - at least
       // assuming aliasing and ordering which have already been checked.
       return true;
     // Storing the same value on every iteration.
     return TheLoop->isLoopInvariant(cast<StoreInst>(I)->getValueOperand());