[LV] Check VPValue operand instead of Cost::isUniformAfterVec (NFC).

[fhahn]

Summary

ILV::scalarizeInstruction still uses the original IR operands to check
if an input value is uniform after vectorization.

There is no need to go back to the cost model to figure that out, as the
information is already explicit in the VPlan. Just check directly
whether the VPValue is defined outside the plan or is a uniform
VPReplicateRecipe.

[fhahn]

rebased after4348cd42c385

[fhahn]

ping :)

[Ayal]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

3036

Whether a VPValue is "uniform after vectorization" or not should probably be a property of VPValue. Currently this can indeed be detected in VPlan by checking if it has no recipe, or has a uniform replicating recipe. Worth wrapping in a method or function? Its implementation may change when VPValues obtain "type" information and/or when additional scalar recipes may be introduced.

Current patch is ok, as a temporary step; would be good to add a comment.

Op may be confused with Operation; perhaps the original Operand would be better.
RepR may be confused with RepRecipe; perhaps OperandRepR would be better.

[fhahn]

Move logic to VPlan::isUniformAfterVectorization, use it

[fhahn]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

3036

Whether a VPValue is "uniform after vectorization" or not should probably be a property of VPValue

Good point! I move the check logic to VPlan::isUniformAfterVectorization, to avoid tying this directly to the information available for a VPV. Was this what you had in mind by moving it into a function?

[Ayal]

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

3036

Yes, it was. Note that in D104254, UAV's (as opposed to truly uniforms) are identified by looking for the address operands of widened memory recipes.

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

void InnerLoopVectorizer::scalarizeInstruction(Instruction *Instr,
   // load/store.
   if (State.MayGeneratePoisonRecipes.count(RepRecipe) > 0)
     Cloned->dropPoisonGeneratingFlags();
 
   State.Builder.SetInsertPoint(Builder.GetInsertBlock(),
                                Builder.GetInsertPoint());
   // Replace the operands of the cloned instructions with their scalar
   // equivalents in the new loop.
-  for (unsigned op = 0, e = RepRecipe->getNumOperands(); op != e; ++op) {
-    auto *Operand = dyn_cast<Instruction>(Instr->getOperand(op));
+  for (auto &I : enumerate(RepRecipe->operands())) {
     auto InputInstance = Instance;
-    if (!Operand || !OrigLoop->contains(Operand) ||
-        (Cost->isUniformAfterVectorization(Operand, State.VF)))
+    VPValue *Operand = I.value();
+    if (State.Plan->isUniformAfterVectorization(Operand))
       InputInstance.Lane = VPLane::getFirstLane();
-    auto *NewOp = State.get(RepRecipe->getOperand(op), InputInstance);
-    Cloned->setOperand(op, NewOp);
+    Cloned->setOperand(I.index(), State.get(Operand, InputInstance));
   }
   addNewMetadata(Cloned, Instr);
 
   // Place the cloned scalar in the new loop.
   Builder.Insert(Cloned);
 
   State.set(RepRecipe, Cloned, Instance);
 

llvm/lib/Transforms/Vectorize/VPlan.h

#endif
   iterator_range<mapped_iterator<Use *, std::function<VPValue *(Value *)>>>
   mapToVPValues(User::op_range Operands) {
     std::function<VPValue *(Value *)> Fn = [this](Value *Op) {
       return getOrAddVPValue(Op);
     };
     return map_range(Operands, Fn);
   }
 
+  /// Returns true if \p VPV is uniform after vectorization.
+  bool isUniformAfterVectorization(VPValue *VPV) const {
+    auto RepR = dyn_cast_or_null<VPReplicateRecipe>(VPV->getDef());
+    return !VPV->getDef() || (RepR && RepR->isUniform());
+  }
+
 private:
   /// Add to the given dominator tree the header block and every new basic block
   /// that was created between it and the latch block, inclusive.
   static void updateDominatorTree(DominatorTree *DT, BasicBlock *LoopLatchBB,
                                   BasicBlock *LoopPreHeaderBB,
                                   BasicBlock *LoopExitBB);
 };