[SLP] Provide an universal interface for FixedVectorType::get. NFC.

[ABataev]

LG

[aeubanks]

this causes crashes on

$ /tmp/a.ll
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define void @f() {
entry:
  %cmp.i.i.i = fcmp olt x86_fp80 0xK00000000000000000000, 0xK00000000000000000000
  %cmp.i6.i.i = fcmp olt x86_fp80 0xK00000000000000000000, 0xK00000000000000000000
  ret void
}
$ opt -p slp-vectorizer -disable-output /tmp/a.ll

I've reverted this for now

[aeubanks]

(also I don't see how this is an improvement)

[HanKuanChen]

Update: Remove isValidElementType assert in makeScalarTyToVectorTy.

[HanKuanChen]

In D154114#4462653, @aeubanks wrote:

(also I don't see how this is an improvement)

The purpose is for revec.

[nikic]

Reverted this again. Please make sure that reviews and commit messages always have a meaningful description.

This change doesn't make sense in isolation, and neither the patch nor the commit include the necessary larger context that could justify this change.

[HanKuanChen]

@aeubanks @nikic I will not add this in the commit message. It is too long.
Let's say we have a IR like this

%in1 = getelementptr inbounds i32, ptr %in0, i64 1
%in2 = getelementptr inbounds i32, ptr %in0, i64 2
%in3 = getelementptr inbounds i32, ptr %in0, i64 3
%v0 = load i32, ptr %in0, align 4
%v1 = load i32, ptr %in1, align 4
%v2 = load i32, ptr %in2, align 4
%v3 = load i32, ptr %in3, align 4

SLP can make this into load <4 x i32>, ptr %in0, align 4. Using FixedVectorType::get(i32, 4) works well.
For revec, we have a IR like this

%in1 = getelementptr inbounds i32, ptr %in0, i64 4
%in2 = getelementptr inbounds i32, ptr %in0, i64 8
%in3 = getelementptr inbounds i32, ptr %in0, i64 12
%v0 = load <4 x i32>, ptr %in0, align 4
%v1 = load <4 x i32>, ptr %in1, align 4
%v2 = load <4 x i32>, ptr %in2, align 4
%v3 = load <4 x i32>, ptr %in3, align 4

SLP can make this into load <16 x i32>, ptr %in0, align 4. But using FixedVectorType::get(<4 x i32>, 4) is incorrect. The actually function call should be FixedVectorType::get(i32, 16).
To make scalar to vector and vector to vector work together, you need

unsigned NumElements = 1;
if (auto *VecTy = dyn_cast<FixedVectorType>(ScalarTy))
  NumElements = VecTy->getNumElements();
FixedVectorType *VecTy = FixedVectorType::get(ScalarTy->getScalarType(), VF * NumElements);

If FixedVectorType::get occurs only few times, do in this way is better. But the fact is we need a function to wrap it.

[nikic]

Here is how your patch description could have looked like:

In the future, we want to support ScalarTy being a vector, which is extended to a VectorTy with more elements. To allow making that change in a central place, introduce a makeScalarTyToVectorTy() helper.

Is that an accurate summary of the motivation?

Maybe together with a note:

This is part up an effort to upstream revec support.

If that is what you're doing.

[vdmitrie]

llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp

1082

my 2 cents

Based on recent comment this is going to be extended to handle vector type as well. In this regard name of the function is not going to sound accurate for its purpose.
I'd suggest to change its signature into something like
FixedVectorType *getWidenedType(const Type *Ty, unsigned VF)

and perhaps place it into
llvm/include/llvm/Analysis/VectorUtils.h
llvm/lib/Analysis/VectorUtils.cpp

llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp

 /// It is so if all either instructions have operands that do not require
 /// scheduling or their users do not require scheduling since they are phis or
 /// in other basic blocks.
 static bool doesNotNeedToSchedule(ArrayRef<Value *> VL) {
   return !VL.empty() &&
          (all_of(VL, isUsedOutsideBlock) || all_of(VL, areAllOperandsNonInsts));
 }
 
+static FixedVectorType *makeScalarTyToVectorTy(Type *ScalarTy, unsigned VF) {
+  return FixedVectorType::get(ScalarTy, VF);
+}
+
 namespace slpvectorizer {
 
 /// Bottom Up SLP Vectorizer.
 class BoUpSLP {
   struct TreeEntry;
   struct ScheduleData;
   class ShuffleCostEstimator;
   class ShuffleInstructionBuilder;
int getShallowScore(Value *V1, Value *V2, Instruction *U1, Instruction *U2,
 
         std::optional<int> Dist = getPointersDiff(
             LI1->getType(), LI1->getPointerOperand(), LI2->getType(),
             LI2->getPointerOperand(), DL, SE, /*StrictCheck=*/true);
         if (!Dist || *Dist == 0) {
           if (getUnderlyingObject(LI1->getPointerOperand()) ==
                   getUnderlyingObject(LI2->getPointerOperand()) &&
               R.TTI->isLegalMaskedGather(
-                  FixedVectorType::get(LI1->getType(), NumLanes),
+                  makeScalarTyToVectorTy(LI1->getType(), NumLanes),
                   LI1->getAlign()))
             return LookAheadHeuristics::ScoreMaskedGatherCandidate;
           return LookAheadHeuristics::ScoreFail;
         }
         // The distance is too large - still may be profitable to use masked
         // loads/gathers.
         if (std::abs(*Dist) > NumLanes / 2)
           return LookAheadHeuristics::ScoreMaskedGatherCandidate;
if (ProfitableGatherPointers || all_of(PointerOps, [IsSorted](Value *P) {
           auto *GEP = dyn_cast<GetElementPtrInst>(P);
           return (IsSorted && !GEP && doesNotNeedToBeScheduled(P)) ||
                  (GEP && GEP->getNumOperands() == 2);
         })) {
       Align CommonAlignment = cast<LoadInst>(VL0)->getAlign();
       for (Value *V : VL)
         CommonAlignment =
             std::min(CommonAlignment, cast<LoadInst>(V)->getAlign());
-      auto *VecTy = FixedVectorType::get(ScalarTy, VL.size());
+      FixedVectorType *VecTy = makeScalarTyToVectorTy(ScalarTy, VL.size());
       if (TTI.isLegalMaskedGather(VecTy, CommonAlignment) &&
           !TTI.forceScalarizeMaskedGather(VecTy, CommonAlignment))
         return LoadsState::ScatterVectorize;
     }
   }
 
   return LoadsState::Gather;
 }
if (TE.State == TreeEntry::NeedToGather) {
     // might be transformed.
     int Sz = TE.Scalars.size();
     if (isSplat(TE.Scalars) && !allConstant(TE.Scalars) &&
         count_if(TE.Scalars, UndefValue::classof) == Sz - 1) {
       const auto *It =
           find_if(TE.Scalars, [](Value *V) { return !isConstant(V); });
       if (It == TE.Scalars.begin())
         return OrdersType();
-      auto *Ty = FixedVectorType::get(TE.Scalars.front()->getType(), Sz);
+      FixedVectorType *Ty =
+          makeScalarTyToVectorTy(TE.Scalars.front()->getType(), Sz);
       if (It != TE.Scalars.end()) {
         OrdersType Order(Sz, Sz);
         unsigned Idx = std::distance(TE.Scalars.begin(), It);
         Order[Idx] = 0;
         fixupOrderingIndices(Order);
         SmallVector<int> Mask;
         inversePermutation(Order, Mask);
         InstructionCost PermuteCost =
if (!ExternalUserReorderIndices.empty()) {
       ExternalUserReorderMap.try_emplace(TE.get(),
                                          std::move(ExternalUserReorderIndices));
     }
 
     // Patterns like [fadd,fsub] can be combined into a single instruction in
     // x86. Reordering them into [fsub,fadd] blocks this pattern. So we need
     // to take into account their order when looking for the most used order.
     if (TE->isAltShuffle()) {
-      VectorType *VecTy =
-          FixedVectorType::get(TE->Scalars[0]->getType(), TE->Scalars.size());
+      FixedVectorType *VecTy =
+          makeScalarTyToVectorTy(TE->Scalars[0]->getType(), TE->Scalars.size());
       unsigned Opcode0 = TE->getOpcode();
       unsigned Opcode1 = TE->getAltOpcode();
       // The opcode mask selects between the two opcodes.
       SmallBitVector OpcodeMask(TE->Scalars.size(), false);
       for (unsigned Lane : seq<unsigned>(0, TE->Scalars.size()))
         if (cast<Instruction>(TE->Scalars[Lane])->getOpcode() == Opcode1)
           OpcodeMask.set(Lane);
       // If this pattern is supported by the target then we consider the order.
if (auto *ST = dyn_cast<StructType>(EltTy)) {
       auto *VT = cast<FixedVectorType>(EltTy);
       N *= VT->getNumElements();
       EltTy = VT->getElementType();
     }
   }
 
   if (!isValidElementType(EltTy))
     return 0;
-  uint64_t VTSize = DL.getTypeStoreSizeInBits(FixedVectorType::get(EltTy, N));
+  uint64_t VTSize = DL.getTypeStoreSizeInBits(makeScalarTyToVectorTy(EltTy, N));
   if (VTSize < MinVecRegSize || VTSize > MaxVecRegSize || VTSize != DL.getTypeStoreSizeInBits(T))
     return 0;
   return N;
 }
 
 bool BoUpSLP::canReuseExtract(ArrayRef<Value *> VL, Value *OpValue,
                               SmallVectorImpl<unsigned> &CurrentOrder) const {
   const auto *It = find_if(VL, [](Value *V) {

 getVectorCallCosts(CallInst *CI, FixedVectorType *VecTy,
                    TargetTransformInfo *TTI, TargetLibraryInfo *TLI) {
   Intrinsic::ID ID = getVectorIntrinsicIDForCall(CI, TLI);
 
   // Calculate the cost of the scalar and vector calls.
   SmallVector<Type *, 4> VecTys;
   for (Use &Arg : CI->args())
     VecTys.push_back(
-        FixedVectorType::get(Arg->getType(), VecTy->getNumElements()));
+        makeScalarTyToVectorTy(Arg->getType(), VecTy->getNumElements()));
   FastMathFlags FMF;
   if (auto *FPCI = dyn_cast<FPMathOperator>(CI))
     FMF = FPCI->getFastMathFlags();
   SmallVector<const Value *> Arguments(CI->args());
   IntrinsicCostAttributes CostAttrs(ID, VecTy, Arguments, VecTys, FMF,
                                     dyn_cast<IntrinsicInst>(CI));
   auto IntrinsicCost =
     TTI->getIntrinsicInstrCost(CostAttrs, TTI::TCK_RecipThroughput);
class BoUpSLP::ShuffleCostEstimator : public BaseShuffleAnalysis {
   ArrayRef<Value *> VectorizedVals;
   BoUpSLP &R;
   SmallPtrSetImpl<Value *> &CheckedExtracts;
   constexpr static TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
 
   InstructionCost getBuildVectorCost(ArrayRef<Value *> VL, Value *Root) {
     if ((!Root && allConstant(VL)) || all_of(VL, UndefValue::classof))
       return TTI::TCC_Free;
-    auto *VecTy = FixedVectorType::get(VL.front()->getType(), VL.size());
+    FixedVectorType *VecTy =
+        makeScalarTyToVectorTy(VL.front()->getType(), VL.size());
     InstructionCost GatherCost = 0;
     SmallVector<Value *> Gathers(VL.begin(), VL.end());
     // Improve gather cost for gather of loads, if we can group some of the
     // loads into vector loads.
     InstructionsState S = getSameOpcode(VL, *R.TLI);
     if (VL.size() > 2 && S.getOpcode() == Instruction::Load &&
         !S.isAltShuffle() &&
         !all_of(Gathers, [&](Value *V) { return R.getTreeEntry(V); }) &&
if (VL.size() > 2 && S.getOpcode() == Instruction::Load &&
         InstructionCost ScalarsCost = 0;
         for (Value *V : VectorizedLoads) {
           auto *LI = cast<LoadInst>(V);
           ScalarsCost +=
               TTI.getMemoryOpCost(Instruction::Load, LI->getType(),
                                   LI->getAlign(), LI->getPointerAddressSpace(),
                                   CostKind, TTI::OperandValueInfo(), LI);
         }
-        auto *LoadTy = FixedVectorType::get(LI->getType(), VF);
+        FixedVectorType *LoadTy = makeScalarTyToVectorTy(LI->getType(), VF);
         Align Alignment = LI->getAlign();
         GatherCost +=
             VectorizedCnt *
             TTI.getMemoryOpCost(Instruction::Load, LoadTy, Alignment,
                                 LI->getPointerAddressSpace(), CostKind,
                                 TTI::OperandValueInfo(), LI);
         GatherCost += ScatterVectorizeCnt *
                       TTI.getGatherScatterOpCost(
return GatherCost +
                 ? TTI::TCC_Free
                 : R.getGatherCost(Gathers, !Root && VL.equals(Gathers)));
   };
 
   /// Compute the cost of creating a vector of type \p VecTy containing the
   /// extracted values from \p VL.
   InstructionCost computeExtractCost(ArrayRef<Value *> VL, ArrayRef<int> Mask,
                                      TTI::ShuffleKind ShuffleKind) {
-    auto *VecTy = FixedVectorType::get(VL.front()->getType(), VL.size());
+    FixedVectorType *VecTy =
+        makeScalarTyToVectorTy(VL.front()->getType(), VL.size());
     unsigned NumOfParts = TTI.getNumberOfParts(VecTy);
 
     if (ShuffleKind != TargetTransformInfo::SK_PermuteSingleSrc ||
         !NumOfParts || VecTy->getNumElements() < NumOfParts)
       return TTI.getShuffleCost(ShuffleKind, VecTy, Mask);
 
     bool AllConsecutive = true;
     unsigned EltsPerVector = VecTy->getNumElements() / NumOfParts;
for (auto *V : VL) {
       if ((Idx + 1) % EltsPerVector != 0 && Idx + 1 != VL.size())
         continue;
 
       // If we have a series of extracts which are not consecutive and hence
       // cannot re-use the source vector register directly, compute the shuffle
       // cost to extract the vector with EltsPerVector elements.
       Cost += TTI.getShuffleCost(
           TargetTransformInfo::SK_PermuteSingleSrc,
-          FixedVectorType::get(VecTy->getElementType(), EltsPerVector),
+          makeScalarTyToVectorTy(VecTy->getElementType(), EltsPerVector),
           RegMask);
     }
     return Cost;
   }
 
   class ShuffleCostBuilder {
     const TargetTransformInfo &TTI;
 
InstructionCost createShuffleVector(Value *V1, Value *,
                                         ArrayRef<int> Mask) const {
       // Empty mask or identity mask are free.
       unsigned VF =
           cast<VectorType>(V1->getType())->getElementCount().getKnownMinValue();
       if (isEmptyOrIdentity(Mask, VF))
         return TTI::TCC_Free;
       return TTI.getShuffleCost(
           TTI::SK_PermuteTwoSrc,
-          FixedVectorType::get(
+          makeScalarTyToVectorTy(
               cast<VectorType>(V1->getType())->getElementType(), Mask.size()),
           Mask);
     }
     InstructionCost createShuffleVector(Value *V1, ArrayRef<int> Mask) const {
       // Empty mask or identity mask are free.
       if (isEmptyOrIdentity(Mask, Mask.size()))
         return TTI::TCC_Free;
       return TTI.getShuffleCost(
           TTI::SK_PermuteSingleSrc,
-          FixedVectorType::get(
+          makeScalarTyToVectorTy(
               cast<VectorType>(V1->getType())->getElementType(), Mask.size()),
           Mask);
     }
     InstructionCost createIdentity(Value *) const { return TTI::TCC_Free; }
     InstructionCost createPoison(Type *Ty, unsigned VF) const {
       return TTI::TCC_Free;
     }
     void resizeToMatch(Value *&, Value *&) const {}
if (!V1) {
         if (V2VF != VF && V2VF == E->Scalars.size())
           VF = E->Scalars.size();
       } else if (!P2.isNull()) {
         const TreeEntry *E2 = P2.get<const TreeEntry *>();
         if (E->Scalars.size() == E2->Scalars.size())
           CommonVF = VF = E->Scalars.size();
       }
       V1 = Constant::getNullValue(
-          FixedVectorType::get(E->Scalars.front()->getType(), VF));
+          makeScalarTyToVectorTy(E->Scalars.front()->getType(), VF));
     }
     if (!V2 && !P2.isNull()) {
       const TreeEntry *E = P2.get<const TreeEntry *>();
       unsigned VF = E->getVectorFactor();
       unsigned V1VF = cast<FixedVectorType>(V1->getType())->getNumElements();
       if (!CommonVF && V1VF == E->Scalars.size())
         CommonVF = E->Scalars.size();
       if (CommonVF)
         VF = CommonVF;
       V2 = Constant::getNullValue(
-          FixedVectorType::get(E->Scalars.front()->getType(), VF));
+          makeScalarTyToVectorTy(E->Scalars.front()->getType(), VF));
     }
     return BaseShuffleAnalysis::createShuffle<InstructionCost>(V1, V2, Mask,
                                                                Builder);
   }
 
 public:
   ShuffleCostEstimator(TargetTransformInfo &TTI,
                        ArrayRef<Value *> VectorizedVals, BoUpSLP &R,
                        SmallPtrSetImpl<Value *> &CheckedExtracts)
       : TTI(TTI), VectorizedVals(VectorizedVals), R(R),
         CheckedExtracts(CheckedExtracts) {}
   Value *adjustExtracts(const TreeEntry *E, ArrayRef<int> Mask,
                         TTI::ShuffleKind ShuffleKind) {
     if (Mask.empty())
       return nullptr;
     Value *VecBase = nullptr;
     ArrayRef<Value *> VL = E->Scalars;
-    auto *VecTy = FixedVectorType::get(VL.front()->getType(), VL.size());
+    FixedVectorType *VecTy =
+        makeScalarTyToVectorTy(VL.front()->getType(), VL.size());
     // If the resulting type is scalarized, do not adjust the cost.
     unsigned VecNumParts = TTI.getNumberOfParts(VecTy);
     if (VecNumParts == VecTy->getNumElements())
       return nullptr;
     DenseMap<Value *, int> ExtractVectorsTys;
     for (auto [I, V] : enumerate(VL)) {
       // Ignore non-extractelement scalars.
       if (isa<UndefValue>(V) || (!Mask.empty() && Mask[I] == PoisonMaskElem))
for (const auto &Data : ExtractVectorsTys) {
           continue;
         if (Idx + NumElts <= EENumElts) {
           Cost += TTI.getShuffleCost(TargetTransformInfo::SK_ExtractSubvector,
                                      EEVTy, std::nullopt, CostKind, Idx, VecTy);
         } else {
           // Need to round up the subvector type vectorization factor to avoid a
           // crash in cost model functions. Make SubVT so that Idx + VF of SubVT
           // <= EENumElts.
-          auto *SubVT =
-              FixedVectorType::get(VecTy->getElementType(), EENumElts - Idx);
+          FixedVectorType *SubVT =
+              makeScalarTyToVectorTy(VecTy->getElementType(), EENumElts - Idx);
           Cost += TTI.getShuffleCost(TargetTransformInfo::SK_ExtractSubvector,
                                      EEVTy, std::nullopt, CostKind, Idx, SubVT);
         }
       } else {
         Cost += TTI.getShuffleCost(TargetTransformInfo::SK_InsertSubvector,
                                    VecTy, std::nullopt, CostKind, 0, EEVTy);
       }
     }
if (Action) {
       }
       for (unsigned Idx = 0, Sz = CommonMask.size(); Idx < Sz; ++Idx)
         if (CommonMask[Idx] != PoisonMaskElem)
           CommonMask[Idx] = Idx;
       assert(VF > 0 &&
              "Expected vector length for the final value before action.");
       Value *V = Vec.dyn_cast<Value *>();
       if (!Vec.isNull() && !V)
-        V = Constant::getNullValue(FixedVectorType::get(
+        V = Constant::getNullValue(makeScalarTyToVectorTy(
             Vec.get<const TreeEntry *>()->Scalars.front()->getType(),
             CommonMask.size()));
       Action(V, CommonMask);
     }
     ::addMask(CommonMask, ExtMask, /*ExtendingManyInputs=*/true);
     if (CommonMask.empty())
       return Cost;
     int Limit = CommonMask.size() * 2;
BoUpSLP::getEntryCost(const TreeEntry *E, ArrayRef<Value *> VectorizedVals,
 
   Type *ScalarTy = VL[0]->getType();
   if (auto *SI = dyn_cast<StoreInst>(VL[0]))
     ScalarTy = SI->getValueOperand()->getType();
   else if (auto *CI = dyn_cast<CmpInst>(VL[0]))
     ScalarTy = CI->getOperand(0)->getType();
   else if (auto *IE = dyn_cast<InsertElementInst>(VL[0]))
     ScalarTy = IE->getOperand(1)->getType();
-  auto *VecTy = FixedVectorType::get(ScalarTy, VL.size());
+  FixedVectorType *VecTy = makeScalarTyToVectorTy(ScalarTy, VL.size());
   TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
 
   // If we have computed a smaller type for the expression, update VecTy so
   // that the costs will be accurate.
   if (MinBWs.count(VL[0]))
-    VecTy = FixedVectorType::get(
+    VecTy = makeScalarTyToVectorTy(
         IntegerType::get(F->getContext(), MinBWs[VL[0]].first), VL.size());
   unsigned EntryVF = E->getVectorFactor();
-  auto *FinalVecTy = FixedVectorType::get(VecTy->getElementType(), EntryVF);
+  FixedVectorType *FinalVecTy =
+      makeScalarTyToVectorTy(VecTy->getElementType(), EntryVF);
 
   bool NeedToShuffleReuses = !E->ReuseShuffleIndices.empty();
   if (E->State == TreeEntry::NeedToGather) {
     if (allConstant(VL))
       return 0;
     if (isa<InsertElementInst>(VL[0]))
       return InstructionCost::getInvalid();
     ShuffleCostEstimator Estimator(*TTI, VectorizedVals, *this,
if (GatherShuffle) {
         Estimator.add(Entries.front(), Mask);
       else
         Estimator.add(Entries.front(), Entries.back(), Mask);
       if (all_of(GatheredScalars, PoisonValue ::classof))
         return Estimator.finalize(E->ReuseShuffleIndices);
       return Estimator.finalize(
           E->ReuseShuffleIndices, E->Scalars.size(),
           [&](Value *&Vec, SmallVectorImpl<int> &Mask) {
-            Vec = Estimator.gather(GatheredScalars,
-                                   Constant::getNullValue(FixedVectorType::get(
-                                       GatheredScalars.front()->getType(),
-                                       GatheredScalars.size())));
+            Vec = Estimator.gather(
+                GatheredScalars, Constant::getNullValue(makeScalarTyToVectorTy(
           });
     }
     if (!all_of(GatheredScalars, PoisonValue::classof)) {
       auto Gathers = ArrayRef(GatheredScalars).take_front(VL.size());
       bool SameGathers = VL.equals(Gathers);
       Value *BV = Estimator.gather(
           Gathers, SameGathers ? nullptr
-                               : Constant::getNullValue(FixedVectorType::get(
+                               : Constant::getNullValue(makeScalarTyToVectorTy(
                                      GatheredScalars.front()->getType(),
                                      GatheredScalars.size())));
       SmallVector<int> ReuseMask(Gathers.size(), PoisonMaskElem);
       std::iota(ReuseMask.begin(), ReuseMask.end(), 0);
       Estimator.add(BV, ReuseMask);
     }
     if (ExtractShuffle)
       Estimator.add(E, std::nullopt);
auto GetScalarCost = [=](unsigned Idx) {
       } else {
         auto *EV = cast<ExtractValueInst>(I);
         Type *AggregateTy = EV->getAggregateOperand()->getType();
         unsigned NumElts;
         if (auto *ATy = dyn_cast<ArrayType>(AggregateTy))
           NumElts = ATy->getNumElements();
         else
           NumElts = AggregateTy->getStructNumElements();
-        SrcVecTy = FixedVectorType::get(ScalarTy, NumElts);
+        SrcVecTy = makeScalarTyToVectorTy(ScalarTy, NumElts);
       }
       if (I->hasOneUse()) {
         Instruction *Ext = I->user_back();
         if ((isa<SExtInst>(Ext) || isa<ZExtInst>(Ext)) &&
             all_of(Ext->users(),
                    [](User *U) { return isa<GetElementPtrInst>(U); })) {
           // Use getExtractWithExtendCost() to calculate the cost of
           // extractelement/ext pair.
case Instruction::InsertElement: {
     Cost -= TTI->getScalarizationOverhead(SrcVecTy, DemandedElts,
                                           /*Insert*/ true, /*Extract*/ false,
                                           CostKind);
 
     // First cost - resize to actual vector size if not identity shuffle or
     // need to shift the vector.
     // Do not calculate the cost if the actual size is the register size and
     // we can merge this shuffle with the following SK_Select.
-    auto *InsertVecTy =
-        FixedVectorType::get(SrcVecTy->getElementType(), InsertVecSz);
+    FixedVectorType *InsertVecTy =
+        makeScalarTyToVectorTy(SrcVecTy->getElementType(), InsertVecSz);
     if (!IsIdentity)
       Cost += TTI->getShuffleCost(TargetTransformInfo::SK_PermuteSingleSrc,
                                   InsertVecTy, Mask);
     auto *FirstInsert = cast<Instruction>(*find_if(E->Scalars, [E](Value *V) {
       return !is_contained(E->Scalars, cast<Instruction>(V)->getOperand(0));
     }));
     // Second cost - permutation with subvector, if some elements are from the
     // initial vector or inserting a subvector.
     // TODO: Implement the analysis of the FirstInsert->getOperand(0)
     // subvector of ActualVecTy.
     SmallBitVector InMask =
         isUndefVector(FirstInsert->getOperand(0),
                       buildUseMask(NumElts, InsertMask, UseMask::UndefsAsMask));
     if (!InMask.all() && NumScalars != NumElts && !IsWholeSubvector) {
       if (InsertVecSz != VecSz) {
-        auto *ActualVecTy =
-            FixedVectorType::get(SrcVecTy->getElementType(), VecSz);
+        FixedVectorType *ActualVecTy =
+            makeScalarTyToVectorTy(SrcVecTy->getElementType(), VecSz);
         Cost += TTI->getShuffleCost(TTI::SK_InsertSubvector, ActualVecTy,
                                     std::nullopt, CostKind, OffsetBeg - Offset,
                                     InsertVecTy);
       } else {
         for (unsigned I = 0, End = OffsetBeg - Offset; I < End; ++I)
           Mask[I] = InMask.test(I) ? PoisonMaskElem : I;
         for (unsigned I = OffsetBeg - Offset, End = OffsetEnd - Offset;
              I <= End; ++I)
case Instruction::BitCast: {
     auto GetScalarCost = [=](unsigned Idx) {
       auto *VI = cast<Instruction>(VL[Idx]);
       return TTI->getCastInstrCost(E->getOpcode(), ScalarTy,
                                    VI->getOperand(0)->getType(),
                                    TTI::getCastContextHint(VI), CostKind, VI);
     };
     auto GetVectorCost = [=](InstructionCost CommonCost) {
       Type *SrcTy = VL0->getOperand(0)->getType();
-      auto *SrcVecTy = FixedVectorType::get(SrcTy, VL.size());
+      FixedVectorType *SrcVecTy = makeScalarTyToVectorTy(SrcTy, VL.size());
       InstructionCost VecCost = CommonCost;
       // Check if the values are candidates to demote.
       if (!MinBWs.count(VL0) || VecTy != SrcVecTy)
         VecCost +=
             TTI->getCastInstrCost(E->getOpcode(), VecTy, SrcVecTy,
                                   TTI::getCastContextHint(VL0), CostKind, VL0);
       return VecCost;
     };
auto GetScalarCost = [&](unsigned Idx) {
                                        ? CmpInst::BAD_FCMP_PREDICATE
                                        : CmpInst::BAD_ICMP_PREDICATE;
 
       return TTI->getCmpSelInstrCost(E->getOpcode(), ScalarTy,
                                      Builder.getInt1Ty(), CurrentPred, CostKind,
                                      VI);
     };
     auto GetVectorCost = [&](InstructionCost CommonCost) {
-      auto *MaskTy = FixedVectorType::get(Builder.getInt1Ty(), VL.size());
+      FixedVectorType *MaskTy =
+          makeScalarTyToVectorTy(Builder.getInt1Ty(), VL.size());
 
       InstructionCost VecCost = TTI->getCmpSelInstrCost(
           E->getOpcode(), VecTy, MaskTy, VecPred, CostKind, VL0);
       // Check if it is possible and profitable to use min/max for selects
       // in VL.
       //
       auto IntrinsicAndUse = canConvertToMinOrMaxIntrinsic(VL);
       if (IntrinsicAndUse.first != Intrinsic::not_intrinsic) {
auto GetVectorCost = [&](InstructionCost) {
         });
         // No need to add new vector costs here since we're going to reuse
         // same main/alternate vector ops, just do different shuffling.
       } else if (Instruction::isBinaryOp(E->getOpcode())) {
         VecCost = TTI->getArithmeticInstrCost(E->getOpcode(), VecTy, CostKind);
         VecCost +=
             TTI->getArithmeticInstrCost(E->getAltOpcode(), VecTy, CostKind);
       } else if (auto *CI0 = dyn_cast<CmpInst>(VL0)) {
-        auto *MaskTy = FixedVectorType::get(Builder.getInt1Ty(), VL.size());
+        FixedVectorType *MaskTy =
+            makeScalarTyToVectorTy(Builder.getInt1Ty(), VL.size());
         VecCost = TTI->getCmpSelInstrCost(E->getOpcode(), VecTy, MaskTy,
                                           CI0->getPredicate(), CostKind, VL0);
         VecCost += TTI->getCmpSelInstrCost(
             E->getOpcode(), VecTy, MaskTy,
             cast<CmpInst>(E->getAltOp())->getPredicate(), CostKind,
             E->getAltOp());
       } else {
         Type *Src0SclTy = E->getMainOp()->getOperand(0)->getType();
         Type *Src1SclTy = E->getAltOp()->getOperand(0)->getType();
-        auto *Src0Ty = FixedVectorType::get(Src0SclTy, VL.size());
-        auto *Src1Ty = FixedVectorType::get(Src1SclTy, VL.size());
+        FixedVectorType *Src0Ty = makeScalarTyToVectorTy(Src0SclTy, VL.size());
+        FixedVectorType *Src1Ty = makeScalarTyToVectorTy(Src1SclTy, VL.size());
         VecCost = TTI->getCastInstrCost(E->getOpcode(), VecTy, Src0Ty,
                                         TTI::CastContextHint::None, CostKind);
         VecCost += TTI->getCastInstrCost(E->getAltOpcode(), VecTy, Src1Ty,
                                          TTI::CastContextHint::None, CostKind);
       }
       if (E->ReuseShuffleIndices.empty()) {
         VecCost +=
             TTI->getShuffleCost(TargetTransformInfo::SK_Select, FinalVecTy);
for (Instruction *Inst : OrderedScalars) {
     }
 
     if (NumCalls) {
       SmallVector<Type *, 4> V;
       for (auto *II : LiveValues) {
         auto *ScalarTy = II->getType();
         if (auto *VectorTy = dyn_cast<FixedVectorType>(ScalarTy))
           ScalarTy = VectorTy->getElementType();
-        V.push_back(FixedVectorType::get(ScalarTy, BundleWidth));
+        V.push_back(makeScalarTyToVectorTy(ScalarTy, BundleWidth));
       }
       Cost += NumCalls * TTI->getCostOfKeepingLiveOverCall(V);
     }
 
     PrevInst = Inst;
   }
 
   return Cost;
if (auto *VU = dyn_cast_or_null<InsertElementInst>(EU.User)) {
           continue;
         }
       }
     }
 
     // If we plan to rewrite the tree in a smaller type, we will need to sign
     // extend the extracted value back to the original type. Here, we account
     // for the extract and the added cost of the sign extend if needed.
-    auto *VecTy = FixedVectorType::get(EU.Scalar->getType(), BundleWidth);
+    FixedVectorType *VecTy =
+        makeScalarTyToVectorTy(EU.Scalar->getType(), BundleWidth);
     TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
     auto *ScalarRoot = VectorizableTree[0]->Scalars[0];
     if (MinBWs.count(ScalarRoot)) {
       auto *MinTy = IntegerType::get(F->getContext(), MinBWs[ScalarRoot].first);
       auto Extend =
           MinBWs[ScalarRoot].second ? Instruction::SExt : Instruction::ZExt;
-      VecTy = FixedVectorType::get(MinTy, BundleWidth);
+      VecTy = makeScalarTyToVectorTy(MinTy, BundleWidth);
       ExtractCost += TTI->getExtractWithExtendCost(Extend, EU.Scalar->getType(),
                                                    VecTy, EU.Lane);
     } else {
       ExtractCost += TTI->getVectorInstrCost(Instruction::ExtractElement, VecTy,
                                              CostKind, EU.Lane);
     }
   }
 
if (VF != VecVF &&
          (all_of(Mask,
                  [VF](int Idx) { return Idx < 2 * static_cast<int>(VF); }) &&
           !ShuffleVectorInst::isIdentityMask(Mask)))) {
       SmallVector<int> OrigMask(VecVF, PoisonMaskElem);
       std::copy(Mask.begin(), std::next(Mask.begin(), std::min(VF, VecVF)),
                 OrigMask.begin());
       C = TTI->getShuffleCost(
           TTI::SK_PermuteSingleSrc,
-          FixedVectorType::get(TE->getMainOp()->getType(), VecVF), OrigMask);
+          makeScalarTyToVectorTy(TE->getMainOp()->getType(), VecVF), OrigMask);
       LLVM_DEBUG(
           dbgs() << "SLP: Adding cost " << C
                  << " for final shuffle of insertelement external users.\n";
           TE->dump(); dbgs() << "SLP: Current total cost = " << Cost << "\n");
       Cost += C;
       return std::make_pair(TE, true);
     }
     return std::make_pair(TE, false);
   };
   // Calculate the cost of the reshuffled vectors, if any.
   for (int I = 0, E = FirstUsers.size(); I < E; ++I) {
     Value *Base = cast<Instruction>(FirstUsers[I].first)->getOperand(0);
     unsigned VF = ShuffleMasks[I].begin()->second.size();
-    auto *FTy = FixedVectorType::get(
+    FixedVectorType *FTy = makeScalarTyToVectorTy(
         cast<VectorType>(FirstUsers[I].first->getType())->getElementType(), VF);
     auto Vector = ShuffleMasks[I].takeVector();
     auto &&EstimateShufflesCost = [this, FTy,
                                    &Cost](ArrayRef<int> Mask,
                                           ArrayRef<const TreeEntry *> TEs) {
       assert((TEs.size() == 1 || TEs.size() == 2) &&
              "Expected exactly 1 or 2 tree entries.");
       if (TEs.size() == 1) {

 }
 
 InstructionCost BoUpSLP::getGatherCost(ArrayRef<Value *> VL,
                                        bool ForPoisonSrc) const {
   // Find the type of the operands in VL.
   Type *ScalarTy = VL[0]->getType();
   if (StoreInst *SI = dyn_cast<StoreInst>(VL[0]))
     ScalarTy = SI->getValueOperand()->getType();
-  auto *VecTy = FixedVectorType::get(ScalarTy, VL.size());
+  FixedVectorType *VecTy = makeScalarTyToVectorTy(ScalarTy, VL.size());
   bool DuplicateNonConst = false;
   // Find the cost of inserting/extracting values from the vector.
   // Check if the same elements are inserted several times and count them as
   // shuffle candidates.
   APInt ShuffledElements = APInt::getZero(VL.size());
   DenseSet<Value *> UniqueElements;
   constexpr TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
   InstructionCost Cost;
if (TreeEntry *Entry = getTreeEntry(V)) {
       // Find which lane we need to extract.
       unsigned FoundLane = Entry->findLaneForValue(V);
       ExternalUses.emplace_back(V, InsElt, FoundLane);
     }
     return Vec;
   };
   Value *Val0 =
       isa<StoreInst>(VL[0]) ? cast<StoreInst>(VL[0])->getValueOperand() : VL[0];
-  FixedVectorType *VecTy = FixedVectorType::get(Val0->getType(), VL.size());
+  FixedVectorType *VecTy = makeScalarTyToVectorTy(Val0->getType(), VL.size());
   Value *Vec = Root ? Root : PoisonValue::get(VecTy);
   SmallVector<int> NonConsts;
   // Insert constant values at first.
   for (int I = 0, E = VL.size(); I < E; ++I) {
     if (PostponedIndices.contains(I))
       continue;
     if (!isConstant(VL[I])) {
       NonConsts.push_back(I);
Value *createShuffleVector(Value *V1, ArrayRef<int> Mask) {
       if (auto *I = dyn_cast<Instruction>(Vec)) {
         GatherShuffleExtractSeq.insert(I);
         CSEBlocks.insert(I->getParent());
       }
       return Vec;
     }
     Value *createIdentity(Value *V) { return V; }
     Value *createPoison(Type *Ty, unsigned VF) {
-      return PoisonValue::get(FixedVectorType::get(Ty, VF));
+      return PoisonValue::get(makeScalarTyToVectorTy(Ty, VF));
     }
     /// Resizes 2 input vector to match the sizes, if the they are not equal
     /// yet. The smallest vector is resized to the size of the larger vector.
     void resizeToMatch(Value *&V1, Value *&V2) {
       if (V1->getType() == V2->getType())
         return;
       int V1VF = cast<FixedVectorType>(V1->getType())->getNumElements();
       int V2VF = cast<FixedVectorType>(V2->getType())->getNumElements();
public:
   /// Checks if the specified entry \p E needs to be delayed because of its
   /// dependency nodes.
   Value *needToDelay(const TreeEntry *E, ArrayRef<const TreeEntry *> Deps) {
     // No need to delay emission if all deps are ready.
     if (all_of(Deps, [](const TreeEntry *TE) { return TE->VectorizedValue; }))
       return nullptr;
     // Postpone gather emission, will be emitted after the end of the
     // process to keep correct order.
-    auto *VecTy = FixedVectorType::get(E->Scalars.front()->getType(),
-                                       E->getVectorFactor());
+    FixedVectorType *VecTy = makeScalarTyToVectorTy(
+        E->Scalars.front()->getType(), E->getVectorFactor());
     return Builder.CreateAlignedLoad(
         VecTy, PoisonValue::get(PointerType::getUnqual(VecTy->getContext())),
         MaybeAlign());
   }
   /// Adds 2 input vectors and the mask for their shuffling.
   void add(Value *V1, Value *V2, ArrayRef<int> Mask) {
     assert(V1 && V2 && !Mask.empty() && "Expected non-empty input vectors.");
     if (InVectors.empty()) {
if (ExtractShuffle) {
         IsNonPoisoned &=
             isGuaranteedNotToBePoison(Vec1) && isGuaranteedNotToBePoison(Vec2);
         ShuffleBuilder.add(Vec1, Vec2, ExtractMask);
       } else if (Vec1) {
         IsUsedInExpr = FindReusedSplat(ExtractMask);
         ShuffleBuilder.add(Vec1, ExtractMask);
         IsNonPoisoned &= isGuaranteedNotToBePoison(Vec1);
       } else {
-        ShuffleBuilder.add(PoisonValue::get(FixedVectorType::get(
+        ShuffleBuilder.add(PoisonValue::get(makeScalarTyToVectorTy(
                                ScalarTy, GatheredScalars.size())),
                            ExtractMask);
       }
     }
     if (GatherShuffle) {
       if (Entries.size() == 1) {
         IsUsedInExpr = FindReusedSplat(Mask);
         ShuffleBuilder.add(Entries.front()->VectorizedValue, Mask);
Value *BoUpSLP::vectorizeTree(TreeEntry *E) {
   unsigned ShuffleOrOp =
       E->isAltShuffle() ? (unsigned)Instruction::ShuffleVector : E->getOpcode();
   Instruction *VL0 = E->getMainOp();
   Type *ScalarTy = VL0->getType();
   if (auto *Store = dyn_cast<StoreInst>(VL0))
     ScalarTy = Store->getValueOperand()->getType();
   else if (auto *IE = dyn_cast<InsertElementInst>(VL0))
     ScalarTy = IE->getOperand(1)->getType();
-  auto *VecTy = FixedVectorType::get(ScalarTy, E->Scalars.size());
+  FixedVectorType *VecTy = makeScalarTyToVectorTy(ScalarTy, E->Scalars.size());
   switch (ShuffleOrOp) {
     case Instruction::PHI: {
       assert((E->ReorderIndices.empty() ||
               E != VectorizableTree.front().get() ||
               !E->UserTreeIndices.empty()) &&
              "PHI reordering is free.");
       auto *PH = cast<PHINode>(VL0);
       Builder.SetInsertPoint(PH->getParent()->getFirstNonPHI());
case Instruction::Call: {
                           VecCallCosts.first <= VecCallCosts.second;
 
       Value *ScalarArg = nullptr;
       std::vector<Value *> OpVecs;
       SmallVector<Type *, 2> TysForDecl;
       // Add return type if intrinsic is overloaded on it.
       if (isVectorIntrinsicWithOverloadTypeAtArg(IID, -1))
         TysForDecl.push_back(
-            FixedVectorType::get(CI->getType(), E->Scalars.size()));
+            makeScalarTyToVectorTy(CI->getType(), E->Scalars.size()));
       for (int j = 0, e = CI->arg_size(); j < e; ++j) {
         ValueList OpVL;
         // Some intrinsics have scalar arguments. This argument should not be
         // vectorized.
         if (UseIntrinsic && isVectorIntrinsicWithScalarOpAtArg(IID, j)) {
           CallInst *CEI = cast<CallInst>(VL0);
           ScalarArg = CEI->getArgOperand(j);
           OpVecs.push_back(CEI->getArgOperand(j));
if (auto *I = dyn_cast<Instruction>(VectorRoot)) {
       // last phi node.
       if (isa<PHINode>(I))
         Builder.SetInsertPoint(&*I->getParent()->getFirstInsertionPt());
       else
         Builder.SetInsertPoint(&*++BasicBlock::iterator(I));
     }
     auto BundleWidth = VectorizableTree[0]->Scalars.size();
     auto *MinTy = IntegerType::get(F->getContext(), MinBWs[ScalarRoot].first);
-    auto *VecTy = FixedVectorType::get(MinTy, BundleWidth);
+    FixedVectorType *VecTy = makeScalarTyToVectorTy(MinTy, BundleWidth);
     auto *Trunc = Builder.CreateTrunc(VectorRoot, VecTy);
     VectorizableTree[0]->VectorizedValue = Trunc;
   }
 
   LLVM_DEBUG(dbgs() << "SLP: Extracting " << ExternalUses.size()
                     << " values .\n");
 
   SmallVector<ShuffledInsertData> ShuffledInserts;
for (int I = 0, E = NewMask.size(); I < E; ++I) {
       if (NewMask[I] == PoisonMaskElem)
         NewMask[I] = SM1[I];
     }
     // Check if the last undefs actually change the final number of used vector
     // registers.
     return SM1.size() - LastUndefsCnt > 1 &&
            TTI->getNumberOfParts(SI1->getType()) ==
                TTI->getNumberOfParts(
-                   FixedVectorType::get(SI1->getType()->getElementType(),
+                   makeScalarTyToVectorTy(SI1->getType()->getElementType(),
                                           SM1.size() - LastUndefsCnt));
   };
   // Perform O(N^2) search over the gather/shuffle sequences and merge identical
   // instructions. TODO: We can further optimize this scan if we split the
   // instructions into different buckets based on the insert lane.
   SmallVector<Instruction *, 16> Visited;
   for (auto I = CSEWorkList.begin(), E = CSEWorkList.end(); I != E; ++I) {
     assert(*I &&
            (I == CSEWorkList.begin() || !DT->dominates(*I, *std::prev(I))) &&
bool SLPVectorizerPass::tryToVectorizeList(ArrayRef<Value *> VL, BoUpSLP &R,
   if (auto *IE = dyn_cast<InsertElementInst>(VL[0]))
     ScalarTy = IE->getOperand(1)->getType();
 
   unsigned NextInst = 0, MaxInst = VL.size();
   for (unsigned VF = MaxVF; NextInst + 1 < MaxInst && VF >= MinVF; VF /= 2) {
     // No actual vectorization should happen, if number of parts is the same as
     // provided vectorization factor (i.e. the scalar type is used for vector
     // code during codegen).
-    auto *VecTy = FixedVectorType::get(ScalarTy, VF);
+    FixedVectorType *VecTy = makeScalarTyToVectorTy(ScalarTy, VF);
     if (TTI->getNumberOfParts(VecTy) == VF)
       continue;
     for (unsigned I = NextInst; I < MaxInst; ++I) {
       unsigned ActualVF = std::min(MaxInst - I, VF);
 
       if (!isPowerOf2_32(ActualVF))
         continue;
 
private:
   /// Calculate the cost of a reduction.
   InstructionCost getReductionCost(TargetTransformInfo *TTI,
                                    ArrayRef<Value *> ReducedVals,
                                    bool IsCmpSelMinMax, unsigned ReduxWidth,
                                    FastMathFlags FMF) {
     TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;
     Value *FirstReducedVal = ReducedVals.front();
     Type *ScalarTy = FirstReducedVal->getType();
-    FixedVectorType *VectorTy = FixedVectorType::get(ScalarTy, ReduxWidth);
+    FixedVectorType *VectorTy = makeScalarTyToVectorTy(ScalarTy, ReduxWidth);
     InstructionCost VectorCost = 0, ScalarCost;
     // If all of the reduced values are constant, the vector cost is 0, since
     // the reduction value can be calculated at the compile time.
     bool AllConsts = allConstant(ReducedVals);
     auto EvaluateScalarCost = [&](function_ref<InstructionCost()> GenCostFn) {
       InstructionCost Cost = 0;
       // Scalar cost is repeated for N-1 elements.
       int Cnt = ReducedVals.size();