[RISCV] Use materialization cost when lowering constant build_vector

[luke]

Summary

[RISCV] Use materialization cost when lowering constant build_vector

When lowering a constant build_vector, we currently match the following
patterns in this order:

1. Splats
2. vid sequences
3. <= 32 bit scalar vmv.s.x
4. Hidden splats
5. Dominant values

So if a build_vector could be expressed by both a vid sequence and a vmv.s.x,
then it will always lower it to a vid. However, the vmv.s.x might be a more
profitable lowering if the constants are cheap to materialize, e.g.:

<4 x i8> <i8 1, i8 3, i8 5, i8 7>

Could be lowered as:

vsetivli zero, 4, e8, mf4, ta, ma
vid.v v8
vadd.vv v8, v8, v8
vadd.vi v8, v8, 1

Or as 3 instructions with:

vsetivli zero, 4, e32, m1, ta, ma
lui a0, 28752
vmv.s.x v8, a0

This patch computes the (rough) cost for vid sequences, scalar vmv.s.x and
hidden splat lowerings, then chooses the cheapest lowering (I've left out
dominant values for now).
An arbitrary maximum cost has been chosen for now, but it should be replaced
with an approximate constant pool load cost.
The main result of this is that we get some more vmv.s.xs where there would be
vids.

[reames]

Please separate the removal of the immediate restriction on the VID into it's own change, either before or after.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

3294

Copying everything by value here to avoid scope issues is rather subtle. I don't spot a scope bug here, but I'm not a huge fan of the code structure.

3361

This is off when we can use a vmv.v.i to perform the insert.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-shuffles.ll

195

Not this change, but there's something odd here. Unless I'm misreading this, we should be doing a splat of 2.0 as double here. That's the hex value 0x4000000000000000 which is a constant we can materialize in two instructions at worst. (1 with zbs).

It looks maybe we've got a problem with how we lower constant splats of floating point types? We should be able to use the integer mat path, and it looks like we're not doing so?

[luke]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

3294

Not a fan either. I admittedly spent some time trying to track down a miscompile due to UB, where I had originally copied some stack variables by reference.

Two possible alternatives:

• Explicitly specify the list of captures
• Create classes for each type of lowering and move DAG logic & cost logic into them

[luke]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

3361

I think I'm missing something, I can't find anywhere where the insert_vector_elt below would get lowered or combined to a vmv.v.i. There doesn't seem to be any vmv.v.i's introduced in the test diff in https://reviews.llvm.org/D157299 either?

[luke]

Split out relxation on vid sequence. (Still need to think about code structure but haven't
come up with any better ideas yet)

[luke]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-shuffles.ll

195

Took a look at this, this seems to be working as intended, it uses the integer mat path only if the cost to materialise the integer < 2. I.e. adding zbs gives:

vrgather_shuffle_vx_v4f64:              # @vrgather_shuffle_vx_v4f64
	.cfi_startproc
# %bb.0:
	vsetivli	zero, 4, e64, m2, ta, ma
	vid.v	v10
	li	a0, 3
	vmul.vx	v12, v10, a0
	bseti	a0, zero, 62
	vmv.v.x	v10, a0
	vsetivli	zero, 1, e8, mf8, ta, ma
	vmv.v.i	v0, 3
	vsetivli	zero, 4, e64, m2, ta, mu
	vrgather.vv	v10, v8, v12, v0.t
	vmv.v.v	v8, v10
	ret

or alternatively, increasing -riscv-lower-fp-imm-cost from the default 2 to 3:

vrgather_shuffle_vx_v4f64:              # @vrgather_shuffle_vx_v4f64
	.cfi_startproc
# %bb.0:
	vsetivli	zero, 4, e64, m2, ta, ma
	vid.v	v10
	li	a0, 3
	vmul.vx	v12, v10, a0
	li	a0, 1
	slli	a0, a0, 62
	vmv.v.x	v10, a0
	vsetivli	zero, 1, e8, mf8, ta, ma
	vmv.v.i	v0, 3
	vsetivli	zero, 4, e64, m2, ta, mu
	vrgather.vv	v10, v8, v12, v0.t
	vmv.v.v	v8, v10
	ret

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

 static SDValue lowerBuildVectorOfConstants(SDValue Op, SelectionDAG &DAG,
                                            const RISCVSubtarget &Subtarget) {
   MVT VT = Op.getSimpleValueType();
   assert(VT.isFixedLengthVector() && "Unexpected vector!");
 
   MVT ContainerVT = getContainerForFixedLengthVector(DAG, VT, Subtarget);
 
   SDLoc DL(Op);
-  auto [Mask, VL] = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget);
+  // TODO: Need to manually binding Mask and VL because they're captured in
+  // lambdas below. Use structured binding if/when we move to C++20.
+  auto DefVLOps = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget);
+  SDValue Mask = DefVLOps.first, VL = DefVLOps.second;
 
   MVT XLenVT = Subtarget.getXLenVT();
   unsigned NumElts = Op.getNumOperands();
 
   if (VT.getVectorElementType() == MVT::i1) {
     if (ISD::isBuildVectorAllZeros(Op.getNode())) {
       SDValue VMClr = DAG.getNode(RISCVISD::VMCLR_VL, DL, ContainerVT, VL);
       return convertFromScalableVector(VT, VMClr, DAG, Subtarget);
static SDValue lowerBuildVectorOfConstants(SDValue Op, SelectionDAG &DAG,
   if (SDValue Splat = cast<BuildVectorSDNode>(Op)->getSplatValue()) {
     unsigned Opc = VT.isFloatingPoint() ? RISCVISD::VFMV_V_F_VL
                                         : RISCVISD::VMV_V_X_VL;
     Splat =
         DAG.getNode(Opc, DL, ContainerVT, DAG.getUNDEF(ContainerVT), Splat, VL);
     return convertFromScalableVector(VT, Splat, DAG, Subtarget);
   }
 
+  std::function<SDValue()> CheapestLowering;
+  // TODO: Substitute this with the cost of a constant pool load.
+  const unsigned MaxCost = 4;
+  auto AddLowering = [&CheapestLowering,
+                      CurCost = -1u](unsigned Cost,
+                                     std::function<SDValue()> Lowering) mutable {
+    if (Cost > MaxCost)
+      return;
+    if (!CheapestLowering || Cost < CurCost) {
+      CheapestLowering = Lowering;
+      CurCost = Cost;
+    }
+  };
+
+  const unsigned EltBitSize = VT.getScalarSizeInBits();
+
   // Try and match index sequences, which we can lower to the vid instruction
   // with optional modifications. An all-undef vector is matched by
   // getSplatValue, above.
   if (auto SimpleVID = isSimpleVIDSequence(Op)) {
     int64_t StepNumerator = SimpleVID->StepNumerator;
     unsigned StepDenominator = SimpleVID->StepDenominator;
     int64_t Addend = SimpleVID->Addend;
 
     assert(StepNumerator != 0 && "Invalid step");
     bool Negate = false;
     int64_t SplatStepVal = StepNumerator;
     unsigned StepOpcode = ISD::MUL;
     if (StepNumerator != 1) {
       if (isPowerOf2_64(std::abs(StepNumerator))) {
         Negate = StepNumerator < 0;
         StepOpcode = ISD::SHL;
         SplatStepVal = Log2_64(std::abs(StepNumerator));
       }
     }
 
+    unsigned Cost = 1; // Base cost of 1 for vid
+    if (Addend || Negate) {
+      Cost++;
+      // Add the constant materialization cost if it won't fit into vadd.vi
+      if (!isInt<5>(Addend))
+        Cost += RISCVMatInt::getIntMatCost(
+            APInt(64, Addend), EltBitSize, Subtarget.getFeatureBits());
+    }
+    if (StepOpcode == ISD::MUL && SplatStepVal != 1) {
+      Cost++;
+      // There's no vmul.vi so always include the materialization cost.
+      Cost +=
+          RISCVMatInt::getIntMatCost(APInt(64, SplatStepVal),
+                                     EltBitSize, Subtarget.getFeatureBits());
+    }
+    if (StepOpcode == ISD::SHL && SplatStepVal != 0) {
+      Cost++;
+      // Add the constant materialization cost if it won't fit into vsll.vi.
+      if (!isUInt<5>(SplatStepVal))
+        Cost +=
+            RISCVMatInt::getIntMatCost(APInt(64, SplatStepVal),
+                                       EltBitSize, Subtarget.getFeatureBits());
+    }
+    // May have to emit a vfwcvt.
+    if (VT.isFloatingPoint())
+      Cost++;
+
     // Only emit VIDs with suitably-small steps/addends. We use imm5 is a
     // threshold since it's the immediate value many RVV instructions accept.
     // There is no vmul.vi instruction so ensure multiply constant can fit in
     // a single addi instruction.
-    if (((StepOpcode == ISD::MUL && isInt<12>(SplatStepVal)) ||
-         (StepOpcode == ISD::SHL && isUInt<5>(SplatStepVal))) &&
-        isPowerOf2_32(StepDenominator) &&
-        (SplatStepVal >= 0 || StepDenominator == 1) && isInt<5>(Addend)) {
+    if (isPowerOf2_32(StepDenominator) && (SplatStepVal >= 0 || StepDenominator == 1)) {
+      AddLowering(Cost, [=, &DAG, &Subtarget]() {
         MVT VIDVT =
             VT.isFloatingPoint() ? VT.changeVectorElementTypeToInteger() : VT;
         MVT VIDContainerVT =
             getContainerForFixedLengthVector(DAG, VIDVT, Subtarget);
-      SDValue VID = DAG.getNode(RISCVISD::VID_VL, DL, VIDContainerVT, Mask, VL);
+        SDValue VID =
+            DAG.getNode(RISCVISD::VID_VL, DL, VIDContainerVT, Mask, VL);
         // Convert right out of the scalable type so we can use standard ISD
         // nodes for the rest of the computation. If we used scalable types with
         // these, we'd lose the fixed-length vector info and generate worse
         // vsetvli code.
         VID = convertFromScalableVector(VIDVT, VID, DAG, Subtarget);
         if ((StepOpcode == ISD::MUL && SplatStepVal != 1) ||
             (StepOpcode == ISD::SHL && SplatStepVal != 0)) {
           SDValue SplatStep = DAG.getSplatBuildVector(
               VIDVT, DL, DAG.getConstant(SplatStepVal, DL, XLenVT));
           VID = DAG.getNode(StepOpcode, DL, VIDVT, VID, SplatStep);
         }
         if (StepDenominator != 1) {
           SDValue SplatStep = DAG.getSplatBuildVector(
               VIDVT, DL, DAG.getConstant(Log2_64(StepDenominator), DL, XLenVT));
           VID = DAG.getNode(ISD::SRL, DL, VIDVT, VID, SplatStep);
         }
         if (Addend != 0 || Negate) {
           SDValue SplatAddend = DAG.getSplatBuildVector(
               VIDVT, DL, DAG.getConstant(Addend, DL, XLenVT));
-        VID = DAG.getNode(Negate ? ISD::SUB : ISD::ADD, DL, VIDVT, SplatAddend,
-                          VID);
+          VID = DAG.getNode(Negate ? ISD::SUB : ISD::ADD, DL, VIDVT,
+                            SplatAddend, VID);
         }
         if (VT.isFloatingPoint()) {
           // TODO: Use vfwcvt to reduce register pressure.
           VID = DAG.getNode(ISD::SINT_TO_FP, DL, VT, VID);
         }
         return VID;
+      });
     }
   }
 
   // For very small build_vectors, use a single scalar insert of a constant.
   // TODO: Base this on constant rematerialization cost, not size.
-  const unsigned EltBitSize = VT.getScalarSizeInBits();
   if (VT.getSizeInBits() <= 32 &&
       ISD::isBuildVectorOfConstantSDNodes(Op.getNode())) {
     MVT ViaIntVT = MVT::getIntegerVT(VT.getSizeInBits());
     assert((ViaIntVT == MVT::i16 || ViaIntVT == MVT::i32) &&
            "Unexpected sequence type");
     // If we can use the original VL with the modified element type, this
     // means we only have a VTYPE toggle, not a VL toggle.  TODO: Should this
     // be moved into InsertVSETVLI?
     unsigned ViaVecLen =
       (Subtarget.getRealMinVLen() >= VT.getSizeInBits() * NumElts) ? NumElts : 1;
     MVT ViaVecVT = MVT::getVectorVT(ViaIntVT, ViaVecLen);
 
     uint64_t EltMask = maskTrailingOnes<uint64_t>(EltBitSize);
     uint64_t SplatValue = 0;
     // Construct the amalgamated value at this larger vector type.
     for (const auto &OpIdx : enumerate(Op->op_values())) {
       const auto &SeqV = OpIdx.value();
       if (!SeqV.isUndef())
         SplatValue |= ((cast<ConstantSDNode>(SeqV)->getZExtValue() & EltMask)
                         << (OpIdx.index() * EltBitSize));
     }
 
     // On RV64, sign-extend from 32 to 64 bits where possible in order to
     // achieve better constant materializion.
     if (Subtarget.is64Bit() && ViaIntVT == MVT::i32)
       SplatValue = SignExtend64<32>(SplatValue);
 
+    // Base cost of 1 for vmv.s.x.
+    unsigned Cost = 1;
+    // We always have to materialize the constant since there's no vmv.s.i.
+    Cost += RISCVMatInt::getIntMatCost(APInt(64, SplatValue), EltBitSize, Subtarget.getFeatureBits());
+
+    AddLowering(Cost, [=, &DAG]() {
       SDValue Vec = DAG.getNode(ISD::INSERT_VECTOR_ELT, DL, ViaVecVT,
                                 DAG.getUNDEF(ViaVecVT),
                                 DAG.getConstant(SplatValue, DL, XLenVT),
                                 DAG.getConstant(0, DL, XLenVT));
       if (ViaVecLen != 1)
         Vec = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL,
                           MVT::getVectorVT(ViaIntVT, 1), Vec,
                           DAG.getConstant(0, DL, XLenVT));
       return DAG.getBitcast(VT, Vec);
+    });
   }
 
 
   // Attempt to detect "hidden" splats, which only reveal themselves as splats
   // when re-interpreted as a vector with a larger element type. For example,
   //   v4i16 = build_vector i16 0, i16 1, i16 0, i16 1
   // could be instead splat as
   //   v2i32 = build_vector i32 0x00010000, i32 0x00010000
if (VT.isInteger() && EltBitSize < 64 &&
 
     // Since we can't introduce illegal i64 types at this stage, we can only
     // perform an i64 splat on RV32 if it is its own sign-extended value. That
     // way we can use RVV instructions to splat.
     assert((ViaIntVT.bitsLE(XLenVT) ||
             (!Subtarget.is64Bit() && ViaIntVT == MVT::i64)) &&
            "Unexpected bitcast sequence");
     if (ViaIntVT.bitsLE(XLenVT) || isInt<32>(SplatValue)) {
+      // Base cost of 1 for vmv.v.x
+      unsigned Cost = 1;
+      if (!isInt<5>(SplatValue))
+  Cost += RISCVMatInt::getIntMatCost(APInt(64, SplatValue), EltBitSize, Subtarget.getFeatureBits());
+
+      AddLowering(Cost, [=, &DAG, &Subtarget]() {
         SDValue ViaVL =
             DAG.getConstant(ViaVecVT.getVectorNumElements(), DL, XLenVT);
         MVT ViaContainerVT =
             getContainerForFixedLengthVector(DAG, ViaVecVT, Subtarget);
         SDValue Splat =
             DAG.getNode(RISCVISD::VMV_V_X_VL, DL, ViaContainerVT,
                         DAG.getUNDEF(ViaContainerVT),
                         DAG.getConstant(SplatValue, DL, XLenVT), ViaVL);
         Splat = convertFromScalableVector(ViaVecVT, Splat, DAG, Subtarget);
         return DAG.getBitcast(VT, Splat);
+      });
     }
   }
 
+  if (CheapestLowering)
+    return CheapestLowering();
+
   if (SDValue Res = lowerBuildVectorViaDominantValues(Op, DAG, Subtarget))
     return Res;
 
   // For constant vectors, use generic constant pool lowering.  Otherwise,
   // we'd have to materialize constants in GPRs just to move them into the
   // vector.
   return SDValue();
 }

llvm/test/CodeGen/RISCV/rvv/active_lane_mask.ll

; CHECK-NEXT:    ret
   %mask = call <8 x i1> @llvm.get.active.lane.mask.v8i1.i64(i64 %index, i64 %tc)
   ret <8 x i1> %mask
 }
 
 define <32 x i1> @fv32(ptr %p, i64 %index, i64 %tc) {
 ; CHECK-LABEL: fv32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    lui a0, %hi(.LCPI8_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI8_0)
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a0, 16
+; CHECK-NEXT:    vadd.vx v16, v8, a0
 ; CHECK-NEXT:    vsaddu.vx v16, v16, a1
-; CHECK-NEXT:    vmsltu.vx v0, v16, a2
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vmsltu.vx v0, v8, a2
 ; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 2
+; CHECK-NEXT:    vslideup.vi v0, v24, 2
 ; CHECK-NEXT:    ret
   %mask = call <32 x i1> @llvm.get.active.lane.mask.v32i1.i64(i64 %index, i64 %tc)
   ret <32 x i1> %mask
 }
 
 define <64 x i1> @fv64(ptr %p, i64 %index, i64 %tc) {
 ; CHECK-LABEL: fv64:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    lui a0, %hi(.LCPI9_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI9_0)
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a0, 16
+; CHECK-NEXT:    vadd.vx v16, v8, a0
 ; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
+; CHECK-NEXT:    vsaddu.vx v16, v8, a1
 ; CHECK-NEXT:    vmsltu.vx v0, v16, a2
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
 ; CHECK-NEXT:    vsetivli zero, 4, e8, mf2, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 2
-; CHECK-NEXT:    lui a0, %hi(.LCPI9_1)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI9_1)
+; CHECK-NEXT:    vslideup.vi v0, v24, 2
+; CHECK-NEXT:    li a0, 32
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 6, e8, mf2, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 4
-; CHECK-NEXT:    lui a0, %hi(.LCPI9_2)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI9_2)
+; CHECK-NEXT:    vslideup.vi v0, v24, 4
+; CHECK-NEXT:    li a0, 48
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
+; CHECK-NEXT:    vadd.vx v8, v8, a0
 ; CHECK-NEXT:    vsaddu.vx v8, v8, a1
 ; CHECK-NEXT:    vmsltu.vx v16, v8, a2
 ; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, ma
 ; CHECK-NEXT:    vslideup.vi v0, v16, 6
 ; CHECK-NEXT:    ret
   %mask = call <64 x i1> @llvm.get.active.lane.mask.v64i1.i64(i64 %index, i64 %tc)
   ret <64 x i1> %mask
 }
 
 define <128 x i1> @fv128(ptr %p, i64 %index, i64 %tc) {
 ; CHECK-LABEL: fv128:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_0)
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a0, 16
+; CHECK-NEXT:    vadd.vx v16, v8, a0
 ; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
+; CHECK-NEXT:    vsaddu.vx v16, v8, a1
 ; CHECK-NEXT:    vmsltu.vx v0, v16, a2
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
 ; CHECK-NEXT:    vsetivli zero, 4, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 2
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_1)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_1)
+; CHECK-NEXT:    vslideup.vi v0, v24, 2
+; CHECK-NEXT:    li a0, 32
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 6, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 4
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_2)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_2)
+; CHECK-NEXT:    vslideup.vi v0, v24, 4
+; CHECK-NEXT:    li a0, 48
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 8, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 6
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_3)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_3)
+; CHECK-NEXT:    vslideup.vi v0, v24, 6
+; CHECK-NEXT:    li a0, 64
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 10, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 8
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_4)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_4)
+; CHECK-NEXT:    vslideup.vi v0, v24, 8
+; CHECK-NEXT:    li a0, 80
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 12, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 10
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_5)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_5)
+; CHECK-NEXT:    vslideup.vi v0, v24, 10
+; CHECK-NEXT:    li a0, 96
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
-; CHECK-NEXT:    vsaddu.vx v8, v8, a1
-; CHECK-NEXT:    vmsltu.vx v16, v8, a2
+; CHECK-NEXT:    vadd.vx v16, v8, a0
+; CHECK-NEXT:    vsaddu.vx v16, v16, a1
+; CHECK-NEXT:    vmsltu.vx v24, v16, a2
 ; CHECK-NEXT:    vsetivli zero, 14, e8, m1, tu, ma
-; CHECK-NEXT:    vslideup.vi v0, v16, 12
-; CHECK-NEXT:    lui a0, %hi(.LCPI10_6)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI10_6)
+; CHECK-NEXT:    vslideup.vi v0, v24, 12
+; CHECK-NEXT:    li a0, 112
 ; CHECK-NEXT:    vsetivli zero, 16, e64, m8, ta, ma
-; CHECK-NEXT:    vle64.v v8, (a0)
+; CHECK-NEXT:    vadd.vx v8, v8, a0
 ; CHECK-NEXT:    vsaddu.vx v8, v8, a1
 ; CHECK-NEXT:    vmsltu.vx v16, v8, a2
 ; CHECK-NEXT:    vsetvli zero, zero, e8, m1, ta, ma
 ; CHECK-NEXT:    vslideup.vi v0, v16, 14
 ; CHECK-NEXT:    ret
   %mask = call <128 x i1> @llvm.get.active.lane.mask.v128i1.i64(i64 %index, i64 %tc)
   ret <128 x i1> %mask
 }
 
 
 declare <vscale x 1 x i1> @llvm.get.active.lane.mask.nxv1i1.i64(i64, i64)
 declare <2 x i1> @llvm.get.active.lane.mask.v2i1.i64(i64, i64)
 declare <8 x i1> @llvm.get.active.lane.mask.v8i1.i64(i64, i64)
 declare <32 x i1> @llvm.get.active.lane.mask.v32i1.i64(i64, i64)
 declare <64 x i1> @llvm.get.active.lane.mask.v64i1.i64(i64, i64)
 declare <128 x i1> @llvm.get.active.lane.mask.v128i1.i64(i64, i64)

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-buildvec.ll

 ; LMULMAX1-NEXT:    vrgather.vi v12, v8, 0
 ; LMULMAX1-NEXT:    vrgather.vi v12, v9, 3, v0.t
 ; LMULMAX1-NEXT:    vsetivli zero, 3, e32, m1, tu, ma
 ; LMULMAX1-NEXT:    vslideup.vi v11, v10, 2
 ; LMULMAX1-NEXT:    vsetivli zero, 2, e32, m1, tu, ma
 ; LMULMAX1-NEXT:    vmv.v.v v11, v12
 ; LMULMAX1-NEXT:    vmv1r.v v8, v11
 ; LMULMAX1-NEXT:    ret
-;
-; LMULMAX2-LABEL: hang_when_merging_stores_after_legalization:
-; LMULMAX2:       # %bb.0:
-; LMULMAX2-NEXT:    vsetivli zero, 8, e32, m2, ta, ma
-; LMULMAX2-NEXT:    vid.v v12
-; LMULMAX2-NEXT:    li a0, 7
-; LMULMAX2-NEXT:    vmul.vx v14, v12, a0
-; LMULMAX2-NEXT:    vrgather.vv v12, v8, v14
-; LMULMAX2-NEXT:    vadd.vi v8, v14, -14
-; LMULMAX2-NEXT:    vsetivli zero, 1, e8, mf8, ta, ma
-; LMULMAX2-NEXT:    vmv.v.i v0, 12
-; LMULMAX2-NEXT:    vsetivli zero, 8, e32, m2, ta, mu
-; LMULMAX2-NEXT:    vrgather.vv v12, v10, v8, v0.t
-; LMULMAX2-NEXT:    vmv1r.v v8, v12
-; LMULMAX2-NEXT:    ret
   %z = shufflevector <8 x float> %x, <8 x float> %y, <4 x i32> <i32 0, i32 7, i32 8, i32 15>
   ret <4 x float> %z
 }
 
 define void @buildvec_dominant0_v2f32(<2 x float>* %x) {
 ; CHECK-LABEL: buildvec_dominant0_v2f32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-interleave.ll

 }
 
 define <64 x float> @interleave_v32f32(<32 x float> %x, <32 x float> %y) {
 ; RV32-V128-LABEL: interleave_v32f32:
 ; RV32-V128:       # %bb.0:
 ; RV32-V128-NEXT:    addi sp, sp, -16
 ; RV32-V128-NEXT:    .cfi_def_cfa_offset 16
 ; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 4
+; RV32-V128-NEXT:    slli a0, a0, 3
 ; RV32-V128-NEXT:    sub sp, sp, a0
-; RV32-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
-; RV32-V128-NEXT:    lui a0, %hi(.LCPI10_0)
-; RV32-V128-NEXT:    addi a0, a0, %lo(.LCPI10_0)
-; RV32-V128-NEXT:    li a1, 32
-; RV32-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV32-V128-NEXT:    vle32.v v0, (a0)
-; RV32-V128-NEXT:    vmv8r.v v24, v8
+; RV32-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x08, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 8 * vlenb
+; RV32-V128-NEXT:    vmv8r.v v0, v8
 ; RV32-V128-NEXT:    addi a0, sp, 16
 ; RV32-V128-NEXT:    vs8r.v v8, (a0) # Unknown-size Folded Spill
-; RV32-V128-NEXT:    vrgather.vv v8, v24, v0
-; RV32-V128-NEXT:    lui a0, %hi(.LCPI10_1)
-; RV32-V128-NEXT:    addi a0, a0, %lo(.LCPI10_1)
-; RV32-V128-NEXT:    vle32.v v24, (a0)
-; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 3
-; RV32-V128-NEXT:    add a0, sp, a0
-; RV32-V128-NEXT:    addi a0, a0, 16
-; RV32-V128-NEXT:    vs8r.v v24, (a0) # Unknown-size Folded Spill
-; RV32-V128-NEXT:    lui a0, 699051
-; RV32-V128-NEXT:    addi a0, a0, -1366
+; RV32-V128-NEXT:    li a0, 32
+; RV32-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV32-V128-NEXT:    vid.v v8
+; RV32-V128-NEXT:    vsrl.vi v8, v8, 1
+; RV32-V128-NEXT:    li a1, 16
+; RV32-V128-NEXT:    vadd.vx v8, v8, a1
+; RV32-V128-NEXT:    vrgather.vv v24, v0, v8
+; RV32-V128-NEXT:    lui a1, 699051
+; RV32-V128-NEXT:    addi a1, a1, -1366
 ; RV32-V128-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-V128-NEXT:    vmv.v.x v0, a0
-; RV32-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
-; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 3
-; RV32-V128-NEXT:    add a0, sp, a0
-; RV32-V128-NEXT:    addi a0, a0, 16
-; RV32-V128-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
-; RV32-V128-NEXT:    vrgather.vv v8, v16, v24, v0.t
-; RV32-V128-NEXT:    vmv.v.v v24, v8
+; RV32-V128-NEXT:    vmv.v.x v0, a1
+; RV32-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV32-V128-NEXT:    vrgather.vv v24, v16, v8, v0.t
 ; RV32-V128-NEXT:    vsetivli zero, 16, e32, m4, ta, ma
 ; RV32-V128-NEXT:    addi a0, sp, 16
 ; RV32-V128-NEXT:    vl8r.v v8, (a0) # Unknown-size Folded Reload
 ; RV32-V128-NEXT:    vwaddu.vv v0, v8, v16
 ; RV32-V128-NEXT:    li a0, -1
 ; RV32-V128-NEXT:    vwmaccu.vx v0, a0, v16
 ; RV32-V128-NEXT:    vmv8r.v v8, v0
 ; RV32-V128-NEXT:    vmv8r.v v16, v24
 ; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 4
+; RV32-V128-NEXT:    slli a0, a0, 3
 ; RV32-V128-NEXT:    add sp, sp, a0
 ; RV32-V128-NEXT:    addi sp, sp, 16
 ; RV32-V128-NEXT:    ret
 ;
 ; RV64-V128-LABEL: interleave_v32f32:
 ; RV64-V128:       # %bb.0:
 ; RV64-V128-NEXT:    addi sp, sp, -16
 ; RV64-V128-NEXT:    .cfi_def_cfa_offset 16
 ; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 4
+; RV64-V128-NEXT:    slli a0, a0, 3
 ; RV64-V128-NEXT:    sub sp, sp, a0
-; RV64-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
-; RV64-V128-NEXT:    lui a0, %hi(.LCPI10_0)
-; RV64-V128-NEXT:    addi a0, a0, %lo(.LCPI10_0)
-; RV64-V128-NEXT:    li a1, 32
-; RV64-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV64-V128-NEXT:    vle32.v v0, (a0)
-; RV64-V128-NEXT:    vmv8r.v v24, v8
+; RV64-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x08, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 8 * vlenb
+; RV64-V128-NEXT:    vmv8r.v v0, v8
 ; RV64-V128-NEXT:    addi a0, sp, 16
 ; RV64-V128-NEXT:    vs8r.v v8, (a0) # Unknown-size Folded Spill
-; RV64-V128-NEXT:    vrgather.vv v8, v24, v0
-; RV64-V128-NEXT:    lui a0, %hi(.LCPI10_1)
-; RV64-V128-NEXT:    addi a0, a0, %lo(.LCPI10_1)
-; RV64-V128-NEXT:    vle32.v v24, (a0)
-; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 3
-; RV64-V128-NEXT:    add a0, sp, a0
-; RV64-V128-NEXT:    addi a0, a0, 16
-; RV64-V128-NEXT:    vs8r.v v24, (a0) # Unknown-size Folded Spill
-; RV64-V128-NEXT:    lui a0, 699051
-; RV64-V128-NEXT:    addiw a0, a0, -1366
+; RV64-V128-NEXT:    li a0, 32
+; RV64-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV64-V128-NEXT:    vid.v v8
+; RV64-V128-NEXT:    vsrl.vi v8, v8, 1
+; RV64-V128-NEXT:    li a1, 16
+; RV64-V128-NEXT:    vadd.vx v8, v8, a1
+; RV64-V128-NEXT:    vrgather.vv v24, v0, v8
+; RV64-V128-NEXT:    lui a1, 699051
+; RV64-V128-NEXT:    addiw a1, a1, -1366
 ; RV64-V128-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV64-V128-NEXT:    vmv.v.x v0, a0
-; RV64-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
-; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 3
-; RV64-V128-NEXT:    add a0, sp, a0
-; RV64-V128-NEXT:    addi a0, a0, 16
-; RV64-V128-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
-; RV64-V128-NEXT:    vrgather.vv v8, v16, v24, v0.t
-; RV64-V128-NEXT:    vmv.v.v v24, v8
+; RV64-V128-NEXT:    vmv.v.x v0, a1
+; RV64-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV64-V128-NEXT:    vrgather.vv v24, v16, v8, v0.t
 ; RV64-V128-NEXT:    vsetivli zero, 16, e32, m4, ta, ma
 ; RV64-V128-NEXT:    addi a0, sp, 16
 ; RV64-V128-NEXT:    vl8r.v v8, (a0) # Unknown-size Folded Reload
 ; RV64-V128-NEXT:    vwaddu.vv v0, v8, v16
 ; RV64-V128-NEXT:    li a0, -1
 ; RV64-V128-NEXT:    vwmaccu.vx v0, a0, v16
 ; RV64-V128-NEXT:    vmv8r.v v8, v0
 ; RV64-V128-NEXT:    vmv8r.v v16, v24
 ; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 4
+; RV64-V128-NEXT:    slli a0, a0, 3
 ; RV64-V128-NEXT:    add sp, sp, a0
 ; RV64-V128-NEXT:    addi sp, sp, 16
 ; RV64-V128-NEXT:    ret
 ;
 ; V512-LABEL: interleave_v32f32:
 ; V512:       # %bb.0:
 ; V512-NEXT:    li a0, 32
 ; V512-NEXT:    vsetvli zero, a0, e32, m2, ta, ma

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-fp-shuffles.ll

 ; RV64-NEXT:    ret
   %s = shufflevector <4 x double> <double 2.0, double 2.0, double 2.0, double 2.0>, <4 x double> %x, <4 x i32> <i32 0, i32 3, i32 6, i32 5>
   ret <4 x double> %s
 }
 
 define <4 x double> @vrgather_shuffle_vx_v4f64(<4 x double> %x) {
 ; RV32-LABEL: vrgather_shuffle_vx_v4f64:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
-; RV32-NEXT:    vid.v v12
-; RV32-NEXT:    li a0, 3
-; RV32-NEXT:    lui a1, %hi(.LCPI8_0)
-; RV32-NEXT:    addi a1, a1, %lo(.LCPI8_0)
-; RV32-NEXT:    vlse64.v v10, (a1), zero
-; RV32-NEXT:    vmul.vx v12, v12, a0
+; RV32-NEXT:    lui a0, %hi(.LCPI8_0)
+; RV32-NEXT:    addi a0, a0, %lo(.LCPI8_0)
+; RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, ma
+; RV32-NEXT:    vlse64.v v10, (a0), zero
+; RV32-NEXT:    lui a0, 48
+; RV32-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; RV32-NEXT:    vmv.v.x v12, a0
 ; RV32-NEXT:    vmv.v.i v0, 3
-; RV32-NEXT:    vsetvli zero, zero, e64, m2, ta, mu
+; RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; RV32-NEXT:    vrgatherei16.vv v10, v8, v12, v0.t
 ; RV32-NEXT:    vmv.v.v v8, v10
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vrgather_shuffle_vx_v4f64:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 4, e64, m2, ta, ma
 ; RV64-NEXT:    vid.v v10
 ; RV64-NEXT:    li a0, 3
 ; RV64-NEXT:    vmul.vx v12, v10, a0
 ; RV64-NEXT:    lui a0, %hi(.LCPI8_0)
 ; RV64-NEXT:    addi a0, a0, %lo(.LCPI8_0)
 ; RV64-NEXT:    vlse64.v v10, (a0), zero
 ; RV64-NEXT:    vsetivli zero, 1, e8, mf8, ta, ma
 ; RV64-NEXT:    vmv.v.i v0, 3
 ; RV64-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; RV64-NEXT:    vrgather.vv v10, v8, v12, v0.t
 ; RV64-NEXT:    vmv.v.v v8, v10
 ; RV64-NEXT:    ret
   %s = shufflevector <4 x double> %x, <4 x double> <double 2.0, double 2.0, double 2.0, double 2.0>, <4 x i32> <i32 0, i32 3, i32 6, i32 5>
   ret <4 x double> %s

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-buildvec.ll

 ; CHECK-NEXT:    ret
   store <16 x i8> <i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15, i8 16, i8 17>, ptr %x
   ret void
 }
 
 define void @buildvec_vid_plus_nonimm_v16i8(ptr %x) {
 ; CHECK-LABEL: buildvec_vid_plus_nonimm_v16i8:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a1, %hi(.LCPI4_0)
-; CHECK-NEXT:    addi a1, a1, %lo(.LCPI4_0)
 ; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, ma
-; CHECK-NEXT:    vle8.v v8, (a1)
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a1, 100
+; CHECK-NEXT:    vadd.vx v8, v8, a1
 ; CHECK-NEXT:    vse8.v v8, (a0)
 ; CHECK-NEXT:    ret
   store <16 x i8> <i8 100, i8 101, i8 102, i8 103, i8 104, i8 105, i8 106, i8 107, i8 108, i8 109, i8 110, i8 111, i8 112, i8 113, i8 114, i8 115>, ptr %x
   ret void
 }
 
 define void @buildvec_vid_mpy_imm_v16i8(ptr %x) {
 ; CHECK-LABEL: buildvec_vid_mpy_imm_v16i8:

 ; CHECK-NEXT:    vadd.vi v8, v8, 1
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 undef, i8 3, i8 5, i8 7>
 }
 
 define <4 x i8> @buildvec_vid_step2_add1_v4i8_undef1() {
 ; CHECK-LABEL: buildvec_vid_step2_add1_v4i8_undef1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; CHECK-NEXT:    vid.v v8
-; CHECK-NEXT:    vadd.vv v8, v8, v8
-; CHECK-NEXT:    vadd.vi v8, v8, 1
+; CHECK-NEXT:    lui a0, 28752
+; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; CHECK-NEXT:    vmv.s.x v8, a0
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 undef, i8 undef, i8 5, i8 7>
 }
 
 define <4 x i8> @buildvec_vid_step2_add1_v4i8_undef2() {
 ; CHECK-LABEL: buildvec_vid_step2_add1_v4i8_undef2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; CHECK-NEXT:    vid.v v8
-; CHECK-NEXT:    vadd.vv v8, v8, v8
-; CHECK-NEXT:    vadd.vi v8, v8, 1
+; CHECK-NEXT:    li a0, 1793
+; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; CHECK-NEXT:    vmv.v.x v8, a0
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 1, i8 undef, i8 undef, i8 7>
 }
 
 define <4 x i8> @buildvec_vid_stepn1_add0_v4i8() {
 ; CHECK-LABEL: buildvec_vid_stepn1_add0_v4i8:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma

 ; CHECK-NEXT:    vrsub.vi v8, v8, 0
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 undef, i8 -2, i8 -4, i8 -6>
 }
 
 define <4 x i8> @buildvec_vid_stepn2_add0_v4i8_undef1() {
 ; CHECK-LABEL: buildvec_vid_stepn2_add0_v4i8_undef1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; CHECK-NEXT:    vid.v v8
-; CHECK-NEXT:    vadd.vv v8, v8, v8
-; CHECK-NEXT:    vrsub.vi v8, v8, 0
+; CHECK-NEXT:    lui a0, 1028032
+; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; CHECK-NEXT:    vmv.s.x v8, a0
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 undef, i8 undef, i8 -4, i8 -6>
 }
 
 define <4 x i8> @buildvec_vid_stepn2_add0_v4i8_undef2() {
 ; CHECK-LABEL: buildvec_vid_stepn2_add0_v4i8_undef2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma

 ; CHECK-NEXT:    vid.v v8
 ; CHECK-NEXT:    vadd.vv v8, v8, v8
 ; CHECK-NEXT:    vrsub.vi v8, v8, 3
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 3, i8 1, i8 -1, i8 -3>
 }
 
 define <4 x i8> @buildvec_vid_stepn3_add3_v4i8() {
-; CHECK-LABEL: buildvec_vid_stepn3_add3_v4i8:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; CHECK-NEXT:    vmv.v.i v9, 3
-; CHECK-NEXT:    vid.v v8
-; CHECK-NEXT:    li a0, -3
-; CHECK-NEXT:    vmadd.vx v8, a0, v9
-; CHECK-NEXT:    ret
+; RV32-LABEL: buildvec_vid_stepn3_add3_v4i8:
+; RV32:       # %bb.0:
+; RV32-NEXT:    lui a0, 1028048
+; RV32-NEXT:    addi a0, a0, 3
+; RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; RV32-NEXT:    vmv.s.x v8, a0
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: buildvec_vid_stepn3_add3_v4i8:
+; RV64:       # %bb.0:
+; RV64-NEXT:    lui a0, 1028048
+; RV64-NEXT:    addiw a0, a0, 3
+; RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; RV64-NEXT:    vmv.s.x v8, a0
+; RV64-NEXT:    ret
   ret <4 x i8> <i8 3, i8 0, i8 -3, i8 -6>
 }
 
 define void @buildvec_vid_stepn3_addn3_v4i32(ptr %z0, ptr %z1, ptr %z2, ptr %z3) {
 ; CHECK-LABEL: buildvec_vid_stepn3_addn3_v4i32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
 ; CHECK-NEXT:    vmv.v.i v8, -3

 ; CHECK-NEXT:    lui a0, 32768
 ; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
 ; CHECK-NEXT:    vmv.s.x v8, a0
 ; CHECK-NEXT:    ret
   ret <4 x i8> <i8 0, i8 undef, i8 undef, i8 8>
 }
 
 define <4 x i8> @buildvec_no_vid_v4i8_3() {
-; RV32-LABEL: buildvec_no_vid_v4i8_3:
-; RV32:       # %bb.0:
-; RV32-NEXT:    lui a0, 28672
-; RV32-NEXT:    addi a0, a0, 255
-; RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
-; RV32-NEXT:    vmv.s.x v8, a0
-; RV32-NEXT:    ret
-;
-; RV64-LABEL: buildvec_no_vid_v4i8_3:
-; RV64:       # %bb.0:
-; RV64-NEXT:    lui a0, 28672
-; RV64-NEXT:    addiw a0, a0, 255
-; RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
-; RV64-NEXT:    vmv.s.x v8, a0
-; RV64-NEXT:    ret
+; CHECK-LABEL: buildvec_no_vid_v4i8_3:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    li a0, 2047
+; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; CHECK-NEXT:    vmv.v.x v8, a0
+; CHECK-NEXT:    ret
   ret <4 x i8> <i8 -1, i8 undef, i8 undef, i8 7>
 }
 
 define <4 x i8> @buildvec_no_vid_v4i8_4() {
 ; CHECK-LABEL: buildvec_no_vid_v4i8_4:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
 ; CHECK-NEXT:    vmv.v.i v8, -2

 ; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, ma
 ; RV64-NEXT:    vse8.v v8, (a0)
 ; RV64-NEXT:    ret
   store <8 x i8> <i8 1, i8 2, i8 3, i8 undef, i8 1, i8 2, i8 3, i8 undef>, ptr %x
   ret void
 }
 
 define void @buildvec_seq_v16i8_v2i64(ptr %x) {
-; RV32-LABEL: buildvec_seq_v16i8_v2i64:
-; RV32:       # %bb.0:
-; RV32-NEXT:    lui a1, %hi(.LCPI42_0)
-; RV32-NEXT:    addi a1, a1, %lo(.LCPI42_0)
-; RV32-NEXT:    vsetivli zero, 16, e8, m1, ta, ma
-; RV32-NEXT:    vle8.v v8, (a1)
-; RV32-NEXT:    vse8.v v8, (a0)
-; RV32-NEXT:    ret
-;
-; RV64-LABEL: buildvec_seq_v16i8_v2i64:
-; RV64:       # %bb.0:
-; RV64-NEXT:    lui a1, %hi(.LCPI42_0)
-; RV64-NEXT:    addi a1, a1, %lo(.LCPI42_0)
-; RV64-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
-; RV64-NEXT:    vlse64.v v8, (a1), zero
-; RV64-NEXT:    vsetivli zero, 16, e8, m1, ta, ma
-; RV64-NEXT:    vse8.v v8, (a0)
-; RV64-NEXT:    ret
+; CHECK-LABEL: buildvec_seq_v16i8_v2i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lui a1, %hi(.LCPI42_0)
+; CHECK-NEXT:    addi a1, a1, %lo(.LCPI42_0)
+; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, ma
+; CHECK-NEXT:    vle8.v v8, (a1)
+; CHECK-NEXT:    vse8.v v8, (a0)
+; CHECK-NEXT:    ret
   store <16 x i8> <i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 8, i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 8>, ptr %x
   ret void
 }
 
 define void @buildvec_seq2_v16i8_v2i64(ptr %x) {
 ; RV32-LABEL: buildvec_seq2_v16i8_v2i64:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    lui a1, 528432

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-interleave.ll

 }
 
 define <64 x i32> @interleave_v32i32(<32 x i32> %x, <32 x i32> %y) {
 ; RV32-V128-LABEL: interleave_v32i32:
 ; RV32-V128:       # %bb.0:
 ; RV32-V128-NEXT:    addi sp, sp, -16
 ; RV32-V128-NEXT:    .cfi_def_cfa_offset 16
 ; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 4
+; RV32-V128-NEXT:    slli a0, a0, 3
 ; RV32-V128-NEXT:    sub sp, sp, a0
-; RV32-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
-; RV32-V128-NEXT:    lui a0, %hi(.LCPI17_0)
-; RV32-V128-NEXT:    addi a0, a0, %lo(.LCPI17_0)
-; RV32-V128-NEXT:    li a1, 32
-; RV32-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV32-V128-NEXT:    vle32.v v0, (a0)
-; RV32-V128-NEXT:    vmv8r.v v24, v8
+; RV32-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x08, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 8 * vlenb
+; RV32-V128-NEXT:    vmv8r.v v0, v8
 ; RV32-V128-NEXT:    addi a0, sp, 16
 ; RV32-V128-NEXT:    vs8r.v v8, (a0) # Unknown-size Folded Spill
-; RV32-V128-NEXT:    vrgather.vv v8, v24, v0
-; RV32-V128-NEXT:    lui a0, %hi(.LCPI17_1)
-; RV32-V128-NEXT:    addi a0, a0, %lo(.LCPI17_1)
-; RV32-V128-NEXT:    vle32.v v24, (a0)
-; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 3
-; RV32-V128-NEXT:    add a0, sp, a0
-; RV32-V128-NEXT:    addi a0, a0, 16
-; RV32-V128-NEXT:    vs8r.v v24, (a0) # Unknown-size Folded Spill
-; RV32-V128-NEXT:    lui a0, 699051
-; RV32-V128-NEXT:    addi a0, a0, -1366
+; RV32-V128-NEXT:    li a0, 32
+; RV32-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV32-V128-NEXT:    vid.v v8
+; RV32-V128-NEXT:    vsrl.vi v8, v8, 1
+; RV32-V128-NEXT:    li a1, 16
+; RV32-V128-NEXT:    vadd.vx v8, v8, a1
+; RV32-V128-NEXT:    vrgather.vv v24, v0, v8
+; RV32-V128-NEXT:    lui a1, 699051
+; RV32-V128-NEXT:    addi a1, a1, -1366
 ; RV32-V128-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-V128-NEXT:    vmv.v.x v0, a0
-; RV32-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
-; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 3
-; RV32-V128-NEXT:    add a0, sp, a0
-; RV32-V128-NEXT:    addi a0, a0, 16
-; RV32-V128-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
-; RV32-V128-NEXT:    vrgather.vv v8, v16, v24, v0.t
-; RV32-V128-NEXT:    vmv.v.v v24, v8
+; RV32-V128-NEXT:    vmv.v.x v0, a1
+; RV32-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV32-V128-NEXT:    vrgather.vv v24, v16, v8, v0.t
 ; RV32-V128-NEXT:    vsetivli zero, 16, e32, m4, ta, ma
 ; RV32-V128-NEXT:    addi a0, sp, 16
 ; RV32-V128-NEXT:    vl8r.v v8, (a0) # Unknown-size Folded Reload
 ; RV32-V128-NEXT:    vwaddu.vv v0, v8, v16
 ; RV32-V128-NEXT:    li a0, -1
 ; RV32-V128-NEXT:    vwmaccu.vx v0, a0, v16
 ; RV32-V128-NEXT:    vmv8r.v v8, v0
 ; RV32-V128-NEXT:    vmv8r.v v16, v24
 ; RV32-V128-NEXT:    csrr a0, vlenb
-; RV32-V128-NEXT:    slli a0, a0, 4
+; RV32-V128-NEXT:    slli a0, a0, 3
 ; RV32-V128-NEXT:    add sp, sp, a0
 ; RV32-V128-NEXT:    addi sp, sp, 16
 ; RV32-V128-NEXT:    ret
 ;
 ; RV64-V128-LABEL: interleave_v32i32:
 ; RV64-V128:       # %bb.0:
 ; RV64-V128-NEXT:    addi sp, sp, -16
 ; RV64-V128-NEXT:    .cfi_def_cfa_offset 16
 ; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 4
+; RV64-V128-NEXT:    slli a0, a0, 3
 ; RV64-V128-NEXT:    sub sp, sp, a0
-; RV64-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
-; RV64-V128-NEXT:    lui a0, %hi(.LCPI17_0)
-; RV64-V128-NEXT:    addi a0, a0, %lo(.LCPI17_0)
-; RV64-V128-NEXT:    li a1, 32
-; RV64-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV64-V128-NEXT:    vle32.v v0, (a0)
-; RV64-V128-NEXT:    vmv8r.v v24, v8
+; RV64-V128-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x08, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 8 * vlenb
+; RV64-V128-NEXT:    vmv8r.v v0, v8
 ; RV64-V128-NEXT:    addi a0, sp, 16
 ; RV64-V128-NEXT:    vs8r.v v8, (a0) # Unknown-size Folded Spill
-; RV64-V128-NEXT:    vrgather.vv v8, v24, v0
-; RV64-V128-NEXT:    lui a0, %hi(.LCPI17_1)
-; RV64-V128-NEXT:    addi a0, a0, %lo(.LCPI17_1)
-; RV64-V128-NEXT:    vle32.v v24, (a0)
-; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 3
-; RV64-V128-NEXT:    add a0, sp, a0
-; RV64-V128-NEXT:    addi a0, a0, 16
-; RV64-V128-NEXT:    vs8r.v v24, (a0) # Unknown-size Folded Spill
-; RV64-V128-NEXT:    lui a0, 699051
-; RV64-V128-NEXT:    addiw a0, a0, -1366
+; RV64-V128-NEXT:    li a0, 32
+; RV64-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV64-V128-NEXT:    vid.v v8
+; RV64-V128-NEXT:    vsrl.vi v8, v8, 1
+; RV64-V128-NEXT:    li a1, 16
+; RV64-V128-NEXT:    vadd.vx v8, v8, a1
+; RV64-V128-NEXT:    vrgather.vv v24, v0, v8
+; RV64-V128-NEXT:    lui a1, 699051
+; RV64-V128-NEXT:    addiw a1, a1, -1366
 ; RV64-V128-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV64-V128-NEXT:    vmv.v.x v0, a0
-; RV64-V128-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
-; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 3
-; RV64-V128-NEXT:    add a0, sp, a0
-; RV64-V128-NEXT:    addi a0, a0, 16
-; RV64-V128-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
-; RV64-V128-NEXT:    vrgather.vv v8, v16, v24, v0.t
-; RV64-V128-NEXT:    vmv.v.v v24, v8
+; RV64-V128-NEXT:    vmv.v.x v0, a1
+; RV64-V128-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV64-V128-NEXT:    vrgather.vv v24, v16, v8, v0.t
 ; RV64-V128-NEXT:    vsetivli zero, 16, e32, m4, ta, ma
 ; RV64-V128-NEXT:    addi a0, sp, 16
 ; RV64-V128-NEXT:    vl8r.v v8, (a0) # Unknown-size Folded Reload
 ; RV64-V128-NEXT:    vwaddu.vv v0, v8, v16
 ; RV64-V128-NEXT:    li a0, -1
 ; RV64-V128-NEXT:    vwmaccu.vx v0, a0, v16
 ; RV64-V128-NEXT:    vmv8r.v v8, v0
 ; RV64-V128-NEXT:    vmv8r.v v16, v24
 ; RV64-V128-NEXT:    csrr a0, vlenb
-; RV64-V128-NEXT:    slli a0, a0, 4
+; RV64-V128-NEXT:    slli a0, a0, 3
 ; RV64-V128-NEXT:    add sp, sp, a0
 ; RV64-V128-NEXT:    addi sp, sp, 16
 ; RV64-V128-NEXT:    ret
 ;
 ; V512-LABEL: interleave_v32i32:
 ; V512:       # %bb.0:
 ; V512-NEXT:    li a0, 32
 ; V512-NEXT:    vsetvli zero, a0, e32, m2, ta, ma
; V512-NEXT:    ret
   ret <8 x i32> %a
 }
 
 ; This interleaves the first 2 elements of a vector in opposite order. With
 ; undefs for the remaining elements. We use to miscompile this.
 define <4 x i8> @unary_interleave_10uu_v4i8(<4 x i8> %x) {
 ; V128-LABEL: unary_interleave_10uu_v4i8:
 ; V128:       # %bb.0:
+; V128-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; V128-NEXT:    vmv.v.i v10, 1
 ; V128-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; V128-NEXT:    vid.v v9
-; V128-NEXT:    vrsub.vi v10, v9, 1
 ; V128-NEXT:    vrgather.vv v9, v8, v10
 ; V128-NEXT:    vmv1r.v v8, v9
 ; V128-NEXT:    ret
 ;
 ; V512-LABEL: unary_interleave_10uu_v4i8:
 ; V512:       # %bb.0:
+; V512-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; V512-NEXT:    vmv.v.i v10, 1
 ; V512-NEXT:    vsetivli zero, 4, e8, mf8, ta, ma
-; V512-NEXT:    vid.v v9
-; V512-NEXT:    vrsub.vi v10, v9, 1
 ; V512-NEXT:    vrgather.vv v9, v8, v10
 ; V512-NEXT:    vmv1r.v v8, v9
 ; V512-NEXT:    ret
   %a = shufflevector <4 x i8> %x, <4 x i8> poison, <4 x i32> <i32 1, i32 0, i32 undef, i32 undef>
   ret <4 x i8> %a
 }
 

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int-shuffles.ll

 ; CHECK-NEXT:    ret
   %s = shufflevector <4 x i16> <i16 5, i16 5, i16 5, i16 5>, <4 x i16> %x, <4 x i32> <i32 0, i32 3, i32 6, i32 5>
   ret <4 x i16> %s
 }
 
 define <4 x i16> @vrgather_shuffle_vx_v4i16(<4 x i16> %x) {
 ; CHECK-LABEL: vrgather_shuffle_vx_v4i16:
 ; CHECK:       # %bb.0:
+; CHECK-NEXT:    lui a0, 48
+; CHECK-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; CHECK-NEXT:    vmv.v.x v10, a0
 ; CHECK-NEXT:    vsetivli zero, 4, e16, mf2, ta, mu
-; CHECK-NEXT:    vid.v v9
-; CHECK-NEXT:    li a0, 3
-; CHECK-NEXT:    vmul.vx v10, v9, a0
 ; CHECK-NEXT:    vmv.v.i v0, 3
 ; CHECK-NEXT:    vmv.v.i v9, 5
 ; CHECK-NEXT:    vrgather.vv v9, v8, v10, v0.t
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %s = shufflevector <4 x i16> %x, <4 x i16> <i16 5, i16 5, i16 5, i16 5>, <4 x i32> <i32 0, i32 3, i32 6, i32 5>
   ret <4 x i16> %s
 }

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-int.ll

 ; RV64-NEXT:    ret
   %a = load <8 x i16>, ptr %x
   %b = sdiv <8 x i16> %a, <i16 -7, i16 7, i16 7, i16 -7, i16 7, i16 -7, i16 -7, i16 7>
   store <8 x i16> %b, ptr %x
   ret void
 }
 
 define void @mulhs_v6i16(ptr %x) {
-; CHECK-LABEL: mulhs_v6i16:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
-; CHECK-NEXT:    vle16.v v8, (a0)
-; CHECK-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
-; CHECK-NEXT:    vmv.v.i v0, 6
-; CHECK-NEXT:    vmv.v.i v9, -7
-; CHECK-NEXT:    vmerge.vim v9, v9, 7, v0
-; CHECK-NEXT:    vdiv.vv v9, v8, v9
-; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
-; CHECK-NEXT:    vmv.v.i v10, 7
-; CHECK-NEXT:    vid.v v11
-; CHECK-NEXT:    li a1, -14
-; CHECK-NEXT:    vmadd.vx v11, a1, v10
-; CHECK-NEXT:    vsetivli zero, 2, e16, m1, ta, ma
-; CHECK-NEXT:    vslidedown.vi v8, v8, 4
-; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
-; CHECK-NEXT:    vdiv.vv v8, v8, v11
-; CHECK-NEXT:    vsetivli zero, 8, e16, m1, ta, ma
-; CHECK-NEXT:    vslideup.vi v9, v8, 4
-; CHECK-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
-; CHECK-NEXT:    vse16.v v9, (a0)
-; CHECK-NEXT:    ret
+; RV32-LABEL: mulhs_v6i16:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
+; RV32-NEXT:    vle16.v v8, (a0)
+; RV32-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
+; RV32-NEXT:    vmv.v.i v0, 6
+; RV32-NEXT:    vmv.v.i v9, -7
+; RV32-NEXT:    vmerge.vim v9, v9, 7, v0
+; RV32-NEXT:    vdiv.vv v9, v8, v9
+; RV32-NEXT:    vsetivli zero, 2, e16, m1, ta, ma
+; RV32-NEXT:    vslidedown.vi v8, v8, 4
+; RV32-NEXT:    lui a1, 1048464
+; RV32-NEXT:    addi a1, a1, 7
+; RV32-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; RV32-NEXT:    vmv.s.x v10, a1
+; RV32-NEXT:    vsetvli zero, zero, e16, mf4, ta, ma
+; RV32-NEXT:    vdiv.vv v8, v8, v10
+; RV32-NEXT:    vsetivli zero, 8, e16, m1, ta, ma
+; RV32-NEXT:    vslideup.vi v9, v8, 4
+; RV32-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
+; RV32-NEXT:    vse16.v v9, (a0)
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: mulhs_v6i16:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
+; RV64-NEXT:    vle16.v v8, (a0)
+; RV64-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
+; RV64-NEXT:    vmv.v.i v0, 6
+; RV64-NEXT:    vmv.v.i v9, -7
+; RV64-NEXT:    vmerge.vim v9, v9, 7, v0
+; RV64-NEXT:    vdiv.vv v9, v8, v9
+; RV64-NEXT:    vsetivli zero, 2, e16, m1, ta, ma
+; RV64-NEXT:    vslidedown.vi v8, v8, 4
+; RV64-NEXT:    lui a1, 1048464
+; RV64-NEXT:    addiw a1, a1, 7
+; RV64-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; RV64-NEXT:    vmv.s.x v10, a1
+; RV64-NEXT:    vsetvli zero, zero, e16, mf4, ta, ma
+; RV64-NEXT:    vdiv.vv v8, v8, v10
+; RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, ma
+; RV64-NEXT:    vslideup.vi v9, v8, 4
+; RV64-NEXT:    vsetivli zero, 6, e16, m1, ta, ma
+; RV64-NEXT:    vse16.v v9, (a0)
+; RV64-NEXT:    ret
   %a = load <6 x i16>, ptr %x
   %b = sdiv <6 x i16> %a, <i16 -7, i16 7, i16 7, i16 -7, i16 7, i16 -7>
   store <6 x i16> %b, ptr %x
   ret void
 }
 
 define void @mulhs_v4i32(ptr %x) {
 ; RV32-LABEL: mulhs_v4i32:

 ; RV32-NEXT:    vadd.vv v8, v8, v9
 ; RV32-NEXT:    vse32.v v8, (a0)
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: mulhs_v4i32:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
 ; RV64-NEXT:    vle32.v v8, (a0)
-; RV64-NEXT:    lui a1, %hi(.LCPI73_0)
-; RV64-NEXT:    addi a1, a1, %lo(.LCPI73_0)
-; RV64-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
-; RV64-NEXT:    vlse64.v v9, (a1), zero
-; RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; RV64-NEXT:    lui a1, 419430
+; RV64-NEXT:    addiw a1, a1, 1639
+; RV64-NEXT:    vmv.v.x v9, a1
+; RV64-NEXT:    vmv.v.i v0, 5
+; RV64-NEXT:    lui a1, 629146
+; RV64-NEXT:    addiw a1, a1, -1639
+; RV64-NEXT:    vmerge.vxm v9, v9, a1, v0
 ; RV64-NEXT:    vmulh.vv v8, v8, v9
 ; RV64-NEXT:    vsra.vi v8, v8, 1
 ; RV64-NEXT:    vsrl.vi v9, v8, 31
 ; RV64-NEXT:    vadd.vv v8, v8, v9
 ; RV64-NEXT:    vse32.v v8, (a0)
 ; RV64-NEXT:    ret
   %a = load <4 x i32>, ptr %x
   %b = sdiv <4 x i32> %a, <i32 -5, i32 5, i32 -5, i32 5>

 ; LMULMAX2-RV32-NEXT:    vadd.vv v8, v8, v10
 ; LMULMAX2-RV32-NEXT:    vse32.v v8, (a0)
 ; LMULMAX2-RV32-NEXT:    ret
 ;
 ; LMULMAX2-RV64-LABEL: mulhs_v8i32:
 ; LMULMAX2-RV64:       # %bb.0:
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e32, m2, ta, ma
 ; LMULMAX2-RV64-NEXT:    vle32.v v8, (a0)
-; LMULMAX2-RV64-NEXT:    lui a1, %hi(.LCPI187_0)
-; LMULMAX2-RV64-NEXT:    addi a1, a1, %lo(.LCPI187_0)
-; LMULMAX2-RV64-NEXT:    vsetivli zero, 4, e64, m2, ta, ma
-; LMULMAX2-RV64-NEXT:    vlse64.v v10, (a1), zero
+; LMULMAX2-RV64-NEXT:    lui a1, 419430
+; LMULMAX2-RV64-NEXT:    addiw a1, a1, 1639
+; LMULMAX2-RV64-NEXT:    vmv.v.x v10, a1
+; LMULMAX2-RV64-NEXT:    li a1, 85
+; LMULMAX2-RV64-NEXT:    vsetivli zero, 1, e8, mf8, ta, ma
+; LMULMAX2-RV64-NEXT:    vmv.v.x v0, a1
+; LMULMAX2-RV64-NEXT:    lui a1, 629146
+; LMULMAX2-RV64-NEXT:    addiw a1, a1, -1639
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e32, m2, ta, ma
+; LMULMAX2-RV64-NEXT:    vmerge.vxm v10, v10, a1, v0
 ; LMULMAX2-RV64-NEXT:    vmulh.vv v8, v8, v10
 ; LMULMAX2-RV64-NEXT:    vsra.vi v8, v8, 1
 ; LMULMAX2-RV64-NEXT:    vsrl.vi v10, v8, 31
 ; LMULMAX2-RV64-NEXT:    vadd.vv v8, v8, v10
 ; LMULMAX2-RV64-NEXT:    vse32.v v8, (a0)
 ; LMULMAX2-RV64-NEXT:    ret
 ;
 ; LMULMAX1-RV32-LABEL: mulhs_v8i32:

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-reduction-int-vp.ll

 
 define signext i8 @vpreduce_mul_v64i8(i8 signext %s, <64 x i8> %v, <64 x i1> %m, i32 zeroext %evl) {
 ; RV32-LABEL: vpreduce_mul_v64i8:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    addi sp, sp, -16
 ; RV32-NEXT:    .cfi_def_cfa_offset 16
 ; RV32-NEXT:    sw ra, 12(sp) # 4-byte Folded Spill
 ; RV32-NEXT:    .cfi_offset ra, -4
-; RV32-NEXT:    lui a2, %hi(.LCPI72_0)
-; RV32-NEXT:    addi a2, a2, %lo(.LCPI72_0)
-; RV32-NEXT:    li a3, 32
-; RV32-NEXT:    vsetvli zero, a3, e32, m8, ta, ma
-; RV32-NEXT:    vle32.v v16, (a2)
 ; RV32-NEXT:    mv a2, a0
-; RV32-NEXT:    vmsltu.vx v12, v16, a1
+; RV32-NEXT:    li a0, 32
+; RV32-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
 ; RV32-NEXT:    vid.v v16
+; RV32-NEXT:    vadd.vx v24, v16, a0
+; RV32-NEXT:    vmsltu.vx v12, v24, a1
 ; RV32-NEXT:    vmsltu.vx v13, v16, a1
 ; RV32-NEXT:    vsetivli zero, 8, e8, mf2, ta, ma
 ; RV32-NEXT:    vslideup.vi v13, v12, 4
-; RV32-NEXT:    li a0, 64
-; RV32-NEXT:    vsetvli zero, a0, e8, m4, ta, ma
+; RV32-NEXT:    li a1, 64
+; RV32-NEXT:    vsetvli zero, a1, e8, m4, ta, ma
 ; RV32-NEXT:    vmand.mm v0, v13, v0
 ; RV32-NEXT:    vmv.v.i v12, 1
 ; RV32-NEXT:    vmerge.vvm v8, v12, v8, v0
-; RV32-NEXT:    vslidedown.vx v12, v8, a3
+; RV32-NEXT:    vslidedown.vx v12, v8, a0
 ; RV32-NEXT:    vmul.vv v8, v8, v12
 ; RV32-NEXT:    vslidedown.vi v12, v8, 16
 ; RV32-NEXT:    vmul.vv v8, v8, v12
 ; RV32-NEXT:    vslidedown.vi v12, v8, 8
 ; RV32-NEXT:    vmul.vv v8, v8, v12
 ; RV32-NEXT:    vslidedown.vi v12, v8, 4
 ; RV32-NEXT:    vmul.vv v8, v8, v12
 ; RV32-NEXT:    vslidedown.vi v12, v8, 2

 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vpreduce_mul_v64i8:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    addi sp, sp, -16
 ; RV64-NEXT:    .cfi_def_cfa_offset 16
 ; RV64-NEXT:    sd ra, 8(sp) # 8-byte Folded Spill
 ; RV64-NEXT:    .cfi_offset ra, -8
-; RV64-NEXT:    lui a2, %hi(.LCPI72_0)
-; RV64-NEXT:    addi a2, a2, %lo(.LCPI72_0)
-; RV64-NEXT:    li a3, 32
-; RV64-NEXT:    vsetvli zero, a3, e32, m8, ta, ma
-; RV64-NEXT:    vle32.v v16, (a2)
 ; RV64-NEXT:    mv a2, a0
-; RV64-NEXT:    vmsltu.vx v12, v16, a1
+; RV64-NEXT:    li a0, 32
+; RV64-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
 ; RV64-NEXT:    vid.v v16
+; RV64-NEXT:    vadd.vx v24, v16, a0
+; RV64-NEXT:    vmsltu.vx v12, v24, a1
 ; RV64-NEXT:    vmsltu.vx v13, v16, a1
 ; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, ma
 ; RV64-NEXT:    vslideup.vi v13, v12, 4
-; RV64-NEXT:    li a0, 64
-; RV64-NEXT:    vsetvli zero, a0, e8, m4, ta, ma
+; RV64-NEXT:    li a1, 64
+; RV64-NEXT:    vsetvli zero, a1, e8, m4, ta, ma
 ; RV64-NEXT:    vmand.mm v0, v13, v0
 ; RV64-NEXT:    vmv.v.i v12, 1
 ; RV64-NEXT:    vmerge.vvm v8, v12, v8, v0
-; RV64-NEXT:    vslidedown.vx v12, v8, a3
+; RV64-NEXT:    vslidedown.vx v12, v8, a0
 ; RV64-NEXT:    vmul.vv v8, v8, v12
 ; RV64-NEXT:    vslidedown.vi v12, v8, 16
 ; RV64-NEXT:    vmul.vv v8, v8, v12
 ; RV64-NEXT:    vslidedown.vi v12, v8, 8
 ; RV64-NEXT:    vmul.vv v8, v8, v12
 ; RV64-NEXT:    vslidedown.vi v12, v8, 4
 ; RV64-NEXT:    vmul.vv v8, v8, v12
 ; RV64-NEXT:    vslidedown.vi v12, v8, 2

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-shuffle-reverse.ll

 ; CHECK-NEXT:    ret
   %res = call <16 x i1> @llvm.experimental.vector.reverse.v16i1(<16 x i1> %a)
   ret <16 x i1> %res
 }
 
 define <32 x i1> @reverse_v32i1(<32 x i1> %a) {
 ; CHECK-LABEL: reverse_v32i1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI4_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI4_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e8, m2, ta, ma
-; CHECK-NEXT:    vle8.v v8, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e8, m2, ta, ma
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v8, v8, a0
 ; CHECK-NEXT:    vmv.v.i v10, 0
 ; CHECK-NEXT:    vmerge.vim v10, v10, 1, v0
 ; CHECK-NEXT:    vrgather.vv v12, v10, v8
 ; CHECK-NEXT:    vmsne.vi v0, v12, 0
 ; CHECK-NEXT:    ret
   %res = call <32 x i1> @llvm.experimental.vector.reverse.v32i1(<32 x i1> %a)
   ret <32 x i1> %res
 }
 
 define <64 x i1> @reverse_v64i1(<64 x i1> %a) {
 ; CHECK-LABEL: reverse_v64i1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI5_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI5_0)
-; CHECK-NEXT:    li a1, 64
-; CHECK-NEXT:    vsetvli zero, a1, e8, m4, ta, ma
-; CHECK-NEXT:    vle8.v v8, (a0)
+; CHECK-NEXT:    li a0, 64
+; CHECK-NEXT:    vsetvli zero, a0, e8, m4, ta, ma
+; CHECK-NEXT:    vid.v v8
+; CHECK-NEXT:    li a0, 63
+; CHECK-NEXT:    vrsub.vx v8, v8, a0
 ; CHECK-NEXT:    vmv.v.i v12, 0
 ; CHECK-NEXT:    vmerge.vim v12, v12, 1, v0
 ; CHECK-NEXT:    vrgather.vv v16, v12, v8
 ; CHECK-NEXT:    vmsne.vi v0, v16, 0
 ; CHECK-NEXT:    ret
   %res = call <64 x i1> @llvm.experimental.vector.reverse.v64i1(<64 x i1> %a)
   ret <64 x i1> %res
 }

 ; CHECK-NEXT:    ret
   %res = call <16 x i8> @llvm.experimental.vector.reverse.v16i8(<16 x i8> %a)
   ret <16 x i8> %res
 }
 
 define <32 x i8> @reverse_v32i8(<32 x i8> %a) {
 ; CHECK-LABEL: reverse_v32i8:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI11_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI11_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e8, m2, ta, ma
-; CHECK-NEXT:    vle8.v v12, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e8, m2, ta, ma
+; CHECK-NEXT:    vid.v v10
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v12, v10, a0
 ; CHECK-NEXT:    vrgather.vv v10, v8, v12
 ; CHECK-NEXT:    vmv.v.v v8, v10
 ; CHECK-NEXT:    ret
   %res = call <32 x i8> @llvm.experimental.vector.reverse.v32i8(<32 x i8> %a)
   ret <32 x i8> %res
 }
 
 define <64 x i8> @reverse_v64i8(<64 x i8> %a) {
 ; CHECK-LABEL: reverse_v64i8:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI12_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI12_0)
-; CHECK-NEXT:    li a1, 64
-; CHECK-NEXT:    vsetvli zero, a1, e8, m4, ta, ma
-; CHECK-NEXT:    vle8.v v16, (a0)
+; CHECK-NEXT:    li a0, 64
+; CHECK-NEXT:    vsetvli zero, a0, e8, m4, ta, ma
+; CHECK-NEXT:    vid.v v12
+; CHECK-NEXT:    li a0, 63
+; CHECK-NEXT:    vrsub.vx v16, v12, a0
 ; CHECK-NEXT:    vrgather.vv v12, v8, v16
 ; CHECK-NEXT:    vmv.v.v v8, v12
 ; CHECK-NEXT:    ret
   %res = call <64 x i8> @llvm.experimental.vector.reverse.v64i8(<64 x i8> %a)
   ret <64 x i8> %res
 }
 
 define <1 x i16> @reverse_v1i16(<1 x i16> %a) {

 ; CHECK-NEXT:    ret
   %res = call <16 x i16> @llvm.experimental.vector.reverse.v16i16(<16 x i16> %a)
   ret <16 x i16> %res
 }
 
 define <32 x i16> @reverse_v32i16(<32 x i16> %a) {
 ; CHECK-LABEL: reverse_v32i16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI18_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI18_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e16, m4, ta, ma
-; CHECK-NEXT:    vle16.v v16, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e16, m4, ta, ma
+; CHECK-NEXT:    vid.v v12
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v16, v12, a0
 ; CHECK-NEXT:    vrgather.vv v12, v8, v16
 ; CHECK-NEXT:    vmv.v.v v8, v12
 ; CHECK-NEXT:    ret
   %res = call <32 x i16> @llvm.experimental.vector.reverse.v32i16(<32 x i16> %a)
   ret <32 x i16> %res
 }
 
 define <1 x i32> @reverse_v1i32(<1 x i32> %a) {

 ; CHECK-NEXT:    ret
   %res = call <16 x half> @llvm.experimental.vector.reverse.v16f16(<16 x half> %a)
   ret <16 x half> %res
 }
 
 define <32 x half> @reverse_v32f16(<32 x half> %a) {
 ; CHECK-LABEL: reverse_v32f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI33_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI33_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e16, m4, ta, ma
-; CHECK-NEXT:    vle16.v v16, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e16, m4, ta, ma
+; CHECK-NEXT:    vid.v v12
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v16, v12, a0
 ; CHECK-NEXT:    vrgather.vv v12, v8, v16
 ; CHECK-NEXT:    vmv.v.v v8, v12
 ; CHECK-NEXT:    ret
   %res = call <32 x half> @llvm.experimental.vector.reverse.v32f16(<32 x half> %a)
   ret <32 x half> %res
 }
 
 define <1 x float> @reverse_v1f32(<1 x float> %a) {

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-store.ll

 ; RV64-NEXT:    ret
   store <4 x i8> <i8 0, i8 1, i8 2, i8 3>, ptr %p
   ret void
 }
 
 define void @store_constant_v2i8_align1(ptr %p) {
 ; CHECK-LABEL: store_constant_v2i8_align1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 2, e8, mf8, ta, ma
-; CHECK-NEXT:    vmv.v.i v8, 3
-; CHECK-NEXT:    vid.v v9
-; CHECK-NEXT:    li a1, 3
-; CHECK-NEXT:    vmadd.vx v9, a1, v8
-; CHECK-NEXT:    vse8.v v9, (a0)
+; CHECK-NEXT:    li a1, 1539
+; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; CHECK-NEXT:    vmv.s.x v8, a1
+; CHECK-NEXT:    vse8.v v8, (a0)
 ; CHECK-NEXT:    ret
   store <2 x i8> <i8 3, i8 6>, ptr %p, align 1
   ret void
 }
 
 define void @store_constant_splat_v2i8(ptr %p) {
 ; CHECK-LABEL: store_constant_splat_v2i8:
 ; CHECK:       # %bb.0:

llvm/test/CodeGen/RISCV/rvv/shuffle-reverse.ll

 ; CHECK-NEXT:    ret
   %v16i8 = shufflevector <16 x i8> %a, <16 x i8> undef, <16 x i32> <i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <16 x i8> %v16i8
 }
 
 define <32 x i8> @v16i8_2(<16 x i8> %a, <16 x i8> %b) {
 ; RV32-LABEL: v16i8_2:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    lui a0, %hi(.LCPI7_0)
-; RV32-NEXT:    addi a0, a0, %lo(.LCPI7_0)
-; RV32-NEXT:    li a1, 32
-; RV32-NEXT:    vsetvli zero, a1, e8, m2, ta, ma
-; RV32-NEXT:    vle8.v v12, (a0)
-; RV32-NEXT:    vmv1r.v v14, v9
-; RV32-NEXT:    vrgather.vv v10, v8, v12
-; RV32-NEXT:    vid.v v8
-; RV32-NEXT:    vrsub.vi v8, v8, 15
-; RV32-NEXT:    lui a0, 16
-; RV32-NEXT:    addi a0, a0, -1
+; RV32-NEXT:    vmv1r.v v12, v9
+; RV32-NEXT:    li a0, 32
+; RV32-NEXT:    vsetvli zero, a0, e8, m2, ta, ma
+; RV32-NEXT:    vid.v v14
+; RV32-NEXT:    li a1, 31
+; RV32-NEXT:    vrsub.vx v16, v14, a1
+; RV32-NEXT:    vrgather.vv v10, v8, v16
+; RV32-NEXT:    vrsub.vi v8, v14, 15
+; RV32-NEXT:    lui a1, 16
+; RV32-NEXT:    addi a1, a1, -1
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-NEXT:    vmv.v.x v0, a0
-; RV32-NEXT:    vsetvli zero, a1, e8, m2, ta, mu
-; RV32-NEXT:    vrgather.vv v10, v14, v8, v0.t
+; RV32-NEXT:    vmv.v.x v0, a1
+; RV32-NEXT:    vsetvli zero, a0, e8, m2, ta, mu
+; RV32-NEXT:    vrgather.vv v10, v12, v8, v0.t
 ; RV32-NEXT:    vmv.v.v v8, v10
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: v16i8_2:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    lui a0, %hi(.LCPI7_0)
-; RV64-NEXT:    addi a0, a0, %lo(.LCPI7_0)
-; RV64-NEXT:    li a1, 32
-; RV64-NEXT:    vsetvli zero, a1, e8, m2, ta, ma
-; RV64-NEXT:    vle8.v v12, (a0)
-; RV64-NEXT:    vmv1r.v v14, v9
-; RV64-NEXT:    vrgather.vv v10, v8, v12
-; RV64-NEXT:    vid.v v8
-; RV64-NEXT:    vrsub.vi v8, v8, 15
-; RV64-NEXT:    lui a0, 16
-; RV64-NEXT:    addiw a0, a0, -1
+; RV64-NEXT:    vmv1r.v v12, v9
+; RV64-NEXT:    li a0, 32
+; RV64-NEXT:    vsetvli zero, a0, e8, m2, ta, ma
+; RV64-NEXT:    vid.v v14
+; RV64-NEXT:    li a1, 31
+; RV64-NEXT:    vrsub.vx v16, v14, a1
+; RV64-NEXT:    vrgather.vv v10, v8, v16
+; RV64-NEXT:    vrsub.vi v8, v14, 15
+; RV64-NEXT:    lui a1, 16
+; RV64-NEXT:    addiw a1, a1, -1
 ; RV64-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV64-NEXT:    vmv.v.x v0, a0
-; RV64-NEXT:    vsetvli zero, a1, e8, m2, ta, mu
-; RV64-NEXT:    vrgather.vv v10, v14, v8, v0.t
+; RV64-NEXT:    vmv.v.x v0, a1
+; RV64-NEXT:    vsetvli zero, a0, e8, m2, ta, mu
+; RV64-NEXT:    vrgather.vv v10, v12, v8, v0.t
 ; RV64-NEXT:    vmv.v.v v8, v10
 ; RV64-NEXT:    ret
   %v32i8 = shufflevector <16 x i8> %a, <16 x i8> %b,  <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <32 x i8> %v32i8
 }
 
 define <2 x i16> @v2i16(<2 x i16> %a) {
 ; CHECK-LABEL: v2i16:

 ; CHECK-NEXT:    ret
   %v16i16 = shufflevector <16 x i16> %a, <16 x i16> undef,  <16 x i32> <i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <16 x i16> %v16i16
 }
 
 define <32 x i16> @v16i16_2(<16 x i16> %a, <16 x i16> %b) {
 ; RV32-LABEL: v16i16_2:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    lui a0, %hi(.LCPI15_0)
-; RV32-NEXT:    addi a0, a0, %lo(.LCPI15_0)
-; RV32-NEXT:    li a1, 32
-; RV32-NEXT:    vsetvli zero, a1, e16, m4, ta, ma
-; RV32-NEXT:    vle16.v v20, (a0)
 ; RV32-NEXT:    vmv2r.v v16, v10
 ; RV32-NEXT:    vmv2r.v v12, v8
-; RV32-NEXT:    vrgather.vv v8, v12, v20
-; RV32-NEXT:    vid.v v12
-; RV32-NEXT:    vrsub.vi v12, v12, 15
-; RV32-NEXT:    lui a0, 16
-; RV32-NEXT:    addi a0, a0, -1
+; RV32-NEXT:    li a0, 32
+; RV32-NEXT:    vsetvli zero, a0, e16, m4, ta, ma
+; RV32-NEXT:    vid.v v20
+; RV32-NEXT:    li a1, 31
+; RV32-NEXT:    vrsub.vx v24, v20, a1
+; RV32-NEXT:    vrgather.vv v8, v12, v24
+; RV32-NEXT:    vrsub.vi v12, v20, 15
+; RV32-NEXT:    lui a1, 16
+; RV32-NEXT:    addi a1, a1, -1
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-NEXT:    vmv.v.x v0, a0
-; RV32-NEXT:    vsetvli zero, a1, e16, m4, ta, mu
+; RV32-NEXT:    vmv.v.x v0, a1
+; RV32-NEXT:    vsetvli zero, a0, e16, m4, ta, mu
 ; RV32-NEXT:    vrgather.vv v8, v16, v12, v0.t
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: v16i16_2:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    lui a0, %hi(.LCPI15_0)
-; RV64-NEXT:    addi a0, a0, %lo(.LCPI15_0)
-; RV64-NEXT:    li a1, 32
-; RV64-NEXT:    vsetvli zero, a1, e16, m4, ta, ma
-; RV64-NEXT:    vle16.v v20, (a0)
 ; RV64-NEXT:    vmv2r.v v16, v10
 ; RV64-NEXT:    vmv2r.v v12, v8
-; RV64-NEXT:    vrgather.vv v8, v12, v20
-; RV64-NEXT:    vid.v v12
-; RV64-NEXT:    vrsub.vi v12, v12, 15
-; RV64-NEXT:    lui a0, 16
-; RV64-NEXT:    addiw a0, a0, -1
+; RV64-NEXT:    li a0, 32
+; RV64-NEXT:    vsetvli zero, a0, e16, m4, ta, ma
+; RV64-NEXT:    vid.v v20
+; RV64-NEXT:    li a1, 31
+; RV64-NEXT:    vrsub.vx v24, v20, a1
+; RV64-NEXT:    vrgather.vv v8, v12, v24
+; RV64-NEXT:    vrsub.vi v12, v20, 15
+; RV64-NEXT:    lui a1, 16
+; RV64-NEXT:    addiw a1, a1, -1
 ; RV64-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV64-NEXT:    vmv.v.x v0, a0
-; RV64-NEXT:    vsetvli zero, a1, e16, m4, ta, mu
+; RV64-NEXT:    vmv.v.x v0, a1
+; RV64-NEXT:    vsetvli zero, a0, e16, m4, ta, mu
 ; RV64-NEXT:    vrgather.vv v8, v16, v12, v0.t
 ; RV64-NEXT:    ret
   %v32i16 = shufflevector <16 x i16> %a, <16 x i16> %b,  <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <32 x i16> %v32i16
 }
 
 define <2 x i32> @v2i32(<2 x i32> %a) {
 ; CHECK-LABEL: v2i32:

 ; CHECK-NEXT:    ret
   %v16i32 = shufflevector <16 x i32> %a, <16 x i32> undef,  <16 x i32> <i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <16 x i32> %v16i32
 }
 
 define <32 x i32> @v16i32_2(<16 x i32> %a, <16 x i32> %b) {
 ; RV32-LABEL: v16i32_2:
 ; RV32:       # %bb.0:
-; RV32-NEXT:    lui a0, %hi(.LCPI23_0)
-; RV32-NEXT:    addi a0, a0, %lo(.LCPI23_0)
-; RV32-NEXT:    li a1, 32
-; RV32-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV32-NEXT:    vle32.v v0, (a0)
-; RV32-NEXT:    vmv4r.v v24, v12
+; RV32-NEXT:    addi sp, sp, -16
+; RV32-NEXT:    .cfi_def_cfa_offset 16
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    slli a0, a0, 4
+; RV32-NEXT:    sub sp, sp, a0
+; RV32-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
+; RV32-NEXT:    vmv4r.v v16, v12
+; RV32-NEXT:    addi a0, sp, 16
+; RV32-NEXT:    vs8r.v v16, (a0) # Unknown-size Folded Spill
 ; RV32-NEXT:    vmv4r.v v16, v8
-; RV32-NEXT:    vrgather.vv v8, v16, v0
-; RV32-NEXT:    vid.v v16
-; RV32-NEXT:    vrsub.vi v16, v16, 15
-; RV32-NEXT:    lui a0, 16
-; RV32-NEXT:    addi a0, a0, -1
+; RV32-NEXT:    li a0, 32
+; RV32-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV32-NEXT:    vid.v v0
+; RV32-NEXT:    li a1, 31
+; RV32-NEXT:    vrsub.vx v8, v0, a1
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    slli a1, a1, 3
+; RV32-NEXT:    add a1, sp, a1
+; RV32-NEXT:    addi a1, a1, 16
+; RV32-NEXT:    vs8r.v v8, (a1) # Unknown-size Folded Spill
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    slli a1, a1, 3
+; RV32-NEXT:    add a1, sp, a1
+; RV32-NEXT:    addi a1, a1, 16
+; RV32-NEXT:    vl8r.v v24, (a1) # Unknown-size Folded Reload
+; RV32-NEXT:    vrgather.vv v8, v16, v24
+; RV32-NEXT:    vrsub.vi v16, v0, 15
+; RV32-NEXT:    lui a1, 16
+; RV32-NEXT:    addi a1, a1, -1
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-NEXT:    vmv.v.x v0, a0
-; RV32-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
+; RV32-NEXT:    vmv.v.x v0, a1
+; RV32-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV32-NEXT:    addi a0, sp, 16
+; RV32-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
 ; RV32-NEXT:    vrgather.vv v8, v24, v16, v0.t
+; RV32-NEXT:    csrr a0, vlenb
+; RV32-NEXT:    slli a0, a0, 4
+; RV32-NEXT:    add sp, sp, a0
+; RV32-NEXT:    addi sp, sp, 16
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: v16i32_2:
 ; RV64:       # %bb.0:
-; RV64-NEXT:    lui a0, %hi(.LCPI23_0)
-; RV64-NEXT:    addi a0, a0, %lo(.LCPI23_0)
-; RV64-NEXT:    li a1, 32
-; RV64-NEXT:    vsetvli zero, a1, e32, m8, ta, ma
-; RV64-NEXT:    vle32.v v0, (a0)
-; RV64-NEXT:    vmv4r.v v24, v12
+; RV64-NEXT:    addi sp, sp, -16
+; RV64-NEXT:    .cfi_def_cfa_offset 16
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    slli a0, a0, 4
+; RV64-NEXT:    sub sp, sp, a0
+; RV64-NEXT:    .cfi_escape 0x0f, 0x0d, 0x72, 0x00, 0x11, 0x10, 0x22, 0x11, 0x10, 0x92, 0xa2, 0x38, 0x00, 0x1e, 0x22 # sp + 16 + 16 * vlenb
+; RV64-NEXT:    vmv4r.v v16, v12
+; RV64-NEXT:    addi a0, sp, 16
+; RV64-NEXT:    vs8r.v v16, (a0) # Unknown-size Folded Spill
 ; RV64-NEXT:    vmv4r.v v16, v8
-; RV64-NEXT:    vrgather.vv v8, v16, v0
-; RV64-NEXT:    vid.v v16
-; RV64-NEXT:    vrsub.vi v16, v16, 15
-; RV64-NEXT:    lui a0, 16
-; RV64-NEXT:    addiw a0, a0, -1
+; RV64-NEXT:    li a0, 32
+; RV64-NEXT:    vsetvli zero, a0, e32, m8, ta, ma
+; RV64-NEXT:    vid.v v0
+; RV64-NEXT:    li a1, 31
+; RV64-NEXT:    vrsub.vx v8, v0, a1
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    slli a1, a1, 3
+; RV64-NEXT:    add a1, sp, a1
+; RV64-NEXT:    addi a1, a1, 16
+; RV64-NEXT:    vs8r.v v8, (a1) # Unknown-size Folded Spill
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    slli a1, a1, 3
+; RV64-NEXT:    add a1, sp, a1
+; RV64-NEXT:    addi a1, a1, 16
+; RV64-NEXT:    vl8r.v v24, (a1) # Unknown-size Folded Reload
+; RV64-NEXT:    vrgather.vv v8, v16, v24
+; RV64-NEXT:    vrsub.vi v16, v0, 15
+; RV64-NEXT:    lui a1, 16
+; RV64-NEXT:    addiw a1, a1, -1
 ; RV64-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV64-NEXT:    vmv.v.x v0, a0
-; RV64-NEXT:    vsetvli zero, a1, e32, m8, ta, mu
+; RV64-NEXT:    vmv.v.x v0, a1
+; RV64-NEXT:    vsetvli zero, a0, e32, m8, ta, mu
+; RV64-NEXT:    addi a0, sp, 16
+; RV64-NEXT:    vl8r.v v24, (a0) # Unknown-size Folded Reload
 ; RV64-NEXT:    vrgather.vv v8, v24, v16, v0.t
+; RV64-NEXT:    csrr a0, vlenb
+; RV64-NEXT:    slli a0, a0, 4
+; RV64-NEXT:    add sp, sp, a0
+; RV64-NEXT:    addi sp, sp, 16
 ; RV64-NEXT:    ret
   %v32i32 = shufflevector <16 x i32> %a, <16 x i32> %b,  <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <32 x i32> %v32i32
 }
 
 define <2 x i64> @v2i64(<2 x i64> %a) {
 ; CHECK-LABEL: v2i64:
 ; CHECK:       # %bb.0:

 ; CHECK-NEXT:    ret
   %v16f16 = shufflevector <16 x half> %a, <16 x half> undef,  <16 x i32> <i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <16 x half> %v16f16
 }
 
 define <32 x half> @v16f16_2(<16 x half> %a) {
 ; CHECK-LABEL: v16f16_2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI35_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI35_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e16, m4, ta, ma
-; CHECK-NEXT:    vle16.v v16, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e16, m4, ta, ma
+; CHECK-NEXT:    vid.v v12
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v16, v12, a0
 ; CHECK-NEXT:    vrgather.vv v12, v8, v16
 ; CHECK-NEXT:    vmv.v.v v8, v12
 ; CHECK-NEXT:    ret
   %v32f16 = shufflevector <16 x half> %a, <16 x half> undef,  <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <32 x half> %v32f16
 }
 
 define <2 x float> @v2f32(<2 x float> %a) {

 ; RV64-NEXT:    ret
   %v8f64 = shufflevector <4 x double> %a, <4 x double> %b, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <8 x double> %v8f64
 }
 
 define <32 x i8> @v32i8(<32 x i8> %a) {
 ; CHECK-LABEL: v32i8:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    lui a0, %hi(.LCPI46_0)
-; CHECK-NEXT:    addi a0, a0, %lo(.LCPI46_0)
-; CHECK-NEXT:    li a1, 32
-; CHECK-NEXT:    vsetvli zero, a1, e8, m2, ta, ma
-; CHECK-NEXT:    vle8.v v12, (a0)
+; CHECK-NEXT:    li a0, 32
+; CHECK-NEXT:    vsetvli zero, a0, e8, m2, ta, ma
+; CHECK-NEXT:    vid.v v10
+; CHECK-NEXT:    li a0, 31
+; CHECK-NEXT:    vrsub.vx v12, v10, a0
 ; CHECK-NEXT:    vrgather.vv v10, v8, v12
 ; CHECK-NEXT:    vmv.v.v v8, v10
 ; CHECK-NEXT:    ret
   %v32i8 = shufflevector <32 x i8> %a, <32 x i8> undef, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <32 x i8> %v32i8
 }