[RISCV] Use v(f)slide1up for shuffle+insert idiom

[reames]

Summary

This is pretty straight forward in the basic form. I did need to move the slideup matching earlier, but that looks generally profitable on it's own.

As follow ups, I plan to explore the v(f)slide1down variants, and see what I can do to canonicalize the shuffle then insert pattern (see _inverse tests at the end of the vslide1up.ll test).

[reames]

llvm/test/CodeGen/RISCV/rvv/fixed-vector-shuffle-vslide1up.ll

65–67

I do need to figure out why these aren't matching. My guess is that we're canonicalizing to a bitcast somewhere, need to track that down. The delta is an improvement even without the vslide1up match, so I think this can be a separate change.

[frasercrmck]

All of the tests are for two-element vectors. Is it worth adding further testing?

I played around with some other examples; should all of these also be lowered to this pattern?

; This one matches
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

; I wouldn't have thought the splat index should matter
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 1, i32 4, i32 5, i32 6>

; Should be the same as the first example, but doesn't match
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v2 = shufflevector <4 x i8> %vb, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

I'm admittedly a bit out of the loop and the shuffle stuff is the first to go, so I may be missing something obvious here, sorry.

[reames]

Rebase over corrected tests, and fix one bug. I'd forgotten to check that the entire tail was being written. We could handle this via the pass through argument, but since we'd have to materialize the splat source it isn't profitable. Instead, just limit the transform to avoid that case. (see neg1 test)

[reames]

In D151468#4375309, @frasercrmck wrote:

All of the tests are for two-element vectors. Is it worth adding further testing?

Actually, there were test for 4 x E vectors, I'd just gotten the shuffle mask wrong on every single one of them. :( Fixed, and rebased, so the tests should make a lot more sense now.

I played around with some other examples; should all of these also be lowered to this pattern?

; This one matches
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

; I wouldn't have thought the splat index should matter
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 1, i32 4, i32 5, i32 6>

This is an artifact of where I put the matching code. This isn't a sub-vector insert, and I'd placed the vslide1up match below that check. Good catch. If you're okay with it, I'd like to land this as is, and then handle this in a separate patch. I need to write dedicated matching for the vslide1down case anyways, and I'd like to have the test coverage in tree already.

; Should be the same as the first example, but doesn't match
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v2 = shufflevector <4 x i8> %vb, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

This looks identical to the new vslide1up_4xi8_swapped test. It does match on RV64, are you maybe testing this on RV32? There's something a bit odd going on with that one; we're widening the op type on the insert. I have not looked into that one yet.

[frasercrmck]

LGTM with a minor typo

In D151468#4376270, @reames wrote:

In D151468#4375309, @frasercrmck wrote:

; This one matches
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

; I wouldn't have thought the splat index should matter
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
%v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 1, i32 4, i32 5, i32 6>

This is an artifact of where I put the matching code. This isn't a sub-vector insert, and I'd placed the vslide1up match below that check. Good catch. If you're okay with it, I'd like to land this as is, and then handle this in a separate patch. I need to write dedicated matching for the vslide1down case anyways, and I'd like to have the test coverage in tree already.

Yes that sounds good.

; Should be the same as the first example, but doesn't match
%vb = insertelement <4 x i8> poison, i8 %b, i64 0
%v2 = shufflevector <4 x i8> %vb, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>

This looks identical to the new vslide1up_4xi8_swapped test. It does match on RV64, are you maybe testing this on RV32? There's something a bit odd going on with that one; we're widening the op type on the insert. I have not looked into that one yet.

Yes good spot - I was testing only with RV32, which I don't normally do when playing around with things.

llvm/test/CodeGen/RISCV/rvv/fixed-vector-shuffle-vslide1up.ll

309

typo: materialize

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

static SDValue lowerVECTOR_SHUFFLEAsVSlideup(const SDLoc &DL, MVT VT,
 
   bool OpsSwapped = Mask[Index] < (int)NumElts;
   SDValue InPlace = OpsSwapped ? V2 : V1;
   SDValue ToInsert = OpsSwapped ? V1 : V2;
 
   MVT XLenVT = Subtarget.getXLenVT();
   MVT ContainerVT = getContainerForFixedLengthVector(DAG, VT, Subtarget);
   auto TrueMask = getDefaultVLOps(VT, ContainerVT, DL, DAG, Subtarget).first;
+  if (Index == 1 && NumSubElts + Index == (int)NumElts &&
+      isa<BuildVectorSDNode>(InPlace)) {
+    if (SDValue Splat = cast<BuildVectorSDNode>(InPlace)->getSplatValue()) {
+      auto OpCode =
+        VT.isFloatingPoint() ? RISCVISD::VFSLIDE1UP_VL : RISCVISD::VSLIDE1UP_VL;
+      auto Vec = DAG.getNode(OpCode, DL, ContainerVT,
+                             DAG.getUNDEF(ContainerVT),
+                             convertToScalableVector(ContainerVT, ToInsert, DAG, Subtarget),
+                             Splat, TrueMask,
+                             DAG.getConstant(NumSubElts + Index, DL, XLenVT));
+      return convertFromScalableVector(VT, Vec, DAG, Subtarget);
+    }
+  }
+
   // We slide up by the index that the subvector is being inserted at, and set
   // VL to the index + the number of elements being inserted.
   unsigned Policy = RISCVII::TAIL_UNDISTURBED_MASK_UNDISTURBED | RISCVII::MASK_AGNOSTIC;
   // If the we're adding a suffix to the in place vector, i.e. inserting right
   // up to the very end of it, then we don't actually care about the tail.
   if (NumSubElts + Index >= (int)NumElts)
     Policy |= RISCVII::TAIL_AGNOSTIC;
   SDValue Slideup = getVSlideup(
static SDValue lowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
 
   // If this is a deinterleave and we can widen the vector, then we can use
   // vnsrl to deinterleave.
   if (isDeinterleaveShuffle(VT, ContainerVT, V1, V2, Mask, Subtarget)) {
     return getDeinterleaveViaVNSRL(DL, VT, V1.getOperand(0), Mask[0] == 0,
                                    Subtarget, DAG);
   }
 
+  if (SDValue V =
+          lowerVECTOR_SHUFFLEAsVSlideup(DL, VT, V1, V2, Mask, Subtarget, DAG))
+    return V;
+
   // Detect an interleave shuffle and lower to
   // (vmaccu.vx (vwaddu.vx lohalf(V1), lohalf(V2)), lohalf(V2), (2^eltbits - 1))
   int EvenSrc, OddSrc;
   if (isInterleaveShuffle(Mask, VT, EvenSrc, OddSrc, Subtarget)) {
     // Extract the halves of the vectors.
     MVT HalfVT = VT.getHalfNumVectorElementsVT();
 
     int Size = Mask.size();
     SDValue EvenV, OddV;
     assert(EvenSrc >= 0 && "Undef source?");
     EvenV = (EvenSrc / Size) == 0 ? V1 : V2;
     EvenV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, HalfVT, EvenV,
                         DAG.getConstant(EvenSrc % Size, DL, XLenVT));
 
     assert(OddSrc >= 0 && "Undef source?");
     OddV = (OddSrc / Size) == 0 ? V1 : V2;
     OddV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, HalfVT, OddV,
                        DAG.getConstant(OddSrc % Size, DL, XLenVT));
 
     return getWideningInterleave(EvenV, OddV, DL, DAG, Subtarget);
   }
 
-  if (SDValue V =
-          lowerVECTOR_SHUFFLEAsVSlideup(DL, VT, V1, V2, Mask, Subtarget, DAG))
-    return V;
-
   // Detect shuffles which can be re-expressed as vector selects; these are
   // shuffles in which each element in the destination is taken from an element
   // at the corresponding index in either source vectors.
   bool IsSelect = all_of(enumerate(Mask), [&](const auto &MaskIdx) {
     int MaskIndex = MaskIdx.value();
     return MaskIndex < 0 || MaskIdx.index() == (unsigned)MaskIndex % NumElts;
   });
 

llvm/test/CodeGen/RISCV/rvv/fixed-vector-shuffle-transpose.ll

 ; CHECK-NEXT:    ret
   %tmp0 = shufflevector <8 x i16> %v0, <8 x i16> %v1, <8 x i32> <i32 1, i32 9, i32 3, i32 11, i32 5, i32 13, i32 7, i32 15>
   ret <8 x i16> %tmp0
 }
 
 define <2 x i32> @trn1.v2i32(<2 x i32> %v0, <2 x i32> %v1) {
 ; CHECK-LABEL: trn1.v2i32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; CHECK-NEXT:    vwaddu.vv v10, v8, v9
-; CHECK-NEXT:    li a0, -1
-; CHECK-NEXT:    vwmaccu.vx v10, a0, v9
-; CHECK-NEXT:    vmv1r.v v8, v10
+; CHECK-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; CHECK-NEXT:    vslideup.vi v8, v9, 1
 ; CHECK-NEXT:    ret
   %tmp0 = shufflevector <2 x i32> %v0, <2 x i32> %v1, <2 x i32> <i32 0, i32 2>
   ret <2 x i32> %tmp0
 }
 
 define <2 x i32> @trn2.v2i32(<2 x i32> %v0, <2 x i32> %v1) {
 ; CHECK-LABEL: trn2.v2i32:
 ; CHECK:       # %bb.0:

 ; CHECK-NEXT:    ret
   %tmp0 = shufflevector <2 x i64> %v0, <2 x i64> %v1, <2 x i32> <i32 1, i32 3>
   ret <2 x i64> %tmp0
 }
 
 define <2 x float> @trn1.v2f32(<2 x float> %v0, <2 x float> %v1) {
 ; CHECK-LABEL: trn1.v2f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; CHECK-NEXT:    vwaddu.vv v10, v8, v9
-; CHECK-NEXT:    li a0, -1
-; CHECK-NEXT:    vwmaccu.vx v10, a0, v9
-; CHECK-NEXT:    vmv1r.v v8, v10
+; CHECK-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; CHECK-NEXT:    vslideup.vi v8, v9, 1
 ; CHECK-NEXT:    ret
   %tmp0 = shufflevector <2 x float> %v0, <2 x float> %v1, <2 x i32> <i32 0, i32 2>
   ret <2 x float> %tmp0
 }
 
 define <2 x float> @trn2.v2f32(<2 x float> %v0, <2 x float> %v1) {
 ; CHECK-LABEL: trn2.v2f32:
 ; CHECK:       # %bb.0:

llvm/test/CodeGen/RISCV/rvv/fixed-vector-shuffle-vslide1up.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc -mtriple=riscv32 -mattr=+m,+v,+f,+d,+zfh,+experimental-zvfh -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32
 ; RUN: llc -mtriple=riscv64 -mattr=+m,+v,+f,+d,+zfh,+experimental-zvfh -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 define <2 x i8> @vslide1up_2xi8(<2 x i8> %v, i8 %b) {
 ; CHECK-LABEL: vslide1up_2xi8:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e8, mf8, ta, ma
-; CHECK-NEXT:    vmv.v.x v10, a0
-; CHECK-NEXT:    vsetivli zero, 1, e8, mf8, ta, ma
-; CHECK-NEXT:    vwaddu.vv v9, v10, v8
-; CHECK-NEXT:    li a0, -1
-; CHECK-NEXT:    vwmaccu.vx v9, a0, v8
+; CHECK-NEXT:    vslide1up.vx v9, v8, a0
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <2 x i8> poison, i8 %b, i64 0
   %v1 = shufflevector <2 x i8> %v, <2 x i8> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x i8> %v1
 }
 
 define <4 x i8> @vslide1up_4xi8(<4 x i8> %v, i8 %b) {
 ; RV32-LABEL: vslide1up_4xi8:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
 ; RV32-NEXT:    vmv.s.x v9, a0
 ; RV32-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
 ; RV32-NEXT:    vslideup.vi v9, v8, 1
 ; RV32-NEXT:    vmv1r.v v8, v9
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_4xi8:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; RV64-NEXT:    vmv.v.x v9, a0
-; RV64-NEXT:    vslideup.vi v9, v8, 1
+; RV64-NEXT:    vslide1up.vx v9, v8, a0
 ; RV64-NEXT:    vmv1r.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <4 x i8> poison, i8 %b, i64 0
   %v1 = shufflevector <4 x i8> %v, <4 x i8> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x i8> %v1
 }
 
 define <4 x i8> @vslide1up_4xi8_swapped(<4 x i8> %v, i8 %b) {
 ; RV32-LABEL: vslide1up_4xi8_swapped:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
 ; RV32-NEXT:    vmv.s.x v9, a0
 ; RV32-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
 ; RV32-NEXT:    vslideup.vi v9, v8, 1
 ; RV32-NEXT:    vmv1r.v v8, v9
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_4xi8_swapped:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 4, e8, mf4, ta, ma
-; RV64-NEXT:    vmv.v.x v9, a0
-; RV64-NEXT:    vslideup.vi v9, v8, 1
+; RV64-NEXT:    vslide1up.vx v9, v8, a0
 ; RV64-NEXT:    vmv1r.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <4 x i8> poison, i8 %b, i64 0
   %v1 = shufflevector <4 x i8> %vb, <4 x i8> %v, <4 x i32> <i32 0, i32 4, i32 5, i32 6>
   ret <4 x i8> %v1
 }
 
 define <2 x i16> @vslide1up_2xi16(<2 x i16> %v, i16 %b) {
 ; RV32-LABEL: vslide1up_2xi16:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-NEXT:    vmv.s.x v10, a0
-; RV32-NEXT:    vsetvli zero, zero, e16, mf4, ta, ma
-; RV32-NEXT:    vwaddu.vv v9, v10, v8
-; RV32-NEXT:    li a0, -1
-; RV32-NEXT:    vwmaccu.vx v9, a0, v8
+; RV32-NEXT:    vmv.s.x v9, a0
+; RV32-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
+; RV32-NEXT:    vslideup.vi v9, v8, 1
 ; RV32-NEXT:    vmv1r.v v8, v9
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_2xi16:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
-; RV64-NEXT:    vmv.v.x v10, a0
-; RV64-NEXT:    vsetivli zero, 1, e16, mf4, ta, ma
-; RV64-NEXT:    vwaddu.vv v9, v10, v8
-; RV64-NEXT:    li a0, -1
-; RV64-NEXT:    vwmaccu.vx v9, a0, v8
+; RV64-NEXT:    vslide1up.vx v9, v8, a0
 ; RV64-NEXT:    vmv1r.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <2 x i16> poison, i16 %b, i64 0
   %v1 = shufflevector <2 x i16> %v, <2 x i16> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x i16> %v1
 }
 
 define <4 x i16> @vslide1up_4xi16(<4 x i16> %v, i16 %b) {
 ; RV32-LABEL: vslide1up_4xi16:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
-; RV32-NEXT:    vmv.v.x v9, a0
-; RV32-NEXT:    vslideup.vi v9, v8, 1
+; RV32-NEXT:    vslide1up.vx v9, v8, a0
 ; RV32-NEXT:    vmv1r.v v8, v9
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_4xi16:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, ma
 ; RV64-NEXT:    vmv.s.x v9, a0
 ; RV64-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
 ; RV64-NEXT:    vslideup.vi v9, v8, 1
 ; RV64-NEXT:    vmv1r.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <4 x i16> poison, i16 %b, i64 0
   %v1 = shufflevector <4 x i16> %v, <4 x i16> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x i16> %v1
 }
 
 define <2 x i32> @vslide1up_2xi32(<2 x i32> %v, i32 %b) {
 ; RV32-LABEL: vslide1up_2xi32:
 ; RV32:       # %bb.0:
 ; RV32-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
-; RV32-NEXT:    vmv.v.x v10, a0
-; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; RV32-NEXT:    vwaddu.vv v9, v10, v8
-; RV32-NEXT:    li a0, -1
-; RV32-NEXT:    vwmaccu.vx v9, a0, v8
+; RV32-NEXT:    vslide1up.vx v9, v8, a0
 ; RV32-NEXT:    vmv1r.v v8, v9
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_2xi32:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, ma
-; RV64-NEXT:    vmv.s.x v10, a0
-; RV64-NEXT:    vsetvli zero, zero, e32, mf2, ta, ma
-; RV64-NEXT:    vwaddu.vv v9, v10, v8
-; RV64-NEXT:    li a0, -1
-; RV64-NEXT:    vwmaccu.vx v9, a0, v8
+; RV64-NEXT:    vmv.s.x v9, a0
+; RV64-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
+; RV64-NEXT:    vslideup.vi v9, v8, 1
 ; RV64-NEXT:    vmv1r.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <2 x i32> poison, i32 %b, i64 0
   %v1 = shufflevector <2 x i32> %v, <2 x i32> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x i32> %v1
 }
 
 define <4 x i32> @vslide1up_4xi32(<4 x i32> %v, i32 %b) {
 ; CHECK-LABEL: vslide1up_4xi32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
-; CHECK-NEXT:    vmv.v.x v9, a0
-; CHECK-NEXT:    vslideup.vi v9, v8, 1
+; CHECK-NEXT:    vslide1up.vx v9, v8, a0
 ; CHECK-NEXT:    vmv.v.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <4 x i32> poison, i32 %b, i64 0
   %v1 = shufflevector <4 x i32> %v, <4 x i32> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x i32> %v1
 }
 
 define <2 x i64> @vslide1up_2xi64(<2 x i64> %v, i64 %b) {

 ; RV32-NEXT:    vslideup.vi v9, v8, 1
 ; RV32-NEXT:    vmv.v.v v8, v9
 ; RV32-NEXT:    addi sp, sp, 16
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_2xi64:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
-; RV64-NEXT:    vmv.v.x v9, a0
-; RV64-NEXT:    vslideup.vi v9, v8, 1
+; RV64-NEXT:    vslide1up.vx v9, v8, a0
 ; RV64-NEXT:    vmv.v.v v8, v9
 ; RV64-NEXT:    ret
   %vb = insertelement <2 x i64> poison, i64 %b, i64 0
   %v1 = shufflevector <2 x i64> %v, <2 x i64> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x i64> %v1
 }
 
 define <4 x i64> @vslide1up_4xi64(<4 x i64> %v, i64 %b) {

 ; RV32-NEXT:    vslideup.vi v10, v8, 1
 ; RV32-NEXT:    vmv.v.v v8, v10
 ; RV32-NEXT:    addi sp, sp, 16
 ; RV32-NEXT:    ret
 ;
 ; RV64-LABEL: vslide1up_4xi64:
 ; RV64:       # %bb.0:
 ; RV64-NEXT:    vsetivli zero, 4, e64, m2, ta, ma
-; RV64-NEXT:    vmv.v.x v10, a0
-; RV64-NEXT:    vslideup.vi v10, v8, 1
+; RV64-NEXT:    vslide1up.vx v10, v8, a0
 ; RV64-NEXT:    vmv.v.v v8, v10
 ; RV64-NEXT:    ret
   %vb = insertelement <4 x i64> poison, i64 %b, i64 0
   %v1 = shufflevector <4 x i64> %v, <4 x i64> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x i64> %v1
 }
 
 define <2 x half> @vslide1up_2xf16(<2 x half> %v, half %b) {
 ; CHECK-LABEL: vslide1up_2xf16:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e16, mf4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v10, fa0
-; CHECK-NEXT:    vsetivli zero, 1, e16, mf4, ta, ma
-; CHECK-NEXT:    vwaddu.vv v9, v10, v8
-; CHECK-NEXT:    li a0, -1
-; CHECK-NEXT:    vwmaccu.vx v9, a0, v8
+; CHECK-NEXT:    vfslide1up.vf v9, v8, fa0
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <2 x half> poison, half %b, i64 0
   %v1 = shufflevector <2 x half> %v, <2 x half> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x half> %v1
 }
 
 define <4 x half> @vslide1up_4xf16(<4 x half> %v, half %b) {
 ; CHECK-LABEL: vslide1up_4xf16:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e16, mf2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vslideup.vi v9, v8, 1
+; CHECK-NEXT:    vfslide1up.vf v9, v8, fa0
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <4 x half> poison, half %b, i64 0
   %v1 = shufflevector <4 x half> %v, <4 x half> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x half> %v1
 }
 
 define <2 x float> @vslide1up_2xf32(<2 x float> %v, float %b) {
 ; CHECK-LABEL: vslide1up_2xf32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e32, mf2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v10, fa0
-; CHECK-NEXT:    vsetivli zero, 1, e32, mf2, ta, ma
-; CHECK-NEXT:    vwaddu.vv v9, v10, v8
-; CHECK-NEXT:    li a0, -1
-; CHECK-NEXT:    vwmaccu.vx v9, a0, v8
+; CHECK-NEXT:    vfslide1up.vf v9, v8, fa0
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <2 x float> poison, float %b, i64 0
   %v1 = shufflevector <2 x float> %v, <2 x float> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x float> %v1
 }
 
 define <4 x float> @vslide1up_4xf32(<4 x float> %v, float %b) {
 ; CHECK-LABEL: vslide1up_4xf32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vslideup.vi v9, v8, 1
+; CHECK-NEXT:    vfslide1up.vf v9, v8, fa0
 ; CHECK-NEXT:    vmv.v.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <4 x float> poison, float %b, i64 0
   %v1 = shufflevector <4 x float> %v, <4 x float> %vb, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
   ret <4 x float> %v1
 }
 
 define <2 x double> @vslide1up_2xf64(<2 x double> %v, double %b) {
 ; CHECK-LABEL: vslide1up_2xf64:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vslideup.vi v9, v8, 1
+; CHECK-NEXT:    vfslide1up.vf v9, v8, fa0
 ; CHECK-NEXT:    vmv.v.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <2 x double> poison, double %b, i64 0
   %v1 = shufflevector <2 x double> %v, <2 x double> %vb, <2 x i32> <i32 2, i32 0>
   ret <2 x double> %v1
 }
 
 define <4 x double> @vslide1up_4xf64(<4 x double> %v, double %b) {
 ; CHECK-LABEL: vslide1up_4xf64:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e64, m2, ta, ma
 ; CHECK-NEXT:    vfmv.v.f v10, fa0
 ; CHECK-NEXT:    vslideup.vi v10, v8, 3
 ; CHECK-NEXT:    vmv.v.v v8, v10
 ; CHECK-NEXT:    ret
   %vb = insertelement <4 x double> poison, double %b, i64 0
   %v1 = shufflevector <4 x double> %v, <4 x double> %vb, <4 x i32> <i32 4, i32 5, i32 6, i32 0>
   ret <4 x double> %v1
 }
 
+define <4 x i8> @vslide1up_4xi8_with_splat(<4 x i8> %v, i8 %b) {
+; CHECK-LABEL: vslide1up_4xi8_with_splat:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    li a1, 14
+; CHECK-NEXT:    vsetivli zero, 4, e8, mf4, ta, mu
+; CHECK-NEXT:    vmv.s.x v0, a1
+; CHECK-NEXT:    vid.v v9
+; CHECK-NEXT:    vadd.vi v10, v9, -1
+; CHECK-NEXT:    vmv.v.x v9, a0
+; CHECK-NEXT:    vrgather.vv v9, v8, v10, v0.t
+; CHECK-NEXT:    vmv1r.v v8, v9
+; CHECK-NEXT:    ret
+  %vb = insertelement <4 x i8> poison, i8 %b, i64 0
+  %v1 = shufflevector <4 x i8> %vb, <4 x i8> poison, <4 x i32> zeroinitializer
+  %v2 = shufflevector <4 x i8> %v1, <4 x i8> %v, <4 x i32> <i32 1, i32 4, i32 5, i32 6>
+  ret <4 x i8> %v2
+}
+
 define <2 x double> @vslide1up_v2f64_inverted(<2 x double> %v, double %b) {
 ; CHECK-LABEL: vslide1up_v2f64_inverted:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 2, e64, m1, ta, ma
 ; CHECK-NEXT:    vrgather.vi v9, v8, 0
 ; CHECK-NEXT:    vsetvli zero, zero, e64, m1, tu, ma
 ; CHECK-NEXT:    vfmv.s.f v9, fa0
 ; CHECK-NEXT:    vmv1r.v v8, v9

 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %v1 = shufflevector <4 x i8> %v, <4 x i8> poison, <4 x i32> <i32 undef, i32 0, i32 1, i32 2>
   %v2 = insertelement <4 x i8> %v1, i8 %b, i64 0
   ret <4 x i8> %v2
 }
 
 
-; The length of the shift is less than the suffix
+; The length of the shift is less than the suffix, since we'd have to
+; materailize the splat, using the vslide1up doesn't help us.
 define <4 x i32> @vslide1up_4xi32_neg1(<4 x i32> %v, i32 %b) {
 ; CHECK-LABEL: vslide1up_4xi32_neg1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
 ; CHECK-NEXT:    vmv.v.x v9, a0
 ; CHECK-NEXT:    vsetivli zero, 3, e32, m1, tu, ma
 ; CHECK-NEXT:    vslideup.vi v9, v8, 1
 ; CHECK-NEXT:    vmv1r.v v8, v9
 ; CHECK-NEXT:    ret
   %vb = insertelement <4 x i32> poison, i32 %b, i64 0
   %vb2 = insertelement <4 x i32> %vb, i32 %b, i64 3
   %v1 = shufflevector <4 x i32> %v, <4 x i32> %vb2, <4 x i32> <i32 4, i32 0, i32 1, i32 7>
   ret <4 x i32> %v1
 }
+
+; We don't know the scalar to do the vslide1up
+define <4 x i32> @vslide1up_4xi32_neg2(<4 x i32> %v1, <4 x i32> %v2) {
+; CHECK-LABEL: vslide1up_4xi32_neg2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetivli zero, 4, e32, m1, ta, ma
+; CHECK-NEXT:    vslideup.vi v9, v8, 1
+; CHECK-NEXT:    vmv.v.v v8, v9
+; CHECK-NEXT:    ret
+  %res = shufflevector <4 x i32> %v1, <4 x i32> %v2, <4 x i32> <i32 4, i32 0, i32 1, i32 2>
+  ret <4 x i32> %res
+}