[RISCV] Add support for VECTOR_REVERSE for scalable vector types.

[craig.topper]

Summary

I've left mask registers to a future patch as we'll need
to convert them to full vectors, shuffle, and then truncate.

[craig.topper]

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

743

This hack is to prevent tablegen from throwing a bunch of warnings about assuming a vector type is not scalable. This occurs because of my use of SDTCisSameSizeAs in the type profile for vrgather_vv_vl in tablegen.

SmallSet defaults to std::less as its comparison function. But TypeSize doesn't implement operator< natively. But it appears to match to the operator< for uint64_t since there's a conversion operator to uint64_t in TypeSize. But that only works for fixed vectors.

To workaround this, I explicitly split up the TypeSize into a std::pair. Adding an operator< to TypeSize didn't seem like a good idea. Another option might be to switch to SmallDenseSet and add a DenseMapInfo for TypeSize.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1519

I suppose we'll have to promote to i16 for this case?

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

743

Hmm. I agree that operator< for TypeSize doesn't sound right. I can see the benefit of having a DenseMapInfo for TypeSize. I guess another option is to overload the comparison for this one case? That's quite short-sighted though.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1519

Yeah, but we don't really have a good way of know if it is needed without just always doing it or deciding based on whether -riscv-vector-bits-max is set and what its value is?

We also can't promote to i16 without splitting for lmul=8.

[craig.topper]

Use a custom comparator struct in tablegen. Thought about a DenseMap, but wasn't sure what to use for empty key and tombstone key.

[khchen]

Overall LGTM, waiting for @frasercrmck's comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1519

Agree, it's really a problem, unfortunately vrgatherei16 can not support i8m8 case directly.

I think maybe compiler could emit a warning for non-workable case if user specific -riscv-vector-bits-max.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1519

Yeah, nothing really works well in this situation. It's just that with <vscale x 64 x i8> being legal it's not a problem for the future - it would be silently generating wrong code right now on many architectures.

What if, for now, we decided not to support i8 unless -riscv-vector-bits-max was passed?

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

718

I think this is an improvement over the std::pair.

[craig.topper]

Handle VLMAX>=256 for SEW=8 by checking -riscv-v-vector-bits-max.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2827

Is there an issue with using CONCAT_VECTORS?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2827

We don’t seem to have implemented CONCAT_VECTORS for scalable vectors yet so it went to the stack.

[frasercrmck]

LGTM

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

2827

Ah of course; we only added it for fixed-length vectors since it's common during legalization.

llvm/lib/Target/RISCV/RISCVISelLowering.h

 //===-- RISCVISelLowering.h - RISCV DAG Lowering Interface ------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
enum NodeType : unsigned {
   VMAND_VL,
   VMOR_VL,
   VMXOR_VL,
 
   // Set mask vector to all zeros or ones.
   VMCLR_VL,
   VMSET_VL,
 
-  // Matches the semantics of vrgather.vx with an extra operand for VL.
+  // Matches the semantics of vrgather.vx and vrgather.vv with an extra operand
+  // for VL.
   VRGATHER_VX_VL,
+  VRGATHER_VV_VL,
 
   // Vector sign/zero extend with additional mask & VL operands.
   VSEXT_VL,
   VZEXT_VL,
 
   // Memory opcodes start here.
   VLE_VL = ISD::FIRST_TARGET_MEMORY_OPCODE,
   VSE_VL,

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

 //===-- RISCVISelLowering.cpp - RISCV DAG Lowering Implementation  --------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
for (MVT VT : IntVecVTs) {
       setOperationAction(ISD::VECREDUCE_XOR, VT, Custom);
       setOperationAction(ISD::VECREDUCE_SMAX, VT, Custom);
       setOperationAction(ISD::VECREDUCE_SMIN, VT, Custom);
       setOperationAction(ISD::VECREDUCE_UMAX, VT, Custom);
       setOperationAction(ISD::VECREDUCE_UMIN, VT, Custom);
 
       setOperationAction(ISD::INSERT_SUBVECTOR, VT, Custom);
       setOperationAction(ISD::EXTRACT_SUBVECTOR, VT, Custom);
+
+      setOperationAction(ISD::VECTOR_REVERSE, VT, Custom);
     }
 
     // Expand various CCs to best match the RVV ISA, which natively supports UNE
     // but no other unordered comparisons, and supports all ordered comparisons
     // except ONE. Additionally, we expand GT,OGT,GE,OGE for optimization
     // purposes; they are expanded to their swapped-operand CCs (LT,OLT,LE,OLE),
     // and we pattern-match those back to the "original", swapping operands once
     // more. This way we catch both operations and both "vf" and "fv" forms with
const auto SetCommonVFPActions = [&](MVT VT) {
         setCondCodeAction(CC, VT, Expand);
 
       setOperationAction(ISD::VECREDUCE_FADD, VT, Custom);
       setOperationAction(ISD::VECREDUCE_SEQ_FADD, VT, Custom);
       setOperationAction(ISD::FCOPYSIGN, VT, Legal);
 
       setOperationAction(ISD::INSERT_SUBVECTOR, VT, Custom);
       setOperationAction(ISD::EXTRACT_SUBVECTOR, VT, Custom);
+
+      setOperationAction(ISD::VECTOR_REVERSE, VT, Custom);
     };
 
     if (Subtarget.hasStdExtZfh())
       for (MVT VT : F16VecVTs)
         SetCommonVFPActions(VT);
 
     if (Subtarget.hasStdExtF())
       for (MVT VT : F32VecVTs)
case ISD::VECREDUCE_XOR:
     return lowerVECREDUCE(Op, DAG);
   case ISD::VECREDUCE_FADD:
   case ISD::VECREDUCE_SEQ_FADD:
     return lowerFPVECREDUCE(Op, DAG);
   case ISD::INSERT_SUBVECTOR:
     return lowerINSERT_SUBVECTOR(Op, DAG);
   case ISD::EXTRACT_SUBVECTOR:
     return lowerEXTRACT_SUBVECTOR(Op, DAG);
+  case ISD::VECTOR_REVERSE: {
+    // Implement vector_reverse using vrgather.vv with indices determined by
+    // subtracting the id of each element from (VLMAX-1). This will convert
+    // the indices like so:
+    // (0, 1,..., VLMAX-2, VLMAX-1) -> (VLMAX-1, VLMAX-2,..., 1, 0).
+    // FIXME: This implementation doesn't work for vectors of more than 256
+    // elements for SEW==8.
+    SDLoc DL(Op);
+    MVT VecVT = Op.getSimpleValueType();
+    MVT IntVT = VecVT.changeVectorElementTypeToInteger();
+    MVT XLenVT = Subtarget.getXLenVT();
+    SDValue Mask, VL;
+    std::tie(Mask, VL) = getDefaultScalableVLOps(VecVT, DL, DAG, Subtarget);
+
+    // Calculate VLMAX-1 for the desired SEW.
+    unsigned MinElts = VecVT.getVectorMinNumElements();
+    SDValue VLMax = DAG.getNode(ISD::VSCALE, DL, XLenVT,
+                                DAG.getConstant(MinElts, DL, XLenVT));
+    SDValue VLMinus1 = DAG.getNode(ISD::SUB, DL, XLenVT, VLMax,
+                                   DAG.getConstant(1, DL, XLenVT));
+
+    // Splat VLMAX-1 taking care to handle SEW==64 on RV32.
+    bool IsRV32E64 =
+        !Subtarget.is64Bit() && IntVT.getVectorElementType() == MVT::i64;
+    SDValue SplatVL;
+    if (!IsRV32E64)
+      SplatVL = DAG.getSplatVector(IntVT, DL, VLMinus1);
+    else
+      SplatVL = DAG.getNode(RISCVISD::SPLAT_VECTOR_I64, DL, IntVT, VLMinus1);
+
+    SDValue VID = DAG.getNode(RISCVISD::VID_VL, DL, IntVT, Mask, VL);
+    SDValue Indices =
+        DAG.getNode(RISCVISD::SUB_VL, DL, IntVT, SplatVL, VID, Mask, VL);
+
+    return DAG.getNode(RISCVISD::VRGATHER_VV_VL, DL, VecVT, Op.getOperand(0),
+                       Indices, Mask, VL);
+  }
   case ISD::BUILD_VECTOR:
     return lowerBUILD_VECTOR(Op, DAG, Subtarget);
   case ISD::VECTOR_SHUFFLE:
     return lowerVECTOR_SHUFFLE(Op, DAG, Subtarget);
   case ISD::CONCAT_VECTORS: {
     // Split CONCAT_VECTORS into a series of INSERT_SUBVECTOR nodes. This is
     // better than going through the stack, as the default expansion does.
     SDLoc DL(Op);
RISCVTargetLowering::lowerFixedLengthVectorStoreToRVV(SDValue Op,
 
   // FIXME: We probably need to zero any extra bits in a byte for mask stores.
   // This is tricky to do.
 
   MVT ContainerVT =
       RISCVTargetLowering::getContainerForFixedLengthVector(DAG, VT, Subtarget);
 
   SDValue VL =
       DAG.getConstant(VT.getVectorNumElements(), DL, Subtarget.getXLenVT());
 
   SDValue NewValue =
       convertToScalableVector(ContainerVT, Store->getValue(), DAG, Subtarget);
   return DAG.getMemIntrinsicNode(
       RISCVISD::VSE_VL, DL, DAG.getVTList(MVT::Other),
       {Store->getChain(), NewValue, Store->getBasePtr(), VL},
       Store->getMemoryVT(), Store->getMemOperand());
 }
#define NODE_NAME_CASE(NODE)                                                   \
   NODE_NAME_CASE(SETCC_VL)
   NODE_NAME_CASE(VSELECT_VL)
   NODE_NAME_CASE(VMAND_VL)
   NODE_NAME_CASE(VMOR_VL)
   NODE_NAME_CASE(VMXOR_VL)
   NODE_NAME_CASE(VMCLR_VL)
   NODE_NAME_CASE(VMSET_VL)
   NODE_NAME_CASE(VRGATHER_VX_VL)
+  NODE_NAME_CASE(VRGATHER_VV_VL)
   NODE_NAME_CASE(VSEXT_VL)
   NODE_NAME_CASE(VZEXT_VL)
   NODE_NAME_CASE(VLE_VL)
   NODE_NAME_CASE(VSE_VL)
   }
   // clang-format on
   return nullptr;
 #undef NODE_NAME_CASE

llvm/lib/Target/RISCV/RISCVInstrInfoVVLPatterns.td

 
 def riscv_vrgather_vx_vl : SDNode<"RISCVISD::VRGATHER_VX_VL",
                                   SDTypeProfile<1, 4, [SDTCisVec<0>,
                                                        SDTCisSameAs<0, 1>,
                                                        SDTCisVT<2, XLenVT>,
                                                        SDTCVecEltisVT<3, i1>,
                                                        SDTCisSameNumEltsAs<0, 3>,
                                                        SDTCisVT<4, XLenVT>]>>;
+def riscv_vrgather_vv_vl : SDNode<"RISCVISD::VRGATHER_VV_VL",
+                                  SDTypeProfile<1, 4, [SDTCisVec<0>,
+                                                       SDTCisSameAs<0, 1>,
+                                                       SDTCisInt<2>,
+                                                       SDTCisSameNumEltsAs<0, 2>,
+                                                       SDTCisSameSizeAs<0, 2>,
+                                                       SDTCVecEltisVT<3, i1>,
+                                                       SDTCisSameNumEltsAs<0, 3>,
+                                                       SDTCisVT<4, XLenVT>]>>;
 
 def riscv_vselect_vl : SDNode<"RISCVISD::VSELECT_VL",
                               SDTypeProfile<1, 4, [SDTCisVec<0>,
                                                    SDTCisVec<1>,
                                                    SDTCisSameNumEltsAs<0, 1>,
                                                    SDTCVecEltisVT<1, i1>,
                                                    SDTCisSameAs<0, 2>,
                                                    SDTCisSameAs<2, 3>,

 // 17.4. Vector Register Gather Instruction
 foreach vti = AllIntegerVectors in {
   def : Pat<(vti.Vector (riscv_vmv_s_xf_vl (vti.Vector vti.RegClass:$merge),
                                            (XLenVT vti.ScalarRegClass:$rs1),
                                            (XLenVT (VLOp GPR:$vl)))),
             (!cast<Instruction>("PseudoVMV_S_X_"#vti.LMul.MX)
                 vti.RegClass:$merge,
                 (vti.Scalar vti.ScalarRegClass:$rs1), GPR:$vl, vti.SEW)>;
+  def : Pat<(vti.Vector (riscv_vrgather_vv_vl vti.RegClass:$rs2,
+                                              (vti.Vector vti.RegClass:$rs1),
+                                              (vti.Mask true_mask),
+                                              (XLenVT (VLOp GPR:$vl)))),
+            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX)
+                 vti.RegClass:$rs2, vti.RegClass:$rs1, GPR:$vl, vti.SEW)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, GPR:$rs1,
                                               (vti.Mask true_mask),
                                               (XLenVT (VLOp GPR:$vl)))),
             (!cast<Instruction>("PseudoVRGATHER_VX_"# vti.LMul.MX)
                  vti.RegClass:$rs2, GPR:$rs1, GPR:$vl, vti.SEW)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, uimm5:$imm,
                                               (vti.Mask true_mask),
                                               (XLenVT (VLOp GPR:$vl)))),
             (!cast<Instruction>("PseudoVRGATHER_VI_"# vti.LMul.MX)
                  vti.RegClass:$rs2, uimm5:$imm, GPR:$vl, vti.SEW)>;
 }
 
 } // Predicates = [HasStdExtV]
 
 let Predicates = [HasStdExtV, HasStdExtF] in {
 
 // 17.2. Floating-Point Scalar Move Instructions
 foreach vti = AllFloatVectors in {
   def : Pat<(vti.Vector (riscv_vmv_s_xf_vl (vti.Vector vti.RegClass:$merge),
                                            vti.ScalarRegClass:$rs1,
                                            (XLenVT (VLOp GPR:$vl)))),
             (!cast<Instruction>("PseudoVFMV_S_"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                 vti.RegClass:$merge,
                 (vti.Scalar vti.ScalarRegClass:$rs1), GPR:$vl, vti.SEW)>;
+  defvar ivti = GetIntVTypeInfo<vti>.Vti;
+  def : Pat<(vti.Vector (riscv_vrgather_vv_vl vti.RegClass:$rs2,
+                                              (ivti.Vector vti.RegClass:$rs1),
+                                              (vti.Mask true_mask),
+                                              (XLenVT (VLOp GPR:$vl)))),
+            (!cast<Instruction>("PseudoVRGATHER_VV_"# vti.LMul.MX)
+                 vti.RegClass:$rs2, vti.RegClass:$rs1, GPR:$vl, vti.SEW)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, GPR:$rs1,
                                               (vti.Mask true_mask),
                                               (XLenVT (VLOp GPR:$vl)))),
             (!cast<Instruction>("PseudoVRGATHER_VX_"# vti.LMul.MX)
                  vti.RegClass:$rs2, GPR:$rs1, GPR:$vl, vti.SEW)>;
   def : Pat<(vti.Vector (riscv_vrgather_vx_vl vti.RegClass:$rs2, uimm5:$imm,
                                               (vti.Mask true_mask),
                                               (XLenVT (VLOp GPR:$vl)))),

llvm/test/CodeGen/RISCV/rvv/named-vector-shuffle-reverse.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -mattr=+m,+experimental-v,+f,+d,+experimental-zfh -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK
+; RUN: llc -mtriple=riscv64 -mattr=+m,+experimental-v,+f,+d,+experimental-zfh -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK
+
+;
+; VECTOR_REVERSE - masks
+;
+; FIXME: Support for mask vectors
+
+;define <vscale x 2 x i1> @reverse_nxv2i1(<vscale x 2 x i1> %a) {
+;  %res = call <vscale x 2 x i1> @llvm.experimental.vector.reverse.nxv2i1(<vscale x 2 x i1> %a)
+;  ret <vscale x 2 x i1> %res
+;}
+;
+;define <vscale x 4 x i1> @reverse_nxv4i1(<vscale x 4 x i1> %a) {
+;  %res = call <vscale x 4 x i1> @llvm.experimental.vector.reverse.nxv4i1(<vscale x 4 x i1> %a)
+;  ret <vscale x 4 x i1> %res
+;}
+;
+;define <vscale x 8 x i1> @reverse_nxv8i1(<vscale x 8 x i1> %a) {
+;  %res = call <vscale x 8 x i1> @llvm.experimental.vector.reverse.nxv8i1(<vscale x 8 x i1> %a)
+;  ret <vscale x 8 x i1> %res
+;}
+;
+;define <vscale x 16 x i1> @reverse_nxv16i1(<vscale x 16 x i1> %a) {
+;  %res = call <vscale x 16 x i1> @llvm.experimental.vector.reverse.nxv16i1(<vscale x 16 x i1> %a)
+;  ret <vscale x 16 x i1> %res
+;}
+;
+;define <vscale x 32 x i1> @reverse_nxv32i1(<vscale x 32 x i1> %a) {
+;  %res = call <vscale x 32 x i1> @llvm.experimental.vector.reverse.nxv32i1(<vscale x 32 x i1> %a)
+;  ret <vscale x 32 x i1> %res
+;}
+;
+;define <vscale x 64 x i1> @reverse_nxv64i1(<vscale x 64 x i1> %a) {
+;  %res = call <vscale x 64 x i1> @llvm.experimental.vector.reverse.nxv64i1(<vscale x 64 x i1> %a)
+;  ret <vscale x 64 x i1> %res
+;}
+
+
+;
+; VECTOR_REVERSE - integer
+;
+
+define <vscale x 1 x i8> @reverse_nxv1i8(<vscale x 1 x i8> %a) {
+; CHECK-LABEL: reverse_nxv1i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf8,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x i8> @llvm.experimental.vector.reverse.nxv1i8(<vscale x 1 x i8> %a)
+  ret <vscale x 1 x i8> %res
+}
+
+define <vscale x 2 x i8> @reverse_nxv2i8(<vscale x 2 x i8> %a) {
+; CHECK-LABEL: reverse_nxv2i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf4,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x i8> @llvm.experimental.vector.reverse.nxv2i8(<vscale x 2 x i8> %a)
+  ret <vscale x 2 x i8> %res
+}
+
+define <vscale x 4 x i8> @reverse_nxv4i8(<vscale x 4 x i8> %a) {
+; CHECK-LABEL: reverse_nxv4i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,mf2,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x i8> @llvm.experimental.vector.reverse.nxv4i8(<vscale x 4 x i8> %a)
+  ret <vscale x 4 x i8> %res
+}
+
+define <vscale x 8 x i8> @reverse_nxv8i8(<vscale x 8 x i8> %a) {
+; CHECK-LABEL: reverse_nxv8i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x i8> @llvm.experimental.vector.reverse.nxv8i8(<vscale x 8 x i8> %a)
+  ret <vscale x 8 x i8> %res
+}
+
+define <vscale x 16 x i8> @reverse_nxv16i8(<vscale x 16 x i8> %a) {
+; CHECK-LABEL: reverse_nxv16i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 16 x i8> @llvm.experimental.vector.reverse.nxv16i8(<vscale x 16 x i8> %a)
+  ret <vscale x 16 x i8> %res
+}
+
+define <vscale x 32 x i8> @reverse_nxv32i8(<vscale x 32 x i8> %a) {
+; CHECK-LABEL: reverse_nxv32i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 32 x i8> @llvm.experimental.vector.reverse.nxv32i8(<vscale x 32 x i8> %a)
+  ret <vscale x 32 x i8> %res
+}
+
+define <vscale x 64 x i8> @reverse_nxv64i8(<vscale x 64 x i8> %a) {
+; CHECK-LABEL: reverse_nxv64i8:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e8,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 64 x i8> @llvm.experimental.vector.reverse.nxv64i8(<vscale x 64 x i8> %a)
+  ret <vscale x 64 x i8> %res
+}
+
+define <vscale x 1 x i16> @reverse_nxv1i16(<vscale x 1 x i16> %a) {
+; CHECK-LABEL: reverse_nxv1i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf4,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x i16> @llvm.experimental.vector.reverse.nxv1i16(<vscale x 1 x i16> %a)
+  ret <vscale x 1 x i16> %res
+}
+
+define <vscale x 2 x i16> @reverse_nxv2i16(<vscale x 2 x i16> %a) {
+; CHECK-LABEL: reverse_nxv2i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf2,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x i16> @llvm.experimental.vector.reverse.nxv2i16(<vscale x 2 x i16> %a)
+  ret <vscale x 2 x i16> %res
+}
+
+define <vscale x 4 x i16> @reverse_nxv4i16(<vscale x 4 x i16> %a) {
+; CHECK-LABEL: reverse_nxv4i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x i16> @llvm.experimental.vector.reverse.nxv4i16(<vscale x 4 x i16> %a)
+  ret <vscale x 4 x i16> %res
+}
+
+define <vscale x 8 x i16> @reverse_nxv8i16(<vscale x 8 x i16> %a) {
+; CHECK-LABEL: reverse_nxv8i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x i16> @llvm.experimental.vector.reverse.nxv8i16(<vscale x 8 x i16> %a)
+  ret <vscale x 8 x i16> %res
+}
+
+define <vscale x 16 x i16> @reverse_nxv16i16(<vscale x 16 x i16> %a) {
+; CHECK-LABEL: reverse_nxv16i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 16 x i16> @llvm.experimental.vector.reverse.nxv16i16(<vscale x 16 x i16> %a)
+  ret <vscale x 16 x i16> %res
+}
+
+define <vscale x 32 x i16> @reverse_nxv32i16(<vscale x 32 x i16> %a) {
+; CHECK-LABEL: reverse_nxv32i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 32 x i16> @llvm.experimental.vector.reverse.nxv32i16(<vscale x 32 x i16> %a)
+  ret <vscale x 32 x i16> %res
+}
+
+define <vscale x 1 x i32> @reverse_nxv1i32(<vscale x 1 x i32> %a) {
+; CHECK-LABEL: reverse_nxv1i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,mf2,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x i32> @llvm.experimental.vector.reverse.nxv1i32(<vscale x 1 x i32> %a)
+  ret <vscale x 1 x i32> %res
+}
+
+define <vscale x 2 x i32> @reverse_nxv2i32(<vscale x 2 x i32> %a) {
+; CHECK-LABEL: reverse_nxv2i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x i32> @llvm.experimental.vector.reverse.nxv2i32(<vscale x 2 x i32> %a)
+  ret <vscale x 2 x i32> %res
+}
+
+define <vscale x 4 x i32> @reverse_nxv4i32(<vscale x 4 x i32> %a) {
+; CHECK-LABEL: reverse_nxv4i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x i32> @llvm.experimental.vector.reverse.nxv4i32(<vscale x 4 x i32> %a)
+  ret <vscale x 4 x i32> %res
+}
+
+define <vscale x 8 x i32> @reverse_nxv8i32(<vscale x 8 x i32> %a) {
+; CHECK-LABEL: reverse_nxv8i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x i32> @llvm.experimental.vector.reverse.nxv8i32(<vscale x 8 x i32> %a)
+  ret <vscale x 8 x i32> %res
+}
+
+define <vscale x 16 x i32> @reverse_nxv16i32(<vscale x 16 x i32> %a) {
+; CHECK-LABEL: reverse_nxv16i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 16 x i32> @llvm.experimental.vector.reverse.nxv16i32(<vscale x 16 x i32> %a)
+  ret <vscale x 16 x i32> %res
+}
+
+define <vscale x 1 x i64> @reverse_nxv1i64(<vscale x 1 x i64> %a) {
+; CHECK-LABEL: reverse_nxv1i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x i64> @llvm.experimental.vector.reverse.nxv1i64(<vscale x 1 x i64> %a)
+  ret <vscale x 1 x i64> %res
+}
+
+define <vscale x 2 x i64> @reverse_nxv2i64(<vscale x 2 x i64> %a) {
+; CHECK-LABEL: reverse_nxv2i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x i64> @llvm.experimental.vector.reverse.nxv2i64(<vscale x 2 x i64> %a)
+  ret <vscale x 2 x i64> %res
+}
+
+define <vscale x 4 x i64> @reverse_nxv4i64(<vscale x 4 x i64> %a) {
+; CHECK-LABEL: reverse_nxv4i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x i64> @llvm.experimental.vector.reverse.nxv4i64(<vscale x 4 x i64> %a)
+  ret <vscale x 4 x i64> %res
+}
+
+define <vscale x 8 x i64> @reverse_nxv8i64(<vscale x 8 x i64> %a) {
+; CHECK-LABEL: reverse_nxv8i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x i64> @llvm.experimental.vector.reverse.nxv8i64(<vscale x 8 x i64> %a)
+  ret <vscale x 8 x i64> %res
+}
+
+;
+; VECTOR_REVERSE - floating point
+;
+
+define <vscale x 1 x half> @reverse_nxv1f16(<vscale x 1 x half> %a) {
+; CHECK-LABEL: reverse_nxv1f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf4,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x half> @llvm.experimental.vector.reverse.nxv1f16(<vscale x 1 x half> %a)
+  ret <vscale x 1 x half> %res
+}
+
+define <vscale x 2 x half> @reverse_nxv2f16(<vscale x 2 x half> %a) {
+; CHECK-LABEL: reverse_nxv2f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,mf2,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x half> @llvm.experimental.vector.reverse.nxv2f16(<vscale x 2 x half> %a)
+  ret <vscale x 2 x half> %res
+}
+
+define <vscale x 4 x half> @reverse_nxv4f16(<vscale x 4 x half> %a) {
+; CHECK-LABEL: reverse_nxv4f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x half> @llvm.experimental.vector.reverse.nxv4f16(<vscale x 4 x half> %a)
+  ret <vscale x 4 x half> %res
+}
+
+define <vscale x 8 x half> @reverse_nxv8f16(<vscale x 8 x half> %a) {
+; CHECK-LABEL: reverse_nxv8f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x half> @llvm.experimental.vector.reverse.nxv8f16(<vscale x 8 x half> %a)
+  ret <vscale x 8 x half> %res
+}
+
+define <vscale x 16 x half> @reverse_nxv16f16(<vscale x 16 x half> %a) {
+; CHECK-LABEL: reverse_nxv16f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 16 x half> @llvm.experimental.vector.reverse.nxv16f16(<vscale x 16 x half> %a)
+  ret <vscale x 16 x half> %res
+}
+
+define <vscale x 32 x half> @reverse_nxv32f16(<vscale x 32 x half> %a) {
+; CHECK-LABEL: reverse_nxv32f16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e16,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 32 x half> @llvm.experimental.vector.reverse.nxv32f16(<vscale x 32 x half> %a)
+  ret <vscale x 32 x half> %res
+}
+
+define <vscale x 1 x float> @reverse_nxv1f32(<vscale x 1 x float> %a) {
+; CHECK-LABEL: reverse_nxv1f32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,mf2,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x float> @llvm.experimental.vector.reverse.nxv1f32(<vscale x 1 x float> %a)
+  ret <vscale x 1 x float> %res
+}
+
+define <vscale x 2 x float> @reverse_nxv2f32(<vscale x 2 x float> %a) {
+; CHECK-LABEL: reverse_nxv2f32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x float> @llvm.experimental.vector.reverse.nxv2f32(<vscale x 2 x float> %a)
+  ret <vscale x 2 x float> %res
+}
+
+define <vscale x 4 x float> @reverse_nxv4f32(<vscale x 4 x float> %a) {
+; CHECK-LABEL: reverse_nxv4f32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x float> @llvm.experimental.vector.reverse.nxv4f32(<vscale x 4 x float> %a)
+  ret <vscale x 4 x float> %res
+}
+
+define <vscale x 8 x float> @reverse_nxv8f32(<vscale x 8 x float> %a) {
+; CHECK-LABEL: reverse_nxv8f32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x float> @llvm.experimental.vector.reverse.nxv8f32(<vscale x 8 x float> %a)
+  ret <vscale x 8 x float> %res
+}
+
+define <vscale x 16 x float> @reverse_nxv16f32(<vscale x 16 x float> %a) {
+; CHECK-LABEL: reverse_nxv16f32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    slli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e32,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 16 x float> @llvm.experimental.vector.reverse.nxv16f32(<vscale x 16 x float> %a)
+  ret <vscale x 16 x float> %res
+}
+
+define <vscale x 1 x double> @reverse_nxv1f64(<vscale x 1 x double> %a) {
+; CHECK-LABEL: reverse_nxv1f64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 3
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m1,ta,mu
+; CHECK-NEXT:    vid.v v25
+; CHECK-NEXT:    vrsub.vx v26, v25, a0
+; CHECK-NEXT:    vrgather.vv v25, v8, v26
+; CHECK-NEXT:    vmv1r.v v8, v25
+; CHECK-NEXT:    ret
+  %res = call <vscale x 1 x double> @llvm.experimental.vector.reverse.nxv1f64(<vscale x 1 x double> %a)
+  ret <vscale x 1 x double> %res
+}
+
+define <vscale x 2 x double> @reverse_nxv2f64(<vscale x 2 x double> %a) {
+; CHECK-LABEL: reverse_nxv2f64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 2
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m2,ta,mu
+; CHECK-NEXT:    vid.v v26
+; CHECK-NEXT:    vrsub.vx v28, v26, a0
+; CHECK-NEXT:    vrgather.vv v26, v8, v28
+; CHECK-NEXT:    vmv2r.v v8, v26
+; CHECK-NEXT:    ret
+  %res = call <vscale x 2 x double> @llvm.experimental.vector.reverse.nxv2f64(<vscale x 2 x double> %a)
+  ret <vscale x 2 x double> %res
+}
+
+define <vscale x 4 x double> @reverse_nxv4f64(<vscale x 4 x double> %a) {
+; CHECK-LABEL: reverse_nxv4f64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    srli a0, a0, 1
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m4,ta,mu
+; CHECK-NEXT:    vid.v v28
+; CHECK-NEXT:    vrsub.vx v12, v28, a0
+; CHECK-NEXT:    vrgather.vv v28, v8, v12
+; CHECK-NEXT:    vmv4r.v v8, v28
+; CHECK-NEXT:    ret
+  %res = call <vscale x 4 x double> @llvm.experimental.vector.reverse.nxv4f64(<vscale x 4 x double> %a)
+  ret <vscale x 4 x double> %res
+}
+
+define <vscale x 8 x double> @reverse_nxv8f64(<vscale x 8 x double> %a) {
+; CHECK-LABEL: reverse_nxv8f64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    csrr a0, vlenb
+; CHECK-NEXT:    addi a0, a0, -1
+; CHECK-NEXT:    vsetvli a1, zero, e64,m8,ta,mu
+; CHECK-NEXT:    vid.v v16
+; CHECK-NEXT:    vrsub.vx v24, v16, a0
+; CHECK-NEXT:    vrgather.vv v16, v8, v24
+; CHECK-NEXT:    vmv8r.v v8, v16
+; CHECK-NEXT:    ret
+  %res = call <vscale x 8 x double> @llvm.experimental.vector.reverse.nxv8f64(<vscale x 8 x double> %a)
+  ret <vscale x 8 x double> %res
+}
+
+declare <vscale x 2 x i1> @llvm.experimental.vector.reverse.nxv2i1(<vscale x 2 x i1>)
+declare <vscale x 4 x i1> @llvm.experimental.vector.reverse.nxv4i1(<vscale x 4 x i1>)
+declare <vscale x 8 x i1> @llvm.experimental.vector.reverse.nxv8i1(<vscale x 8 x i1>)
+declare <vscale x 16 x i1> @llvm.experimental.vector.reverse.nxv16i1(<vscale x 16 x i1>)
+declare <vscale x 32 x i1> @llvm.experimental.vector.reverse.nxv32i1(<vscale x 32 x i1>)
+declare <vscale x 64 x i1> @llvm.experimental.vector.reverse.nxv64i1(<vscale x 64 x i1>)
+declare <vscale x 1 x i8> @llvm.experimental.vector.reverse.nxv1i8(<vscale x 1 x i8>)
+declare <vscale x 2 x i8> @llvm.experimental.vector.reverse.nxv2i8(<vscale x 2 x i8>)
+declare <vscale x 4 x i8> @llvm.experimental.vector.reverse.nxv4i8(<vscale x 4 x i8>)
+declare <vscale x 8 x i8> @llvm.experimental.vector.reverse.nxv8i8(<vscale x 8 x i8>)
+declare <vscale x 16 x i8> @llvm.experimental.vector.reverse.nxv16i8(<vscale x 16 x i8>)
+declare <vscale x 32 x i8> @llvm.experimental.vector.reverse.nxv32i8(<vscale x 32 x i8>)
+declare <vscale x 64 x i8> @llvm.experimental.vector.reverse.nxv64i8(<vscale x 64 x i8>)
+declare <vscale x 1 x i16> @llvm.experimental.vector.reverse.nxv1i16(<vscale x 1 x i16>)
+declare <vscale x 2 x i16> @llvm.experimental.vector.reverse.nxv2i16(<vscale x 2 x i16>)
+declare <vscale x 4 x i16> @llvm.experimental.vector.reverse.nxv4i16(<vscale x 4 x i16>)
+declare <vscale x 8 x i16> @llvm.experimental.vector.reverse.nxv8i16(<vscale x 8 x i16>)
+declare <vscale x 16 x i16> @llvm.experimental.vector.reverse.nxv16i16(<vscale x 16 x i16>)
+declare <vscale x 32 x i16> @llvm.experimental.vector.reverse.nxv32i16(<vscale x 32 x i16>)
+declare <vscale x 1 x i32> @llvm.experimental.vector.reverse.nxv1i32(<vscale x 1 x i32>)
+declare <vscale x 2 x i32> @llvm.experimental.vector.reverse.nxv2i32(<vscale x 2 x i32>)
+declare <vscale x 4 x i32> @llvm.experimental.vector.reverse.nxv4i32(<vscale x 4 x i32>)
+declare <vscale x 8 x i32> @llvm.experimental.vector.reverse.nxv8i32(<vscale x 8 x i32>)
+declare <vscale x 16 x i32> @llvm.experimental.vector.reverse.nxv16i32(<vscale x 16 x i32>)
+declare <vscale x 1 x i64> @llvm.experimental.vector.reverse.nxv1i64(<vscale x 1 x i64>)
+declare <vscale x 2 x i64> @llvm.experimental.vector.reverse.nxv2i64(<vscale x 2 x i64>)
+declare <vscale x 4 x i64> @llvm.experimental.vector.reverse.nxv4i64(<vscale x 4 x i64>)
+declare <vscale x 8 x i64> @llvm.experimental.vector.reverse.nxv8i64(<vscale x 8 x i64>)
+declare <vscale x 1 x half> @llvm.experimental.vector.reverse.nxv1f16(<vscale x 1 x half>)
+declare <vscale x 2 x half> @llvm.experimental.vector.reverse.nxv2f16(<vscale x 2 x half>)
+declare <vscale x 4 x half> @llvm.experimental.vector.reverse.nxv4f16(<vscale x 4 x half>)
+declare <vscale x 8 x half> @llvm.experimental.vector.reverse.nxv8f16(<vscale x 8 x half>)
+declare <vscale x 16 x half> @llvm.experimental.vector.reverse.nxv16f16(<vscale x 16 x half>)
+declare <vscale x 32 x half> @llvm.experimental.vector.reverse.nxv32f16(<vscale x 32 x half>)
+declare <vscale x 1 x float> @llvm.experimental.vector.reverse.nxv1f32(<vscale x 1 x float>)
+declare <vscale x 2 x float> @llvm.experimental.vector.reverse.nxv2f32(<vscale x 2 x float>)
+declare <vscale x 4 x float> @llvm.experimental.vector.reverse.nxv4f32(<vscale x 4 x float>)
+declare <vscale x 8 x float> @llvm.experimental.vector.reverse.nxv8f32(<vscale x 8 x float>)
+declare <vscale x 16 x float> @llvm.experimental.vector.reverse.nxv16f32(<vscale x 16 x float>)
+declare <vscale x 1 x double> @llvm.experimental.vector.reverse.nxv1f64(<vscale x 1 x double>)
+declare <vscale x 2 x double> @llvm.experimental.vector.reverse.nxv2f64(<vscale x 2 x double>)
+declare <vscale x 4 x double> @llvm.experimental.vector.reverse.nxv4f64(<vscale x 4 x double>)
+declare <vscale x 8 x double> @llvm.experimental.vector.reverse.nxv8f64(<vscale x 8 x double>)

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

 //===- CodeGenDAGPatterns.cpp - Read DAG patterns from .td file -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
for (MVT T : WS)
       WN.insert(T.isVector() ? T.getVectorNumElements() : 0);
 
     Changed |= berase_if(VS, std::bind(NoLength, WN, std::placeholders::_1));
     Changed |= berase_if(WS, std::bind(NoLength, VN, std::placeholders::_1));
   }
   return Changed;
 }
 
+namespace {
+struct TypeSizeComparator {
+  bool operator()(const TypeSize &LHS, const TypeSize &RHS) const {
+    return std::make_tuple(LHS.isScalable(), LHS.getKnownMinValue()) <
+           std::make_tuple(RHS.isScalable(), RHS.getKnownMinValue());
+  }
+};
+} // end anonymous namespace
+
 /// 1. Ensure that for each type T in A, there exists a type U in B,
 ///    such that T and U have equal size in bits.
 /// 2. Ensure that for each type U in B, there exists a type T in A
 ///    such that T and U have equal size in bits (reverse of 1).
 bool TypeInfer::EnforceSameSize(TypeSetByHwMode &A, TypeSetByHwMode &B) {
   ValidateOnExit _1(A, *this), _2(B, *this);
   if (TP.hasError())
     return false;
   bool Changed = false;
   if (A.empty())
     Changed |= EnforceAny(A);
   if (B.empty())
     Changed |= EnforceAny(B);
 
-  auto NoSize = [](const SmallSet<TypeSize, 2> &Sizes, MVT T) -> bool {
+  typedef SmallSet<TypeSize, 2, TypeSizeComparator> TypeSizeSet;
+
+  auto NoSize = [](const TypeSizeSet &Sizes, MVT T) -> bool {
     return !Sizes.count(T.getSizeInBits());
   };
 
   for (unsigned M : union_modes(A, B)) {
     TypeSetByHwMode::SetType &AS = A.get(M);
     TypeSetByHwMode::SetType &BS = B.get(M);
-    SmallSet<TypeSize, 2> AN, BN;
+    TypeSizeSet AN, BN;
 
     for (MVT T : AS)
       AN.insert(T.getSizeInBits());
     for (MVT T : BS)
       BN.insert(T.getSizeInBits());
 
     Changed |= berase_if(AS, std::bind(NoSize, BN, std::placeholders::_1));
     Changed |= berase_if(BS, std::bind(NoSize, AN, std::placeholders::_1));