[RISCV] Lower experimental_get_vector_length intrinsic to vsetvli for some cases.

[craig.topper]

Summary

This patch lowers to vsetvli when the AVL is i32 or XLenVT and
the VF is a power of 2 in the range [1, 64]. VLEN=32 is not supported
as we don't have a valid type mapping for that. VF=1 is not supported
with Zve32* only.

The element width is used to set the SEW for the vsetvli if possible.
Otherwise we use SEW=8.

[asb]

I think you meant to mark this as dependent on D149916.

[craig.topper]

Rebase on parent patch

[reames]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1185

For fixed vectors, we should be able to statically compute this when the ElementWidth * VF is less than VLEN right? It's fine to do that in a different patch, just want to make sure I'm not missing something.

1188

Why not i16? I don't see anything in the implementation which couldn't be handled via an extend?

6664

The "if possible" bit is bugging me here. Isn't this a hard requirement? VLMAX should be a function of SEW and LMUL. Given the result here depends on VLMAX, don't we have a hard requirement on having a recognized SEW/LMUL combination?

6673

encodeSEW

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1185

The ElementWidth doesn't have any real meaning. The vectorizer already picked a VF. The get_vector_length is asking, given that VF, how many elements fit in that type. The width of the elements or VLEN doesn't change that answer.

I guess we could return a lower number but that means we'd never use the entire type the vectorizer is emitting. That seems like we a cost model issue if we actually want to use less than the type the vectorizer picks.

1188

No strong reason. Might need to add setOperationAction(ISD::INTRINSIC_WO_CHAIN, MVT::i16, Custom) to get the legalization to be called.

[reames]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

1185

I see your point for fixed length vectors. Essentially, we can let the vectorizer pick one via cost modeling, and then split as needed. So here, we can report the number of elements even if splitting might be required.

[craig.topper]

Rebase after removing element width from the intrinsic.

[reames]

LGTM w/comment addressed.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6660

This depends very heavily on the fact that VLMAX for 1 x i8 and VLMAX for 1 x i64 are the same. It took me a while to convince myself that this was actually true. The former maps to a fractional lmul, the later maps to m1. And as a result, the element width is irrelevant.

Can you add a comment which explains this in a bit more detail? If this is already well explained elsewhere, you can simply point to that comment.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6660

A lot of this needs to be rewritten using RVVBitsPerBlock. I'll update.

[craig.topper]

Use RVVBitsPerBlock to define the math.
Add explanatory comments.

[craig.topper]

Requesting review to make sure the new comments are sufficient.

[reames]

Ok, yeah, that's a lot easier to follow. Thanks!

LGTM

llvm/lib/Target/RISCV/RISCVISelLowering.h

private:
   void validateCCReservedRegs(
       const SmallVectorImpl<std::pair<llvm::Register, llvm::SDValue>> &Regs,
       MachineFunction &MF) const;
 
   bool useRVVForFixedLengthVectorVT(MVT VT) const;
 
   MVT getVPExplicitVectorLengthTy() const override;
 
+  bool shouldExpandGetVectorLength(EVT TripCountVT, unsigned VF,
+                                   bool IsScalable) const override;
+
   /// RVV code generation for fixed length vectors does not lower all
   /// BUILD_VECTORs. This makes BUILD_VECTOR legalisation a source of stores to
   /// merge. However, merging them creates a BUILD_VECTOR that is just as
   /// illegal as the original, thus leading to an infinite legalisation loop.
   /// NOTE: Once BUILD_VECTOR can be custom lowered for all legal vector types,
   /// this override can be removed.
   bool mergeStoresAfterLegalization(EVT VT) const override;
 

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

if (Subtarget.hasVInstructions() &&
     return EVT::getVectorVT(Context, MVT::i1, VT.getVectorElementCount());
   return VT.changeVectorElementTypeToInteger();
 }
 
 MVT RISCVTargetLowering::getVPExplicitVectorLengthTy() const {
   return Subtarget.getXLenVT();
 }
 
+// Return false if we can lower get_vector_length to a vsetvli intrinsic.
+bool RISCVTargetLowering::shouldExpandGetVectorLength(EVT TripCountVT,
+                                                      unsigned VF,
+                                                      bool IsScalable) const {
+  if (!Subtarget.hasVInstructions())
+    return true;
+
+  if (!IsScalable)
+    return true;
+
+  if (TripCountVT != MVT::i32 && TripCountVT != Subtarget.getXLenVT())
+    return true;
+
+  // Don't allow VF=1 if those types are't legal.
+  if (VF < RISCV::RVVBitsPerBlock / Subtarget.getELEN())
+    return true;
+
+  // VLEN=32 support is incomplete.
+  if (Subtarget.getRealMinVLen() < RISCV::RVVBitsPerBlock)
+    return true;
+
+  // The maximum VF is for the smallest element width with LMUL=8.
+  // VF must be a power of 2.
+  unsigned MaxVF = (RISCV::RVVBitsPerBlock / 8) * 8;
+  return VF > MaxVF || !isPowerOf2_32(VF);
+}
+
 bool RISCVTargetLowering::getTgtMemIntrinsic(IntrinsicInfo &Info,
                                              const CallInst &I,
                                              MachineFunction &MF,
                                              unsigned Intrinsic) const {
   auto &DL = I.getModule()->getDataLayout();
 
   auto SetRVVLoadStoreInfo = [&](unsigned PtrOp, bool IsStore,
                                  bool IsUnitStrided) {
static SDValue lowerVectorIntrinsicScalars(SDValue Op, SelectionDAG &DAG,
 
   // We need to convert the scalar to a splat vector.
   SDValue VL = getVLOperand(Op);
   assert(VL.getValueType() == XLenVT);
   ScalarOp = splatSplitI64WithVL(DL, VT, SDValue(), ScalarOp, VL, DAG);
   return DAG.getNode(Op->getOpcode(), DL, Op->getVTList(), Operands);
 }
 
+// Lower the llvm.get.vector.length intrinsic to vsetvli. We only support
+// scalable vector llvm.get.vector.length for now.
+//
+// We need to convert from a scalable VF to a vsetvli with VLMax equal to
+// (vscale * VF). The vscale and VF are independent of element width. We use
+// SEW=8 for the vsetvli because it is the only element width that supports all
+// fractional LMULs. The LMUL is choosen so that with SEW=8 the VLMax is
+// (vscale * VF). Where vscale is defined as VLEN/RVVBitsPerBlock. The
+// InsertVSETVLI pass can fix up the vtype of the vsetvli if a different
+// SEW and LMUL are better for the surrounding vector instructions.
+static SDValue lowerGetVectorLength(SDNode *N, SelectionDAG &DAG,
+                                    const RISCVSubtarget &Subtarget) {
+  MVT XLenVT = Subtarget.getXLenVT();
+
+  // The smallest LMUL is only valid for the smallest element width.
+  const unsigned ElementWidth = 8;
+
+  // Determine the VF that corresponds to LMUL 1 for ElementWidth.
+  unsigned LMul1VF = RISCV::RVVBitsPerBlock / ElementWidth;
+  // We don't support VF==1 with ELEN==32.
+  unsigned MinVF = RISCV::RVVBitsPerBlock / Subtarget.getELEN();
+
+  unsigned VF = N->getConstantOperandVal(2);
+  assert(VF >= MinVF && VF <= (LMul1VF * 8) && isPowerOf2_32(VF) &&
+         "Unexpected VF");
+
+  bool Fractional = VF < LMul1VF;
+  unsigned LMulVal = Fractional ? LMul1VF / VF : VF / LMul1VF;
+  unsigned VLMUL = (unsigned)RISCVVType::encodeLMUL(LMulVal, Fractional);
+  unsigned VSEW = RISCVVType::encodeSEW(ElementWidth);
+
+  SDLoc DL(N);
+
+  SDValue LMul = DAG.getTargetConstant(VLMUL, DL, XLenVT);
+  SDValue Sew = DAG.getTargetConstant(VSEW, DL, XLenVT);
+
+  SDValue AVL = DAG.getNode(ISD::ZERO_EXTEND, DL, XLenVT, N->getOperand(1));
+
+  SDValue ID = DAG.getTargetConstant(Intrinsic::riscv_vsetvli, DL, XLenVT);
+  return DAG.getNode(ISD::INTRINSIC_WO_CHAIN, DL, XLenVT, ID, AVL, Sew, LMul);
+}
+
 SDValue RISCVTargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op,
                                                      SelectionDAG &DAG) const {
   unsigned IntNo = Op.getConstantOperandVal(0);
   SDLoc DL(Op);
   MVT XLenVT = Subtarget.getXLenVT();
 
   switch (IntNo) {
   default:
case Intrinsic::riscv_brev8: {
     return DAG.getNode(Opc, DL, XLenVT, Op.getOperand(1));
   }
   case Intrinsic::riscv_zip:
   case Intrinsic::riscv_unzip: {
     unsigned Opc =
         IntNo == Intrinsic::riscv_zip ? RISCVISD::ZIP : RISCVISD::UNZIP;
     return DAG.getNode(Opc, DL, XLenVT, Op.getOperand(1));
   }
+  case Intrinsic::experimental_get_vector_length:
+    return lowerGetVectorLength(Op.getNode(), DAG, Subtarget);
   case Intrinsic::riscv_vmv_x_s:
     assert(Op.getValueType() == XLenVT && "Unexpected VT!");
     return DAG.getNode(RISCVISD::VMV_X_S, DL, Op.getValueType(),
                        Op.getOperand(1));
   case Intrinsic::riscv_vfmv_f_s:
     return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, Op.getValueType(),
                        Op.getOperand(1), DAG.getConstant(0, DL, XLenVT));
   case Intrinsic::riscv_vmv_v_x:
case ISD::EXTRACT_VECTOR_ELT: {
     break;
   }
   case ISD::INTRINSIC_WO_CHAIN: {
     unsigned IntNo = cast<ConstantSDNode>(N->getOperand(0))->getZExtValue();
     switch (IntNo) {
     default:
       llvm_unreachable(
           "Don't know how to custom type legalize this intrinsic!");
+    case Intrinsic::experimental_get_vector_length: {
+      SDValue Res = lowerGetVectorLength(N, DAG, Subtarget);
+      Results.push_back(DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, Res));
+      return;
+    }
     case Intrinsic::riscv_orc_b: {
       SDValue NewOp =
           DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(1));
       SDValue Res = DAG.getNode(RISCVISD::ORC_B, DL, MVT::i64, NewOp);
       Results.push_back(DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, Res));
       return;
     }
     case Intrinsic::riscv_vmv_x_s: {

llvm/test/CodeGen/RISCV/rvv/get_vector_length.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
-; RUN: sed 's/iXLen/i32/g' %s | llc -mtriple=riscv32 -mattr=+v -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK,RV32
-; RUN: sed 's/iXLen/i32/g' %s | llc -mtriple=riscv64 -mattr=+v -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK,RV64
+; RUN: sed 's/iXLen/i32/g' %s | llc -mtriple=riscv32 -mattr=+m,+v -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK,RV32
+; RUN: sed 's/iXLen/i32/g' %s | llc -mtriple=riscv64 -mattr=+m,+v -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK,RV64
 
 declare i32 @llvm.experimental.get.vector.length.i16(i16, i32, i1)
 declare i32 @llvm.experimental.get.vector.length.i32(i32, i32, i1)
 declare i32 @llvm.experimental.get.vector.length.i64(i64, i32, i1)
 
 define i32 @vector_length_i16(i16 zeroext %tc) {
 ; CHECK-LABEL: vector_length_i16:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    csrr a1, vlenb
 ; CHECK-NEXT:    srli a1, a1, 2
 ; CHECK-NEXT:    bltu a0, a1, .LBB0_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    mv a0, a1
 ; CHECK-NEXT:  .LBB0_2:
 ; CHECK-NEXT:    ret
   %a = call i32 @llvm.experimental.get.vector.length.i16(i16 %tc, i32 2, i1 true)
   ret i32 %a
 }
 
 define i32 @vector_length_i32(i32 zeroext %tc) {
-; RV32-LABEL: vector_length_i32:
-; RV32:       # %bb.0:
-; RV32-NEXT:    csrr a1, vlenb
-; RV32-NEXT:    srli a1, a1, 2
-; RV32-NEXT:    bltu a0, a1, .LBB1_2
-; RV32-NEXT:  # %bb.1:
-; RV32-NEXT:    mv a0, a1
-; RV32-NEXT:  .LBB1_2:
-; RV32-NEXT:    ret
-;
-; RV64-LABEL: vector_length_i32:
-; RV64:       # %bb.0:
-; RV64-NEXT:    sext.w a0, a0
-; RV64-NEXT:    csrr a1, vlenb
-; RV64-NEXT:    srli a1, a1, 2
-; RV64-NEXT:    bltu a0, a1, .LBB1_2
-; RV64-NEXT:  # %bb.1:
-; RV64-NEXT:    mv a0, a1
-; RV64-NEXT:  .LBB1_2:
-; RV64-NEXT:    ret
+; CHECK-LABEL: vector_length_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf4, ta, ma
+; CHECK-NEXT:    ret
   %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 2, i1 true)
   ret i32 %a
 }
 
 define i32 @vector_length_XLen(iXLen zeroext %tc) {
-; RV32-LABEL: vector_length_XLen:
-; RV32:       # %bb.0:
-; RV32-NEXT:    csrr a1, vlenb
-; RV32-NEXT:    srli a1, a1, 2
-; RV32-NEXT:    bltu a0, a1, .LBB2_2
-; RV32-NEXT:  # %bb.1:
-; RV32-NEXT:    mv a0, a1
-; RV32-NEXT:  .LBB2_2:
-; RV32-NEXT:    ret
-;
-; RV64-LABEL: vector_length_XLen:
-; RV64:       # %bb.0:
-; RV64-NEXT:    sext.w a0, a0
-; RV64-NEXT:    csrr a1, vlenb
-; RV64-NEXT:    srli a1, a1, 2
-; RV64-NEXT:    bltu a0, a1, .LBB2_2
-; RV64-NEXT:  # %bb.1:
-; RV64-NEXT:    mv a0, a1
-; RV64-NEXT:  .LBB2_2:
-; RV64-NEXT:    ret
+; CHECK-LABEL: vector_length_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf4, ta, ma
+; CHECK-NEXT:    ret
   %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 2, i1 true)
   ret i32 %a
 }
 
 define i32 @vector_length_i16_fixed(i16 zeroext %tc) {
 ; CHECK-LABEL: vector_length_i16_fixed:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    li a1, 2

 ; RV64-NEXT:    bltu a0, a1, .LBB5_2
 ; RV64-NEXT:  # %bb.1:
 ; RV64-NEXT:    li a0, 2
 ; RV64-NEXT:  .LBB5_2:
 ; RV64-NEXT:    ret
   %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 2, i1 false)
   ret i32 %a
 }
+
+define i32 @vector_length_vf1_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf1_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf8, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 1, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf1_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf1_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf8, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 1, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf2_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf2_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf4, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 2, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf2_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf2_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf4, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 2, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf4_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf4_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf2, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 4, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf4_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf4_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, mf2, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 4, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf8_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf8_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m1, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 8, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf8_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf8_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m1, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 8, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf16_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf16_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m2, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 16, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf16_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf16_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m2, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 16, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf32_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf32_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m4, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 32, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf32_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf32_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m4, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 32, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf64_i32(i32 zeroext %tc) {
+; CHECK-LABEL: vector_length_vf64_i32:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m8, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 64, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf64_XLen(iXLen zeroext %tc) {
+; CHECK-LABEL: vector_length_vf64_XLen:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vsetvli a0, a0, e8, m8, ta, ma
+; CHECK-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 64, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf128_i32(i32 zeroext %tc) {
+; RV32-LABEL: vector_length_vf128_i32:
+; RV32:       # %bb.0:
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    slli a1, a1, 4
+; RV32-NEXT:    bltu a0, a1, .LBB20_2
+; RV32-NEXT:  # %bb.1:
+; RV32-NEXT:    mv a0, a1
+; RV32-NEXT:  .LBB20_2:
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: vector_length_vf128_i32:
+; RV64:       # %bb.0:
+; RV64-NEXT:    sext.w a0, a0
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    slli a1, a1, 4
+; RV64-NEXT:    bltu a0, a1, .LBB20_2
+; RV64-NEXT:  # %bb.1:
+; RV64-NEXT:    mv a0, a1
+; RV64-NEXT:  .LBB20_2:
+; RV64-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 128, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf128_XLen(iXLen zeroext %tc) {
+; RV32-LABEL: vector_length_vf128_XLen:
+; RV32:       # %bb.0:
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    slli a1, a1, 4
+; RV32-NEXT:    bltu a0, a1, .LBB21_2
+; RV32-NEXT:  # %bb.1:
+; RV32-NEXT:    mv a0, a1
+; RV32-NEXT:  .LBB21_2:
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: vector_length_vf128_XLen:
+; RV64:       # %bb.0:
+; RV64-NEXT:    sext.w a0, a0
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    slli a1, a1, 4
+; RV64-NEXT:    bltu a0, a1, .LBB21_2
+; RV64-NEXT:  # %bb.1:
+; RV64-NEXT:    mv a0, a1
+; RV64-NEXT:  .LBB21_2:
+; RV64-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 128, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf3_i32(i32 zeroext %tc) {
+; RV32-LABEL: vector_length_vf3_i32:
+; RV32:       # %bb.0:
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    srli a1, a1, 3
+; RV32-NEXT:    slli a2, a1, 1
+; RV32-NEXT:    add a1, a2, a1
+; RV32-NEXT:    bltu a0, a1, .LBB22_2
+; RV32-NEXT:  # %bb.1:
+; RV32-NEXT:    mv a0, a1
+; RV32-NEXT:  .LBB22_2:
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: vector_length_vf3_i32:
+; RV64:       # %bb.0:
+; RV64-NEXT:    sext.w a0, a0
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    srli a1, a1, 3
+; RV64-NEXT:    slli a2, a1, 1
+; RV64-NEXT:    add a1, a2, a1
+; RV64-NEXT:    bltu a0, a1, .LBB22_2
+; RV64-NEXT:  # %bb.1:
+; RV64-NEXT:    mv a0, a1
+; RV64-NEXT:  .LBB22_2:
+; RV64-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.i32(i32 %tc, i32 3, i1 true)
+  ret i32 %a
+}
+
+define i32 @vector_length_vf3_XLen(iXLen zeroext %tc) {
+; RV32-LABEL: vector_length_vf3_XLen:
+; RV32:       # %bb.0:
+; RV32-NEXT:    csrr a1, vlenb
+; RV32-NEXT:    srli a1, a1, 3
+; RV32-NEXT:    slli a2, a1, 1
+; RV32-NEXT:    add a1, a2, a1
+; RV32-NEXT:    bltu a0, a1, .LBB23_2
+; RV32-NEXT:  # %bb.1:
+; RV32-NEXT:    mv a0, a1
+; RV32-NEXT:  .LBB23_2:
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: vector_length_vf3_XLen:
+; RV64:       # %bb.0:
+; RV64-NEXT:    sext.w a0, a0
+; RV64-NEXT:    csrr a1, vlenb
+; RV64-NEXT:    srli a1, a1, 3
+; RV64-NEXT:    slli a2, a1, 1
+; RV64-NEXT:    add a1, a2, a1
+; RV64-NEXT:    bltu a0, a1, .LBB23_2
+; RV64-NEXT:  # %bb.1:
+; RV64-NEXT:    mv a0, a1
+; RV64-NEXT:  .LBB23_2:
+; RV64-NEXT:    ret
+  %a = call i32 @llvm.experimental.get.vector.length.iXLen(iXLen %tc, i32 3, i1 true)
+  ret i32 %a
+}