[RISCV] Improve extract_vector_elt for fixed mask registers.

[jacquesguan]

Summary

Now the backend promotes mask vector to an i8 vector and extract element from that. We could bitcast to a widen element vector, and extract from it to GPR, then use I instruction to extract the certain bit.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4100

Use unsigned instead of auto

4102

Ele->Elt would be a more common shortening.

4106

getSizeBits(). XLenVT is always scalar.

4113

getSizeBits(). XLenVT is always scalar.

4116

getSizeBits(). XLenVT is always scalar.

4118

This shift pair can be an AND right?

4133

I don't think you need to pass false here.

4135

Use SDValue instead of auto.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4112

This won't work with Zve32 on RV64. A vector XLen elements wouldn't be legal.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

I don't think I understand what this subtract is doing.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4100

Done.

4102

Done.

4106

Done

4112

I add MaxEEW to get the right largest vector element width that we could have on current target, but we do not support Zve extension now, so I just set it to 64 and add a TODO to remind changing it after having Zve.

4113

Done

4116

Done

4118

Done

4133

Done.

4135

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

We will use vmv.x.s to extract the elment to GPR, and if eew is less than XLEN, the value will be signed extend. For example, for 8 x i1 mask vector, we will have:

GPR bit    | XLEN -1 |--------------| 6 | 5 | 4 | 3 | 2 | 1 | 0 |
mask index |    0    |--------------| 1 | 2 | 3 | 4 | 5 | 6 | 7 |

So if we want to extract the 0th element, we should set extract bit index to XLEN - 1, and otherwise the index = element width - 1 - mask index.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

Why would the mask bits be in reversed order?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

It is my mistake, I thought things in wrong way. Here should not be in reversed order. I fixed it.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4127

elment -> element

4131

Sign extend means that bit 7 and above all have the same value. So why can't we just check bit 7?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4112

You don't need Zve to test this. Before Zve was added to the spec we added a command line option "-riscv-v-fixed-length-vector-elen-max" to clip the ELEN.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4112

Done.

4127

Done.

4131

Done.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4106

Use unsigned. Nothing here needs 64-bits.

[jacquesguan]

Address comment

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4106

Done.

[jacquesguan]

ping.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4097

This comment is out of date.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4097

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4099

nit: Nums is a little unconventional. NumElts or something? You can also use getVectorNumElements because we know it's a fixed-length vector.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

You'll need to rebase and swap +experimental-v for +v

272

This doesn't look like it's doing the right thing. We haven't modified the original extract index a1 at this point so we could be shifting an i32 right by up to 127 places? Have I missed something?

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Here we want to extract the (idx % 32)th bit from the GPR, and a1 owns the value of idx, so (idx % 32) is a1[4-0]. And because the shift instruction only uses the 0 - (log2(xlen)-1) bits of rs2, so we actually do not need to get a1[4-0], we could just use a1. So I think here is OK?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4099

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4109

I think asserting that NumElts is a power of two (or just skipping this optimization) would be useful just in case we ever support other vector types. This is quite an edge case so won't be well covered, and could silently do some weird things.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Ah yes, that's what I missed. Thanks! I see there's an AND being generated anyway which should be doing the right thing.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4109

Thanks, I add an assert for this.

[frasercrmck]

LGTM, thanks

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
   SDLoc DL(Op);
   SDValue Idx = Op.getOperand(1);
   SDValue Vec = Op.getOperand(0);
   EVT EltVT = Op.getValueType();
   MVT VecVT = Vec.getSimpleValueType();
   MVT XLenVT = Subtarget.getXLenVT();
 
   if (VecVT.getVectorElementType() == MVT::i1) {
     // FIXME: For now we just promote to an i8 vector and extract from that,
     // but this is probably not optimal.
+    if (VecVT.isFixedLengthVector()) {
+      unsigned Nums = VecVT.getVectorMinNumElements();
+      if (Nums >= 8) {
+        MVT WideEltVT;
+        unsigned WidenVecLen;
+        SDValue ExtractElementIdx;
+        SDValue ExtractBitIdx;
+        uint64_t MaxEEW = Subtarget.getMaxELENForFixedLengthVectors();
+        MVT LargestEltVT = MVT::getIntegerVT(
+            std::min(MaxEEW, uint64_t(XLenVT.getSizeInBits())));
+        if (Nums <= LargestEltVT.getSizeInBits()) {
+          WideEltVT = MVT::getIntegerVT(Nums);
+          WidenVecLen = 1;
+          ExtractElementIdx = DAG.getConstant(0, DL, XLenVT);
+          ExtractBitIdx = Idx;
+        } else {
+          WideEltVT = LargestEltVT;
+          WidenVecLen = Nums / WideEltVT.getSizeInBits();
+          // extract element index = index / element width
+          ExtractElementIdx = DAG.getNode(
+              ISD::SRL, DL, XLenVT, Idx,
+              DAG.getConstant(Log2_64(WideEltVT.getSizeInBits()), DL, XLenVT));
+          // mask bit index = index % element width
+          ExtractBitIdx = DAG.getNode(
+              ISD::AND, DL, XLenVT, Idx,
+              DAG.getConstant(WideEltVT.getSizeInBits() - 1, DL, XLenVT));
+        }
+        MVT WideVT = MVT::getVectorVT(WideEltVT, WidenVecLen);
+        Vec = DAG.getNode(ISD::BITCAST, DL, WideVT, Vec);
+        SDValue ExtractElt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, XLenVT,
+                                         Vec, ExtractElementIdx);
+        // Extract the bit from GPR.
+        SDValue ShiftRight =
+            DAG.getNode(ISD::SRL, DL, XLenVT, ExtractElt, ExtractBitIdx);
+        return DAG.getNode(ISD::AND, DL, XLenVT, ShiftRight,
+                           DAG.getConstant(1, DL, XLenVT));
+      }
+    }
+
     MVT WideVT = MVT::getVectorVT(MVT::i8, VecVT.getVectorElementCount());
     Vec = DAG.getNode(ISD::ZERO_EXTEND, DL, WideVT, Vec);
     return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, EltVT, Vec, Idx);
   }
 
   // If this is a fixed vector, we need to convert it to a scalable vector.
   MVT ContainerVT = VecVT;
   if (VecVT.isFixedLengthVector()) {

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+experimental-v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32
 ; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+experimental-v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64
 
+; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+experimental-v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32ZBS
+; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+experimental-v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64ZBS
+
 define i1 @extractelt_v1i1(<1 x i8>* %x, i64 %idx) nounwind {
 ; CHECK-LABEL: extractelt_v1i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 1, e8, mf8, ta, mu
 ; CHECK-NEXT:    vle8.v v8, (a0)
 ; CHECK-NEXT:    vmseq.vi v0, v8, 0
 ; CHECK-NEXT:    vmv.v.i v8, 0
 ; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0

 ; CHECK-NEXT:    ret
   %a = load <4 x i8>, <4 x i8>* %x
   %b = icmp eq <4 x i8> %a, zeroinitializer
   %c = extractelement <4 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v8i1(<8 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v8i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, mf2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v8i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v8i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v8i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v8i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <8 x i8>, <8 x i8>* %x
   %b = icmp eq <8 x i8> %a, zeroinitializer
   %c = extractelement <8 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v16i1(<16 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v16i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m1, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v16i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v16i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v16i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v16i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <16 x i8>, <16 x i8>* %x
   %b = icmp eq <16 x i8> %a, zeroinitializer
   %c = extractelement <16 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v32i1(<32 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v32i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 32
-; CHECK-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v32i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 32
+; RV32-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v10, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v10
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v32i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 32
+; RV64-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v10, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v10
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v32i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 32
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v10
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v32i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 32
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v10
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <32 x i8>, <32 x i8>* %x
   %b = icmp eq <32 x i8> %a, zeroinitializer
   %c = extractelement <32 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v64i1(<64 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v64i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 64
-; CHECK-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m4, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v64i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 64
+; RV32-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v12, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v12, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v64i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 64
+; RV64-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v12, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v12
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v64i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 64
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v12, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v64i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 64
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v12
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <64 x i8>, <64 x i8>* %x
   %b = icmp eq <64 x i8> %a, zeroinitializer
   %c = extractelement <64 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v128i1(<128 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v128i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 128
-; CHECK-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m8, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v128i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 128
+; RV32-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v16, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v16, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v128i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 128
+; RV64-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v16, v8, 0
+; RV64-NEXT:    srli a0, a1, 6
+; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64-NEXT:    vslidedown.vx v8, v16, a0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v128i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 128
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v128i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 128
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV64ZBS-NEXT:    srli a0, a1, 6
+; RV64ZBS-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <128 x i8>, <128 x i8>* %x
   %b = icmp eq <128 x i8> %a, zeroinitializer
   %c = extractelement <128 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v256i1(<256 x i8>* %x, i64 %idx) nounwind {
 ; RV32-LABEL: extractelt_v256i1:

 ; RV64-NEXT:    addi a1, sp, 128
 ; RV64-NEXT:    vse8.v v8, (a1)
 ; RV64-NEXT:    lb a0, 0(a0)
 ; RV64-NEXT:    addi sp, s0, -512
 ; RV64-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    addi sp, sp, 512
 ; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v256i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    addi sp, sp, -512
+; RV32ZBS-NEXT:    sw ra, 508(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    sw s0, 504(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    addi s0, sp, 512
+; RV32ZBS-NEXT:    andi sp, sp, -128
+; RV32ZBS-NEXT:    andi a1, a1, 255
+; RV32ZBS-NEXT:    addi a2, a0, 128
+; RV32ZBS-NEXT:    li a3, 128
+; RV32ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v16, (a0)
+; RV32ZBS-NEXT:    vle8.v v24, (a2)
+; RV32ZBS-NEXT:    addi a0, sp, 128
+; RV32ZBS-NEXT:    add a0, a0, a1
+; RV32ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV32ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV32ZBS-NEXT:    vmv.v.i v16, 0
+; RV32ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 256
+; RV32ZBS-NEXT:    vse8.v v24, (a1)
+; RV32ZBS-NEXT:    vmv1r.v v0, v8
+; RV32ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 128
+; RV32ZBS-NEXT:    vse8.v v8, (a1)
+; RV32ZBS-NEXT:    lb a0, 0(a0)
+; RV32ZBS-NEXT:    addi sp, s0, -512
+; RV32ZBS-NEXT:    lw ra, 508(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    lw s0, 504(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    addi sp, sp, 512
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v256i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    addi sp, sp, -512
+; RV64ZBS-NEXT:    sd ra, 504(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    sd s0, 496(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    addi s0, sp, 512
+; RV64ZBS-NEXT:    andi sp, sp, -128
+; RV64ZBS-NEXT:    andi a1, a1, 255
+; RV64ZBS-NEXT:    addi a2, a0, 128
+; RV64ZBS-NEXT:    li a3, 128
+; RV64ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v16, (a0)
+; RV64ZBS-NEXT:    vle8.v v24, (a2)
+; RV64ZBS-NEXT:    addi a0, sp, 128
+; RV64ZBS-NEXT:    add a0, a0, a1
+; RV64ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV64ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV64ZBS-NEXT:    vmv.v.i v16, 0
+; RV64ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 256
+; RV64ZBS-NEXT:    vse8.v v24, (a1)
+; RV64ZBS-NEXT:    vmv1r.v v0, v8
+; RV64ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 128
+; RV64ZBS-NEXT:    vse8.v v8, (a1)
+; RV64ZBS-NEXT:    lb a0, 0(a0)
+; RV64ZBS-NEXT:    addi sp, s0, -512
+; RV64ZBS-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    addi sp, sp, 512
+; RV64ZBS-NEXT:    ret
   %a = load <256 x i8>, <256 x i8>* %x
   %b = icmp eq <256 x i8> %a, zeroinitializer
   %c = extractelement <256 x i1> %b, i64 %idx
   ret i1 %c
 }