[RISCV] Improve extract_vector_elt for fixed mask registers.

[jacquesguan]

Summary

Now the backend promotes mask vector to an i8 vector and extract element from that. We could bitcast to a widen element vector, and extract from it to GPR, then use I instruction to extract the certain bit.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4293

Use unsigned instead of auto

4295

Ele->Elt would be a more common shortening.

4299

getSizeBits(). XLenVT is always scalar.

4306

getSizeBits(). XLenVT is always scalar.

4309

getSizeBits(). XLenVT is always scalar.

4311

This shift pair can be an AND right?

4326

I don't think you need to pass false here.

4328

Use SDValue instead of auto.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4305

This won't work with Zve32 on RV64. A vector XLen elements wouldn't be legal.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

I don't think I understand what this subtract is doing.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4293

Done.

4295

Done.

4299

Done

4305

I add MaxEEW to get the right largest vector element width that we could have on current target, but we do not support Zve extension now, so I just set it to 64 and add a TODO to remind changing it after having Zve.

4306

Done

4309

Done

4311

Done

4326

Done.

4328

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

We will use vmv.x.s to extract the elment to GPR, and if eew is less than XLEN, the value will be signed extend. For example, for 8 x i1 mask vector, we will have:

GPR bit    | XLEN -1 |--------------| 6 | 5 | 4 | 3 | 2 | 1 | 0 |
mask index |    0    |--------------| 1 | 2 | 3 | 4 | 5 | 6 | 7 |

So if we want to extract the 0th element, we should set extract bit index to XLEN - 1, and otherwise the index = element width - 1 - mask index.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

Why would the mask bits be in reversed order?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

It is my mistake, I thought things in wrong way. Here should not be in reversed order. I fixed it.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4320

elment -> element

4324

Sign extend means that bit 7 and above all have the same value. So why can't we just check bit 7?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4305

You don't need Zve to test this. Before Zve was added to the spec we added a command line option "-riscv-v-fixed-length-vector-elen-max" to clip the ELEN.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4305

Done.

4320

Done.

4324

Done.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4299

Use unsigned. Nothing here needs 64-bits.

[jacquesguan]

Address comment

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4299

Done.

[jacquesguan]

ping.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4290–4291

This comment is out of date.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4290–4291

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4292

nit: Nums is a little unconventional. NumElts or something? You can also use getVectorNumElements because we know it's a fixed-length vector.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

You'll need to rebase and swap +experimental-v for +v

272

This doesn't look like it's doing the right thing. We haven't modified the original extract index a1 at this point so we could be shifting an i32 right by up to 127 places? Have I missed something?

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Here we want to extract the (idx % 32)th bit from the GPR, and a1 owns the value of idx, so (idx % 32) is a1[4-0]. And because the shift instruction only uses the 0 - (log2(xlen)-1) bits of rs2, so we actually do not need to get a1[4-0], we could just use a1. So I think here is OK?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4292

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4302

I think asserting that NumElts is a power of two (or just skipping this optimization) would be useful just in case we ever support other vector types. This is quite an edge case so won't be well covered, and could silently do some weird things.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Ah yes, that's what I missed. Thanks! I see there's an AND being generated anyway which should be doing the right thing.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4302

Thanks, I add an assert for this.

[frasercrmck]

LGTM, thanks

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
                                                      SelectionDAG &DAG) const {
   SDLoc DL(Op);
   SDValue Idx = Op.getOperand(1);
   SDValue Vec = Op.getOperand(0);
   EVT EltVT = Op.getValueType();
   MVT VecVT = Vec.getSimpleValueType();
   MVT XLenVT = Subtarget.getXLenVT();
 
   if (VecVT.getVectorElementType() == MVT::i1) {
-    // FIXME: For now we just promote to an i8 vector and extract from that,
-    // but this is probably not optimal.
+    if (VecVT.isFixedLengthVector()) {
+      unsigned NumElts = VecVT.getVectorNumElements();
+      if (NumElts >= 8) {
+        MVT WideEltVT;
+        unsigned WidenVecLen;
+        SDValue ExtractElementIdx;
+        SDValue ExtractBitIdx;
+        unsigned MaxEEW = Subtarget.getMaxELENForFixedLengthVectors();
+        MVT LargestEltVT = MVT::getIntegerVT(
+            std::min(MaxEEW, unsigned(XLenVT.getSizeInBits())));
+        if (NumElts <= LargestEltVT.getSizeInBits()) {
+          assert(isPowerOf2_32(NumElts) &&
+                 "the number of elements should be power of 2");
+          WideEltVT = MVT::getIntegerVT(NumElts);
+          WidenVecLen = 1;
+          ExtractElementIdx = DAG.getConstant(0, DL, XLenVT);
+          ExtractBitIdx = Idx;
+        } else {
+          WideEltVT = LargestEltVT;
+          WidenVecLen = NumElts / WideEltVT.getSizeInBits();
+          // extract element index = index / element width
+          ExtractElementIdx = DAG.getNode(
+              ISD::SRL, DL, XLenVT, Idx,
+              DAG.getConstant(Log2_64(WideEltVT.getSizeInBits()), DL, XLenVT));
+          // mask bit index = index % element width
+          ExtractBitIdx = DAG.getNode(
+              ISD::AND, DL, XLenVT, Idx,
+              DAG.getConstant(WideEltVT.getSizeInBits() - 1, DL, XLenVT));
+        }
+        MVT WideVT = MVT::getVectorVT(WideEltVT, WidenVecLen);
+        Vec = DAG.getNode(ISD::BITCAST, DL, WideVT, Vec);
+        SDValue ExtractElt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, XLenVT,
+                                         Vec, ExtractElementIdx);
+        // Extract the bit from GPR.
+        SDValue ShiftRight =
+            DAG.getNode(ISD::SRL, DL, XLenVT, ExtractElt, ExtractBitIdx);
+        return DAG.getNode(ISD::AND, DL, XLenVT, ShiftRight,
+                           DAG.getConstant(1, DL, XLenVT));
+      }
+    }
+    // Otherwise, promote to an i8 vector and extract from that.
     MVT WideVT = MVT::getVectorVT(MVT::i8, VecVT.getVectorElementCount());
     Vec = DAG.getNode(ISD::ZERO_EXTEND, DL, WideVT, Vec);
     return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, EltVT, Vec, Idx);
   }
 
   // If this is a fixed vector, we need to convert it to a scalable vector.
   MVT ContainerVT = VecVT;
   if (VecVT.isFixedLengthVector()) {

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32
 ; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64
 
+; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32ZBS
+; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64ZBS
+
 define i1 @extractelt_v1i1(<1 x i8>* %x, i64 %idx) nounwind {
 ; CHECK-LABEL: extractelt_v1i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 1, e8, mf8, ta, mu
 ; CHECK-NEXT:    vle8.v v8, (a0)
 ; CHECK-NEXT:    vmseq.vi v0, v8, 0
 ; CHECK-NEXT:    vmv.v.i v8, 0
 ; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0

 ; CHECK-NEXT:    ret
   %a = load <4 x i8>, <4 x i8>* %x
   %b = icmp eq <4 x i8> %a, zeroinitializer
   %c = extractelement <4 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v8i1(<8 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v8i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, mf2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v8i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v8i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v8i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v8i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <8 x i8>, <8 x i8>* %x
   %b = icmp eq <8 x i8> %a, zeroinitializer
   %c = extractelement <8 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v16i1(<16 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v16i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m1, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v16i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v16i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v16i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v16i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <16 x i8>, <16 x i8>* %x
   %b = icmp eq <16 x i8> %a, zeroinitializer
   %c = extractelement <16 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v32i1(<32 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v32i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 32
-; CHECK-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v32i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 32
+; RV32-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v10, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v10
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v32i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 32
+; RV64-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v10, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v10
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v32i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 32
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v10
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v32i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 32
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v10
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <32 x i8>, <32 x i8>* %x
   %b = icmp eq <32 x i8> %a, zeroinitializer
   %c = extractelement <32 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v64i1(<64 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v64i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 64
-; CHECK-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m4, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v64i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 64
+; RV32-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v12, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v12, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v64i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 64
+; RV64-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v12, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v12
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v64i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 64
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v12, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v64i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 64
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v12
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <64 x i8>, <64 x i8>* %x
   %b = icmp eq <64 x i8> %a, zeroinitializer
   %c = extractelement <64 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v128i1(<128 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v128i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 128
-; CHECK-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m8, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v128i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 128
+; RV32-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v16, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v16, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v128i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 128
+; RV64-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v16, v8, 0
+; RV64-NEXT:    srli a0, a1, 6
+; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64-NEXT:    vslidedown.vx v8, v16, a0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v128i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 128
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v128i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 128
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV64ZBS-NEXT:    srli a0, a1, 6
+; RV64ZBS-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <128 x i8>, <128 x i8>* %x
   %b = icmp eq <128 x i8> %a, zeroinitializer
   %c = extractelement <128 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v256i1(<256 x i8>* %x, i64 %idx) nounwind {
 ; RV32-LABEL: extractelt_v256i1:

 ; RV64-NEXT:    addi a1, sp, 128
 ; RV64-NEXT:    vse8.v v8, (a1)
 ; RV64-NEXT:    lb a0, 0(a0)
 ; RV64-NEXT:    addi sp, s0, -512
 ; RV64-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    addi sp, sp, 512
 ; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v256i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    addi sp, sp, -512
+; RV32ZBS-NEXT:    sw ra, 508(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    sw s0, 504(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    addi s0, sp, 512
+; RV32ZBS-NEXT:    andi sp, sp, -128
+; RV32ZBS-NEXT:    andi a1, a1, 255
+; RV32ZBS-NEXT:    addi a2, a0, 128
+; RV32ZBS-NEXT:    li a3, 128
+; RV32ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v16, (a0)
+; RV32ZBS-NEXT:    vle8.v v24, (a2)
+; RV32ZBS-NEXT:    addi a0, sp, 128
+; RV32ZBS-NEXT:    add a0, a0, a1
+; RV32ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV32ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV32ZBS-NEXT:    vmv.v.i v16, 0
+; RV32ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 256
+; RV32ZBS-NEXT:    vse8.v v24, (a1)
+; RV32ZBS-NEXT:    vmv1r.v v0, v8
+; RV32ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 128
+; RV32ZBS-NEXT:    vse8.v v8, (a1)
+; RV32ZBS-NEXT:    lb a0, 0(a0)
+; RV32ZBS-NEXT:    addi sp, s0, -512
+; RV32ZBS-NEXT:    lw ra, 508(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    lw s0, 504(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    addi sp, sp, 512
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v256i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    addi sp, sp, -512
+; RV64ZBS-NEXT:    sd ra, 504(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    sd s0, 496(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    addi s0, sp, 512
+; RV64ZBS-NEXT:    andi sp, sp, -128
+; RV64ZBS-NEXT:    andi a1, a1, 255
+; RV64ZBS-NEXT:    addi a2, a0, 128
+; RV64ZBS-NEXT:    li a3, 128
+; RV64ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v16, (a0)
+; RV64ZBS-NEXT:    vle8.v v24, (a2)
+; RV64ZBS-NEXT:    addi a0, sp, 128
+; RV64ZBS-NEXT:    add a0, a0, a1
+; RV64ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV64ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV64ZBS-NEXT:    vmv.v.i v16, 0
+; RV64ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 256
+; RV64ZBS-NEXT:    vse8.v v24, (a1)
+; RV64ZBS-NEXT:    vmv1r.v v0, v8
+; RV64ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 128
+; RV64ZBS-NEXT:    vse8.v v8, (a1)
+; RV64ZBS-NEXT:    lb a0, 0(a0)
+; RV64ZBS-NEXT:    addi sp, s0, -512
+; RV64ZBS-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    addi sp, sp, 512
+; RV64ZBS-NEXT:    ret
   %a = load <256 x i8>, <256 x i8>* %x
   %b = icmp eq <256 x i8> %a, zeroinitializer
   %c = extractelement <256 x i1> %b, i64 %idx
   ret i1 %c
 }