[RISCV] Improve extract_vector_elt for fixed mask registers.

[jacquesguan]

Summary

Now the backend promotes mask vector to an i8 vector and extract element from that. We could bitcast to a widen element vector, and extract from it to GPR, then use I instruction to extract the certain bit.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4296

Use unsigned instead of auto

4298

Ele->Elt would be a more common shortening.

4302

getSizeBits(). XLenVT is always scalar.

4309

getSizeBits(). XLenVT is always scalar.

4312

getSizeBits(). XLenVT is always scalar.

4314

This shift pair can be an AND right?

4329

I don't think you need to pass false here.

4331

Use SDValue instead of auto.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4308

This won't work with Zve32 on RV64. A vector XLen elements wouldn't be legal.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

I don't think I understand what this subtract is doing.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4296

Done.

4298

Done.

4302

Done

4308

I add MaxEEW to get the right largest vector element width that we could have on current target, but we do not support Zve extension now, so I just set it to 64 and add a TODO to remind changing it after having Zve.

4309

Done

4312

Done

4314

Done

4329

Done.

4331

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

We will use vmv.x.s to extract the elment to GPR, and if eew is less than XLEN, the value will be signed extend. For example, for 8 x i1 mask vector, we will have:

GPR bit    | XLEN -1 |--------------| 6 | 5 | 4 | 3 | 2 | 1 | 0 |
mask index |    0    |--------------| 1 | 2 | 3 | 4 | 5 | 6 | 7 |

So if we want to extract the 0th element, we should set extract bit index to XLEN - 1, and otherwise the index = element width - 1 - mask index.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

Why would the mask bits be in reversed order?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

It is my mistake, I thought things in wrong way. Here should not be in reversed order. I fixed it.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4323

elment -> element

4327

Sign extend means that bit 7 and above all have the same value. So why can't we just check bit 7?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4308

You don't need Zve to test this. Before Zve was added to the spec we added a command line option "-riscv-v-fixed-length-vector-elen-max" to clip the ELEN.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4308

Done.

4323

Done.

4327

Done.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4302

Use unsigned. Nothing here needs 64-bits.

[jacquesguan]

Address comment

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4302

Done.

[jacquesguan]

ping.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4293–4294

This comment is out of date.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4293–4294

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4295

nit: Nums is a little unconventional. NumElts or something? You can also use getVectorNumElements because we know it's a fixed-length vector.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

You'll need to rebase and swap +experimental-v for +v

272

This doesn't look like it's doing the right thing. We haven't modified the original extract index a1 at this point so we could be shifting an i32 right by up to 127 places? Have I missed something?

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Here we want to extract the (idx % 32)th bit from the GPR, and a1 owns the value of idx, so (idx % 32) is a1[4-0]. And because the shift instruction only uses the 0 - (log2(xlen)-1) bits of rs2, so we actually do not need to get a1[4-0], we could just use a1. So I think here is OK?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4295

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4305

I think asserting that NumElts is a power of two (or just skipping this optimization) would be useful just in case we ever support other vector types. This is quite an edge case so won't be well covered, and could silently do some weird things.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Ah yes, that's what I missed. Thanks! I see there's an AND being generated anyway which should be doing the right thing.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4305

Thanks, I add an assert for this.

[frasercrmck]

LGTM, thanks

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
                                                      SelectionDAG &DAG) const {
   SDLoc DL(Op);
   SDValue Idx = Op.getOperand(1);
   SDValue Vec = Op.getOperand(0);
   EVT EltVT = Op.getValueType();
   MVT VecVT = Vec.getSimpleValueType();
   MVT XLenVT = Subtarget.getXLenVT();
 
   if (VecVT.getVectorElementType() == MVT::i1) {
-    // FIXME: For now we just promote to an i8 vector and extract from that,
-    // but this is probably not optimal.
+    if (VecVT.isFixedLengthVector()) {
+      unsigned NumElts = VecVT.getVectorNumElements();
+      if (NumElts >= 8) {
+        MVT WideEltVT;
+        unsigned WidenVecLen;
+        SDValue ExtractElementIdx;
+        SDValue ExtractBitIdx;
+        unsigned MaxEEW = Subtarget.getMaxELENForFixedLengthVectors();
+        MVT LargestEltVT = MVT::getIntegerVT(
+            std::min(MaxEEW, unsigned(XLenVT.getSizeInBits())));
+        if (NumElts <= LargestEltVT.getSizeInBits()) {
+          assert(isPowerOf2_32(NumElts) &&
+                 "the number of elements should be power of 2");
+          WideEltVT = MVT::getIntegerVT(NumElts);
+          WidenVecLen = 1;
+          ExtractElementIdx = DAG.getConstant(0, DL, XLenVT);
+          ExtractBitIdx = Idx;
+        } else {
+          WideEltVT = LargestEltVT;
+          WidenVecLen = NumElts / WideEltVT.getSizeInBits();
+          // extract element index = index / element width
+          ExtractElementIdx = DAG.getNode(
+              ISD::SRL, DL, XLenVT, Idx,
+              DAG.getConstant(Log2_64(WideEltVT.getSizeInBits()), DL, XLenVT));
+          // mask bit index = index % element width
+          ExtractBitIdx = DAG.getNode(
+              ISD::AND, DL, XLenVT, Idx,
+              DAG.getConstant(WideEltVT.getSizeInBits() - 1, DL, XLenVT));
+        }
+        MVT WideVT = MVT::getVectorVT(WideEltVT, WidenVecLen);
+        Vec = DAG.getNode(ISD::BITCAST, DL, WideVT, Vec);
+        SDValue ExtractElt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, XLenVT,
+                                         Vec, ExtractElementIdx);
+        // Extract the bit from GPR.
+        SDValue ShiftRight =
+            DAG.getNode(ISD::SRL, DL, XLenVT, ExtractElt, ExtractBitIdx);
+        return DAG.getNode(ISD::AND, DL, XLenVT, ShiftRight,
+                           DAG.getConstant(1, DL, XLenVT));
+      }
+    }
+    // Otherwise, promote to an i8 vector and extract from that.
     MVT WideVT = MVT::getVectorVT(MVT::i8, VecVT.getVectorElementCount());
     Vec = DAG.getNode(ISD::ZERO_EXTEND, DL, WideVT, Vec);
     return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, EltVT, Vec, Idx);
   }
 
   // If this is a fixed vector, we need to convert it to a scalable vector.
   MVT ContainerVT = VecVT;
   if (VecVT.isFixedLengthVector()) {

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32
 ; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64
 
+; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32ZBS
+; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64ZBS
+
 define i1 @extractelt_v1i1(<1 x i8>* %x, i64 %idx) nounwind {
 ; CHECK-LABEL: extractelt_v1i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 1, e8, mf8, ta, mu
 ; CHECK-NEXT:    vle8.v v8, (a0)
 ; CHECK-NEXT:    vmseq.vi v0, v8, 0
 ; CHECK-NEXT:    vmv.v.i v8, 0
 ; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0

 ; CHECK-NEXT:    ret
   %a = load <4 x i8>, <4 x i8>* %x
   %b = icmp eq <4 x i8> %a, zeroinitializer
   %c = extractelement <4 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v8i1(<8 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v8i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, mf2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v8i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v8i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v8i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v8i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <8 x i8>, <8 x i8>* %x
   %b = icmp eq <8 x i8> %a, zeroinitializer
   %c = extractelement <8 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v16i1(<16 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v16i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m1, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v16i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v16i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v16i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v16i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <16 x i8>, <16 x i8>* %x
   %b = icmp eq <16 x i8> %a, zeroinitializer
   %c = extractelement <16 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v32i1(<32 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v32i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 32
-; CHECK-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v32i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 32
+; RV32-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v10, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v10
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v32i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 32
+; RV64-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v10, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v10
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v32i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 32
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v10
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v32i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 32
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v10
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <32 x i8>, <32 x i8>* %x
   %b = icmp eq <32 x i8> %a, zeroinitializer
   %c = extractelement <32 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v64i1(<64 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v64i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 64
-; CHECK-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m4, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v64i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 64
+; RV32-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v12, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v12, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v64i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 64
+; RV64-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v12, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v12
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v64i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 64
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v12, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v64i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 64
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v12
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <64 x i8>, <64 x i8>* %x
   %b = icmp eq <64 x i8> %a, zeroinitializer
   %c = extractelement <64 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v128i1(<128 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v128i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 128
-; CHECK-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m8, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v128i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 128
+; RV32-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v16, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v16, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v128i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 128
+; RV64-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v16, v8, 0
+; RV64-NEXT:    srli a0, a1, 6
+; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64-NEXT:    vslidedown.vx v8, v16, a0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v128i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 128
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v128i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 128
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV64ZBS-NEXT:    srli a0, a1, 6
+; RV64ZBS-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <128 x i8>, <128 x i8>* %x
   %b = icmp eq <128 x i8> %a, zeroinitializer
   %c = extractelement <128 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v256i1(<256 x i8>* %x, i64 %idx) nounwind {
 ; RV32-LABEL: extractelt_v256i1:

 ; RV64-NEXT:    addi a1, sp, 128
 ; RV64-NEXT:    vse8.v v8, (a1)
 ; RV64-NEXT:    lb a0, 0(a0)
 ; RV64-NEXT:    addi sp, s0, -512
 ; RV64-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    addi sp, sp, 512
 ; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v256i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    addi sp, sp, -512
+; RV32ZBS-NEXT:    sw ra, 508(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    sw s0, 504(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    addi s0, sp, 512
+; RV32ZBS-NEXT:    andi sp, sp, -128
+; RV32ZBS-NEXT:    andi a1, a1, 255
+; RV32ZBS-NEXT:    addi a2, a0, 128
+; RV32ZBS-NEXT:    li a3, 128
+; RV32ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v16, (a0)
+; RV32ZBS-NEXT:    vle8.v v24, (a2)
+; RV32ZBS-NEXT:    addi a0, sp, 128
+; RV32ZBS-NEXT:    add a0, a0, a1
+; RV32ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV32ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV32ZBS-NEXT:    vmv.v.i v16, 0
+; RV32ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 256
+; RV32ZBS-NEXT:    vse8.v v24, (a1)
+; RV32ZBS-NEXT:    vmv1r.v v0, v8
+; RV32ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 128
+; RV32ZBS-NEXT:    vse8.v v8, (a1)
+; RV32ZBS-NEXT:    lb a0, 0(a0)
+; RV32ZBS-NEXT:    addi sp, s0, -512
+; RV32ZBS-NEXT:    lw ra, 508(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    lw s0, 504(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    addi sp, sp, 512
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v256i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    addi sp, sp, -512
+; RV64ZBS-NEXT:    sd ra, 504(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    sd s0, 496(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    addi s0, sp, 512
+; RV64ZBS-NEXT:    andi sp, sp, -128
+; RV64ZBS-NEXT:    andi a1, a1, 255
+; RV64ZBS-NEXT:    addi a2, a0, 128
+; RV64ZBS-NEXT:    li a3, 128
+; RV64ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v16, (a0)
+; RV64ZBS-NEXT:    vle8.v v24, (a2)
+; RV64ZBS-NEXT:    addi a0, sp, 128
+; RV64ZBS-NEXT:    add a0, a0, a1
+; RV64ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV64ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV64ZBS-NEXT:    vmv.v.i v16, 0
+; RV64ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 256
+; RV64ZBS-NEXT:    vse8.v v24, (a1)
+; RV64ZBS-NEXT:    vmv1r.v v0, v8
+; RV64ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 128
+; RV64ZBS-NEXT:    vse8.v v8, (a1)
+; RV64ZBS-NEXT:    lb a0, 0(a0)
+; RV64ZBS-NEXT:    addi sp, s0, -512
+; RV64ZBS-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    addi sp, sp, 512
+; RV64ZBS-NEXT:    ret
   %a = load <256 x i8>, <256 x i8>* %x
   %b = icmp eq <256 x i8> %a, zeroinitializer
   %c = extractelement <256 x i1> %b, i64 %idx
   ret i1 %c
 }