[RISCV] Improve extract_vector_elt for fixed mask registers.

[jacquesguan]

Summary

Now the backend promotes mask vector to an i8 vector and extract element from that. We could bitcast to a widen element vector, and extract from it to GPR, then use I instruction to extract the certain bit.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4261

Use unsigned instead of auto

4263

Ele->Elt would be a more common shortening.

4267

getSizeBits(). XLenVT is always scalar.

4274

getSizeBits(). XLenVT is always scalar.

4277

getSizeBits(). XLenVT is always scalar.

4279

This shift pair can be an AND right?

4294

I don't think you need to pass false here.

4296

Use SDValue instead of auto.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4273

This won't work with Zve32 on RV64. A vector XLen elements wouldn't be legal.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

I don't think I understand what this subtract is doing.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4261

Done.

4263

Done.

4267

Done

4273

I add MaxEEW to get the right largest vector element width that we could have on current target, but we do not support Zve extension now, so I just set it to 64 and add a TODO to remind changing it after having Zve.

4274

Done

4277

Done

4279

Done

4294

Done.

4296

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

We will use vmv.x.s to extract the elment to GPR, and if eew is less than XLEN, the value will be signed extend. For example, for 8 x i1 mask vector, we will have:

GPR bit    | XLEN -1 |--------------| 6 | 5 | 4 | 3 | 2 | 1 | 0 |
mask index |    0    |--------------| 1 | 2 | 3 | 4 | 5 | 6 | 7 |

So if we want to extract the 0th element, we should set extract bit index to XLEN - 1, and otherwise the index = element width - 1 - mask index.

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

Why would the mask bits be in reversed order?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

68

It is my mistake, I thought things in wrong way. Here should not be in reversed order. I fixed it.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4288

elment -> element

4292

Sign extend means that bit 7 and above all have the same value. So why can't we just check bit 7?

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4273

You don't need Zve to test this. Before Zve was added to the spec we added a command line option "-riscv-v-fixed-length-vector-elen-max" to clip the ELEN.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4273

Done.

4288

Done.

4292

Done.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4267

Use unsigned. Nothing here needs 64-bits.

[jacquesguan]

Address comment

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4267

Done.

[jacquesguan]

ping.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4258–4259

This comment is out of date.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4258–4259

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4260

nit: Nums is a little unconventional. NumElts or something? You can also use getVectorNumElements because we know it's a fixed-length vector.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

You'll need to rebase and swap +experimental-v for +v

272

This doesn't look like it's doing the right thing. We haven't modified the original extract index a1 at this point so we could be shifting an i32 right by up to 127 places? Have I missed something?

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Here we want to extract the (idx % 32)th bit from the GPR, and a1 owns the value of idx, so (idx % 32) is a1[4-0]. And because the shift instruction only uses the 0 - (log2(xlen)-1) bits of rs2, so we actually do not need to get a1[4-0], we could just use a1. So I think here is OK?

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4260

Done.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

5

Done.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4270

I think asserting that NumElts is a power of two (or just skipping this optimization) would be useful just in case we ever support other vector types. This is quite an edge case so won't be well covered, and could silently do some weird things.

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

272

Ah yes, that's what I missed. Thanks! I see there's an AND being generated anyway which should be doing the right thing.

[jacquesguan]

Address comment.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4270

Thanks, I add an assert for this.

[frasercrmck]

LGTM, thanks

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

SDValue RISCVTargetLowering::lowerEXTRACT_VECTOR_ELT(SDValue Op,
                                                      SelectionDAG &DAG) const {
   SDLoc DL(Op);
   SDValue Idx = Op.getOperand(1);
   SDValue Vec = Op.getOperand(0);
   EVT EltVT = Op.getValueType();
   MVT VecVT = Vec.getSimpleValueType();
   MVT XLenVT = Subtarget.getXLenVT();
 
   if (VecVT.getVectorElementType() == MVT::i1) {
-    // FIXME: For now we just promote to an i8 vector and extract from that,
-    // but this is probably not optimal.
+    if (VecVT.isFixedLengthVector()) {
+      unsigned NumElts = VecVT.getVectorNumElements();
+      if (NumElts >= 8) {
+        MVT WideEltVT;
+        unsigned WidenVecLen;
+        SDValue ExtractElementIdx;
+        SDValue ExtractBitIdx;
+        unsigned MaxEEW = Subtarget.getMaxELENForFixedLengthVectors();
+        MVT LargestEltVT = MVT::getIntegerVT(
+            std::min(MaxEEW, unsigned(XLenVT.getSizeInBits())));
+        if (NumElts <= LargestEltVT.getSizeInBits()) {
+          WideEltVT = MVT::getIntegerVT(NumElts);
+          WidenVecLen = 1;
+          ExtractElementIdx = DAG.getConstant(0, DL, XLenVT);
+          ExtractBitIdx = Idx;
+        } else {
+          WideEltVT = LargestEltVT;
+          WidenVecLen = NumElts / WideEltVT.getSizeInBits();
+          // extract element index = index / element width
+          ExtractElementIdx = DAG.getNode(
+              ISD::SRL, DL, XLenVT, Idx,
+              DAG.getConstant(Log2_64(WideEltVT.getSizeInBits()), DL, XLenVT));
+          // mask bit index = index % element width
+          ExtractBitIdx = DAG.getNode(
+              ISD::AND, DL, XLenVT, Idx,
+              DAG.getConstant(WideEltVT.getSizeInBits() - 1, DL, XLenVT));
+        }
+        MVT WideVT = MVT::getVectorVT(WideEltVT, WidenVecLen);
+        Vec = DAG.getNode(ISD::BITCAST, DL, WideVT, Vec);
+        SDValue ExtractElt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, XLenVT,
+                                         Vec, ExtractElementIdx);
+        // Extract the bit from GPR.
+        SDValue ShiftRight =
+            DAG.getNode(ISD::SRL, DL, XLenVT, ExtractElt, ExtractBitIdx);
+        return DAG.getNode(ISD::AND, DL, XLenVT, ShiftRight,
+                           DAG.getConstant(1, DL, XLenVT));
+      }
+    }
+    // Otherwise, promote to an i8 vector and extract from that.
     MVT WideVT = MVT::getVectorVT(MVT::i8, VecVT.getVectorElementCount());
     Vec = DAG.getNode(ISD::ZERO_EXTEND, DL, WideVT, Vec);
     return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, EltVT, Vec, Idx);
   }
 
   // If this is a fixed vector, we need to convert it to a scalable vector.
   MVT ContainerVT = VecVT;
   if (VecVT.isFixedLengthVector()) {

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-extract-i1.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32
 ; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64
 
+; RUN: llc -mtriple=riscv32 -target-abi=ilp32d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV32ZBS
+; RUN: llc -mtriple=riscv64 -target-abi=lp64d -mattr=+v,+zfh,+f,+d,+zbs -riscv-v-vector-bits-min=128 -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,RV64ZBS
+
 define i1 @extractelt_v1i1(<1 x i8>* %x, i64 %idx) nounwind {
 ; CHECK-LABEL: extractelt_v1i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vsetivli zero, 1, e8, mf8, ta, mu
 ; CHECK-NEXT:    vle8.v v8, (a0)
 ; CHECK-NEXT:    vmseq.vi v0, v8, 0
 ; CHECK-NEXT:    vmv.v.i v8, 0
 ; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0

 ; CHECK-NEXT:    ret
   %a = load <4 x i8>, <4 x i8>* %x
   %b = icmp eq <4 x i8> %a, zeroinitializer
   %c = extractelement <4 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v8i1(<8 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v8i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, mf2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v8i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v8i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v8i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v8i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 8, e8, mf2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <8 x i8>, <8 x i8>* %x
   %b = icmp eq <8 x i8> %a, zeroinitializer
   %c = extractelement <8 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v16i1(<16 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v16i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m1, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v16i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v8, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v16i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v8, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v16i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v16i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    vsetivli zero, 16, e8, m1, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v8, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e16, mf4, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <16 x i8>, <16 x i8>* %x
   %b = icmp eq <16 x i8> %a, zeroinitializer
   %c = extractelement <16 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v32i1(<32 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v32i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 32
-; CHECK-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m2, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v32i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 32
+; RV32-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v10, v8, 0
+; RV32-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32-NEXT:    vmv.x.s a0, v10
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v32i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 32
+; RV64-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v10, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v10
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v32i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 32
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV32ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vmv.x.s a0, v10
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v32i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 32
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m2, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v10, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e32, mf2, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v10
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <32 x i8>, <32 x i8>* %x
   %b = icmp eq <32 x i8> %a, zeroinitializer
   %c = extractelement <32 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v64i1(<64 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v64i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 64
-; CHECK-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m4, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v64i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 64
+; RV32-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v12, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v12, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v64i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 64
+; RV64-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v12, v8, 0
+; RV64-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64-NEXT:    vmv.x.s a0, v12
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v64i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 64
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, mf2, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v12, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v64i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 64
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m4, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v12, v8, 0
+; RV64ZBS-NEXT:    vsetivli zero, 0, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vmv.x.s a0, v12
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <64 x i8>, <64 x i8>* %x
   %b = icmp eq <64 x i8> %a, zeroinitializer
   %c = extractelement <64 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v128i1(<128 x i8>* %x, i64 %idx) nounwind {
-; CHECK-LABEL: extractelt_v128i1:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    li a2, 128
-; CHECK-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
-; CHECK-NEXT:    vle8.v v8, (a0)
-; CHECK-NEXT:    vmseq.vi v0, v8, 0
-; CHECK-NEXT:    vmv.v.i v8, 0
-; CHECK-NEXT:    vmerge.vim v8, v8, 1, v0
-; CHECK-NEXT:    vsetivli zero, 1, e8, m8, ta, mu
-; CHECK-NEXT:    vslidedown.vx v8, v8, a1
-; CHECK-NEXT:    vmv.x.s a0, v8
-; CHECK-NEXT:    ret
+; RV32-LABEL: extractelt_v128i1:
+; RV32:       # %bb.0:
+; RV32-NEXT:    li a2, 128
+; RV32-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32-NEXT:    vle8.v v8, (a0)
+; RV32-NEXT:    vmseq.vi v16, v8, 0
+; RV32-NEXT:    srli a0, a1, 5
+; RV32-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32-NEXT:    vslidedown.vx v8, v16, a0
+; RV32-NEXT:    vmv.x.s a0, v8
+; RV32-NEXT:    srl a0, a0, a1
+; RV32-NEXT:    andi a0, a0, 1
+; RV32-NEXT:    ret
+;
+; RV64-LABEL: extractelt_v128i1:
+; RV64:       # %bb.0:
+; RV64-NEXT:    li a2, 128
+; RV64-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64-NEXT:    vle8.v v8, (a0)
+; RV64-NEXT:    vmseq.vi v16, v8, 0
+; RV64-NEXT:    srli a0, a1, 6
+; RV64-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64-NEXT:    vslidedown.vx v8, v16, a0
+; RV64-NEXT:    vmv.x.s a0, v8
+; RV64-NEXT:    srl a0, a0, a1
+; RV64-NEXT:    andi a0, a0, 1
+; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v128i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    li a2, 128
+; RV32ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v8, (a0)
+; RV32ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV32ZBS-NEXT:    srli a0, a1, 5
+; RV32ZBS-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
+; RV32ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV32ZBS-NEXT:    vmv.x.s a0, v8
+; RV32ZBS-NEXT:    bext a0, a0, a1
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v128i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    li a2, 128
+; RV64ZBS-NEXT:    vsetvli zero, a2, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v8, (a0)
+; RV64ZBS-NEXT:    vmseq.vi v16, v8, 0
+; RV64ZBS-NEXT:    srli a0, a1, 6
+; RV64ZBS-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
+; RV64ZBS-NEXT:    vslidedown.vx v8, v16, a0
+; RV64ZBS-NEXT:    vmv.x.s a0, v8
+; RV64ZBS-NEXT:    bext a0, a0, a1
+; RV64ZBS-NEXT:    ret
   %a = load <128 x i8>, <128 x i8>* %x
   %b = icmp eq <128 x i8> %a, zeroinitializer
   %c = extractelement <128 x i1> %b, i64 %idx
   ret i1 %c
 }
 
 define i1 @extractelt_v256i1(<256 x i8>* %x, i64 %idx) nounwind {
 ; RV32-LABEL: extractelt_v256i1:

 ; RV64-NEXT:    addi a1, sp, 128
 ; RV64-NEXT:    vse8.v v8, (a1)
 ; RV64-NEXT:    lb a0, 0(a0)
 ; RV64-NEXT:    addi sp, s0, -512
 ; RV64-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
 ; RV64-NEXT:    addi sp, sp, 512
 ; RV64-NEXT:    ret
+;
+; RV32ZBS-LABEL: extractelt_v256i1:
+; RV32ZBS:       # %bb.0:
+; RV32ZBS-NEXT:    addi sp, sp, -512
+; RV32ZBS-NEXT:    sw ra, 508(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    sw s0, 504(sp) # 4-byte Folded Spill
+; RV32ZBS-NEXT:    addi s0, sp, 512
+; RV32ZBS-NEXT:    andi sp, sp, -128
+; RV32ZBS-NEXT:    andi a1, a1, 255
+; RV32ZBS-NEXT:    addi a2, a0, 128
+; RV32ZBS-NEXT:    li a3, 128
+; RV32ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV32ZBS-NEXT:    vle8.v v16, (a0)
+; RV32ZBS-NEXT:    vle8.v v24, (a2)
+; RV32ZBS-NEXT:    addi a0, sp, 128
+; RV32ZBS-NEXT:    add a0, a0, a1
+; RV32ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV32ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV32ZBS-NEXT:    vmv.v.i v16, 0
+; RV32ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 256
+; RV32ZBS-NEXT:    vse8.v v24, (a1)
+; RV32ZBS-NEXT:    vmv1r.v v0, v8
+; RV32ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV32ZBS-NEXT:    addi a1, sp, 128
+; RV32ZBS-NEXT:    vse8.v v8, (a1)
+; RV32ZBS-NEXT:    lb a0, 0(a0)
+; RV32ZBS-NEXT:    addi sp, s0, -512
+; RV32ZBS-NEXT:    lw ra, 508(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    lw s0, 504(sp) # 4-byte Folded Reload
+; RV32ZBS-NEXT:    addi sp, sp, 512
+; RV32ZBS-NEXT:    ret
+;
+; RV64ZBS-LABEL: extractelt_v256i1:
+; RV64ZBS:       # %bb.0:
+; RV64ZBS-NEXT:    addi sp, sp, -512
+; RV64ZBS-NEXT:    sd ra, 504(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    sd s0, 496(sp) # 8-byte Folded Spill
+; RV64ZBS-NEXT:    addi s0, sp, 512
+; RV64ZBS-NEXT:    andi sp, sp, -128
+; RV64ZBS-NEXT:    andi a1, a1, 255
+; RV64ZBS-NEXT:    addi a2, a0, 128
+; RV64ZBS-NEXT:    li a3, 128
+; RV64ZBS-NEXT:    vsetvli zero, a3, e8, m8, ta, mu
+; RV64ZBS-NEXT:    vle8.v v16, (a0)
+; RV64ZBS-NEXT:    vle8.v v24, (a2)
+; RV64ZBS-NEXT:    addi a0, sp, 128
+; RV64ZBS-NEXT:    add a0, a0, a1
+; RV64ZBS-NEXT:    vmseq.vi v8, v16, 0
+; RV64ZBS-NEXT:    vmseq.vi v0, v24, 0
+; RV64ZBS-NEXT:    vmv.v.i v16, 0
+; RV64ZBS-NEXT:    vmerge.vim v24, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 256
+; RV64ZBS-NEXT:    vse8.v v24, (a1)
+; RV64ZBS-NEXT:    vmv1r.v v0, v8
+; RV64ZBS-NEXT:    vmerge.vim v8, v16, 1, v0
+; RV64ZBS-NEXT:    addi a1, sp, 128
+; RV64ZBS-NEXT:    vse8.v v8, (a1)
+; RV64ZBS-NEXT:    lb a0, 0(a0)
+; RV64ZBS-NEXT:    addi sp, s0, -512
+; RV64ZBS-NEXT:    ld ra, 504(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    ld s0, 496(sp) # 8-byte Folded Reload
+; RV64ZBS-NEXT:    addi sp, sp, 512
+; RV64ZBS-NEXT:    ret
   %a = load <256 x i8>, <256 x i8>* %x
   %b = icmp eq <256 x i8> %a, zeroinitializer
   %c = extractelement <256 x i1> %b, i64 %idx
   ret i1 %c
 }