[RISCV] Add codegen pattern matching for bit manipulation assembly instructions.

[PaoloS]

Summary

This patch provides optimization of single block bit manipulation operations by enabling the +b target feature.
The matching patterns are reduced to equivalent single assembly bit manipulation instructions.

This patch is based on Clifford Wolf's proposal of the bit manipulation extension for RISCV:
https://github.com/riscv/riscv-bitmanip/blob/master/bitmanip-0.92.pdf

[PaoloS]

Updated tests that failed due to latest commit in LLVM.

[PaoloS]

Removed new lines and trailing spaces.

[PaoloS]

Added matching of LLVM bit manipulation instrinsics like bswap, bitreverse, fshl and fshr to the corresponding asm instructions in the RISC-V bit manipulation ISA extension. Added codegen tests accordingly. Added codegen tests for the intrinsics ctlz, cttz and ctpop, to check that they are matched with the assembly instructions clz, ctz and pcnt.

[edward-jones]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

450

Is there a way to use a symbolic value for the CC here instead of the (i32 20) magic number? I notice that other backends appear to use "SETLT", "SETULT" and similar in their DAG patterns.

llvm/test/CodeGen/RISCV/rv32Zbb.ll

14

Checking for the exact codegen for ctlz without the Zbb extension seems like it could be fragile. Is it necessary to test the non-Zbb expansion of ctlz given that it is just the default lowering?

If the test exists to make sure that "clz" isn't generated unless Zbb is present then it might be easier to change this test into a single "RV32I-NOT clz".

[PaoloS]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

450

I thought that too, but if I use the symbolic name the pattern doesn't match. Instead the CC Constant is turned into a TargetConstant and the pattern is matched with a generic select with CC. It seems that at the early stage in which I apply the pattern recognition for min/max/minu/maxu the symbolic names SETLT, SETULT, SETEQ... don't match.

[PaoloS]

Update codegen pattern matching to RISCV BitManip v0.92:

Add pattern matching for packu, packh, sext.b, sext.h.
Add correspondent tests.
Remove pattern matching for bfp. It is very unlikely to find an exact user implementation of such operation in the source code that can be matched automatically. And it is fragile to maintain.
Revert the lowering of select CC instructions into cmov. cmov would be used for any select CC instruction regardless the condition code and such condition wouldn't be computed.
Changed the tests in a way that they check just whether the bitmanip instruction is generated or not. There's no need for this patch to check the correctness of the selection without the B extension.

[simoncook]

Rebase on other bitmanip patches

[simoncook]

Update Target Features to be of form zb<x> rather than just b<x>.

[simoncook]

Update to keep in sync with feature names in D65649

[simoncook]

Rebase on updated D65649

[lewis-revill]

All looks good apart from some test nitpicks. Also add more reviewers

llvm/test/CodeGen/RISCV/rv32Zbb.ll

2

This comment is not the case for these files anymore right?

llvm/test/CodeGen/RISCV/rv64Zbs.ll

10

I'm not certain what the underscore is for, assuming it's to avoid clashing with LLVM intrinsics? If so shouldn't all LLVM intrinsics which cause a clash have a lowering?

[simoncook]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

551

nitpick: can you remove these double spaces after instruction names

563

Looking at the other RISC-V InstrInfo files, we don't indent here and add comments at the closing block indicating which predicates no longer apply, could you update to be consistent

566

These are inconsistently rewrapped to avoid hitting 80 columns, can you wrap those that aren't correctly done so

[simoncook]

Rebase on updated MC patch

[PaoloS]

llvm/test/CodeGen/RISCV/rv64Zbs.ll

10

That was a very old design choice of mine to be sure to avoid conflicts, but I should have removed it after verifying that it doesn't clash at all. I'll remove the underscore and check again.

[PaoloS]

Fixed the order of the patterns according to follow the order in the opcode table in the specs.
Same for the tests.
Fixed the flags of the tests.
Removed duplicate patterns.
Added pattern matching for rev8.h from bswap.
Removed underscores from names of the functions in the tests.
Wrapped indentation to fit into 80 columns.

[simoncook]

Looking at the structure this is looking good, it's probably worth renaming the codegen tests to match what I did with the MC patch before commiting ('Z'->'z') but other than a few formatting changes, this seems good. I haven't yet read through all the patterns, I'll add more comments as I go through it.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

159

missing a space here after if

165

missing a space here after if

169

missing a space here after if

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

641

I think the patterns shouldn't be indented after a let, the other backend tablegen files don't indent here, so we should be consistent.

[simoncook]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

660

These seem inconsistent as to whether they put GPR:$rs1 or the other argument first.

672

Isn't this the same pattern as on line 664?

681

Are these doing the operations in the right order? By my reading the shifts and the and are the wrong way around.

Isn't this the pattern you're trying to match?

(or GPR:$rs1, (or (shl (and GPR:$rs1, (i32 0x55555555), (i32 1))),
                  (srl (and GPR:$rs1, (i32 0xaaaaaaaa), (i32 1)))))

(The same applies to all the GREVI/GORCI instructions)

774

Can we do anything neater than raw ISD::CondCode values here?

794

This line and the one below should be split out to a HasStdExtZbb, IsRV32 like the 64-bit variants below

818

These patterns are the same as the ones in the block above?

850

I think this might have the same shift/and issue and GREVI/GORCI?

[PaoloS]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

551

This feature belongs to the MC patch, but I haven't seen this comment here. I'll fix that for the next commit on the encodings, or hopefully once we'll have the official release 1.00

681

Fair point, but LLVM seems to prefer to give priority to the shift. Therefore it changes e.g. 0x55555555 to 0xaaaaaaaa and vice versa in order to compensate the fact that the shift happens before.
It does the same operation.
In the case of GORCI there's also an exchange of the order of the two operands of the OR operation. The result of the operation is eventually ORed with rs1 itself and it seems that LLVM prefers to do this OR only with one of the two operands, the first one, that eventually will be ORed anyway with the other.
Here again, the outcome is the same.

Just as an example I implemented a test following the exact C syntax of the spec:

uint32_t _grev1(uint32_t a) {
  return ((a & 0x55555555) << 1) | ((a & 0xaaaaaaaa) >> 1);
}

And the pattern that is created is the one I described that privileges the shifts.

I also implemented in C the version that does the shifts first:

uint32_t _grev1b(uint32_t a) {
  return ((a << 1) & 0xaaaaaaaa) | ((a >> 1) & 0x55555555);
}

And the pattern was the same, with the shifts coming first.

774

If I add for instance the ConCode from the enum, like SETEQ, it breaks.
I could maybe add some macros with similar names, like RVB_SETEQ in this case. But I'm not sure this would be a preferred way to do it. Would it?

Besides if I use the labels SETEQ and similar other riscv codegen tests not related to bitmanip fail.

By having a brief look at other targets like for instance PowerPC I've seen that they use values like SETEQ and others in the pattern matching as I do, but they actually rely on selectcc while RISCV uses its own riscv_selectcc.

818

Yes same pattern, but LLVM uses a 64 bit value for the CondCode and complains if I don't specify the type in the pattern of riscv_selectcc. for this reason I have to use 2 versions of the same pattern, one with the 32 bit CondCode and the other with the 64 bit one.
If I'm missing something here of course any suggestion is welcome.

850

It is similar to the case of GREVI/GORCI. In this case though the difference with the description of the operation in the spec:

uint32_t shuffle32_stage(uint32_t src, uint32_t maskL, uint32_t maskR, int N)
{
  uint32_t x = src & ~(maskL | maskR);
  x |= ((src << N) & maskL) | ((src >> N) & maskR);
  return x;
}

is that the OR-increment of x is carried out not at the end, but with one of the operands of the OR on the right (the first or the second depending on the parenthesis in the implementation), and always with an equivalent outcome. So instead of having something like this:

src & ~(maskL | maskR);
 x |= ((src << N) & maskL) | ((src >> N) & maskR);

we have this:

uint32_t x = src & ~(maskL | maskR);
x = (x | ((src << N) & maskL)) | ((src >> N) & maskR);

Actually I noticed that what happens is that x is ORed with the second operand of the OR:

uint32_t x = src & ~(maskL | maskR);
x = ((src << N) & maskL) | (x | ((src >> N) & maskR));

but then I verified that of course that doesn't matter as it is commutative and LLVM selects correctly

shfli a0 8

llvm/test/CodeGen/RISCV/rv32Zbb.ll

2

Well, I used the tool for the tests, and I see that many other tests left that message. If it doesn't do any harm...

[simoncook]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

681

Ok, I hadn't noticed that the constants were the other way round to what I'd expect. If this is the canonical pattern SelectionDAG wants to match then indeed, use that pattern.

[PaoloS]

llvm/test/CodeGen/RISCV/rv32Zbb.ll

2

Correction, I edited the tests after generating them, so yes you were right. Thank you!

[PaoloS]

Fixed indentation.
Regrouped/rearranged the patterns to follow the order of the instructions in the encoding table in the specs.
Removed autogenerated comments in the tests.

[asb]

Hi Paolo. I'm sorry this has been left hanging for some time. On the one hand, with this being an experimental feature and purely additive the bar for merging is slightly lower than e.g. a rewrite of all our existing codegen patterns (which could cause new regressions). On the other hand, this pre-commit review is realistically going to be the time when the codegen patterns and associated tests get most scrutiny and it would be a shame to skip that.

I think this patch is currently of a size where it's quite difficult to review all in one go (I've certainly sat down several times to try to do so, and failed). So I'd like to propose that you split it up so we can incrementally review and merge the bitmanip subsets individually. I think we can do this pretty quickly, e.g. as long as there are no unforeseen issues found I'd imagine we might get through at least one a day over the next week (possibly more).

Does that sound like a reasonable path forwards for you?

[PaoloS]

Thanks Alex.
Yes splitting it sounds reasonable. It is indeed quite a big monolithic patch and I struggle to review it myself sometimes.
Splitting it into subsets sounds a practical approach, will do.

[PaoloS]

As suggested I split the patch by subextension.
You can find in order the pieces that add pattern-matching for zbb, zbp, zbbp, zbs and zbt here:

https://reviews.llvm.org/D79870
https://reviews.llvm.org/D79871
https://reviews.llvm.org/D79873
https://reviews.llvm.org/D79874
https://reviews.llvm.org/D79875

I'll keep this revision as a reference as long as the pieces are reviewed.
Thank you again for the time and the useful comments.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
   setOperationAction(ISD::UDIVREM, XLenVT, Expand);
   setOperationAction(ISD::SMUL_LOHI, XLenVT, Expand);
   setOperationAction(ISD::UMUL_LOHI, XLenVT, Expand);
 
   setOperationAction(ISD::SHL_PARTS, XLenVT, Custom);
   setOperationAction(ISD::SRL_PARTS, XLenVT, Custom);
   setOperationAction(ISD::SRA_PARTS, XLenVT, Custom);
 
+  if (!(Subtarget.hasStdExtZbb() || Subtarget.hasStdExtZbp())) {
     setOperationAction(ISD::ROTL, XLenVT, Expand);
     setOperationAction(ISD::ROTR, XLenVT, Expand);
+  }
+
+  if (!Subtarget.hasStdExtZbp())
     setOperationAction(ISD::BSWAP, XLenVT, Expand);
+
+  if (!Subtarget.hasStdExtZbb()) {
     setOperationAction(ISD::CTTZ, XLenVT, Expand);
     setOperationAction(ISD::CTLZ, XLenVT, Expand);
     setOperationAction(ISD::CTPOP, XLenVT, Expand);
+  }
+
+  if (Subtarget.hasStdExtZbp()) {
+    setOperationAction(ISD::BITREVERSE, XLenVT, Legal);
+  }
+
+  if (Subtarget.hasStdExtZbt()) {
+    setOperationAction(ISD::FSHL, XLenVT, Legal);
+    setOperationAction(ISD::FSHR, XLenVT, Legal);
+  }
 
   ISD::CondCode FPCCToExtend[] = {
       ISD::SETOGT, ISD::SETOGE, ISD::SETONE, ISD::SETUEQ, ISD::SETUGT,
       ISD::SETUGE, ISD::SETULT, ISD::SETULE, ISD::SETUNE, ISD::SETGT,
       ISD::SETGE,  ISD::SETNE};
 
   ISD::NodeType FPOpToExtend[] = {
       ISD::FSIN, ISD::FCOS, ISD::FSINCOS, ISD::FPOW, ISD::FREM, ISD::FP16_TO_FP,

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

def : InstAlias<"rev.n $rd, $rs",  (GREVI GPR:$rd, GPR:$rs, 0b00011)>,
       Sched<[]>;
 def : InstAlias<"rev4.b $rd, $rs", (GREVI GPR:$rd, GPR:$rs, 0b00100)>,
       Sched<[]>;
 def : InstAlias<"rev2.b $rd, $rs", (GREVI GPR:$rd, GPR:$rs, 0b00110)>,
       Sched<[]>;
 def : InstAlias<"rev.b $rd, $rs",  (GREVI GPR:$rd, GPR:$rs, 0b00111)>,
       Sched<[]>;
 def : InstAlias<"rev8.h $rd, $rs", (GREVI GPR:$rd, GPR:$rs, 0b01000)>,
       Sched<[]>;
 def : InstAlias<"rev4.h $rd, $rs", (GREVI GPR:$rd, GPR:$rs, 0b01100)>,
       Sched<[]>;
 def : InstAlias<"rev2.h $rd, $rs", (GREVI GPR:$rd, GPR:$rs, 0b01110)>,
       Sched<[]>;
 def : InstAlias<"rev.h $rd, $rs",  (GREVI GPR:$rd, GPR:$rs, 0b01111)>,
       Sched<[]>;
 
 def : InstAlias<"zip.n $rd, $rs",    (SHFLI   GPR:$rd, GPR:$rs, 0b0001)>,

 def : InstAlias<"rev2.w $rd, $rs",  (GREVI GPR:$rd, GPR:$rs, 0b011110)>,
       Sched<[]>;
 def : InstAlias<"rev.w $rd, $rs",   (GREVI GPR:$rd, GPR:$rs, 0b011111)>,
       Sched<[]>;
 def : InstAlias<"rev32 $rd, $rs",   (GREVI GPR:$rd, GPR:$rs, 0b100000)>,
       Sched<[]>;
 def : InstAlias<"rev16 $rd, $rs",   (GREVI GPR:$rd, GPR:$rs, 0b110000)>,
       Sched<[]>;
 def : InstAlias<"rev8 $rd, $rs",    (GREVI GPR:$rd, GPR:$rs, 0b111000)>,
       Sched<[]>;
 def : InstAlias<"rev4 $rd, $rs",    (GREVI GPR:$rd, GPR:$rs, 0b111100)>,
       Sched<[]>;
 def : InstAlias<"rev2 $rd, $rs",    (GREVI GPR:$rd, GPR:$rs, 0b111110)>,
       Sched<[]>;
 def : InstAlias<"rev $rd, $rs",     (GREVI GPR:$rd, GPR:$rs, 0b111111)>,
       Sched<[]>;
 
 def : InstAlias<"zip8.w $rd, $rs",   (SHFLI   GPR:$rd, GPR:$rs, 0b01000)>,
       Sched<[]>;
 def : InstAlias<"unzip8.w $rd, $rs", (UNSHFLI GPR:$rd, GPR:$rs, 0b01000)>,
       Sched<[]>;
 def : InstAlias<"zip4.w $rd, $rs",   (SHFLI   GPR:$rd, GPR:$rs, 0b01100)>,
       Sched<[]>;
 def : InstAlias<"unzip4.w $rd, $rs", (UNSHFLI GPR:$rd, GPR:$rs, 0b01100)>,
       Sched<[]>;
 def : InstAlias<"zip2.w $rd, $rs",   (SHFLI   GPR:$rd, GPR:$rs, 0b01110)>,
       Sched<[]>;
 def : InstAlias<"unzip2.w $rd, $rs", (UNSHFLI GPR:$rd, GPR:$rs, 0b01110)>,
       Sched<[]>;
 def : InstAlias<"zip.w $rd, $rs",    (SHFLI   GPR:$rd, GPR:$rs, 0b01111)>,
       Sched<[]>;
 def : InstAlias<"unzip.w $rd, $rs",  (UNSHFLI GPR:$rd, GPR:$rs, 0b01111)>,

 def : CompressPat<(SUB GPRC:$rs1, X0, GPRC:$rs1),
                   (C_NEG GPRC:$rs1)>;
 } // Predicates = [HasStdExtZbproposedc, HasStdExtC]
 
 let Predicates = [HasStdExtZbproposedc, HasStdExtZbbOrZbp, HasStdExtC, IsRV64] in {
 def : CompressPat<(PACK GPRC:$rs1, GPRC:$rs1, X0),
                   (C_ZEXTW GPRC:$rs1)>;
 } // Predicates = [HasStdExtZbproposedc, HasStdExtC, IsRV64]
+
+//===----------------------------------------------------------------------===//
+// Codegen patterns
+//===----------------------------------------------------------------------===//
+
+let Predicates = [HasStdExtZbb] in {
+  def : Pat<(ctlz GPR:$rs1), (CLZ GPR:$rs1)>;
+  def : Pat<(cttz GPR:$rs1), (CTZ GPR:$rs1)>;
+  def : Pat<(ctpop GPR:$rs1), (PCNT GPR:$rs1)>;
+  def : Pat<(xor (shl (xor GPR:$rs1, -1), GPR:$rs2), -1), (SLO GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(xor (srl (xor GPR:$rs1, -1), GPR:$rs2), -1), (SRO GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(smin GPR:$rs1, GPR:$rs2), (MIN  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs1, GPR:$rs2, (i32 20), GPR:$rs1, GPR:$rs2),
+            (MIN  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(umin GPR:$rs1, GPR:$rs2), (MINU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs1, GPR:$rs2, (i32 12), GPR:$rs1, GPR:$rs2),
+            (MINU  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(smax GPR:$rs1, GPR:$rs2), (MAX  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs2, GPR:$rs1, (i32 20), GPR:$rs1, GPR:$rs2),
+            (MAX  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(umax GPR:$rs1, GPR:$rs2), (MAXU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs2, GPR:$rs1, (i32 12), GPR:$rs1, GPR:$rs2),
+            (MAXU  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(sra (shl GPR:$rs1, (i32 24)), (i32 24)), (SEXTB GPR:$rs1)>;
+  def : Pat<(sra (shl GPR:$rs1, (i32 16)), (i32 16)), (SEXTH GPR:$rs1)>;
+}
+
+let Predicates = [HasStdExtZbb, IsRV64] in {
+  def : Pat<(and (add GPR:$rs1, GPR:$rs2), 0xFFFFFFFF), (ADDWU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (sub GPR:$rs1, GPR:$rs2), 0xFFFFFFFF), (SUBWU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (add GPR:$rs, simm12:$simm12), 0xFFFFFFFF),
+            (ADDIWU GPR:$rs, simm12:$simm12)>;
+  def : Pat<(add GPR:$rs1, (and GPR:$rs2, 0xFFFFFFFF)), (ADDUW GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(sub GPR:$rs1, (and GPR:$rs2, 0xFFFFFFFF)), (SUBUW GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs1, GPR:$rs2, (i64 20), GPR:$rs1, GPR:$rs2),
+            (MIN  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs1, GPR:$rs2, (i64 12), GPR:$rs1, GPR:$rs2),
+            (MINU  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs2, GPR:$rs1, (i64 20), GPR:$rs1, GPR:$rs2),
+            (MAX  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(riscv_selectcc GPR:$rs2, GPR:$rs1, (i64 12), GPR:$rs1, GPR:$rs2),
+            (MAXU  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(sra (shl GPR:$rs1, (i64 56)), (i64 56)), (SEXTB GPR:$rs1)>;
+  def : Pat<(sra (shl GPR:$rs1, (i64 48)), (i64 48)), (SEXTH GPR:$rs1)>;
+}
+
+let Predicates = [HasStdExtZbbOrZbp] in {
+  def : Pat<(and GPR:$rs1, (not GPR:$rs2)), (ANDN GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(or  GPR:$rs1, (not GPR:$rs2)), (ORN  GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(xor GPR:$rs1, (not GPR:$rs2)), (XNOR GPR:$rs1, GPR:$rs2)>;
+
+  def : Pat<(rotl GPR:$rs1, GPR:$rs2), (ROL GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(rotr GPR:$rs1, GPR:$rs2), (ROR GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(fshl GPR:$rs1, GPR:$rs1, GPR:$rs2), (ROL GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(fshr GPR:$rs1, GPR:$rs1, GPR:$rs2), (ROR GPR:$rs1, GPR:$rs2)>;
+}
+
+let Predicates = [HasStdExtZbbOrZbp, IsRV32] in {
+  def : Pat<(or (and GPR:$rs1, 0xFFFF), (shl GPR:$rs2, (i32 16))),
+            (PACK GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(or (and GPR:$rs2, 0xFFFF0000), (srl GPR:$rs1, (i32 16))),
+            (PACKU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(or (and (shl GPR:$rs2, (i32 8)), 0xFF00),
+                (and GPR:$rs1, 0x00FF)),
+            (PACKH GPR:$rs1, GPR:$rs2)>;
+}
+
+let Predicates = [HasStdExtZbbOrZbp, IsRV64] in {
+  def : Pat<(or (and GPR:$rs1, 0xFFFFFFFF), (shl GPR:$rs2, (i64 32))),
+            (PACK GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(or (and GPR:$rs2, 0xFFFFFFFF00000000), (srl GPR:$rs1, (i64 32))),
+            (PACKU GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(or (and (shl GPR:$rs2, (i64 8)), 0xFF00),
+                (and GPR:$rs1, 0x00FF)),
+            (PACKH GPR:$rs1, GPR:$rs2)>;
+}
+
+let Predicates = [HasStdExtZbp, IsRV32] in {
+  def : Pat<(or (and (shl GPR:$rs1, (i32 1)), (i32 0xAAAAAAAA)),
+                (and (srl GPR:$rs1, (i32 1)), (i32 0x55555555))),
+            (GREVI GPR:$rs1, (i32 1))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i32 2)), (i32 0xCCCCCCCC)),
+                (and (srl GPR:$rs1, (i32 2)), (i32 0x33333333))),
+            (GREVI GPR:$rs1, (i32 2))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i32 4)), (i32 0xF0F0F0F0)),
+                (and (srl GPR:$rs1, (i32 4)), (i32 0x0F0F0F0F))),
+            (GREVI GPR:$rs1, (i32 4))>;
+  def : Pat<(or (shl GPR:$rs1, (i32 16)), (srl GPR:$rs1, (i32 16))),
+            (GREVI GPR:$rs1, (i32 16))>;
+  def : Pat<(rotl GPR:$rs1, (i32 16)), (GREVI GPR:$rs1, (i32 16))>;
+
+  def : Pat<(rotr (bswap GPR:$rs1), (i32 16)), (GREVI GPR:$rs1, (i32 8))>;
+  def : Pat<(bswap GPR:$rs1), (GREVI GPR:$rs1, (i32 24))>;
+  def : Pat<(bitreverse GPR:$rs1), (GREVI GPR:$rs1, (i32 31))>;
+
+  def : Pat<(or (or (and (srl GPR:$rs1, (i32 1)), (i32 0x55555555)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i32 1)), (i32 0xAAAAAAAA))),
+            (GORCI GPR:$rs1, (i32 1))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i32 2)), (i32 0x33333333)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i32 2)), (i32 0xCCCCCCCC))),
+            (GORCI GPR:$rs1, (i32 2))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i32 4)), (i32 0x0F0F0F0F)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i32 4)), (i32 0xF0F0F0F0))),
+            (GORCI GPR:$rs1, (i32 4))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i32 8)), (i32 0x00FF00FF)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i32 8)), (i32 0xFF00FF00))),
+            (GORCI GPR:$rs1, (i32 8))>;
+  def : Pat<(or (or (srl GPR:$rs1, (i32 16)), GPR:$rs1),
+                (shl GPR:$rs1, (i32 16))),
+            (GORCI GPR:$rs1, (i32 16))>;
+
+  def : Pat<(or (or (and (shl GPR:$rs1, (i32 8)), (i32 0x00FF0000)),
+                     (and GPR:$rs1, (i32 0xFF0000FF))),
+                (and (srl GPR:$rs1, (i32 8)), (i32 0x0000FF00))),
+            (SHFLI GPR:$rs1, (i32 8))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i32 4)), (i32 0x0F000F00)),
+                     (and GPR:$rs1, (i32 0xF00FF00F))),
+                (and (srl GPR:$rs1, (i32 4)), (i32 0x00F000F0))),
+            (SHFLI GPR:$rs1, (i32 4))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i32 2)), (i32 0x30303030)),
+                     (and GPR:$rs1, (i32 0xC3C3C3C3))),
+                (and (srl GPR:$rs1, (i32 2)), (i32 0x0C0C0C0C))),
+            (SHFLI GPR:$rs1, (i32 2))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i32 1)), (i32 0x44444444)),
+                     (and GPR:$rs1, (i32 0x99999999))),
+                (and (srl GPR:$rs1, (i32 1)), (i32 0x22222222))),
+            (SHFLI GPR:$rs1, (i32 1))>;
+}
+
+let Predicates = [HasStdExtZbp, IsRV64] in {
+  def : Pat<(or (and (shl GPR:$rs1, (i64 1)), (i64 0xAAAAAAAAAAAAAAAA)),
+                (and (srl GPR:$rs1, (i64 1)), (i64 0x5555555555555555))),
+            (GREVI GPR:$rs1, (i64 1))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i64 2)), (i64 0xCCCCCCCCCCCCCCCC)),
+                (and (srl GPR:$rs1, (i64 2)), (i64 0x3333333333333333))),
+            (GREVI GPR:$rs1, (i64 2))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i64 4)), (i64 0xF0F0F0F0F0F0F0F0)),
+                (and (srl GPR:$rs1, (i64 4)), (i64 0x0F0F0F0F0F0F0F0F))),
+            (GREVI GPR:$rs1, (i64 4))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i64 8)), (i64 0xFF00FF00FF00FF00)),
+                (and (srl GPR:$rs1, (i64 8)), (i64 0x00FF00FF00FF00FF))),
+            (GREVI GPR:$rs1, (i64 8))>;
+  def : Pat<(or (and (shl GPR:$rs1, (i64 16)), (i64 0xFFFF0000FFFF0000)),
+                (and (srl GPR:$rs1, (i64 16)), (i64 0x0000FFFF0000FFFF))),
+            (GREVI GPR:$rs1, (i64 16))>;
+  def : Pat<(or (shl GPR:$rs1, (i64 16)), (srl GPR:$rs1, (i64 16))),
+            (GREVI GPR:$rs1, (i64 16))>;
+  def : Pat<(or (shl GPR:$rs1, (i64 32)), (srl GPR:$rs1, (i64 32))),
+            (GREVI GPR:$rs1, (i64 32))>;
+  def : Pat<(rotl GPR:$rs1, (i64 32)), (GREVI GPR:$rs1, (i64 32))>;
+
+  def : Pat<(bswap GPR:$rs1), (GREVI GPR:$rs1, (i64 56))>;
+  def : Pat<(bitreverse GPR:$rs1), (GREVI GPR:$rs1, (i64 63))>;
+
+  def : Pat<(or (or (and (srl GPR:$rs1, (i64 1)), (i64 0x5555555555555555)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i64 1)), (i64 0xAAAAAAAAAAAAAAAA))),
+            (GORCI GPR:$rs1, (i64 1))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i64 2)), (i64 0x3333333333333333)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i64 2)), (i64 0xCCCCCCCCCCCCCCCC))),
+            (GORCI GPR:$rs1, (i64 2))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i64 4)), (i64 0x0F0F0F0F0F0F0F0F)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i64 4)), (i64 0xF0F0F0F0F0F0F0F0))),
+            (GORCI GPR:$rs1, (i64 4))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i64 8)), (i64 0x00FF00FF00FF00FF)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i64 8)), (i64 0xFF00FF00FF00FF00))),
+            (GORCI GPR:$rs1, (i64 8))>;
+  def : Pat<(or (or (and (srl GPR:$rs1, (i64 16)), (i64 0x0000FFFF0000FFFF)), GPR:$rs1),
+                (and (shl GPR:$rs1, (i64 16)), (i64 0xFFFF0000FFFF0000))),
+            (GORCI GPR:$rs1, (i64 16))>;
+  def : Pat<(or (or (srl GPR:$rs1, (i64 32)), GPR:$rs1),
+                (shl GPR:$rs1, (i64 32))),
+            (GORCI GPR:$rs1, (i64 32))>;
+
+  def : Pat<(or (or (and (shl GPR:$rs1, (i64 16)), (i64 0x0000FFFF00000000)),
+                     (and GPR:$rs1, (i64 0xFFFF00000000FFFF))),
+                (and (srl GPR:$rs1, (i64 16)), (i64 0x00000000FFFF0000))),
+            (SHFLI GPR:$rs1, (i64 16))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i64 8)), (i64 0x00FF000000FF0000)),
+                     (and GPR:$rs1, (i64 0xFF0000FFFF0000FF))),
+                (and (srl GPR:$rs1, (i64 8)), (i64 0x0000FF000000FF00))),
+            (SHFLI GPR:$rs1, (i64 8))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i64 4)), (i64 0x0F000F000F000F00)),
+                     (and GPR:$rs1, (i64 0xF00FF00FF00FF00F))),
+                (and (srl GPR:$rs1, (i64 4)), (i64 0x00F000F000F000F0))),
+            (SHFLI GPR:$rs1, (i64 4))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i64 2)), (i64 0x3030303030303030)),
+                     (and GPR:$rs1, (i64 0xC3C3C3C3C3C3C3C3))),
+                (and (srl GPR:$rs1, (i64 2)), (i64 0x0C0C0C0C0C0C0C0C))),
+            (SHFLI GPR:$rs1, (i64 2))>;
+  def : Pat<(or (or (and (shl GPR:$rs1, (i64 1)), (i64 0x4444444444444444)),
+                     (and GPR:$rs1, (i64 0x9999999999999999))),
+                (and (srl GPR:$rs1, (i64 1)), (i64 0x2222222222222222))),
+            (SHFLI GPR:$rs1, (i64 1))>;
+}
+
+let Predicates = [HasStdExtZbs] in {
+  def : Pat<(or (shl 1, GPR:$rs2), GPR:$rs1), (SBSET GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (xor (shl 1, GPR:$rs2), -1), GPR:$rs1), (SBCLR GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (rotl -2, GPR:$rs2), GPR:$rs1), (SBCLR GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(xor GPR:$rs1, (shl 1, GPR:$rs2)), (SBINV GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (srl GPR:$rs1, GPR:$rs2), 1), (SBEXT GPR:$rs1, GPR:$rs2)>;
+}
+
+let Predicates = [HasStdExtZbs, IsRV64] in {
+  def : Pat<(or (riscv_sllw 1, GPR:$rs2), GPR:$rs1), (SBSET GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (xor (riscv_sllw 1, GPR:$rs2), -1), GPR:$rs1), (SBCLR GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(and (rotl -2, GPR:$rs2), GPR:$rs1), (SBCLR GPR:$rs1, GPR:$rs2)>;
+  def : Pat<(xor GPR:$rs1, (riscv_sllw 1, GPR:$rs2)), (SBINV GPR:$rs1, GPR:$rs2)>;
+}
+
+let Predicates = [HasStdExtZbt] in {
+  def : Pat<(or (and (xor GPR:$rs2, -1), GPR:$rs3), (and GPR:$rs2, GPR:$rs1)),
+            (CMIX GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+
+  def : Pat<(fshl GPR:$rs1, GPR:$rs2, GPR:$rs3), (FSL GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+  def : Pat<(fshr GPR:$rs1, GPR:$rs2, GPR:$rs3), (FSR GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+}
+
+let Predicates = [HasStdExtZbt, IsRV32] in {
+  def : Pat<(riscv_selectcc GPR:$rs2, (i32 0), (i32 17), GPR:$rs3, GPR:$rs1),
+            (CMOV GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+}
+
+let Predicates = [HasStdExtZbt, IsRV64] in {
+  def : Pat<(riscv_selectcc GPR:$rs2, (i64 0), (i64 17), GPR:$rs3, GPR:$rs1),
+            (CMOV GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+}

llvm/test/CodeGen/RISCV/rv32Zbb.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32I
+; RUN: llc -mtriple=riscv32 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbb -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+
+declare i32 @llvm.ctlz.i32(i32, i1)
+
+define i32 @_ctlz_i32(i32 %a) nounwind {
+; RV32I-NOT:    clz a0, a0
+;
+; RV32IB-LABEL: _ctlz_i32:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    beqz a0, .LBB0_2
+; RV32IB-NEXT:  # %bb.1: # %cond.false
+; RV32IB-NEXT:    clz a0, a0
+; RV32IB-NEXT:    ret
+; RV32IB-NEXT:  .LBB0_2:
+; RV32IB-NEXT:    addi a0, zero, 32
+; RV32IB-NEXT:    ret
+  %1 = call i32 @llvm.ctlz.i32(i32 %a, i1 false)
+  ret i32 %1
+}
+
+declare i32 @llvm.cttz.i32(i32, i1)
+
+define i32 @_cttz_i32(i32 %a) nounwind {
+; RV32I-NOT:    ctz a0, a0
+;
+; RV32IB-LABEL: _cttz_i32:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    beqz a0, .LBB1_2
+; RV32IB-NEXT:  # %bb.1: # %cond.false
+; RV32IB-NEXT:    ctz a0, a0
+; RV32IB-NEXT:    ret
+; RV32IB-NEXT:  .LBB1_2:
+; RV32IB-NEXT:    addi a0, zero, 32
+; RV32IB-NEXT:    ret
+  %1 = call i32 @llvm.cttz.i32(i32 %a, i1 false)
+  ret i32 %1
+}
+
+declare i32 @llvm.ctpop.i32(i32)
+
+define i32 @_ctpop_i32(i32 %a) nounwind {
+; RV32I-NOT:    pcnt a0, a0
+;
+; RV32IB-LABEL: _ctpop_i32:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    pcnt a0, a0
+; RV32IB-NEXT:    ret
+  %1 = call i32 @llvm.ctpop.i32(i32 %a)
+  ret i32 %1
+}
+
+define i32 @_slo(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    slo a0, a0, a1
+;
+; RV32IB-LABEL: _slo:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    slo a0, a0, a1
+; RV32IB-NEXT:    ret
+  %neg = xor i32 %a, -1
+  %shl = shl i32 %neg, %b
+  %neg1 = xor i32 %shl, -1
+  ret i32 %neg1
+}
+
+define i32 @_sro(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    sro a0, a0, a1
+;
+; RV32IB-LABEL: _sro:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sro a0, a0, a1
+; RV32IB-NEXT:    ret
+  %neg = xor i32 %a, -1
+  %shr = lshr i32 %neg, %b
+  %neg1 = xor i32 %shr, -1
+  ret i32 %neg1
+}
+
+define i32 @_min(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    min a0, a0, a1
+;
+; RV32IB-LABEL: _min:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    min a0, a0, a1
+; RV32IB-NEXT:    ret
+  %cmp = icmp slt i32 %a, %b
+  %cond = select i1 %cmp, i32 %a, i32 %b
+  ret i32 %cond
+}
+
+define i32 @_max(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    max a0, a0, a1
+;
+; RV32IB-LABEL: _max:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    max a0, a0, a1
+; RV32IB-NEXT:    ret
+  %cmp = icmp sgt i32 %a, %b
+  %cond = select i1 %cmp, i32 %a, i32 %b
+  ret i32 %cond
+}
+
+define i32 @_minu(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    minu a0, a0, a1
+;
+; RV32IB-LABEL: _minu:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    minu a0, a0, a1
+; RV32IB-NEXT:    ret
+  %cmp = icmp ult i32 %a, %b
+  %cond = select i1 %cmp, i32 %a, i32 %b
+  ret i32 %cond
+}
+
+define i32 @_maxu(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    maxu a0, a0, a1
+;
+; RV32IB-LABEL: _maxu:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    maxu a0, a0, a1
+; RV32IB-NEXT:    ret
+  %cmp = icmp ugt i32 %a, %b
+  %cond = select i1 %cmp, i32 %a, i32 %b
+  ret i32 %cond
+}
+
+define i32 @_sextb(i32 %a) nounwind {
+; RV32I-NOT:    sext.b a0, a0
+;
+; RV32IB-LABEL: _sextb:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sext.b a0, a0
+; RV32IB-NEXT:    ret
+  %shl = shl i32 %a, 24
+  %shr = ashr exact i32 %shl, 24
+  ret i32 %shr
+}
+
+define i32 @_sexth(i32 %a) nounwind {
+; RV32I-NOT:    sext.h a0, a0
+;
+; RV32IB-LABEL: _sexth:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sext.h a0, a0
+; RV32IB-NEXT:    ret
+  %shl = shl i32 %a, 16
+  %shr = ashr exact i32 %shl, 16
+  ret i32 %shr
+}

llvm/test/CodeGen/RISCV/rv32Zbbp.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32I
+; RUN: llc -mtriple=riscv32 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbb -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbp -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+
+define i32 @_pack(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    pack a0, a0, a1
+;
+; RV32IB-LABEL: _pack:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    pack a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shl = and i32 %a, 65535
+  %shl1 = shl i32 %b, 16
+  %or = or i32 %shl1, %shl
+  ret i32 %or
+}
+
+define i32 @_packu(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    packu a0, a0, a1
+;
+; RV32IB-LABEL: _packu:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    packu a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shr = lshr i32 %a, 16
+  %shr1 = and i32 %b, -65536
+  %or = or i32 %shr1, %shr
+  ret i32 %or
+}
+
+define i32 @_packh(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    packh a0, a0, a1
+;
+; RV32IB-LABEL: _packh:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    packh a0, a0, a1
+; RV32IB-NEXT:    ret
+  %and = and i32 %a, 255
+  %and1 = shl i32 %b, 8
+  %shl = and i32 %and1, 65280
+  %or = or i32 %shl, %and
+  ret i32 %or
+}
+
+define i32 @_andn(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    andn a0, a0, a1
+;
+; RV32IB-LABEL: _andn:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    andn a0, a0, a1
+; RV32IB-NEXT:    ret
+  %neg = xor i32 %b, -1
+  %and = and i32 %neg, %a
+  ret i32 %and
+}
+
+define i32 @_orn(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    orn a0, a0, a1
+;
+; RV32IB-LABEL: _orn:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orn a0, a0, a1
+; RV32IB-NEXT:    ret
+  %neg = xor i32 %b, -1
+  %or = or i32 %neg, %a
+  ret i32 %or
+}
+
+define i32 @_xnor(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    xnor a0, a0, a1
+;
+; RV32IB-LABEL: _xnor:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    xnor a0, a0, a1
+; RV32IB-NEXT:    ret
+  %neg = xor i32 %a, -1
+  %xor = xor i32 %neg, %b
+  ret i32 %xor
+}
+
+declare i32 @llvm.fshl.i32(i32, i32, i32)
+
+define i32 @_rol(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    rol a0, a0, a1
+;
+; RV32IB-LABEL: _rol:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rol a0, a0, a1
+; RV32IB-NEXT:    ret
+  %or = tail call i32 @llvm.fshl.i32(i32 %a, i32 %a, i32 %b)
+  ret i32 %or
+}
+
+declare i32 @llvm.fshr.i32(i32, i32, i32)
+
+define i32 @_ror(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    ror a0, a0, a1
+;
+; RV32IB-LABEL: _ror:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    ror a0, a0, a1
+; RV32IB-NEXT:    ret
+  %or = tail call i32 @llvm.fshr.i32(i32 %a, i32 %a, i32 %b)
+  ret i32 %or
+}

llvm/test/CodeGen/RISCV/rv32Zbp.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32I
+; RUN: llc -mtriple=riscv32 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbp -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+
+define i32 @_grev1(i32 %a) nounwind {
+; RV32I-NOT:    rev.p a0, a0
+;
+; RV32IB-LABEL: _grev1:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev.p a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 1
+  %shl = and i32 %and, -1431655766
+  %and1 = lshr i32 %a, 1
+  %shr = and i32 %and1, 1431655765
+  %or = or i32 %shl, %shr
+  ret i32 %or
+}
+
+define i32 @_grev2(i32 %a) nounwind {
+; RV32I-NOT:    rev2.n a0, a0
+;
+; RV32IB-LABEL: _grev2:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev2.n a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 2
+  %shl = and i32 %and, -858993460
+  %and1 = lshr i32 %a, 2
+  %shr = and i32 %and1, 858993459
+  %or = or i32 %shl, %shr
+  ret i32 %or
+}
+
+define i32 @_grev4(i32 %a) nounwind {
+; RV32I-NOT:    rev4.b a0, a0
+;
+; RV32IB-LABEL: _grev4:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev4.b a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 4
+  %shl = and i32 %and, -252645136
+  %and1 = lshr i32 %a, 4
+  %shr = and i32 %and1, 252645135
+  %or = or i32 %shl, %shr
+  ret i32 %or
+}
+
+define i32 @_grev8(i32 %a) nounwind {
+; RV32I-NOT:    rev8.h a0, a0
+;
+; RV32IB-LABEL: _grev8:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev8.h a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 8
+  %shl = and i32 %and, -16711936
+  %and1 = lshr i32 %a, 8
+  %shr = and i32 %and1, 16711935
+  %or = or i32 %shl, %shr
+  ret i32 %or
+}
+
+define i32 @_grev16(i32 %a) nounwind {
+; RV32I-NOT:    rev16 a0, a0
+;
+; RV32IB-LABEL: _grev16:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev16 a0, a0
+; RV32IB-NEXT:    ret
+  %shl = shl i32 %a, 16
+  %shr = lshr i32 %a, 16
+  %or = or i32 %shl, %shr
+  ret i32 %or
+}
+
+declare i32 @llvm.bswap.i32(i32)
+
+define i32 @_bswap_i32(i32 %a) nounwind {
+; RV32I-NOT:    rev8 a0, a0
+;
+; RV32IB-LABEL: _bswap_i32:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev8 a0, a0
+; RV32IB-NEXT:    ret
+  %1 = tail call i32 @llvm.bswap.i32(i32 %a)
+  ret i32 %1
+}
+
+declare i32 @llvm.bitreverse.i32(i32)
+
+define i32 @_bitreverse_i32(i32 %a) nounwind {
+; RV32I-NOT:    rev a0, a0
+;
+; RV32IB-LABEL: _bitreverse_i32:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    rev a0, a0
+; RV32IB-NEXT:    ret
+  %1 = tail call i32 @llvm.bitreverse.i32(i32 %a)
+  ret i32 %1
+}
+
+define i32 @_gorc1(i32 %a) nounwind {
+; RV32I-NOT:    orc.p a0, a0
+;
+; RV32IB-LABEL: _gorc1:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orc.p a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 1
+  %shl = and i32 %and, -1431655766
+  %and1 = lshr i32 %a, 1
+  %shr = and i32 %and1, 1431655765
+  %or = or i32 %shr, %a
+  %or2 = or i32 %or, %shl
+  ret i32 %or2
+}
+
+define i32 @_gorc2(i32 %a) nounwind {
+; RV32I-NOT:    orc2.n a0, a0
+;
+; RV32IB-LABEL: _gorc2:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orc2.n a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 2
+  %shl = and i32 %and, -858993460
+  %and1 = lshr i32 %a, 2
+  %shr = and i32 %and1, 858993459
+  %or = or i32 %shr, %a
+  %or2 = or i32 %or, %shl
+  ret i32 %or2
+}
+
+define i32 @_gorc4(i32 %a) nounwind {
+; RV32I-NOT:    orc4.b a0, a0
+;
+; RV32IB-LABEL: _gorc4:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orc4.b a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 4
+  %shl = and i32 %and, -252645136
+  %and1 = lshr i32 %a, 4
+  %shr = and i32 %and1, 252645135
+  %or = or i32 %shr, %a
+  %or2 = or i32 %or, %shl
+  ret i32 %or2
+}
+
+define i32 @_gorc8(i32 %a) nounwind {
+; RV32I-NOT:    orc8.h a0, a0
+;
+; RV32IB-LABEL: _gorc8:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orc8.h a0, a0
+; RV32IB-NEXT:    ret
+  %and = shl i32 %a, 8
+  %shl = and i32 %and, -16711936
+  %and1 = lshr i32 %a, 8
+  %shr = and i32 %and1, 16711935
+  %or = or i32 %shr, %a
+  %or2 = or i32 %or, %shl
+  ret i32 %or2
+}
+
+define i32 @_gorc16(i32 %a) nounwind {
+; RV32I-NOT:    orc16 a0, a0
+;
+; RV32IB-LABEL: _gorc16:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    orc16 a0, a0
+; RV32IB-NEXT:    ret
+  %shl = shl i32 %a, 16
+  %shr = lshr i32 %a, 16
+  %or = or i32 %shr, %a
+  %or2 = or i32 %or, %shl
+  ret i32 %or2
+}
+
+define i32 @_shfl1(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    zip.n a0, a0
+;
+; RV32IB-LABEL: _shfl1:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    zip.n a0, a0
+; RV32IB-NEXT:    ret
+  %and = and i32 %a, -1717986919
+  %shl = shl i32 %a, 1
+  %and1 = and i32 %shl, 1145324612
+  %or = or i32 %and1, %and
+  %shr = lshr i32 %a, 1
+  %and2 = and i32 %shr, 572662306
+  %or3 = or i32 %or, %and2
+  ret i32 %or3
+}
+
+define i32 @_shfl2(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    zip2.b a0, a0
+;
+; RV32IB-LABEL: _shfl2:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    zip2.b a0, a0
+; RV32IB-NEXT:    ret
+  %and = and i32 %a, -1010580541
+  %shl = shl i32 %a, 2
+  %and1 = and i32 %shl, 808464432
+  %or = or i32 %and1, %and
+  %shr = lshr i32 %a, 2
+  %and2 = and i32 %shr, 202116108
+  %or3 = or i32 %or, %and2
+  ret i32 %or3
+}
+
+define i32 @_shfl4(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    zip4.h a0, a0
+;
+; RV32IB-LABEL: _shfl4:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    zip4.h a0, a0
+; RV32IB-NEXT:    ret
+  %and = and i32 %a, -267390961
+  %shl = shl i32 %a, 4
+  %and1 = and i32 %shl, 251662080
+  %or = or i32 %and1, %and
+  %shr = lshr i32 %a, 4
+  %and2 = and i32 %shr, 15728880
+  %or3 = or i32 %or, %and2
+  ret i32 %or3
+}
+
+define i32 @_shfl8(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    zip8 a0, a0
+;
+; RV32IB-LABEL: _shfl8:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    zip8 a0, a0
+; RV32IB-NEXT:    ret
+  %and = and i32 %a, -16776961
+  %shl = shl i32 %a, 8
+  %and1 = and i32 %shl, 16711680
+  %or = or i32 %and1, %and
+  %shr = lshr i32 %a, 8
+  %and2 = and i32 %shr, 65280
+  %or3 = or i32 %or, %and2
+  ret i32 %or3
+}

llvm/test/CodeGen/RISCV/rv32Zbs.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32I
+; RUN: llc -mtriple=riscv32 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbs -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+
+define i32 @sbset(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    sbset a0, a0, a1
+;
+; RV32IB-LABEL: sbset:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sbset a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shl = shl i32 1, %b
+  %or = or i32 %shl, %a
+  ret i32 %or
+}
+
+define i32 @sbclr(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    sbclr a0, a0, a1
+;
+; RV32IB-LABEL: sbclr:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sbclr a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shl = shl i32 1, %b
+  %neg = xor i32 %shl, -1
+  %and = and i32 %neg, %a
+  ret i32 %and
+}
+
+define dso_local i32 @sbinv(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    sbinv a0, a0, a1
+;
+; RV32IB-LABEL: sbinv:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sbinv a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shl = shl i32 1, %b
+  %xor = xor i32 %shl, %a
+  ret i32 %xor
+}
+
+define i32 @sbext(i32 %a, i32 %b) nounwind {
+; RV32I-NOT:    sbext a0, a0, a1
+;
+; RV32IB-LABEL: sbext:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sbext a0, a0, a1
+; RV32IB-NEXT:    ret
+  %shr = lshr i32 %a, %b
+  %and = and i32 %shr, 1
+  ret i32 %and
+}

llvm/test/CodeGen/RISCV/rv32Zbt.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32I
+; RUN: llc -mtriple=riscv32 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+; RUN: llc -mtriple=riscv32 -mattr=experimental-zbt -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IB
+
+define i32 @_cmix(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32I-NOT:    cmix a0, a1, a0, a2
+;
+; RV32IB-LABEL: _cmix:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    cmix a0, a1, a0, a2
+; RV32IB-NEXT:    ret
+  %and = and i32 %b, %a
+  %neg = xor i32 %b, -1
+  %and1 = and i32 %neg, %c
+  %or = or i32 %and1, %and
+  ret i32 %or
+}
+
+define i32 @_cmov(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32I-NOT:    cmov a0, a1, a0, a2
+;
+; RV32IB-LABEL: _cmov:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    cmov a0, a1, a0, a2
+; RV32IB-NEXT:    ret
+  %tobool = icmp eq i32 %b, 0
+  %cond = select i1 %tobool, i32 %c, i32 %a
+  ret i32 %cond
+}
+
+declare i32 @llvm.fshl.i32(i32, i32, i32)
+
+define i32 @_fshl(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32I-NOT:    fsl a0, a0, a2, a1
+;
+; RV32IB-LABEL: _fshl:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    fsl a0, a0, a2, a1
+; RV32IB-NEXT:    ret
+  %1 = tail call i32 @llvm.fshl.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}
+
+declare i32 @llvm.fshr.i32(i32, i32, i32)
+
+define i32 @_fshr(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32I-NOT:    fsr a0, a0, a2, a1
+;
+; RV32IB-LABEL: _fshr:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    fsr a0, a0, a2, a1
+; RV32IB-NEXT:    ret
+  %1 = tail call i32 @llvm.fshr.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}

llvm/test/CodeGen/RISCV/rv64Zbb.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64I
+; RUN: llc -mtriple=riscv64 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbb -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+
+declare i64 @llvm.ctlz.i64(i64, i1)
+
+define i64 @_ctlz_i64(i64 %a) nounwind {
+; RV64I-NOT:    clz a0, a0
+;
+; RV64IB-LABEL: _ctlz_i64:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    beqz a0, .LBB0_2
+; RV64IB-NEXT:  # %bb.1: # %cond.false
+; RV64IB-NEXT:    clz a0, a0
+; RV64IB-NEXT:    ret
+; RV64IB-NEXT:  .LBB0_2:
+; RV64IB-NEXT:    addi a0, zero, 64
+; RV64IB-NEXT:    ret
+  %1 = call i64 @llvm.ctlz.i64(i64 %a, i1 false)
+  ret i64 %1
+}
+
+declare i64 @llvm.cttz.i64(i64, i1)
+
+define i64 @_cttz_i64(i64 %a) nounwind {
+; RV64I-NOT:    ctz a0, a0
+;
+; RV64IB-LABEL: _cttz_i64:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    beqz a0, .LBB1_2
+; RV64IB-NEXT:  # %bb.1: # %cond.false
+; RV64IB-NEXT:    ctz a0, a0
+; RV64IB-NEXT:    ret
+; RV64IB-NEXT:  .LBB1_2:
+; RV64IB-NEXT:    addi a0, zero, 64
+; RV64IB-NEXT:    ret
+  %1 = call i64 @llvm.cttz.i64(i64 %a, i1 false)
+  ret i64 %1
+}
+
+declare i64 @llvm.ctpop.i64(i64)
+
+define i64 @_ctpop_i64(i64 %a) nounwind {
+; RV64I-NOT:    pcnt a0, a0
+;
+; RV64IB-LABEL: _ctpop_i64:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    pcnt a0, a0
+; RV64IB-NEXT:    ret
+  %1 = call i64 @llvm.ctpop.i64(i64 %a)
+  ret i64 %1
+}
+
+define i64 @_slo(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    slo a0, a0, a1
+;
+; RV64IB-LABEL: _slo:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    slo a0, a0, a1
+; RV64IB-NEXT:    ret
+  %neg = xor i64 %a, -1
+  %shl = shl i64 %neg, %b
+  %neg1 = xor i64 %shl, -1
+  ret i64 %neg1
+}
+
+define i64 @_sro(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    sro a0, a0, a1
+;
+; RV64IB-LABEL: _sro:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sro a0, a0, a1
+; RV64IB-NEXT:    ret
+  %neg = xor i64 %a, -1
+  %shr = lshr i64 %neg, %b
+  %neg1 = xor i64 %shr, -1
+  ret i64 %neg1
+}
+
+define i64 @_min(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    min a0, a0, a1
+;
+; RV64IB-LABEL: _min:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    min a0, a0, a1
+; RV64IB-NEXT:    ret
+  %cmp = icmp slt i64 %a, %b
+  %cond = select i1 %cmp, i64 %a, i64 %b
+  ret i64 %cond
+}
+
+define i64 @_max(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    max a0, a0, a1
+;
+; RV64IB-LABEL: _max:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    max a0, a0, a1
+; RV64IB-NEXT:    ret
+  %cmp = icmp sgt i64 %a, %b
+  %cond = select i1 %cmp, i64 %a, i64 %b
+  ret i64 %cond
+}
+
+define i64 @_minu(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    minu a0, a0, a1
+;
+; RV64IB-LABEL: _minu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    minu a0, a0, a1
+; RV64IB-NEXT:    ret
+  %cmp = icmp ult i64 %a, %b
+  %cond = select i1 %cmp, i64 %a, i64 %b
+  ret i64 %cond
+}
+
+define i64 @_maxu(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    maxu a0, a0, a1
+;
+; RV64IB-LABEL: _maxu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    maxu a0, a0, a1
+; RV64IB-NEXT:    ret
+  %cmp = icmp ugt i64 %a, %b
+  %cond = select i1 %cmp, i64 %a, i64 %b
+  ret i64 %cond
+}
+
+define i64 @_addwu(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    addwu a0, a1, a0
+;
+; RV64IB-LABEL: _addwu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    addwu a0, a1, a0
+; RV64IB-NEXT:    ret
+  %add = add i64 %b, %a
+  %conv1 = and i64 %add, 4294967295
+  ret i64 %conv1
+}
+
+define i64 @_subwu(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    subwu a0, a0, a1
+;
+; RV64IB-LABEL: _subwu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    subwu a0, a0, a1
+; RV64IB-NEXT:    ret
+  %sub = sub i64 %a, %b
+  %conv1 = and i64 %sub, 4294967295
+  ret i64 %conv1
+}
+
+define i64 @_addiwu(i64 %a) nounwind {
+; RV64I-NOT:    addiwu a0, a0, 1
+;
+; RV64IB-LABEL: _addiwu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    addiwu a0, a0, 1
+; RV64IB-NEXT:    ret
+  %conv = add i64 %a, 1
+  %conv1 = and i64 %conv, 4294967295
+  ret i64 %conv1
+}
+
+define i64 @_adduw(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    addu.w a0, a0, a1
+;
+; RV64IB-LABEL: _adduw:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    addu.w a0, a0, a1
+; RV64IB-NEXT:    ret
+  %and = and i64 %b, 4294967295
+  %add = add i64 %and, %a
+  ret i64 %add
+}
+
+define i64 @_subuw(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    subu.w a0, a0, a1
+;
+; RV64IB-LABEL: _subuw:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    subu.w a0, a0, a1
+; RV64IB-NEXT:    ret
+  %and = and i64 %b, 4294967295
+  %sub = sub i64 %a, %and
+  ret i64 %sub
+}
+
+define i64 @_sextb(i64 %a) nounwind {
+; RV64I-NOT:    sext.b a0, a0
+;
+; RV64IB-LABEL: _sextb:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sext.b a0, a0
+; RV64IB-NEXT:    ret
+  %shl = shl i64 %a, 56
+  %shr = ashr exact i64 %shl, 56
+  ret i64 %shr
+}
+
+define i64 @_sexth(i64 %a) nounwind {
+; RV64I-NOT:    sext.h a0, a0
+;
+; RV64IB-LABEL: _sexth:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sext.h a0, a0
+; RV64IB-NEXT:    ret
+  %shl = shl i64 %a, 48
+  %shr = ashr exact i64 %shl, 48
+  ret i64 %shr
+}

llvm/test/CodeGen/RISCV/rv64Zbbp.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64I
+; RUN: llc -mtriple=riscv64 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbb -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbp -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+
+define i64 @_pack(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    pack a0, a0, a1
+;
+; RV64IB-LABEL: _pack:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    pack a0, a0, a1
+; RV64IB-NEXT:    ret
+  %shl = and i64 %a, 4294967295
+  %shl1 = shl i64 %b, 32
+  %or = or i64 %shl1, %shl
+  ret i64 %or
+}
+
+define i64 @_packu(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    packu a0, a0, a1
+;
+; RV64IB-LABEL: _packu:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    packu a0, a0, a1
+; RV64IB-NEXT:    ret
+  %shr = lshr i64 %a, 32
+  %shr1 = and i64 %b, -4294967296
+  %or = or i64 %shr1, %shr
+  ret i64 %or
+}
+
+define i64 @_packh(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    packh a0, a0, a1
+;
+; RV64IB-LABEL: _packh:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    packh a0, a0, a1
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, 255
+  %and1 = shl i64 %b, 8
+  %shl = and i64 %and1, 65280
+  %or = or i64 %shl, %and
+  ret i64 %or
+}
+
+define i64 @_andn(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    andn a0, a0, a1
+;
+; RV64IB-LABEL: _andn:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    andn a0, a0, a1
+; RV64IB-NEXT:    ret
+  %neg = xor i64 %b, -1
+  %and = and i64 %neg, %a
+  ret i64 %and
+}
+
+define i64 @_orn(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    orn a0, a0, a1
+;
+; RV64IB-LABEL: _orn:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orn a0, a0, a1
+; RV64IB-NEXT:    ret
+  %neg = xor i64 %b, -1
+  %or = or i64 %neg, %a
+  ret i64 %or
+}
+
+define i64 @_xnor(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    xnor a0, a0, a1
+;
+; RV64IB-LABEL: _xnor:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    xnor a0, a0, a1
+; RV64IB-NEXT:    ret
+  %neg = xor i64 %a, -1
+  %xor = xor i64 %neg, %b
+  ret i64 %xor
+}
+
+declare i64 @llvm.fshl.i64(i64, i64, i64)
+
+define i64 @_rol(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    rol a0, a0, a1
+;
+; RV64IB-LABEL: _rol:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rol a0, a0, a1
+; RV64IB-NEXT:    ret
+  %or = tail call i64 @llvm.fshl.i64(i64 %a, i64 %a, i64 %b)
+  ret i64 %or
+}
+
+declare i64 @llvm.fshr.i64(i64, i64, i64)
+
+define i64 @_ror(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    ror a0, a0, a1
+;
+; RV64IB-LABEL: _ror:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    ror a0, a0, a1
+; RV64IB-NEXT:    ret
+  %or = tail call i64 @llvm.fshr.i64(i64 %a, i64 %a, i64 %b)
+  ret i64 %or
+}

llvm/test/CodeGen/RISCV/rv64Zbp.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64I
+; RUN: llc -mtriple=riscv64 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbp -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+
+define i64 @_grev1(i64 %a) nounwind {
+; RV64I-NOT:    rev.p a0, a0
+;
+; RV64IB-LABEL: _grev1:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev.p a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 1
+  %shl = and i64 %and, -6148914691236517206
+  %and1 = lshr i64 %a, 1
+  %shr = and i64 %and1, 6148914691236517205
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+define i64 @_grev2(i64 %a) nounwind {
+; RV64I-NOT:    rev2.n a0, a0
+;
+; RV64IB-LABEL: _grev2:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev2.n a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 2
+  %shl = and i64 %and, -3689348814741910324
+  %and1 = lshr i64 %a, 2
+  %shr = and i64 %and1, 3689348814741910323
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+define i64 @_grev4(i64 %a) nounwind {
+; RV64I-NOT:    rev4.b a0, a0
+;
+; RV64IB-LABEL: _grev4:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev4.b a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 4
+  %shl = and i64 %and, -1085102592571150096
+  %and1 = lshr i64 %a, 4
+  %shr = and i64 %and1, 1085102592571150095
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+define i64 @_grev8(i64 %a) nounwind {
+; RV64I-NOT:    rev8.h a0, a0
+;
+; RV64IB-LABEL: _grev8:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev8.h a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 8
+  %shl = and i64 %and, -71777214294589696
+  %and1 = lshr i64 %a, 8
+  %shr = and i64 %and1, 71777214294589695
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+define i64 @_grev16(i64 %a) nounwind {
+; RV64I-NOT:    rev16.w a0, a0
+;
+; RV64IB-LABEL: _grev16:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev16.w a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 16
+  %shl = and i64 %and, -281470681808896
+  %and1 = lshr i64 %a, 16
+  %shr = and i64 %and1, 281470681808895
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+define i64 @_grev32(i64 %a) nounwind {
+; RV64I-NOT:    rev32 a0, a0
+;
+; RV64IB-LABEL: _grev32:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev32 a0, a0
+; RV64IB-NEXT:    ret
+  %shl = shl i64 %a, 32
+  %shr = lshr i64 %a, 32
+  %or = or i64 %shl, %shr
+  ret i64 %or
+}
+
+declare i64 @llvm.bswap.i64(i64)
+
+define i64 @_bswap_i64(i64 %a) {
+; RV64I-NOT:    rev8 a0, a0
+;
+; RV64IB-LABEL: _bswap_i64:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev8 a0, a0
+; RV64IB-NEXT:    ret
+  %1 = call i64 @llvm.bswap.i64(i64 %a)
+  ret i64 %1
+}
+
+declare i64 @llvm.bitreverse.i64(i64)
+
+define i64 @_bitreverse_i64(i64 %a) nounwind {
+; RV64IB-NOT:    rev a0, a0
+;
+; RV64IB-LABEL: _bitreverse_i64:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    rev a0, a0
+; RV64IB-NEXT:    ret
+  %1 = call i64 @llvm.bitreverse.i64(i64 %a)
+  ret i64 %1
+}
+
+define i64 @_gorc1(i64 %a) nounwind {
+; RV64I-NOT:    orc.p a0, a0
+;
+; RV64IB-LABEL: _gorc1:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc.p a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 1
+  %shl = and i64 %and, -6148914691236517206
+  %and1 = lshr i64 %a, 1
+  %shr = and i64 %and1, 6148914691236517205
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_gorc2(i64 %a) nounwind {
+; RV64I-NOT:    orc2.n a0, a0
+;
+; RV64IB-LABEL: _gorc2:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc2.n a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 2
+  %shl = and i64 %and, -3689348814741910324
+  %and1 = lshr i64 %a, 2
+  %shr = and i64 %and1, 3689348814741910323
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_gorc4(i64 %a) nounwind {
+; RV64I-NOT:    orc4.b a0, a0
+;
+; RV64IB-LABEL: _gorc4:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc4.b a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 4
+  %shl = and i64 %and, -1085102592571150096
+  %and1 = lshr i64 %a, 4
+  %shr = and i64 %and1, 1085102592571150095
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_gorc8(i64 %a) nounwind {
+; RV64I-NOT:    orc8.h a0, a0
+;
+; RV64IB-LABEL: _gorc8:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc8.h a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 8
+  %shl = and i64 %and, -71777214294589696
+  %and1 = lshr i64 %a, 8
+  %shr = and i64 %and1, 71777214294589695
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_gorc16(i64 %a) nounwind {
+; RV64I-NOT:    orc16.w a0, a0
+;
+; RV64IB-LABEL: _gorc16:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc16.w a0, a0
+; RV64IB-NEXT:    ret
+  %and = shl i64 %a, 16
+  %shl = and i64 %and, -281470681808896
+  %and1 = lshr i64 %a, 16
+  %shr = and i64 %and1, 281470681808895
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_gorc32(i64 %a) nounwind {
+; RV64I-NOT:    orc32 a0, a0
+;
+; RV64IB-LABEL: _gorc32:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    orc32 a0, a0
+; RV64IB-NEXT:    ret
+  %shl = shl i64 %a, 32
+  %shr = lshr i64 %a, 32
+  %or = or i64 %shr, %a
+  %or2 = or i64 %or, %shl
+  ret i64 %or2
+}
+
+define i64 @_shfl1(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    zip.n a0, a0
+;
+; RV64IB-LABEL: _shfl1:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    zip.n a0, a0
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, -7378697629483820647
+  %shl = shl i64 %a, 1
+  %and1 = and i64 %shl, 4919131752989213764
+  %or = or i64 %and1, %and
+  %shr = lshr i64 %a, 1
+  %and2 = and i64 %shr, 2459565876494606882
+  %or3 = or i64 %or, %and2
+  ret i64 %or3
+}
+
+define i64 @_shfl2(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    zip2.b a0, a0
+;
+; RV64IB-LABEL: _shfl2:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    zip2.b a0, a0
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, -4340410370284600381
+  %shl = shl i64 %a, 2
+  %and1 = and i64 %shl, 3472328296227680304
+  %or = or i64 %and1, %and
+  %shr = lshr i64 %a, 2
+  %and2 = and i64 %shr, 868082074056920076
+  %or3 = or i64 %or, %and2
+  ret i64 %or3
+}
+
+define i64 @_shfl4(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    zip4.h a0, a0
+;
+; RV64IB-LABEL: _shfl4:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    zip4.h a0, a0
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, -1148435428713435121
+  %shl = shl i64 %a, 4
+  %and1 = and i64 %shl, 1080880403494997760
+  %or = or i64 %and1, %and
+  %shr = lshr i64 %a, 4
+  %and2 = and i64 %shr, 67555025218437360
+  %or3 = or i64 %or, %and2
+  ret i64 %or3
+}
+
+define i64 @_shfl8(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    zip8.w a0, a0
+;
+; RV64IB-LABEL: _shfl8:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    zip8.w a0, a0
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, -72056494543077121
+  %shl = shl i64 %a, 8
+  %and1 = and i64 %shl, 71776119077928960
+  %or = or i64 %and1, %and
+  %shr = lshr i64 %a, 8
+  %and2 = and i64 %shr, 280375465148160
+  %or3 = or i64 %or, %and2
+  ret i64 %or3
+}
+
+define i64 @_shfl16(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    zip16 a0, a0
+;
+; RV64IB-LABEL: _shfl16:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    zip16 a0, a0
+; RV64IB-NEXT:    ret
+  %and = and i64 %a, -281474976645121
+  %shl = shl i64 %a, 16
+  %and1 = and i64 %shl, 281470681743360
+  %or = or i64 %and1, %and
+  %shr = lshr i64 %a, 16
+  %and2 = and i64 %shr, 4294901760
+  %or3 = or i64 %or, %and2
+  ret i64 %or3
+}

llvm/test/CodeGen/RISCV/rv64Zbs.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64I
+; RUN: llc -mtriple=riscv64 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbs -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+
+define i64 @_sbset(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    sbset a0, a0, a1
+;
+; RV64IB-LABEL: _sbset:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sbset a0, a0, a1
+; RV64IB-NEXT:    ret
+  %sh_prom = trunc i64 %b to i32
+  %shl = shl i32 1, %sh_prom
+  %conv = sext i32 %shl to i64
+  %or = or i64 %conv, %a
+  ret i64 %or
+}
+
+define i64 @_sbclr(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    sbclr a0, a0, a1
+;
+; RV64IB-LABEL: _sbclr:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sbclr a0, a0, a1
+; RV64IB-NEXT:    ret
+  %sh_prom = trunc i64 %b to i32
+  %shl = shl i32 1, %sh_prom
+  %neg = xor i32 %shl, -1
+  %conv = sext i32 %neg to i64
+  %and = and i64 %conv, %a
+  ret i64 %and
+}
+
+define i64 @sbinv(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    sbinv a0, a0, a1
+;
+; RV64IB-LABEL: sbinv:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sbinv a0, a0, a1
+; RV64IB-NEXT:    ret
+  %sh_prom = trunc i64 %b to i32
+  %shl = shl i32 1, %sh_prom
+  %conv = sext i32 %shl to i64
+  %xor = xor i64 %conv, %a
+  ret i64 %xor
+}
+
+define i64 @sbext(i64 %a, i64 %b) nounwind {
+; RV64I-NOT:    sbext a0, a0, a1
+;
+; RV64IB-LABEL: sbext:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    sbext a0, a0, a1
+; RV64IB-NEXT:    ret
+  %shr = lshr i64 %a, %b
+  %and = and i64 %shr, 1
+  ret i64 %and
+}

llvm/test/CodeGen/RISCV/rv64Zbt.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64I
+; RUN: llc -mtriple=riscv64 -mattr=experimental-b -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+; RUN: llc -mtriple=riscv64 -mattr=experimental-zbt -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IB
+
+define i64 @_cmix(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64I-NOT:    cmix a0, a1, a0, a2
+;
+; RV64IB-LABEL: _cmix:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    cmix a0, a1, a0, a2
+; RV64IB-NEXT:    ret
+  %and = and i64 %b, %a
+  %neg = xor i64 %b, -1
+  %and1 = and i64 %neg, %c
+  %or = or i64 %and1, %and
+  ret i64 %or
+}
+
+define i64 @_cmov(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64I-NOT:    cmov a0, a1, a0, a2
+;
+; RV64IB-LABEL: _cmov:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    cmov a0, a1, a0, a2
+; RV64IB-NEXT:    ret
+  %tobool = icmp eq i64 %b, 0
+  %cond = select i1 %tobool, i64 %c, i64 %a
+  ret i64 %cond
+}
+
+declare i64 @llvm.fshl.i64(i64, i64, i64)
+
+define i64 @_fshl(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64I-NOT:    fsl a0, a0, a2, a1
+;
+; RV64IB-LABEL: _fshl:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    fsl a0, a0, a2, a1
+; RV64IB-NEXT:    ret
+  %1 = tail call i64 @llvm.fshl.i64(i64 %a, i64 %b, i64 %c)
+  ret i64 %1
+}
+
+declare i64 @llvm.fshr.i64(i64, i64, i64)
+
+define i64 @_fshr(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64I-NOT:    fsr a0, a0, a2, a1
+;
+; RV64IB-LABEL: _fshr:
+; RV64IB:       # %bb.0:
+; RV64IB-NEXT:    fsr a0, a0, a2, a1
+; RV64IB-NEXT:    ret
+  %1 = tail call i64 @llvm.fshr.i64(i64 %a, i64 %b, i64 %c)
+  ret i64 %1
+}