[RISCV] Add intrinsic for Zbt extension

[Chenbing.Zheng]

Summary

RV32: fsl, fsr, fsri

RV64: fsl, fsr, fsri, fslw, fsrw, fsriw

[craig.topper]

clang/include/clang/Basic/BuiltinsRISCV.def

65

cmov and cmix don't builtins don't seem very useful. The compiler can already match them from IR operations that can be expressed in C.

clang/lib/CodeGen/CGBuiltin.cpp

18967

We shouldn't have an fsri builtin. The fsr builtin is sufficient. And riscv_fsri looks unhandled in the backend.

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

907 ↗

(On Diff #400496)

Need to also handle fsl with a constant argument.

[Chenbing.Zheng]

del cmov，cmix，fsri in clang

[Chenbing.Zheng]

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

907 ↗

(On Diff #400496)

Emm.... but there is no fsli in zbt.

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

907 ↗

(On Diff #400496)

Yes because the compiler can change the constant to turn fsli into fsri. We do this on line 903.

If the user uses fsl with a constant we should give them good code by turning it into fsri.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4262

The operand order for RISCVISD::FSL/FSR match llvm.fshl and llvm.fshr rather than rs1, rs2, rs3 order. I think the operand order you want for riscv.fsl and riscv.fsr should be rs1, rs2, rs3.

And the assembly printing for fsl/fsr prints $rd, $rs1, $rs3, $rs2 makes this even more confusing.

I'll put up a patch to change RISCVISD::FSL/FSR and RISCVISD::FSLW/FSRW order to be in rs1, rs2, rs3 order.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4262

i just noticed the proposed intrinsic order here https://github.com/riscv/riscv-bitmanip/blob/main-history/cproofs/rvintrin.h uses rs1, rs3, rs2/shamt ordering. Which is different than the order used in the spec pseudo code. But rs1, rs3, rs2/shamt matches how the instructions are printed.

So I guess we should use rs1, rs3, rs2/shamt order. I'll update the RISCVISD opcodes accordingly.

[craig.topper]

Out of curiosity, what is your interest in Zbt? Do you work for a company that is implementing this extension in hardware?

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4262

I pushed a patch to change the operand order of RISCVISD::FSL/FSR

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

900 ↗

(On Diff #400768)

I made this change in the repo and handled the riscv_fsl with constant case.

[Chenbing.Zheng]

rebase main

[Chenbing.Zheng]

In D117468#3253493, @craig.topper wrote:

Out of curiosity, what is your interest in Zbt? Do you work for a company that is implementing this extension in hardware?

My company has not implementing this extension，but may be in the future. I'm just doing a pre-research.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

4262

I think change swap op2 and op3 in Pat, so shall i keep the order op1, op2, op3 here?

def : Pat<(riscv_fsl GPR:$rs1, GPR:$rs3, GPR:$rs2),

(FSL GPR:$rs1, GPR:$rs2, GPR:$rs3)>;

def : Pat<(riscv_fsr GPR:$rs1, GPR:$rs3, GPR:$rs2),

(FSR GPR:$rs1, GPR:$rs2, GPR:$rs3)>;

[craig.topper]

llvm/test/CodeGen/RISCV/rv32zbt-intrinsic.ll

22

This should have the same register order as the fsl test case. I think the test just needs to be re-generated.

28

Please add an fsli test case using llvm.riscv.fsl.i32 and a constant shift amount

31

This should be fsri a0, a0, a1, 5

llvm/test/CodeGen/RISCV/rv64zbt-intrinsic.ll

28

Please add an fsli test case.

59

Please add a fsri test case.

[craig.topper]

In D117468#3253879, @Chenbing.Zheng wrote:

In D117468#3253493, @craig.topper wrote:

Out of curiosity, what is your interest in Zbt? Do you work for a company that is implementing this extension in hardware?

My company has not implementing this extension，but may be in the future. I'm just doing a pre-research.

The Zbt extensions and the other un-ratified extension from the old Bitmanip spec have not seen any active discussion for at least the last 6-9 months. If that continues, it starts to raise questions about whether llvm should continue to support and extension that does not appear to be on a path to ratification. I've asked for a discussion on this topic at the next RISCV LLVM bi-weekly sync meeting tomorrow morning US time. calendar link here https://llvm.org/docs/GettingInvolved.html#online-sync-ups

[Chenbing.Zheng]

modify tests

[Chenbing.Zheng]

In D117468#3255546, @craig.topper wrote:

In D117468#3253879, @Chenbing.Zheng wrote:

In D117468#3253493, @craig.topper wrote:

Out of curiosity, what is your interest in Zbt? Do you work for a company that is implementing this extension in hardware?

My company has not implementing this extension，but may be in the future. I'm just doing a pre-research.

The Zbt extensions and the other un-ratified extension from the old Bitmanip spec have not seen any active discussion for at least the last 6-9 months. If that continues, it starts to raise questions about whether llvm should continue to support and extension that does not appear to be on a path to ratification. I've asked for a discussion on this topic at the next RISCV LLVM bi-weekly sync meeting tomorrow morning US time. calendar link here https://llvm.org/docs/GettingInvolved.html#online-sync-ups

Thanks a lot, I will keep watching it.

[craig.topper]

LGTM

clang/include/clang/Basic/BuiltinsRISCV.def

 TARGET_BUILTIN(__builtin_riscv_crc32_h, "LiLi", "nc", "experimental-zbr")
 TARGET_BUILTIN(__builtin_riscv_crc32_w, "LiLi", "nc", "experimental-zbr")
 TARGET_BUILTIN(__builtin_riscv_crc32c_b, "LiLi", "nc", "experimental-zbr")
 TARGET_BUILTIN(__builtin_riscv_crc32c_h, "LiLi", "nc", "experimental-zbr")
 TARGET_BUILTIN(__builtin_riscv_crc32c_w, "LiLi", "nc", "experimental-zbr")
 TARGET_BUILTIN(__builtin_riscv_crc32_d, "LiLi", "nc", "experimental-zbr,64bit")
 TARGET_BUILTIN(__builtin_riscv_crc32c_d, "LiLi", "nc", "experimental-zbr,64bit")
 
+// Zbt extension
+TARGET_BUILTIN(__builtin_riscv_fsl_32, "LiLiLiLi", "nc", "experimental-zbt")
+TARGET_BUILTIN(__builtin_riscv_fsr_32, "LiLiLiLi", "nc", "experimental-zbt")
+TARGET_BUILTIN(__builtin_riscv_fsl_64, "WiWiWiWi", "nc", "experimental-zbt,64bit")
+TARGET_BUILTIN(__builtin_riscv_fsr_64, "WiWiWiWi", "nc", "experimental-zbt,64bit")
+
 #undef BUILTIN
 #undef TARGET_BUILTIN

clang/lib/CodeGen/CGBuiltin.cpp

Value *CodeGenFunction::EmitRISCVBuiltinExpr(unsigned BuiltinID,
   case RISCV::BI__builtin_riscv_xperm_w:
   case RISCV::BI__builtin_riscv_crc32_b:
   case RISCV::BI__builtin_riscv_crc32_h:
   case RISCV::BI__builtin_riscv_crc32_w:
   case RISCV::BI__builtin_riscv_crc32_d:
   case RISCV::BI__builtin_riscv_crc32c_b:
   case RISCV::BI__builtin_riscv_crc32c_h:
   case RISCV::BI__builtin_riscv_crc32c_w:
-  case RISCV::BI__builtin_riscv_crc32c_d: {
+  case RISCV::BI__builtin_riscv_crc32c_d:
+  case RISCV::BI__builtin_riscv_fsl_32:
+  case RISCV::BI__builtin_riscv_fsr_32:
+  case RISCV::BI__builtin_riscv_fsl_64:
+  case RISCV::BI__builtin_riscv_fsr_64: {
     switch (BuiltinID) {
     default: llvm_unreachable("unexpected builtin ID");
     // Zbb
     case RISCV::BI__builtin_riscv_orc_b_32:
     case RISCV::BI__builtin_riscv_orc_b_64:
       ID = Intrinsic::riscv_orc_b;
       break;
 
case RISCV::BI__builtin_riscv_crc32c_h:
       ID = Intrinsic::riscv_crc32c_h;
       break;
     case RISCV::BI__builtin_riscv_crc32c_w:
       ID = Intrinsic::riscv_crc32c_w;
       break;
     case RISCV::BI__builtin_riscv_crc32c_d:
       ID = Intrinsic::riscv_crc32c_d;
       break;
+
+    // Zbt
+    case RISCV::BI__builtin_riscv_fsl_32:
+    case RISCV::BI__builtin_riscv_fsl_64:
+      ID = Intrinsic::riscv_fsl;
+      break;
+    case RISCV::BI__builtin_riscv_fsr_32:
+    case RISCV::BI__builtin_riscv_fsr_64:
+      ID = Intrinsic::riscv_fsr;
+      break;
     }
 
     IntrinsicTypes = {ResultType};
     break;
   }
   // Vector builtins are handled from here.
 #include "clang/Basic/riscv_vector_builtin_cg.inc"
   }
 
   assert(ID != Intrinsic::not_intrinsic);
 
   llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
   return Builder.CreateCall(F, Ops, "");
 }

clang/test/CodeGen/RISCV/rvb-intrinsics/riscv32-zbt.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple riscv32 -target-feature +experimental-zbt -emit-llvm %s -o - \
+// RUN:     | FileCheck %s  -check-prefix=RV32ZBT
+
+// RV32ZBT-LABEL: @fsl(
+// RV32ZBT-NEXT:  entry:
+// RV32ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    store i32 [[RS3:%.*]], i32* [[RS3_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP2:%.*]] = load i32, i32* [[RS3_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP3:%.*]] = call i32 @llvm.riscv.fsl.i32(i32 [[TMP0]], i32 [[TMP1]], i32 [[TMP2]])
+// RV32ZBT-NEXT:    ret i32 [[TMP3]]
+//
+int fsl(int rs1, int rs2, int rs3) {
+  return __builtin_riscv_fsl_32(rs1, rs2, rs3);
+}
+
+// RV32ZBT-LABEL: @fsr(
+// RV32ZBT-NEXT:  entry:
+// RV32ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    store i32 [[RS3:%.*]], i32* [[RS3_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP2:%.*]] = load i32, i32* [[RS3_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP3:%.*]] = call i32 @llvm.riscv.fsr.i32(i32 [[TMP0]], i32 [[TMP1]], i32 [[TMP2]])
+// RV32ZBT-NEXT:    ret i32 [[TMP3]]
+//
+int fsr(int rs1, int rs2, int rs3) {
+  return __builtin_riscv_fsr_32(rs1, rs2, rs3);
+}
+
+// RV32ZBT-LABEL: @fsri(
+// RV32ZBT-NEXT:  entry:
+// RV32ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV32ZBT-NEXT:    [[TMP2:%.*]] = call i32 @llvm.riscv.fsr.i32(i32 [[TMP0]], i32 [[TMP1]], i32 15)
+// RV32ZBT-NEXT:    ret i32 [[TMP2]]
+//
+int fsri(int rs1, int rs2) {
+  return __builtin_riscv_fsr_32(rs1, rs2, 15);
+}

clang/test/CodeGen/RISCV/rvb-intrinsics/riscv64-zbt.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple riscv64 -target-feature +experimental-zbt -emit-llvm %s -o - \
+// RUN:     | FileCheck %s  -check-prefix=RV64ZBT
+
+// RV64ZBT-LABEL: @fsl(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    store i32 [[RS3:%.*]], i32* [[RS3_ADDR]], align 4
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV:%.*]] = sext i32 [[TMP0]] to i64
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV1:%.*]] = sext i32 [[TMP1]] to i64
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = load i32, i32* [[RS3_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV2:%.*]] = sext i32 [[TMP2]] to i64
+// RV64ZBT-NEXT:    [[TMP3:%.*]] = call i64 @llvm.riscv.fsl.i64(i64 [[CONV]], i64 [[CONV1]], i64 [[CONV2]])
+// RV64ZBT-NEXT:    [[CONV3:%.*]] = trunc i64 [[TMP3]] to i32
+// RV64ZBT-NEXT:    ret i32 [[CONV3]]
+//
+int fsl(int rs1, int rs2, int rs3) {
+  return __builtin_riscv_fsl_32(rs1, rs2, rs3);
+}
+
+// RV64ZBT-LABEL: @fsr(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    store i32 [[RS3:%.*]], i32* [[RS3_ADDR]], align 4
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV:%.*]] = sext i32 [[TMP0]] to i64
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV1:%.*]] = sext i32 [[TMP1]] to i64
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = load i32, i32* [[RS3_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV2:%.*]] = sext i32 [[TMP2]] to i64
+// RV64ZBT-NEXT:    [[TMP3:%.*]] = call i64 @llvm.riscv.fsr.i64(i64 [[CONV]], i64 [[CONV1]], i64 [[CONV2]])
+// RV64ZBT-NEXT:    [[CONV3:%.*]] = trunc i64 [[TMP3]] to i32
+// RV64ZBT-NEXT:    ret i32 [[CONV3]]
+//
+int fsr(int rs1, int rs2, int rs3) {
+  return __builtin_riscv_fsr_32(rs1, rs2, rs3);
+}
+
+// RV64ZBT-LABEL: @fsri(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i32, align 4
+// RV64ZBT-NEXT:    store i32 [[RS1:%.*]], i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    store i32 [[RS2:%.*]], i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i32, i32* [[RS1_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV:%.*]] = sext i32 [[TMP0]] to i64
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i32, i32* [[RS2_ADDR]], align 4
+// RV64ZBT-NEXT:    [[CONV1:%.*]] = sext i32 [[TMP1]] to i64
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = call i64 @llvm.riscv.fsr.i64(i64 [[CONV]], i64 [[CONV1]], i64 15)
+// RV64ZBT-NEXT:    [[CONV2:%.*]] = trunc i64 [[TMP2]] to i32
+// RV64ZBT-NEXT:    ret i32 [[CONV2]]
+//
+int fsri(int rs1, int rs2) {
+  return __builtin_riscv_fsr_32(rs1, rs2, 15);
+}
+
+// RV64ZBT-LABEL: @fslw(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    store i64 [[RS1:%.*]], i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    store i64 [[RS2:%.*]], i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    store i64 [[RS3:%.*]], i64* [[RS3_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i64, i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i64, i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = load i64, i64* [[RS3_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP3:%.*]] = call i64 @llvm.riscv.fsl.i64(i64 [[TMP0]], i64 [[TMP1]], i64 [[TMP2]])
+// RV64ZBT-NEXT:    ret i64 [[TMP3]]
+//
+long fslw(long rs1, long rs2, long rs3) {
+  return __builtin_riscv_fsl_64(rs1, rs2, rs3);
+}
+
+// RV64ZBT-LABEL: @fsrw(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    [[RS3_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    store i64 [[RS1:%.*]], i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    store i64 [[RS2:%.*]], i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    store i64 [[RS3:%.*]], i64* [[RS3_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i64, i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i64, i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = load i64, i64* [[RS3_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP3:%.*]] = call i64 @llvm.riscv.fsr.i64(i64 [[TMP0]], i64 [[TMP1]], i64 [[TMP2]])
+// RV64ZBT-NEXT:    ret i64 [[TMP3]]
+//
+long fsrw(long rs1, long rs2, long rs3) {
+  return __builtin_riscv_fsr_64(rs1, rs2, rs3);
+}
+
+// RV64ZBT-LABEL: @fsriw(
+// RV64ZBT-NEXT:  entry:
+// RV64ZBT-NEXT:    [[RS1_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    [[RS2_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBT-NEXT:    store i64 [[RS1:%.*]], i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    store i64 [[RS2:%.*]], i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP0:%.*]] = load i64, i64* [[RS1_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP1:%.*]] = load i64, i64* [[RS2_ADDR]], align 8
+// RV64ZBT-NEXT:    [[TMP2:%.*]] = call i64 @llvm.riscv.fsr.i64(i64 [[TMP0]], i64 [[TMP1]], i64 15)
+// RV64ZBT-NEXT:    ret i64 [[TMP2]]
+//
+long fsriw(long rs1, long rs2) {
+  return __builtin_riscv_fsr_64(rs1, rs2, 15);
+}

llvm/include/llvm/IR/IntrinsicsRISCV.td

let TargetPrefix = "riscv" in {
   class BitManipGPRIntrinsics
       : Intrinsic<[llvm_any_ty],
                   [LLVMMatchType<0>],
                   [IntrNoMem, IntrSpeculatable, IntrWillReturn]>;
   class BitManipGPRGPRIntrinsics
       : Intrinsic<[llvm_any_ty],
                   [LLVMMatchType<0>, LLVMMatchType<0>],
                   [IntrNoMem, IntrSpeculatable, IntrWillReturn]>;
+  class BitManipGPRGPRGRIntrinsics
+      : Intrinsic<[llvm_any_ty],
+                  [LLVMMatchType<0>, LLVMMatchType<0>, LLVMMatchType<0>],
+                  [IntrNoMem, IntrSpeculatable, IntrWillReturn]>;
 
   // Zbb
   def int_riscv_orc_b : BitManipGPRIntrinsics;
 
   // Zbc
   def int_riscv_clmul  : BitManipGPRGPRIntrinsics;
   def int_riscv_clmulh : BitManipGPRGPRIntrinsics;
   def int_riscv_clmulr : BitManipGPRGPRIntrinsics;
let TargetPrefix = "riscv" in {
   def int_riscv_crc32_b : BitManipGPRIntrinsics;
   def int_riscv_crc32_h : BitManipGPRIntrinsics;
   def int_riscv_crc32_w : BitManipGPRIntrinsics;
   def int_riscv_crc32_d : BitManipGPRIntrinsics;
   def int_riscv_crc32c_b : BitManipGPRIntrinsics;
   def int_riscv_crc32c_h : BitManipGPRIntrinsics;
   def int_riscv_crc32c_w : BitManipGPRIntrinsics;
   def int_riscv_crc32c_d : BitManipGPRIntrinsics;
+
+  // Zbt
+  def int_riscv_fsl : BitManipGPRGPRGRIntrinsics;
+  def int_riscv_fsr : BitManipGPRGPRGRIntrinsics;
 } // TargetPrefix = "riscv"
 
 //===----------------------------------------------------------------------===//
 // Vectors
 
 // The intrinsic does not have any operand that must be extended.
 defvar NoSplatOperand = 0xF;
 

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

SDValue RISCVTargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op,
   case Intrinsic::riscv_bdecompress: {
     unsigned Opc = IntNo == Intrinsic::riscv_bcompress ? RISCVISD::BCOMPRESS
                                                        : RISCVISD::BDECOMPRESS;
     return DAG.getNode(Opc, DL, XLenVT, Op.getOperand(1), Op.getOperand(2));
   }
   case Intrinsic::riscv_bfp:
     return DAG.getNode(RISCVISD::BFP, DL, XLenVT, Op.getOperand(1),
                        Op.getOperand(2));
+  case Intrinsic::riscv_fsl:
+    return DAG.getNode(RISCVISD::FSL, DL, XLenVT, Op.getOperand(1),
+                       Op.getOperand(2), Op.getOperand(3));
+  case Intrinsic::riscv_fsr:
+    return DAG.getNode(RISCVISD::FSR, DL, XLenVT, Op.getOperand(1),
+                       Op.getOperand(2), Op.getOperand(3));
   case Intrinsic::riscv_vmv_x_s:
     assert(Op.getValueType() == XLenVT && "Unexpected VT!");
     return DAG.getNode(RISCVISD::VMV_X_S, DL, Op.getValueType(),
                        Op.getOperand(1));
   case Intrinsic::riscv_vmv_v_x:
     return lowerScalarSplat(Op.getOperand(1), Op.getOperand(2),
                             Op.getSimpleValueType(), DL, DAG, Subtarget);
   case Intrinsic::riscv_vfmv_v_f:
static RISCVISD::NodeType getRISCVWOpcodeByIntr(unsigned IntNo) {
   case Intrinsic::riscv_gorc:
     return RISCVISD::GORCW;
   case Intrinsic::riscv_bcompress:
     return RISCVISD::BCOMPRESSW;
   case Intrinsic::riscv_bdecompress:
     return RISCVISD::BDECOMPRESSW;
   case Intrinsic::riscv_bfp:
     return RISCVISD::BFPW;
+  case Intrinsic::riscv_fsl:
+    return RISCVISD::FSLW;
+  case Intrinsic::riscv_fsr:
+    return RISCVISD::FSRW;
   }
 }
 
 // Converts the given intrinsic to a i64 operation with any extension.
 static SDValue customLegalizeToWOpByIntr(SDNode *N, SelectionDAG &DAG,
                                          unsigned IntNo) {
   SDLoc DL(N);
   RISCVISD::NodeType WOpcode = getRISCVWOpcodeByIntr(IntNo);
case ISD::INTRINSIC_WO_CHAIN: {
     case Intrinsic::riscv_bcompress:
     case Intrinsic::riscv_bdecompress:
     case Intrinsic::riscv_bfp: {
       assert(N->getValueType(0) == MVT::i32 && Subtarget.is64Bit() &&
              "Unexpected custom legalisation");
       Results.push_back(customLegalizeToWOpByIntr(N, DAG, IntNo));
       break;
     }
+    case Intrinsic::riscv_fsl:
+    case Intrinsic::riscv_fsr: {
+      assert(N->getValueType(0) == MVT::i32 && Subtarget.is64Bit() &&
+             "Unexpected custom legalisation");
+      SDValue NewOp1 =
+          DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(1));
+      SDValue NewOp2 =
+          DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(2));
+      SDValue NewOp3 =
+          DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(3));
+      unsigned Opc = getRISCVWOpcodeByIntr(IntNo);
+      SDValue Res = DAG.getNode(Opc, DL, MVT::i64, NewOp1, NewOp2, NewOp3);
+      Results.push_back(DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, Res));
+      break;
+    }
     case Intrinsic::riscv_orc_b: {
       // Lower to the GORCI encoding for orc.b with the operand extended.
       SDValue NewOp =
           DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, N->getOperand(1));
       // If Zbp is enabled, use GORCIW which will sign extend the result.
       unsigned Opc =
           Subtarget.hasStdExtZbp() ? RISCVISD::GORCW : RISCVISD::GORC;
       SDValue Res = DAG.getNode(Opc, DL, MVT::i64, NewOp,

llvm/test/CodeGen/RISCV/rv32zbt-intrinsic.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zbt -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32ZBT
+
+declare i32 @llvm.riscv.fsl.i32(i32, i32, i32)
+
+define i32 @fsl_i32(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32ZBT-LABEL: fsl_i32:
+; RV32ZBT:       # %bb.0:
+; RV32ZBT-NEXT:    fsl a0, a0, a1, a2
+; RV32ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsl.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}
+
+declare i32 @llvm.riscv.fsr.i32(i32, i32, i32)
+
+define i32 @fsr_i32(i32 %a, i32 %b, i32 %c) nounwind {
+; RV32ZBT-LABEL: fsr_i32:
+; RV32ZBT:       # %bb.0:
+; RV32ZBT-NEXT:    fsr a0, a0, a1, a2
+; RV32ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsr.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}
+
+define i32 @fsli_i32(i32 %a, i32 %b) nounwind {
+; RV32ZBT-LABEL: fsli_i32:
+; RV32ZBT:       # %bb.0:
+; RV32ZBT-NEXT:    fsri a0, a1, a0, 27
+; RV32ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsl.i32(i32 %a, i32 %b, i32 5)
+  ret i32 %1
+}
+
+define i32 @fsri_i32(i32 %a, i32 %b) nounwind {
+; RV32ZBT-LABEL: fsri_i32:
+; RV32ZBT:       # %bb.0:
+; RV32ZBT-NEXT:    fsri a0, a0, a1, 15
+; RV32ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsr.i32(i32 %a, i32 %b, i32 15)
+  ret i32 %1
+}

llvm/test/CodeGen/RISCV/rv64zbt-intrinsic.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zbt -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64ZBT
+
+declare i32 @llvm.riscv.fsl.i32(i32, i32, i32)
+
+define i32 @fsl_i32(i32 %a, i32 %b, i32 %c) nounwind {
+; RV64ZBT-LABEL: fsl_i32:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fslw a0, a0, a1, a2
+; RV64ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsl.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}
+
+declare i32 @llvm.riscv.fsr.i32(i32, i32, i32)
+
+define i32 @fsr_i32(i32 %a, i32 %b, i32 %c) nounwind {
+; RV64ZBT-LABEL: fsr_i32:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsrw a0, a0, a1, a2
+; RV64ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsr.i32(i32 %a, i32 %b, i32 %c)
+  ret i32 %1
+}
+
+define i32 @fsli_i32(i32 %a, i32 %b) nounwind {
+; RV64ZBT-LABEL: fsli_i32:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsriw a0, a1, a0, 27
+; RV64ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsl.i32(i32 %a, i32 %b, i32 5)
+  ret i32 %1
+}
+
+define i32 @fsri_i32(i32 %a, i32 %b) nounwind {
+; RV64ZBT-LABEL: fsri_i32:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsriw a0, a0, a1, 15
+; RV64ZBT-NEXT:    ret
+  %1 = call i32 @llvm.riscv.fsr.i32(i32 %a, i32 %b, i32 15)
+  ret i32 %1
+}
+
+declare i64 @llvm.riscv.fsl.i64(i64, i64, i64)
+
+define i64 @fsl_i64(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64ZBT-LABEL: fsl_i64:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsl a0, a0, a1, a2
+; RV64ZBT-NEXT:    ret
+  %1 = call i64 @llvm.riscv.fsl.i64(i64 %a, i64 %b, i64 %c)
+  ret i64 %1
+}
+
+declare i64 @llvm.riscv.fsr.i64(i64, i64, i64)
+
+define i64 @fsr_i64(i64 %a, i64 %b, i64 %c) nounwind {
+; RV64ZBT-LABEL: fsr_i64:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsr a0, a0, a1, a2
+; RV64ZBT-NEXT:    ret
+  %1 = call i64 @llvm.riscv.fsr.i64(i64 %a, i64 %b, i64 %c)
+  ret i64 %1
+}
+
+define i64 @fsli_i64(i64 %a, i64 %b) nounwind {
+; RV64ZBT-LABEL: fsli_i64:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsri a0, a1, a0, 49
+; RV64ZBT-NEXT:    ret
+  %1 = call i64 @llvm.riscv.fsl.i64(i64 %a, i64 %b, i64 15)
+  ret i64 %1
+}
+
+define i64 @fsri_i64(i64 %a, i64 %b) nounwind {
+; RV64ZBT-LABEL: fsri_i64:
+; RV64ZBT:       # %bb.0:
+; RV64ZBT-NEXT:    fsri a0, a0, a1, 5
+; RV64ZBT-NEXT:    ret
+  %1 = call i64 @llvm.riscv.fsr.i64(i64 %a, i64 %b, i64 5)
+  ret i64 %1
+}