[RISCV] Adjust predicates and update intrinsic for clmul and clmulh in Zbkc extension

[Jimerlife]

Summary

Adjust predicates and update intrinsic for clmul and clmulh in Zbkc extension

[kito-cheng]

Actually this patch is NOT NFC, you fixed an issue that is: zbkc isn't include cmul and clmulh.

So I would suggest you:

1. Remove NFC from the title.
2. Add test case to test cmul and clmulh is available for zbkc after this patch.

[craig.topper]

Will you also be updating clang's RISCVBuiltins.def?

Can you add tests?

[liaolucy]

Do you add intrinsic?

we add init support https://reviews.llvm.org/D102310

[Jimerlife]

1. Add IR tests for clmul and clmulh in zbkc extension.
2. Updating BuiltinsRISCV.def, I rename "builtinriscvclmul" to "builtin_riscv_clmul_kc" in zbkc extension, because cannot share same name "builtinriscv_clmul" in zbc and zbkc extension.
3. Add C tests for clmul and clmulh in zbkc extension

[Jimerlife]

In D118011#3265071, @craig.topper wrote:

Will you also be updating clang's RISCVBuiltins.def?

Can you add tests?

I update BuiltinsRISCV.def and add "__builtin_riscv_clmul_kc" intrinsic for zbkc extension, but I am not sure if the name is appropriate.

[Jimerlife]

In D118011#3265069, @kito-cheng wrote:

Actually this patch is NOT NFC, you fixed an issue that is: zbkc isn't include cmul and clmulh.

So I would suggest you:

1. Remove NFC from the title.
2. Add test case to test cmul and clmulh is available for zbkc after this patch.

Thank you. I adjust patch according your suggestions.

[craig.topper]

llvm/include/llvm/IR/IntrinsicsRISCV.td

91

Add spaces around “or”

[Jimerlife]

add spaces around "or"

[liaolucy]

clang Zbkc patch: https://reviews.llvm.org/D112774， If there are any mistakes, you can help to point them out

[alextsao1999]

In D118011#3265374, @Jimerlife wrote:

In D118011#3265071, @craig.topper wrote:

Will you also be updating clang's RISCVBuiltins.def?

Can you add tests?

I update BuiltinsRISCV.def and add "__builtin_riscv_clmul_kc" intrinsic for zbkc extension, but I am not sure if the name is appropriate.

https://github.com/rvkrypto/rvkrypto-fips/blob/main/rvkintrin.md
Is the name as __builtin_riscv_clmul_32 or __builtin_riscv_clmul_64 better for conflict resolution?

[VincentWu]

Hi, Jimerlife

it is a great work to implement the intrinsic of zbkb.
however, I don't know whether you have noticed that there is already to patch https://reviews.llvm.org/D112774 and https://reviews.llvm.org/D102310.
these two patches have included almost all the work you did in your patch.

I think we should better avoid overlap as much as possible.

therefore, could you please comment on the difference between these two patches as you see it?
let's focus on only one version.

I strongly suggest you to join the bi-weekly LLVM RISC-V meetings to discuss with us if possible. Almost all contributors on RISC-V backend would attend.

[craig.topper]

In D118011#3266014, @alextsao1999 wrote:

In D118011#3265374, @Jimerlife wrote:

In D118011#3265071, @craig.topper wrote:

Will you also be updating clang's RISCVBuiltins.def?

Can you add tests?

I update BuiltinsRISCV.def and add "__builtin_riscv_clmul_kc" intrinsic for zbkc extension, but I am not sure if the name is appropriate.

https://github.com/rvkrypto/rvkrypto-fips/blob/main/rvkintrin.md
Is the name as __builtin_riscv_clmul_32 or __builtin_riscv_clmul_64 better for conflict resolution?

The builtins for clmul have been in for months without _32 or _64 since they operate on the "long" data type.

[craig.topper]

clang/include/clang/Basic/BuiltinsRISCV.def

28

After D117854, it should be possible to use

TARGET_BUILTIN(__builtin_riscv_clmul, "LiLiLi", "nc", "zbc|zbkc")
TARGET_BUILTIN(__builtin_riscv_clmulh, "LiLiLi", "nc", "zbc|zbkc")

clang/include/clang/Basic/BuiltinsRISCV.def

 TARGET_BUILTIN(__builtin_riscv_orc_b_32, "ZiZi", "nc", "zbb")
 TARGET_BUILTIN(__builtin_riscv_orc_b_64, "WiWi", "nc", "zbb,64bit")
 
 // Zbc extension
 TARGET_BUILTIN(__builtin_riscv_clmul, "LiLiLi", "nc", "zbc")
 TARGET_BUILTIN(__builtin_riscv_clmulh, "LiLiLi", "nc", "zbc")
 TARGET_BUILTIN(__builtin_riscv_clmulr, "LiLiLi", "nc", "zbc")
 
+// Zbkc extension
+TARGET_BUILTIN(__builtin_riscv_clmul_kc, "LiLiLi", "nc", "zbkc")
+TARGET_BUILTIN(__builtin_riscv_clmulh_kc, "LiLiLi", "nc", "zbkc")
+
 // Zbe extension
 TARGET_BUILTIN(__builtin_riscv_bcompress_32, "ZiZiZi", "nc", "experimental-zbe")
 TARGET_BUILTIN(__builtin_riscv_bcompress_64, "WiWiWi", "nc",
                "experimental-zbe,64bit")
 TARGET_BUILTIN(__builtin_riscv_bdecompress_32, "ZiZiZi", "nc",
                "experimental-zbe")
 TARGET_BUILTIN(__builtin_riscv_bdecompress_64, "WiWiWi", "nc",
                "experimental-zbe,64bit")

clang/lib/CodeGen/CGBuiltin.cpp

Value *CodeGenFunction::EmitRISCVBuiltinExpr(unsigned BuiltinID,
   // Required for overloaded intrinsics.
   llvm::SmallVector<llvm::Type *, 2> IntrinsicTypes;
   switch (BuiltinID) {
   default: llvm_unreachable("unexpected builtin ID");
   case RISCV::BI__builtin_riscv_orc_b_32:
   case RISCV::BI__builtin_riscv_orc_b_64:
   case RISCV::BI__builtin_riscv_clmul:
   case RISCV::BI__builtin_riscv_clmulh:
+  case RISCV::BI__builtin_riscv_clmul_kc:
+  case RISCV::BI__builtin_riscv_clmulh_kc:
   case RISCV::BI__builtin_riscv_clmulr:
   case RISCV::BI__builtin_riscv_bcompress_32:
   case RISCV::BI__builtin_riscv_bcompress_64:
   case RISCV::BI__builtin_riscv_bdecompress_32:
   case RISCV::BI__builtin_riscv_bdecompress_64:
   case RISCV::BI__builtin_riscv_bfp_32:
   case RISCV::BI__builtin_riscv_bfp_64:
   case RISCV::BI__builtin_riscv_grev_32:
case RISCV::BI__builtin_riscv_fsr_64: {
     switch (BuiltinID) {
     default: llvm_unreachable("unexpected builtin ID");
     // Zbb
     case RISCV::BI__builtin_riscv_orc_b_32:
     case RISCV::BI__builtin_riscv_orc_b_64:
       ID = Intrinsic::riscv_orc_b;
       break;
 
-    // Zbc
+    // Zbc or Zbkc
     case RISCV::BI__builtin_riscv_clmul:
+    case RISCV::BI__builtin_riscv_clmul_kc:
       ID = Intrinsic::riscv_clmul;
       break;
     case RISCV::BI__builtin_riscv_clmulh:
+    case RISCV::BI__builtin_riscv_clmulh_kc:
       ID = Intrinsic::riscv_clmulh;
       break;
+
+    // Zbc
     case RISCV::BI__builtin_riscv_clmulr:
       ID = Intrinsic::riscv_clmulr;
       break;
 
     // Zbe
     case RISCV::BI__builtin_riscv_bcompress_32:
     case RISCV::BI__builtin_riscv_bcompress_64:
       ID = Intrinsic::riscv_bcompress;

clang/test/CodeGen/RISCV/rvb-intrinsics/riscv32-zbkc.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple riscv32 -target-feature +zbkc -emit-llvm %s -o - \
+// RUN:     | FileCheck %s  -check-prefix=RV32ZBKC
+
+// RV32ZBKC-LABEL: @clmul(
+// RV32ZBKC-NEXT:  entry:
+// RV32ZBKC-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBKC-NEXT:    [[B_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBKC-NEXT:    store i32 [[A:%.*]], i32* [[A_ADDR]], align 4
+// RV32ZBKC-NEXT:    store i32 [[B:%.*]], i32* [[B_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP0:%.*]] = load i32, i32* [[A_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP1:%.*]] = load i32, i32* [[B_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP2:%.*]] = call i32 @llvm.riscv.clmul.i32(i32 [[TMP0]], i32 [[TMP1]])
+// RV32ZBKC-NEXT:    ret i32 [[TMP2]]
+//
+long clmul(long a, long b) {
+  return __builtin_riscv_clmul_kc(a, b);
+}
+
+// RV32ZBKC-LABEL: @clmulh(
+// RV32ZBKC-NEXT:  entry:
+// RV32ZBKC-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBKC-NEXT:    [[B_ADDR:%.*]] = alloca i32, align 4
+// RV32ZBKC-NEXT:    store i32 [[A:%.*]], i32* [[A_ADDR]], align 4
+// RV32ZBKC-NEXT:    store i32 [[B:%.*]], i32* [[B_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP0:%.*]] = load i32, i32* [[A_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP1:%.*]] = load i32, i32* [[B_ADDR]], align 4
+// RV32ZBKC-NEXT:    [[TMP2:%.*]] = call i32 @llvm.riscv.clmulh.i32(i32 [[TMP0]], i32 [[TMP1]])
+// RV32ZBKC-NEXT:    ret i32 [[TMP2]]
+//
+long clmulh(long a, long b) {
+  return __builtin_riscv_clmulh_kc(a, b);
+}

clang/test/CodeGen/RISCV/rvb-intrinsics/riscv64-zbkc.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple riscv64 -target-feature +zbkc -emit-llvm %s -o - \
+// RUN:     | FileCheck %s  -check-prefix=RV64ZBKC
+
+// RV64ZBKC-LABEL: @clmul(
+// RV64ZBKC-NEXT:  entry:
+// RV64ZBKC-NEXT:    [[A_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBKC-NEXT:    [[B_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBKC-NEXT:    store i64 [[A:%.*]], i64* [[A_ADDR]], align 8
+// RV64ZBKC-NEXT:    store i64 [[B:%.*]], i64* [[B_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP0:%.*]] = load i64, i64* [[A_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP1:%.*]] = load i64, i64* [[B_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP2:%.*]] = call i64 @llvm.riscv.clmul.i64(i64 [[TMP0]], i64 [[TMP1]])
+// RV64ZBKC-NEXT:    ret i64 [[TMP2]]
+//
+long clmul(long a, long b) {
+  return __builtin_riscv_clmul_kc(a, b);
+}
+
+// RV64ZBKC-LABEL: @clmulh(
+// RV64ZBKC-NEXT:  entry:
+// RV64ZBKC-NEXT:    [[A_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBKC-NEXT:    [[B_ADDR:%.*]] = alloca i64, align 8
+// RV64ZBKC-NEXT:    store i64 [[A:%.*]], i64* [[A_ADDR]], align 8
+// RV64ZBKC-NEXT:    store i64 [[B:%.*]], i64* [[B_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP0:%.*]] = load i64, i64* [[A_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP1:%.*]] = load i64, i64* [[B_ADDR]], align 8
+// RV64ZBKC-NEXT:    [[TMP2:%.*]] = call i64 @llvm.riscv.clmulh.i64(i64 [[TMP0]], i64 [[TMP1]])
+// RV64ZBKC-NEXT:    ret i64 [[TMP2]]
+//
+long clmulh(long a, long b) {
+  return __builtin_riscv_clmulh_kc(a, b);
+}

llvm/include/llvm/IR/IntrinsicsRISCV.td

let TargetPrefix = "riscv" in {
   class BitManipGPRGPRGRIntrinsics
       : Intrinsic<[llvm_any_ty],
                   [LLVMMatchType<0>, LLVMMatchType<0>, LLVMMatchType<0>],
                   [IntrNoMem, IntrSpeculatable, IntrWillReturn]>;
 
   // Zbb
   def int_riscv_orc_b : BitManipGPRIntrinsics;
 
-  // Zbc
+  // Zbc or Zbkc
   def int_riscv_clmul  : BitManipGPRGPRIntrinsics;
   def int_riscv_clmulh : BitManipGPRGPRIntrinsics;
+
+  // Zbc
   def int_riscv_clmulr : BitManipGPRGPRIntrinsics;
 
   // Zbe
   def int_riscv_bcompress   : BitManipGPRGPRIntrinsics;
   def int_riscv_bdecompress : BitManipGPRGPRIntrinsics;
 
   // Zbf
   def int_riscv_bfp  : BitManipGPRGPRIntrinsics;

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

 def : Pat<(i64 (or (sext_inreg (shl GPR:$rs2, (i64 16)), i32),
                    (and GPR:$rs1, 0x000000000000FFFF))),
           (PACKW GPR:$rs1, GPR:$rs2)>;
 def : Pat<(i64 (or (and (assertsexti32 GPR:$rs2), 0xFFFFFFFFFFFF0000),
                    (srl (and GPR:$rs1, 0xFFFFFFFF), (i64 16)))),
           (PACKUW GPR:$rs1, GPR:$rs2)>;
 } // Predicates = [HasStdExtZbp, IsRV64]
 
-let Predicates = [HasStdExtZbc] in {
+let Predicates = [HasStdExtZbcOrZbkc] in {
 def : PatGprGpr<int_riscv_clmul, CLMUL>;
 def : PatGprGpr<int_riscv_clmulh, CLMULH>;
+} // Predicates = [HasStdExtZbcOrZbkc]
+
+let Predicates = [HasStdExtZbc] in
 def : PatGprGpr<int_riscv_clmulr, CLMULR>;
-} // Predicates = [HasStdExtZbc]
 
 let Predicates = [HasStdExtZbe] in {
 def : PatGprGpr<riscv_bcompress, BCOMPRESS>;
 def : PatGprGpr<riscv_bdecompress, BDECOMPRESS>;
 } // Predicates = [HasStdExtZbe]
 
 let Predicates = [HasStdExtZbe, IsRV64] in {
 def : PatGprGpr<riscv_bcompressw, BCOMPRESSW>;

llvm/test/CodeGen/RISCV/rv32zbc-zbkc-intrinsic.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv32 -mattr=+zbc -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32ZBC
+; RUN: llc -mtriple=riscv32 -mattr=+zbkc -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32ZBKC
+
+declare i32 @llvm.riscv.clmul.i32(i32 %a, i32 %b)
+
+define i32 @clmul32(i32 %a, i32 %b) nounwind {
+; RV32ZBC-LABEL: clmul32:
+; RV32ZBC:       # %bb.0:
+; RV32ZBC-NEXT:    clmul a0, a0, a1
+; RV32ZBC-NEXT:    ret
+;
+; RV32ZBKC-LABEL: clmul32:
+; RV32ZBKC:       # %bb.0:
+; RV32ZBKC-NEXT:    clmul a0, a0, a1
+; RV32ZBKC-NEXT:    ret
+  %tmp = call i32 @llvm.riscv.clmul.i32(i32 %a, i32 %b)
+ ret i32 %tmp
+}
+
+declare i32 @llvm.riscv.clmulh.i32(i32 %a, i32 %b)
+
+define i32 @clmul32h(i32 %a, i32 %b) nounwind {
+; RV32ZBC-LABEL: clmul32h:
+; RV32ZBC:       # %bb.0:
+; RV32ZBC-NEXT:    clmulh a0, a0, a1
+; RV32ZBC-NEXT:    ret
+;
+; RV32ZBKC-LABEL: clmul32h:
+; RV32ZBKC:       # %bb.0:
+; RV32ZBKC-NEXT:    clmulh a0, a0, a1
+; RV32ZBKC-NEXT:    ret
+  %tmp = call i32 @llvm.riscv.clmulh.i32(i32 %a, i32 %b)
+ ret i32 %tmp
+}

llvm/test/CodeGen/RISCV/rv64zbc-zbkc-intrinsic.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+zbc -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64ZBC
+; RUN: llc -mtriple=riscv64 -mattr=+zbkc -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64ZBKC
+
+declare i64 @llvm.riscv.clmul.i64(i64 %a, i64 %b)
+
+define i64 @clmul64(i64 %a, i64 %b) nounwind {
+; RV64ZBC-LABEL: clmul64:
+; RV64ZBC:       # %bb.0:
+; RV64ZBC-NEXT:    clmul a0, a0, a1
+; RV64ZBC-NEXT:    ret
+;
+; RV64ZBKC-LABEL: clmul64:
+; RV64ZBKC:       # %bb.0:
+; RV64ZBKC-NEXT:    clmul a0, a0, a1
+; RV64ZBKC-NEXT:    ret
+  %tmp = call i64 @llvm.riscv.clmul.i64(i64 %a, i64 %b)
+ ret i64 %tmp
+}
+
+declare i64 @llvm.riscv.clmulh.i64(i64 %a, i64 %b)
+
+define i64 @clmul64h(i64 %a, i64 %b) nounwind {
+; RV64ZBC-LABEL: clmul64h:
+; RV64ZBC:       # %bb.0:
+; RV64ZBC-NEXT:    clmulh a0, a0, a1
+; RV64ZBC-NEXT:    ret
+;
+; RV64ZBKC-LABEL: clmul64h:
+; RV64ZBKC:       # %bb.0:
+; RV64ZBKC-NEXT:    clmulh a0, a0, a1
+; RV64ZBKC-NEXT:    ret
+  %tmp = call i64 @llvm.riscv.clmulh.i64(i64 %a, i64 %b)
+ ret i64 %tmp
+}