[RISCV][CodeGen] Implement IR Intrinsic support for K extension

[VincentWu]

Summary

This revision implements IR Intrinsic support for RISCV Scalar Crypto extension according to the specification of version 1.0
Co-author：@ksyx & @VincentWu & @lihongliang & @achieveartificialintelligence

[ksyx]

fix: patch does not apply (retrigger ci after adding parent revision)

[Jim]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1254

one space before ":"
And the same case in the following.

[ksyx]

refactor: remove redundant spaces around colons

[ksyx]

refactor: fix spaces

[ksyx]

fix: types of immargs to better fit the proposal

[achieveartificialintelligence]

Updated to k-1.0-rc4

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

259 ↗

(On Diff #381463)

This should be done in the else block where hasStdExtZbb() is handled. Zbp is for GREVI.

2427 ↗

(On Diff #381463)

This should not be needed with the other change.

llvm/lib/Target/RISCV/RISCVInstrInfoZk.td

24 ↗

(On Diff #381463)

This needs to be rebased on the current Zk patch.

211 ↗

(On Diff #381463)

There are no _K instructions in the most recent Zk patches.

All Zb* related changes should be in RISCVInstrInfoZb.td

[VincentWu]

rebase and update

[VincentWu]

rename testcase

[VincentWu]

fix intrinsic

[VincentWu]

fix intrinsic and testcase

[craig.topper]

llvm/include/llvm/IR/IntrinsicsRISCV.td

116

Capital Z

117

rev8 is the same as llvm.bswap.i32 or llvm.bswap.i64. We don't need an intrinsic for it.

1371

Capitalize Z in the comments.

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

282 ↗

(On Diff #402875)

You want Zbkb in addition to Zbb here.

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

892 ↗

(On Diff #402875)

This should stay Zbp only.

901 ↗

(On Diff #402875)

This should stay Zbp only.

1196 ↗

(On Diff #402875)

We don't need int_riscv_rev8. We can use bswap

llvm/test/CodeGen/RISCV/rv32zbc-zbkc-intrinsic.ll

5 ↗

(On Diff #402875)

Can we use a single check-prefix?

llvm/test/CodeGen/RISCV/rv32zbp-zbkb.ll

7 ↗

(On Diff #402875)

Can we use one check-prefix for Zbp and Zbkb?

llvm/test/CodeGen/RISCV/rv64zbb-zbp-zbkb.ll

9 ↗

(On Diff #402875)

Can we use one check-prefix for Zbb, Zbp, and Zbkb?

llvm/test/CodeGen/RISCV/rv64zbc-zbkc-intrinsic.ll

5 ↗

(On Diff #402875)

Can we use a single check-prefix?

llvm/test/CodeGen/RISCV/rv64zbp-zbkb.ll

7 ↗

(On Diff #402875)

Can we use one check-prefix for Zbp and Zbkb?

llvm/test/CodeGen/RISCV/rv64zksed-intrinsic.ll

13 ↗

(On Diff #402875)

Something went wrong here

29 ↗

(On Diff #402875)

And here

[VincentWu]

address comments

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

282 ↗

(On Diff #402875)

Does this line exceed 80 columns?

llvm/lib/Target/RISCV/RISCVInstrInfoZb.td

890 ↗

(On Diff #403134)

Revert this

892 ↗

(On Diff #403134)

Revert this

899 ↗

(On Diff #403134)

Revert this

[VincentWu]

remove duplicate code & clang-format

[craig.topper]

llvm/lib/Target/RISCV/RISCV.td

175 ↗

(On Diff #403139)

The string for Zbb was recently updated. Please update to match.

llvm/test/CodeGen/RISCV/rv32zbb-zbp-zbkb.ll

9 ↗

(On Diff #403139)

Can we share a prefix for Zbb/Zbp/Zbkb?

llvm/test/CodeGen/RISCV/rv64zbb-zbp-zbkb.ll

507 ↗

(On Diff #403139)

This a bug that Zbp doesn't match Zbb. I'll fix it. You'll need to add Zbkb to RISCVTargetLowering::hasAndNotCompare in this patch.

[VincentWu]

update & address comment

[craig.topper]

LGTM

[VincentWu]

rebase & read to submit

[VincentWu]

In D102310#3274695, @craig.topper wrote:

LGTM

Tks.

[craig.topper]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1370

Why does this intrinsic have the Returned attribute?

[ksyx]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1370

Maybe this is following the old 0.9.0 version that mixes the use of rs1/rd registers.

[craig.topper]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1370

I see, but that was still a mistake. The Returned property means that the *value* passed into the argument is the *value* that is returned. Even if rd/rs1 are the same register, the operation still changes the value in that register.

[ksyx]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1370

Thanks for pointing out this as I may misunderstood this while implementing.

[achieveartificialintelligence]

llvm/include/llvm/IR/IntrinsicsRISCV.td

1370

Thanks! I've committed in 352e19c.

llvm/include/llvm/IR/IntrinsicsRISCV.td

let TargetPrefix = "riscv" in {
   def int_riscv_crc32_h : BitManipGPRIntrinsics;
   def int_riscv_crc32_w : BitManipGPRIntrinsics;
   def int_riscv_crc32_d : BitManipGPRIntrinsics;
   def int_riscv_crc32c_b : BitManipGPRIntrinsics;
   def int_riscv_crc32c_h : BitManipGPRIntrinsics;
   def int_riscv_crc32c_w : BitManipGPRIntrinsics;
   def int_riscv_crc32c_d : BitManipGPRIntrinsics;
 } // TargetPrefix = "riscv"
 
 //===----------------------------------------------------------------------===//
 // Vectors
 
 class RISCVVIntrinsic {
   // These intrinsics may accept illegal integer values in their llvm_any_ty
   // operand, so they have to be extended. If set to zero then the intrinsic
   // does not have any operand that must be extended.
   Intrinsic IntrinsicID = !cast<Intrinsic>(NAME);
   bits<4> SplatOperand = 0;
foreach nf = [2, 3, 4, 5, 6, 7, 8] in {
     defm vloxseg # nf : RISCVISegLoad<nf>;
     defm vluxseg # nf : RISCVISegLoad<nf>;
     defm vsseg # nf : RISCVUSSegStore<nf>;
     defm vssseg # nf : RISCVSSegStore<nf>;
     defm vsoxseg # nf : RISCVISegStore<nf>;
     defm vsuxseg # nf : RISCVISegStore<nf>;
   }
 
+//===----------------------------------------------------------------------===//
+// Scalar Cryptography
+
+// These intrinsics will lower directly into the corresponding instructions
+// added by the scalar cyptography extension, if the extension is present.
+
+class ScalarCryptoByteSelect32 : Intrinsic<[llvm_i32_ty],
+                                           [llvm_i32_ty,
+                                            llvm_i32_ty,
+                                            llvm_i32_ty],
+                                           [IntrNoMem, IntrSpeculatable,
+                                            IntrWillReturn,
+                                            ImmArg<ArgIndex<2>>,
+                                            Returned<ArgIndex<0>>]>;
+
+def int_riscv_aes32dsi : ScalarCryptoByteSelect32;
+def int_riscv_aes32esi : ScalarCryptoByteSelect32;
+def int_riscv_aes32dsmi : ScalarCryptoByteSelect32;
+def int_riscv_aes32esmi : ScalarCryptoByteSelect32;
+
+class ScalarCryptoByteSelectAny : Intrinsic<[llvm_anyint_ty],
+                                            [LLVMMatchType<0>,
+                                              LLVMMatchType<0>,
+                                              llvm_anyint_ty],
+                                            [IntrNoMem, IntrSpeculatable,
+                                              IntrWillReturn,
+                                              ImmArg<ArgIndex<2>>,
+                                              Returned<ArgIndex<0>>]>;
+
+def int_riscv_sm4ks : ScalarCryptoByteSelectAny;
+def int_riscv_sm4ed : ScalarCryptoByteSelectAny;
+
+class ScalarCryptoGprGprIntrinsic32 : Intrinsic<[llvm_i32_ty],
+                                                [llvm_i32_ty, llvm_i32_ty],
+                                                [IntrNoMem, IntrSpeculatable]>;
+
+def int_riscv_sha512sig0l : ScalarCryptoGprGprIntrinsic32;
+def int_riscv_sha512sig0h : ScalarCryptoGprGprIntrinsic32;
+def int_riscv_sha512sig1l : ScalarCryptoGprGprIntrinsic32;
+def int_riscv_sha512sig1h : ScalarCryptoGprGprIntrinsic32;
+def int_riscv_sha512sum0r : ScalarCryptoGprGprIntrinsic32;
+def int_riscv_sha512sum1r : ScalarCryptoGprGprIntrinsic32;
+
+class ScalarCryptoGprGprIntrinsic64 : Intrinsic<[llvm_i64_ty],
+                                                [llvm_i64_ty, llvm_i64_ty],
+                                                [IntrNoMem, IntrSpeculatable]>;
+
+def int_riscv_aes64ds : ScalarCryptoGprGprIntrinsic64;
+def int_riscv_aes64es : ScalarCryptoGprGprIntrinsic64;
+def int_riscv_aes64dsm : ScalarCryptoGprGprIntrinsic64;
+def int_riscv_aes64ks2 : ScalarCryptoGprGprIntrinsic64;
+def int_riscv_aes64esm : ScalarCryptoGprGprIntrinsic64;
+
+def int_riscv_aes64ks1i : Intrinsic<[llvm_i64_ty], [llvm_i64_ty, llvm_i64_ty],
+                                    [IntrNoMem, IntrSpeculatable,
+                                     IntrWillReturn, ImmArg<ArgIndex<1>>]>;
+
+class ScalarCryptoGprIntrinsic64 : Intrinsic<[llvm_i64_ty], [llvm_i64_ty],
+                                             [IntrNoMem, IntrSpeculatable]>;
+
+def int_riscv_sha512sig0 : ScalarCryptoGprIntrinsic64;
+def int_riscv_sha512sig1 : ScalarCryptoGprIntrinsic64;
+def int_riscv_sha512sum0 : ScalarCryptoGprIntrinsic64;
+def int_riscv_sha512sum1 : ScalarCryptoGprIntrinsic64;
+def int_riscv_aes64im : ScalarCryptoGprIntrinsic64;
+
+class ScalarCryptoGprIntrinsicAny : Intrinsic<[llvm_anyint_ty],
+                                              [LLVMMatchType<0>],
+                                              [IntrNoMem, IntrSpeculatable]>;
+
+def int_riscv_sha256sig0 : ScalarCryptoGprIntrinsicAny;
+def int_riscv_sha256sig1 : ScalarCryptoGprIntrinsicAny;
+def int_riscv_sha256sum0 : ScalarCryptoGprIntrinsicAny;
+def int_riscv_sha256sum1 : ScalarCryptoGprIntrinsicAny;
+def int_riscv_sm3p0 : ScalarCryptoGprIntrinsicAny;
+def int_riscv_sm3p1 : ScalarCryptoGprIntrinsicAny;
+
+class ScalarCryptoEntropyIntrinsicAny : Intrinsic<[llvm_anyint_ty], [],
+                                                  [IntrNoMem, IntrSpeculatable]>;
+
+def int_riscv_getnoise : ScalarCryptoEntropyIntrinsicAny;
+def int_riscv_pollentropy : ScalarCryptoEntropyIntrinsicAny;
 } // TargetPrefix = "riscv"

llvm/lib/Target/RISCV/RISCVInstrInfo.cpp

if (OpType >= RISCVOp::OPERAND_FIRST_RISCV_IMM &&
         OpType <= RISCVOp::OPERAND_LAST_RISCV_IMM) {
       const MachineOperand &MO = MI.getOperand(OI.index());
       if (MO.isImm()) {
         int64_t Imm = MO.getImm();
         bool Ok;
         switch (OpType) {
         default:
           llvm_unreachable("Unexpected operand type");
+        case RISCVOp::OPERAND_UIMM2:
+          Ok = isUInt<2>(Imm);
+          break;
         case RISCVOp::OPERAND_UIMM4:
           Ok = isUInt<4>(Imm);
           break;
         case RISCVOp::OPERAND_UIMM5:
           Ok = isUInt<5>(Imm);
           break;
         case RISCVOp::OPERAND_UIMM12:
           Ok = isUInt<12>(Imm);

llvm/lib/Target/RISCV/RISCVInstrInfo.td

let MCOperandPredicate = [{
     if (STI.getTargetTriple().isArch64Bit())
       return isUInt<6>(Imm);
     return isUInt<5>(Imm);
   }];
   let OperandType = "OPERAND_UIMMLOG2XLEN";
   let OperandNamespace = "RISCVOp";
 }
 
-def uimm2 : Operand<XLenVT>, ImmLeaf<XLenVT, [{return isUInt<2>(Imm);}]> {
+def uimm2 : Operand<XLenVT>, TImmLeaf<XLenVT, [{return isUInt<2>(Imm);}]> {
   let ParserMatchClass = UImmAsmOperand<2>;
   let DecoderMethod = "decodeUImmOperand<2>";
   let OperandType = "OPERAND_UIMM2";
   let OperandNamespace = "RISCVOp";
 }
 
 def uimm5 : Operand<XLenVT>, ImmLeaf<XLenVT, [{return isUInt<5>(Imm);}]> {
   let ParserMatchClass = UImmAsmOperand<5>;

llvm/lib/Target/RISCV/RISCVInstrInfoK.td

 //===----------------------------------------------------------------------===//
 
 def RconArg : AsmOperandClass {
   let Name = "RconArg";
   let RenderMethod = "addImmOperands";
   let DiagnosticType = "InvalidRconArg";
 }
 
-def rcon : Operand<XLenVT>, ImmLeaf<XLenVT, [{return isUInt<4>(Imm);}]> {
+def rcon : Operand<XLenVT>, TImmLeaf<XLenVT, [{return isUInt<4>(Imm);}]> {
   let ParserMatchClass = RconArg;
   let EncoderMethod = "getImmOpValue";
   let DecoderMethod = "decodeUImmOperand<4>";
   let OperandType = "OPERAND_UIMM4";
   let OperandNamespace = "RISCVOp";
 }
 
 //===----------------------------------------------------------------------===//

 def SM4ED : RVKByteSelect<0b11000, "sm4ed">;
 def SM4KS : RVKByteSelect<0b11010, "sm4ks">;
 } // Predicates = [HasStdExtZksed]
 
 let Predicates = [HasStdExtZksh] in {
 def SM3P0 : RVKUnary<0b0001000, 0b01000, 0b001, "sm3p0">;
 def SM3P1 : RVKUnary<0b0001000, 0b01001, 0b001, "sm3p1">;
 } // Predicates = [HasStdExtZksh]
+
+//===----------------------------------------------------------------------===//
+// Intrinsics
+//===----------------------------------------------------------------------===//
+
+class PatGprGprByteSelect<SDPatternOperator OpNode, RVInst Inst>
+    : Pat<(OpNode GPR:$rs1, GPR:$rs2, uimm2:$imm),
+          (Inst GPR:$rs1, GPR:$rs2, uimm2:$imm)>;
+
+let Predicates = [HasStdExtZkn, IsRV32] in {
+def : PatGprGprByteSelect<int_riscv_aes32dsi, AES32DSI>;
+def : PatGprGprByteSelect<int_riscv_aes32dsmi, AES32DSMI>;
+def : PatGprGprByteSelect<int_riscv_aes32esi, AES32ESI>;
+def : PatGprGprByteSelect<int_riscv_aes32esmi, AES32ESMI>;
+}
+
+let Predicates = [HasStdExtZknh, IsRV32] in {
+def : PatGprGpr<int_riscv_sha512sig0l, SHA512SIG0L>;
+def : PatGprGpr<int_riscv_sha512sig0h, SHA512SIG0H>;
+def : PatGprGpr<int_riscv_sha512sig1l, SHA512SIG1L>;
+def : PatGprGpr<int_riscv_sha512sig1h, SHA512SIG1H>;
+def : PatGprGpr<int_riscv_sha512sum0r, SHA512SUM0R>;
+def : PatGprGpr<int_riscv_sha512sum1r, SHA512SUM1R>;
+}
+
+let Predicates = [HasStdExtZknh, IsRV64] in {
+def : PatGpr<int_riscv_sha512sig0, SHA512SIG0>;
+def : PatGpr<int_riscv_sha512sig1, SHA512SIG1>;
+def : PatGpr<int_riscv_sha512sum0, SHA512SUM0>;
+def : PatGpr<int_riscv_sha512sum1, SHA512SUM1>;
+}
+
+let Predicates = [HasStdExtZknd, IsRV64] in {
+def : PatGprGpr<int_riscv_aes64ds, AES64DS>;
+def : PatGprGpr<int_riscv_aes64dsm, AES64DSM>;
+def : PatGpr<int_riscv_aes64im, AES64IM>;
+}
+
+let Predicates = [HasStdExtZkne, IsRV64] in {
+def : PatGprGpr<int_riscv_aes64es, AES64ES>;
+def : PatGprGpr<int_riscv_aes64esm, AES64ESM>;
+def : Pat<(XLenVT (int_riscv_aes64ks1i (XLenVT GPR:$rs1), rcon:$rcon)),
+          (AES64KS1I GPR:$rs1, rcon:$rcon)>;
+def : PatGprGpr<int_riscv_aes64ks2, AES64KS2>;
+}
+
+let Predicates = [HasStdExtZknh] in {
+def : PatGpr<int_riscv_sha256sig0, SHA256SIG0>;
+def : PatGpr<int_riscv_sha256sig1, SHA256SIG1>;
+def : PatGpr<int_riscv_sha256sum0, SHA256SUM0>;
+def : PatGpr<int_riscv_sha256sum1, SHA256SUM1>;
+}
+
+let Predicates = [HasStdExtZksh] in {
+def : PatGpr<int_riscv_sm3p0, SM3P0>;
+def : PatGpr<int_riscv_sm3p1, SM3P1>;
+}
+
+let Predicates = [HasStdExtZksed, IsRV32] in {
+def : PatGprGprByteSelect<int_riscv_sm4ks, SM4KS>;
+def : PatGprGprByteSelect<int_riscv_sm4ed, SM4ED>;
+}
+
+let Predicates = [HasStdExtZksed, IsRV64] in {
+def : PatGprGprByteSelect<int_riscv_sm4ks, SM4KS>;
+def : PatGprGprByteSelect<int_riscv_sm4ed, SM4ED>;
+}
+
+let Predicates = [HasStdExtZkr] in {
+class PatEntropy<SDPatternOperator OpNode, string name>
+    : Pat<(OpNode), (Pseudo<(outs GPR:$rd), (ins), [], name, "$rd">)>;
+
+def : PatEntropy<int_riscv_getnoise, "getnoise">;
+def : PatEntropy<int_riscv_pollentropy, "pollentropy">;
+}

llvm/test/CodeGen/RISCV/rv32Zkn.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-k -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IK
+
+declare i32 @llvm.riscv.aes32dsi(i32, i32, i32);
+
+define i32 @aes32dsi(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: aes32dsi
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: aes32dsi a{{[0-9]+}}, a{{[0-9]+}}, 0
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.aes32dsi(i32 %a, i32 %b, i32 0)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.aes32dsmi(i32, i32, i32);
+
+define i32 @aes32dsmi(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: aes32dsmi
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: aes32dsmi a{{[0-9]+}}, a{{[0-9]+}}, 1
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.aes32dsmi(i32 %a, i32 %b, i32 1)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.aes32esi(i32, i32, i32);
+
+define i32 @aes32esi(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: aes32esi
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: aes32esi a{{[0-9]+}}, a{{[0-9]+}}, 2
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.aes32esi(i32 %a, i32 %b, i32 2)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.aes32esmi(i32, i32, i32);
+
+define i32 @aes32esmi(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: aes32esmi
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: aes32esmi a{{[0-9]+}}, a{{[0-9]+}}, 3
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.aes32esmi(i32 %a, i32 %b, i32 3)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha256sig0(i32);
+
+define i32 @sha256sig0(i32 %a) nounwind {
+; RV32IK-LABEL: sha256sig0
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha256sig0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha256sig0(i32 %a)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha256sig1(i32);
+
+define i32 @sha256sig1(i32 %a) nounwind {
+; RV32IK-LABEL: sha256sig1
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha256sig1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha256sig1(i32 %a)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha256sum0(i32);
+
+define i32 @sha256sum0(i32 %a) nounwind {
+; RV32IK-LABEL: sha256sum0
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha256sum0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha256sum0(i32 %a)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha256sum1(i32);
+
+define i32 @sha256sum1(i32 %a) nounwind {
+; RV32IK-LABEL: sha256sum1
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha256sum1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha256sum1(i32 %a)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sig0l(i32, i32);
+
+define i32 @sha512sig0l(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sig0l
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sig0l a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sig0l(i32 %a, i32 %b)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sig0h(i32, i32);
+
+define i32 @sha512sig0h(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sig0h
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sig0h a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sig0h(i32 %a, i32 %b)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sig1l(i32, i32);
+
+define i32 @sha512sig1l(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sig1l
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sig1l a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sig1l(i32 %a, i32 %b)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sig1h(i32, i32);
+
+define i32 @sha512sig1h(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sig1h
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sig1h a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sig1h(i32 %a, i32 %b)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sum0r(i32, i32);
+
+define i32 @sha512sum0r(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sum0r
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sum0r a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sum0r(i32 %a, i32 %b)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sha512sum1r(i32, i32);
+
+define i32 @sha512sum1r(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sha512sum1r
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sha512sum1r a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sha512sum1r(i32 %a, i32 %b)
+    ret i32 %val
+}

llvm/test/CodeGen/RISCV/rv32Zkr.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-k -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IK
+
+declare i32 @llvm.riscv.getnoise();
+
+define i32 @getnoise() nounwind {
+; RV32IK-LABEL: getnoise
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: getnoise a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.getnoise()
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.pollentropy();
+
+define i32 @pollentropy() nounwind {
+; RV32IK-LABEL: pollentropy
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: pollentropy a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.pollentropy()
+    ret i32 %val
+}

llvm/test/CodeGen/RISCV/rv32Zksed.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zksed -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IK
+
+declare i32 @llvm.riscv.sm4ks(i32, i32, i32);
+
+define i32 @sm4ks(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sm4ks
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sm4ks a{{[0-9]+}}, a{{[0-9]+}}, 2
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sm4ks(i32 %a, i32 %b, i32 2)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sm4ed(i32, i32, i32);
+
+define i32 @sm4ed(i32 %a, i32 %b) nounwind {
+; RV32IK-LABEL: sm4ed
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sm4ed a{{[0-9]+}}, a{{[0-9]+}}, 3
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sm4ed(i32 %a, i32 %b, i32 3)
+    ret i32 %val
+}

llvm/test/CodeGen/RISCV/rv32Zksh.ll

+; RUN: llc -mtriple=riscv32 -mattr=+experimental-zksh -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV32IK
+
+declare i32 @llvm.riscv.sm3p0(i32);
+
+define i32 @sm3p0(i32 %a) nounwind {
+; RV32IK-LABEL: sm3p0
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sm3p0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sm3p0(i32 %a)
+    ret i32 %val
+}
+
+declare i32 @llvm.riscv.sm3p1(i32);
+
+define i32 @sm3p1(i32 %a) nounwind {
+; RV32IK-LABEL: sm3p1
+; RV32IK: # %bb.0:
+; RV32IK-NEXT: sm3p1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV32IK-NEXT: ret
+    %val = call i32 @llvm.riscv.sm3p1(i32 %a)
+    ret i32 %val
+}

llvm/test/CodeGen/RISCV/rv64Zkn.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-k -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IK
+
+declare i64 @llvm.riscv.aes64es(i64, i64);
+
+define i64 @aes64es(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: aes64es
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64es a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64es(i64 %a, i64 %b)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64esm(i64, i64);
+
+define i64 @aes64esm(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: aes64esm
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64esm a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64esm(i64 %a, i64 %b)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64ds(i64, i64);
+
+define i64 @aes64ds(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: aes64ds
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64ds a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64ds(i64 %a, i64 %b)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64dsm(i64, i64);
+
+define i64 @aes64dsm(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: aes64dsm
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64dsm a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64dsm(i64 %a, i64 %b)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64ks1i(i64, i64);
+
+define i64 @aes64ks1i(i64 %a) nounwind {
+; RV64IK-LABEL: aes64ks1i
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64ks1i a{{[0-9]+}}, a{{[0-9]+}}, {{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64ks1i(i64 %a, i64 10)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64ks2(i64, i64);
+
+define i64 @aes64ks2(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: aes64ks2
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64ks2 a{{[0-9]+}}, a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64ks2(i64 %a, i64 %b)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.aes64im(i64);
+
+define i64 @aes64im(i64 %a) nounwind {
+; RV64IK-LABEL: aes64im
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: aes64im a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.aes64im(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha256sig0(i64);
+
+define i64 @sha256sig0(i64 %a) nounwind {
+; RV64IK-LABEL: sha256sig0
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha256sig0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha256sig0(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha256sig1(i64);
+
+define i64 @sha256sig1(i64 %a) nounwind {
+; RV64IK-LABEL: sha256sig1
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha256sig1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha256sig1(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha256sum0(i64);
+
+define i64 @sha256sum0(i64 %a) nounwind {
+; RV64IK-LABEL: sha256sum0
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha256sum0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha256sum0(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha256sum1(i64);
+
+define i64 @sha256sum1(i64 %a) nounwind {
+; RV64IK-LABEL: sha256sum1
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha256sum1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha256sum1(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha512sig0(i64);
+
+define i64 @sha512sig0(i64 %a) nounwind {
+; RV64IK-LABEL: sha512sig0
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha512sig0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha512sig0(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha512sig1(i64);
+
+define i64 @sha512sig1(i64 %a) nounwind {
+; RV64IK-LABEL: sha512sig1
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha512sig1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha512sig1(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha512sum0(i64);
+
+define i64 @sha512sum0(i64 %a) nounwind {
+; RV64IK-LABEL: sha512sum0
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha512sum0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha512sum0(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sha512sum1(i64);
+
+define i64 @sha512sum1(i64 %a) nounwind {
+; RV64IK-LABEL: sha512sum1
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sha512sum1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sha512sum1(i64 %a)
+    ret i64 %val
+}

llvm/test/CodeGen/RISCV/rv64Zkr.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-k -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IK
+
+declare i64 @llvm.riscv.getnoise();
+
+define i64 @getnoise() nounwind {
+; RV64IK-LABEL: getnoise
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: getnoise a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.getnoise()
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.pollentropy();
+
+define i64 @pollentropy() nounwind {
+; RV64IK-LABEL: pollentropy
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: pollentropy a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.pollentropy()
+    ret i64 %val
+}

llvm/test/CodeGen/RISCV/rv64Zksed.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zksed -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IK
+
+declare i64 @llvm.riscv.sm4ks(i64, i64, i64);
+
+define i64 @sm4ks(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: sm4ks
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sm4ks a{{[0-9]+}}, a{{[0-9]+}}, 0
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sm4ks(i64 %a, i64 %b, i64 0)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sm4ed(i64, i64, i64);
+
+define i64 @sm4ed(i64 %a, i64 %b) nounwind {
+; RV64IK-LABEL: sm4ed
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sm4ed a{{[0-9]+}}, a{{[0-9]+}}, 1
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sm4ed(i64 %a, i64 %b, i64 1)
+    ret i64 %val
+}

llvm/test/CodeGen/RISCV/rv64Zksh.ll

+; RUN: llc -mtriple=riscv64 -mattr=+experimental-zksh -verify-machineinstrs < %s \
+; RUN:   | FileCheck %s -check-prefix=RV64IK
+
+declare i64 @llvm.riscv.sm3p0(i64);
+
+define i64 @sm3p0(i64 %a) nounwind {
+; RV64IK-LABEL: sm3p0
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sm3p0 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sm3p0(i64 %a)
+    ret i64 %val
+}
+
+declare i64 @llvm.riscv.sm3p1(i64);
+
+define i64 @sm3p1(i64 %a) nounwind {
+; RV64IK-LABEL: sm3p1
+; RV64IK: # %bb.0:
+; RV64IK-NEXT: sm3p1 a{{[0-9]+}}, a{{[0-9]+}}
+; RV64IK-NEXT: ret
+    %val = call i64 @llvm.riscv.sm3p1(i64 %a)
+    ret i64 %val
+}