[AArch64] Adding SHA3 Intrinsics support

[rsanthir.quic]

Summary

This patch adds the following SHA3 Intrinsics:
    vsha512hq_u64,
    vsha512h2q_u64,
    vsha512su0q_u64,
    vsha512su1q_u64
    veor3q_u8
    veor3q_u16
    veor3q_u32
    veor3q_u64
    veor3q_s8
    veor3q_s16
    veor3q_s32
    veor3q_s64
    vrax1q_u64
    vxarq_u64
    vbcaxq_u8
    vbcaxq_u16
    vbcaxq_u32
    vbcaxq_u64
    vbcaxq_s8
    vbcaxq_s16
    vbcaxq_s32
    vbcaxq_s64

Note need to include +sha3 and +crypto when building from the front-end

[rsanthir.quic]

This is the second of three patches to address the following:
https://bugs.llvm.org/show_bug.cgi?id=47828

[apazos]

clang/utils/TableGen/NeonEmitter.cpp

2118

Consider alphabetizing the check. move isVXAR check after isVCT_N

[rsanthir.quic]

alphabetized check in NeonEmitter

[apazos]

This looks like a straightforward implementation. The only caveat is that the XAR immediate does not represent a lane, and hence the need for a custom immediate range check. Looks sensible to me.
@labrinea and others at ARM, do have any other comment before this is merged?

[DavidSpickett]

One nit for now, I'll take a proper look tomorrow. Thanks for your work on these!

clang/test/CodeGen/aarch64-neon-sha3.c

26

Nit: remove empty lines in this and the following

[rsanthir.quic]

Removed extra whitespace

[DavidSpickett]

The approach for xar looks fine to me, matches how we handled vcvt_n_* (https://developer.arm.com/architectures/instruction-sets/simd-isas/neon/intrinsics?search=vcvt_n_).

clang/include/clang/Basic/arm_neon.td

1138

Put the second set in c-s-i-l order like the first.

llvm/lib/Target/AArch64/AArch64InstrInfo.td

928

This is unused.

llvm/test/CodeGen/AArch64/neon-sha3.ll

106

I'm not sure you need to declare variants you aren't using but in any case you should test the missing ones e.g. crypto.eor3u.v8i16.
Maybe there's an argument that it isn't very useful to test them all like this but the other files follow this pattern so might as well.

[rsanthir.quic]

Minor corrections and removed unused code, also added complete testing

[rsanthir.quic]

llvm/lib/Target/AArch64/AArch64InstrInfo.td

928

a good catch thank you!

[DavidSpickett]

LGTM

[rsanthir.quic]

Thank you for reviewing this @DavidSpickett ! If you get a chance could you commit this for me? I do not have commit access yet.

[DavidSpickett]

clang/test/CodeGen/aarch64-neon-range-checks.c

31

I added a "}" here. Please run ninja check-clang and/or check-llvm before you update patches.

clang/include/clang/Basic/arm_neon.td

 
 def SHA1C : SInst<"vsha1c", "..1.", "QUi">;
 def SHA1P : SInst<"vsha1p", "..1.", "QUi">;
 def SHA1M : SInst<"vsha1m", "..1.", "QUi">;
 def SHA1SU0 : SInst<"vsha1su0", "....", "QUi">;
 def SHA256H : SInst<"vsha256h", "....", "QUi">;
 def SHA256H2 : SInst<"vsha256h2", "....", "QUi">;
 def SHA256SU1 : SInst<"vsha256su1", "....", "QUi">;
+
+def BCAX : SInst<"vbcax", "....", "QUcQUsQUiQUlQsQcQiQl">;
+def EOR3 : SInst<"veor3", "....", "QUcQUsQUiQUlQsQcQiQl">;
+def RAX1 : SInst<"vrax1", "...", "QUl">;
+
+let isVXAR = 1 in {
+def XAR :  SInst<"vxar", "...I", "QUl">;
+}
+
+def SHA512SU0 : SInst<"vsha512su0", "...", "QUl">;
+def SHA512su1 : SInst<"vsha512su1", "....", "QUl">;
+def SHA512H : SInst<"vsha512h", "....", "QUl">;
+def SHA512H2 : SInst<"vsha512h2", "....", "QUl">;
 }
 
 ////////////////////////////////////////////////////////////////////////////////
 // Float -> Int conversions with explicit rounding mode
 
 let ArchGuard = "__ARM_ARCH >= 8" in {
 def FCVTNS_S32 : SInst<"vcvtn_s32", "S.", "fQf">;
 def FCVTNU_S32 : SInst<"vcvtn_u32", "U.", "fQf">;

clang/include/clang/Basic/arm_neon_incl.td

class Inst <string n, string p, string t, Operation o> {
   string ArchGuard = "";
 
   Operation Operation = o;
   bit BigEndianSafe = 0;
   bit isShift = 0;
   bit isScalarShift = 0;
   bit isScalarNarrowShift = 0;
   bit isVCVT_N = 0;
+  bit isVXAR = 0;
   // For immediate checks: the immediate will be assumed to specify the lane of
   // a Q register. Only used for intrinsics which end up calling polymorphic
   // builtins.
   bit isLaneQ = 0;
 
   // Certain intrinsics have different names than their representative
   // instructions. This field allows us to handle this correctly when we
   // are generating tests.

clang/lib/CodeGen/CGBuiltin.cpp

static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP0(splatq_laneq_v),
   NEONMAP1(vabs_v, aarch64_neon_abs, 0),
   NEONMAP1(vabsq_v, aarch64_neon_abs, 0),
   NEONMAP0(vaddhn_v),
   NEONMAP1(vaesdq_v, aarch64_crypto_aesd, 0),
   NEONMAP1(vaeseq_v, aarch64_crypto_aese, 0),
   NEONMAP1(vaesimcq_v, aarch64_crypto_aesimc, 0),
   NEONMAP1(vaesmcq_v, aarch64_crypto_aesmc, 0),
+  NEONMAP2(vbcaxq_v, aarch64_crypto_bcaxu, aarch64_crypto_bcaxs, Add1ArgType | UnsignedAlts),
   NEONMAP1(vbfdot_v, aarch64_neon_bfdot, 0),
   NEONMAP1(vbfdotq_v, aarch64_neon_bfdot, 0),
   NEONMAP1(vbfmlalbq_v, aarch64_neon_bfmlalb, 0),
   NEONMAP1(vbfmlaltq_v, aarch64_neon_bfmlalt, 0),
   NEONMAP1(vbfmmlaq_v, aarch64_neon_bfmmla, 0),
   NEONMAP1(vcadd_rot270_v, aarch64_neon_vcadd_rot270, Add1ArgType),
   NEONMAP1(vcadd_rot90_v, aarch64_neon_vcadd_rot90, Add1ArgType),
   NEONMAP1(vcaddq_rot270_v, aarch64_neon_vcadd_rot270, Add1ArgType),
static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP1(vcvtq_n_s32_v, aarch64_neon_vcvtfp2fxs, 0),
   NEONMAP1(vcvtq_n_s64_v, aarch64_neon_vcvtfp2fxs, 0),
   NEONMAP1(vcvtq_n_u16_v, aarch64_neon_vcvtfp2fxu, 0),
   NEONMAP1(vcvtq_n_u32_v, aarch64_neon_vcvtfp2fxu, 0),
   NEONMAP1(vcvtq_n_u64_v, aarch64_neon_vcvtfp2fxu, 0),
   NEONMAP1(vcvtx_f32_v, aarch64_neon_fcvtxn, AddRetType | Add1ArgType),
   NEONMAP2(vdot_v, aarch64_neon_udot, aarch64_neon_sdot, 0),
   NEONMAP2(vdotq_v, aarch64_neon_udot, aarch64_neon_sdot, 0),
+  NEONMAP2(veor3q_v, aarch64_crypto_eor3u, aarch64_crypto_eor3s, Add1ArgType | UnsignedAlts),
   NEONMAP0(vext_v),
   NEONMAP0(vextq_v),
   NEONMAP0(vfma_v),
   NEONMAP0(vfmaq_v),
   NEONMAP1(vfmlal_high_v, aarch64_neon_fmlal2, 0),
   NEONMAP1(vfmlal_low_v, aarch64_neon_fmlal, 0),
   NEONMAP1(vfmlalq_high_v, aarch64_neon_fmlal2, 0),
   NEONMAP1(vfmlalq_low_v, aarch64_neon_fmlal, 0),
static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP2(vqshl_v, aarch64_neon_uqshl, aarch64_neon_sqshl, Add1ArgType | UnsignedAlts),
   NEONMAP2(vqshlq_n_v, aarch64_neon_uqshl, aarch64_neon_sqshl,UnsignedAlts),
   NEONMAP2(vqshlq_v, aarch64_neon_uqshl, aarch64_neon_sqshl, Add1ArgType | UnsignedAlts),
   NEONMAP1(vqshlu_n_v, aarch64_neon_sqshlu, 0),
   NEONMAP1(vqshluq_n_v, aarch64_neon_sqshlu, 0),
   NEONMAP2(vqsub_v, aarch64_neon_uqsub, aarch64_neon_sqsub, Add1ArgType | UnsignedAlts),
   NEONMAP2(vqsubq_v, aarch64_neon_uqsub, aarch64_neon_sqsub, Add1ArgType | UnsignedAlts),
   NEONMAP1(vraddhn_v, aarch64_neon_raddhn, Add1ArgType),
+  NEONMAP1(vrax1q_v, aarch64_crypto_rax1, 0),
   NEONMAP2(vrecpe_v, aarch64_neon_frecpe, aarch64_neon_urecpe, 0),
   NEONMAP2(vrecpeq_v, aarch64_neon_frecpe, aarch64_neon_urecpe, 0),
   NEONMAP1(vrecps_v, aarch64_neon_frecps, Add1ArgType),
   NEONMAP1(vrecpsq_v, aarch64_neon_frecps, Add1ArgType),
   NEONMAP2(vrhadd_v, aarch64_neon_urhadd, aarch64_neon_srhadd, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrhaddq_v, aarch64_neon_urhadd, aarch64_neon_srhadd, Add1ArgType | UnsignedAlts),
   NEONMAP0(vrndi_v),
   NEONMAP0(vrndiq_v),
   NEONMAP2(vrshl_v, aarch64_neon_urshl, aarch64_neon_srshl, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrshlq_v, aarch64_neon_urshl, aarch64_neon_srshl, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrshr_n_v, aarch64_neon_urshl, aarch64_neon_srshl, UnsignedAlts),
   NEONMAP2(vrshrq_n_v, aarch64_neon_urshl, aarch64_neon_srshl, UnsignedAlts),
   NEONMAP2(vrsqrte_v, aarch64_neon_frsqrte, aarch64_neon_ursqrte, 0),
   NEONMAP2(vrsqrteq_v, aarch64_neon_frsqrte, aarch64_neon_ursqrte, 0),
   NEONMAP1(vrsqrts_v, aarch64_neon_frsqrts, Add1ArgType),
   NEONMAP1(vrsqrtsq_v, aarch64_neon_frsqrts, Add1ArgType),
   NEONMAP1(vrsubhn_v, aarch64_neon_rsubhn, Add1ArgType),
   NEONMAP1(vsha1su0q_v, aarch64_crypto_sha1su0, 0),
   NEONMAP1(vsha1su1q_v, aarch64_crypto_sha1su1, 0),
   NEONMAP1(vsha256h2q_v, aarch64_crypto_sha256h2, 0),
   NEONMAP1(vsha256hq_v, aarch64_crypto_sha256h, 0),
   NEONMAP1(vsha256su0q_v, aarch64_crypto_sha256su0, 0),
   NEONMAP1(vsha256su1q_v, aarch64_crypto_sha256su1, 0),
+  NEONMAP1(vsha512h2q_v, aarch64_crypto_sha512h2, 0),
+  NEONMAP1(vsha512hq_v, aarch64_crypto_sha512h, 0),
+  NEONMAP1(vsha512su0q_v, aarch64_crypto_sha512su0, 0),
+  NEONMAP1(vsha512su1q_v, aarch64_crypto_sha512su1, 0),
   NEONMAP0(vshl_n_v),
   NEONMAP2(vshl_v, aarch64_neon_ushl, aarch64_neon_sshl, Add1ArgType | UnsignedAlts),
   NEONMAP0(vshll_n_v),
   NEONMAP0(vshlq_n_v),
   NEONMAP2(vshlq_v, aarch64_neon_ushl, aarch64_neon_sshl, Add1ArgType | UnsignedAlts),
   NEONMAP0(vshr_n_v),
   NEONMAP0(vshrn_n_v),
   NEONMAP0(vshrq_n_v),
   NEONMAP1(vst1_x2_v, aarch64_neon_st1x2, 0),
   NEONMAP1(vst1_x3_v, aarch64_neon_st1x3, 0),
   NEONMAP1(vst1_x4_v, aarch64_neon_st1x4, 0),
   NEONMAP1(vst1q_x2_v, aarch64_neon_st1x2, 0),
   NEONMAP1(vst1q_x3_v, aarch64_neon_st1x3, 0),
   NEONMAP1(vst1q_x4_v, aarch64_neon_st1x4, 0),
   NEONMAP0(vsubhn_v),
   NEONMAP0(vtst_v),
   NEONMAP0(vtstq_v),
   NEONMAP1(vusdot_v, aarch64_neon_usdot, 0),
   NEONMAP1(vusdotq_v, aarch64_neon_usdot, 0),
   NEONMAP1(vusmmlaq_v, aarch64_neon_usmmla, 0),
+  NEONMAP1(vxarq_v, aarch64_crypto_xar, 0),
 };
 
 static const ARMVectorIntrinsicInfo AArch64SISDIntrinsicMap[] = {
   NEONMAP1(vabdd_f64, aarch64_sisd_fabd, Add1ArgType),
   NEONMAP1(vabds_f32, aarch64_sisd_fabd, Add1ArgType),
   NEONMAP1(vabsd_s64, aarch64_neon_abs, Add1ArgType),
   NEONMAP1(vaddlv_s32, aarch64_neon_saddlv, AddRetType | Add1ArgType),
   NEONMAP1(vaddlv_u32, aarch64_neon_uaddlv, AddRetType | Add1ArgType),
case NEON::BI__builtin_neon_vrndiq_v:
     Int = Builder.getIsFPConstrained()
               ? Intrinsic::experimental_constrained_nearbyint
               : Intrinsic::nearbyint;
     return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, NameHint);
   case NEON::BI__builtin_neon_vrshr_n_v:
   case NEON::BI__builtin_neon_vrshrq_n_v:
     return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrshr_n",
                         1, true);
+  case NEON::BI__builtin_neon_vsha512hq_v:
+  case NEON::BI__builtin_neon_vsha512h2q_v:
+  case NEON::BI__builtin_neon_vsha512su0q_v:
+  case NEON::BI__builtin_neon_vsha512su1q_v: {
+    Function *F = CGM.getIntrinsic(Int);
+    return EmitNeonCall(F, Ops, "");
+  }
   case NEON::BI__builtin_neon_vshl_n_v:
   case NEON::BI__builtin_neon_vshlq_n_v:
     Ops[1] = EmitNeonShiftVector(Ops[1], Ty, false);
     return Builder.CreateShl(Builder.CreateBitCast(Ops[0],Ty), Ops[1],
                              "vshl_n");
   case NEON::BI__builtin_neon_vshll_n_v: {
     llvm::FixedVectorType *SrcTy =
         llvm::FixedVectorType::getTruncatedElementVectorType(VTy);
for (unsigned vi = 0; vi != 2; ++vi) {
         Indices.push_back(2*i+vi);
 
       Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
       SV = Builder.CreateShuffleVector(Ops[1], Ops[2], Indices, "vuzp");
       SV = Builder.CreateDefaultAlignedStore(SV, Addr);
     }
     return SV;
   }
+  case NEON::BI__builtin_neon_vxarq_v: {
+    Function *F = CGM.getIntrinsic(Int);
+    Ops[2] = Builder.CreateZExt(Ops[2], Int64Ty);
+    return EmitNeonCall(F, Ops, "");
+  }
   case NEON::BI__builtin_neon_vzip_v:
   case NEON::BI__builtin_neon_vzipq_v: {
     Ops[0] = Builder.CreateBitCast(Ops[0], llvm::PointerType::getUnqual(Ty));
     Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
     Ops[2] = Builder.CreateBitCast(Ops[2], Ty);
     Value *SV = nullptr;
 
     for (unsigned vi = 0; vi != 2; ++vi) {

clang/test/CodeGen/aarch64-neon-range-checks.c

+// RUN: %clang_cc1 -triple aarch64-linux-gnu -target-feature +neon -target-feature +crypto -verify %s
+
+#include <arm_neon.h>
+
+void test_range_check_xar(uint64x2_t a, uint64x2_t b) {
+  vxarq_u64(a, b, -1); // expected-error {{argument value -1 is outside the valid range [0, 63]}}
+  vxarq_u64(a, b, 64); // expected-error {{argument value 64 is outside the valid range [0, 63]}}
+  vxarq_u64(a, b, 0);
+  vxarq_u64(a, b, 63);
+}
+

clang/test/CodeGen/aarch64-neon-sha3.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -target-feature +neon \
+// RUN:  -target-feature +crypto -S -emit-llvm -o - %s \
+// RUN:  | FileCheck %s
+
+#include <arm_neon.h>
+
+// CHECK-LABEL: @test_vsha512h(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.sha512h
+//
+void test_vsha512h(uint64x2_t hash_ed, uint64x2_t hash_gf, uint64x2_t kwh_kwh2) {
+  uint64x2_t result = vsha512hq_u64(hash_ed, hash_gf, kwh_kwh2);
+}
+
+// CHECK-LABEL: @test_vsha512h2(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.sha512h2
+//
+void test_vsha512h2(uint64x2_t sum_ab, uint64x2_t hash_c_, uint64x2_t hash_ab) {
+  uint64x2_t result = vsha512h2q_u64(sum_ab, hash_c_, hash_ab);
+}
+
+// CHECK-LABEL: @test_vsha512su0(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.sha512su0
+//
+void test_vsha512su0(uint64x2_t w0_1, uint64x2_t w2_) {
+
+  uint64x2_t result = vsha512su0q_u64(w0_1, w2_);
+}
+
+// CHECK-LABEL: @test_vsha512su1(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.sha512su1
+//
+void test_vsha512su1(uint64x2_t s01_s02, uint64x2_t w14_15, uint64x2_t w9_10) {
+
+  uint64x2_t result = vsha512su1q_u64(s01_s02, w14_15, w9_10);
+}
+
+// CHECK-LABEL: @test_vrax1(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.rax1
+//
+void test_vrax1(uint64x2_t a, uint64x2_t b) {
+
+  uint64x2_t result = vrax1q_u64(a, b);
+}
+
+
+// CHECK-LABEL: @test_xar(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.xar
+//
+void test_xar(uint64x2_t a, uint64x2_t b) {
+
+  uint64x2_t result = vxarq_u64(a, b, 10);
+}
+
+
+// CHECK-LABEL: @test_vbcax_u8(
+// CHECK:    call <16 x i8> @llvm.aarch64.crypto.bcaxu.v16i8
+//
+void test_vbcax_u8(uint8x16_t a, uint8x16_t b, uint8x16_t c) {
+  uint8x16_t result = vbcaxq_u8(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_u16(
+// CHECK:    call <8 x i16> @llvm.aarch64.crypto.bcaxu.v8i16
+//
+void test_vbcax_u16(uint16x8_t a, uint16x8_t b, uint16x8_t c) {
+  uint16x8_t result = vbcaxq_u16(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_u32(
+// CHECK:    call <4 x i32> @llvm.aarch64.crypto.bcaxu.v4i32
+//
+void test_vbcax_u32(uint32x4_t a, uint32x4_t b, uint32x4_t c) {
+  uint32x4_t result = vbcaxq_u32(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_u64(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.bcaxu.v2i64
+//
+void test_vbcax_u64(uint64x2_t a, uint64x2_t b, uint64x2_t c) {
+  uint64x2_t result = vbcaxq_u64(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_s8(
+// CHECK:    call <16 x i8> @llvm.aarch64.crypto.bcaxs.v16i8
+//
+void test_vbcax_s8(int8x16_t a, int8x16_t b, int8x16_t c) {
+  int8x16_t result = vbcaxq_s8(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_s16(
+// CHECK:    call <8 x i16> @llvm.aarch64.crypto.bcaxs.v8i16
+//
+void test_vbcax_s16(int16x8_t a, int16x8_t b, int16x8_t c) {
+  int16x8_t result = vbcaxq_s16(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_s32(
+// CHECK:    call <4 x i32> @llvm.aarch64.crypto.bcaxs.v4i32
+//
+void test_vbcax_s32(int32x4_t a, int32x4_t b, int32x4_t c) {
+  int32x4_t result = vbcaxq_s32(a, b, c);
+}
+
+// CHECK-LABEL: @test_vbcax_s64(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.bcaxs.v2i64
+//
+void test_vbcax_s64(int64x2_t a, int64x2_t b, int64x2_t c) {
+  int64x2_t result = vbcaxq_s64(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_u8(
+// CHECK:    call <16 x i8> @llvm.aarch64.crypto.eor3u.v16i8
+//
+void test_veor3_u8(uint8x16_t a, uint8x16_t b, uint8x16_t c) {
+  uint8x16_t result = veor3q_u8(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_u16(
+// CHECK:    call <8 x i16> @llvm.aarch64.crypto.eor3u.v8i16
+//
+void test_veor3_u16(uint16x8_t a, uint16x8_t b, uint16x8_t c) {
+  uint16x8_t result = veor3q_u16(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_u32(
+// CHECK:    call <4 x i32> @llvm.aarch64.crypto.eor3u.v4i32
+//
+void test_veor3_u32(uint32x4_t a, uint32x4_t b, uint32x4_t c) {
+  uint32x4_t result = veor3q_u32(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_u64(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.eor3u.v2i64
+//
+void test_veor3_u64(uint64x2_t a, uint64x2_t b, uint64x2_t c) {
+  uint64x2_t result = veor3q_u64(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_s8(
+// CHECK:    call <16 x i8> @llvm.aarch64.crypto.eor3s.v16i8
+//
+void test_veor3_s8(int8x16_t a, int8x16_t b, int8x16_t c) {
+  int8x16_t result = veor3q_s8(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_s16(
+// CHECK:     call <8 x i16> @llvm.aarch64.crypto.eor3s.v8i16
+//
+void test_veor3_s16(int16x8_t a, int16x8_t b, int16x8_t c) {
+  int16x8_t result = veor3q_s16(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_s32(
+// CHECK:    call <4 x i32> @llvm.aarch64.crypto.eor3s.v4i32
+//
+void test_veor3_s32(int32x4_t a, int32x4_t b, int32x4_t c) {
+  int32x4_t result = veor3q_s32(a, b, c);
+}
+
+// CHECK-LABEL: @test_veor3_s64(
+// CHECK:    call <2 x i64> @llvm.aarch64.crypto.eor3s.v2i64
+//
+void test_veor3_s64(int64x2_t a, int64x2_t b, int64x2_t c) {
+  int64x2_t result = veor3q_s64(a, b, c);
+}

clang/utils/TableGen/NeonEmitter.cpp

for (auto *Def : Defs) {
     if (!Def->hasImmediate())
       continue;
     if (Emitted.find(Def->getMangledName()) != Emitted.end())
       continue;
 
     std::string LowerBound, UpperBound;
 
     Record *R = Def->getRecord();
     if (R->getValueAsBit("isVCVT_N")) {
       // VCVT between floating- and fixed-point values takes an immediate
       // in the range [1, 32) for f32 or [1, 64) for f64 or [1, 16) for f16.
       LowerBound = "1";
     if (Def->getBaseType().getElementSizeInBits() == 16 ||
       Def->getName().find('h') != std::string::npos)
     // VCVTh operating on FP16 intrinsics in range [1, 16)
     UpperBound = "15";
     else if (Def->getBaseType().getElementSizeInBits() == 32)
         UpperBound = "31";
     else
         UpperBound = "63";
+    } else if (R->getValueAsBit("isVXAR")) {
+      // VXAR takes an immediate in the range [0, 63]
+      LowerBound = "0";
+      UpperBound = "63";
     } else if (R->getValueAsBit("isScalarShift")) {
       // Right shifts have an 'r' in the name, left shifts do not. Convert
       // instructions have the same bounds and right shifts.
       if (Def->getName().find('r') != std::string::npos ||
           Def->getName().find("cvt") != std::string::npos)
         LowerBound = "1";
 
       UpperBound = utostr(Def->getReturnType().getElementSizeInBits() - 1);

llvm/include/llvm/IR/IntrinsicsAArch64.td

let TargetPrefix = "aarch64" in {
   class Crypto_SHA_12Schedule_Intrinsic
     : DefaultAttrsIntrinsic<[llvm_v4i32_ty], [llvm_v4i32_ty, llvm_v4i32_ty, llvm_v4i32_ty],
                 [IntrNoMem]>;
 
   // SHA intrinsic taking 8 words of the hash and 4 of the schedule.
   class Crypto_SHA_8Hash4Schedule_Intrinsic
     : DefaultAttrsIntrinsic<[llvm_v4i32_ty], [llvm_v4i32_ty, llvm_v4i32_ty, llvm_v4i32_ty],
                 [IntrNoMem]>;
+
+  // SHA512 intrinsic taking 2 arguments
+  class Crypto_SHA512_2Arg_Intrinsic
+    : DefaultAttrsIntrinsic<[llvm_v2i64_ty], [llvm_v2i64_ty, llvm_v2i64_ty], [IntrNoMem]>;
+
+  // SHA512 intrinsic taking 3 Arguments
+  class Crypto_SHA512_3Arg_Intrinsic
+    : DefaultAttrsIntrinsic<[llvm_v2i64_ty], [llvm_v2i64_ty, llvm_v2i64_ty, llvm_v2i64_ty],
+                [IntrNoMem]>;
+
+  // SHA3 Intrinsics taking 3 arguments
+  class Crypto_SHA3_3Arg_Intrinsic
+    : DefaultAttrsIntrinsic<[llvm_anyvector_ty],
+               [LLVMMatchType<0>, LLVMMatchType<0>, LLVMMatchType<0>],
+               [IntrNoMem]>;
+
+  // SHA3 Intrinsic taking 2 arguments
+  class Crypto_SHA3_2Arg_Intrinsic
+    : DefaultAttrsIntrinsic<[llvm_v2i64_ty], [llvm_v2i64_ty, llvm_v2i64_ty],
+               [IntrNoMem]>;
+
+  // SHA3 Intrinsic taking 3 Arguments 1 immediate
+  class Crypto_SHA3_2ArgImm_Intrinsic
+    : DefaultAttrsIntrinsic<[llvm_v2i64_ty], [llvm_v2i64_ty, llvm_v2i64_ty, llvm_i64_ty],
+               [IntrNoMem, ImmArg<ArgIndex<2>>]>;
 }
 
 // AES
 def int_aarch64_crypto_aese   : Crypto_AES_DataKey_Intrinsic;
 def int_aarch64_crypto_aesd   : Crypto_AES_DataKey_Intrinsic;
 def int_aarch64_crypto_aesmc  : Crypto_AES_Data_Intrinsic;
 def int_aarch64_crypto_aesimc : Crypto_AES_Data_Intrinsic;
 
 // SHA1
 def int_aarch64_crypto_sha1c  : Crypto_SHA_5Hash4Schedule_Intrinsic;
 def int_aarch64_crypto_sha1p  : Crypto_SHA_5Hash4Schedule_Intrinsic;
 def int_aarch64_crypto_sha1m  : Crypto_SHA_5Hash4Schedule_Intrinsic;
 def int_aarch64_crypto_sha1h  : Crypto_SHA_1Hash_Intrinsic;
 
 def int_aarch64_crypto_sha1su0 : Crypto_SHA_12Schedule_Intrinsic;
 def int_aarch64_crypto_sha1su1 : Crypto_SHA_8Schedule_Intrinsic;
 
 // SHA256
 def int_aarch64_crypto_sha256h   : Crypto_SHA_8Hash4Schedule_Intrinsic;
 def int_aarch64_crypto_sha256h2  : Crypto_SHA_8Hash4Schedule_Intrinsic;
 def int_aarch64_crypto_sha256su0 : Crypto_SHA_8Schedule_Intrinsic;
 def int_aarch64_crypto_sha256su1 : Crypto_SHA_12Schedule_Intrinsic;
 
+//SHA3
+def int_aarch64_crypto_eor3s : Crypto_SHA3_3Arg_Intrinsic;
+def int_aarch64_crypto_eor3u : Crypto_SHA3_3Arg_Intrinsic;
+def int_aarch64_crypto_bcaxs : Crypto_SHA3_3Arg_Intrinsic;
+def int_aarch64_crypto_bcaxu : Crypto_SHA3_3Arg_Intrinsic;
+def int_aarch64_crypto_rax1 : Crypto_SHA3_2Arg_Intrinsic;
+def int_aarch64_crypto_xar : Crypto_SHA3_2ArgImm_Intrinsic;
+
+// SHA512
+def int_aarch64_crypto_sha512h : Crypto_SHA512_3Arg_Intrinsic;
+def int_aarch64_crypto_sha512h2 : Crypto_SHA512_3Arg_Intrinsic;
+def int_aarch64_crypto_sha512su0 : Crypto_SHA512_2Arg_Intrinsic;
+def int_aarch64_crypto_sha512su1 : Crypto_SHA512_3Arg_Intrinsic;
+
 //===----------------------------------------------------------------------===//
 // CRC32
 
 let TargetPrefix = "aarch64" in {
 
 def int_aarch64_crc32b  : DefaultAttrsIntrinsic<[llvm_i32_ty], [llvm_i32_ty, llvm_i32_ty],
     [IntrNoMem]>;
 def int_aarch64_crc32cb : DefaultAttrsIntrinsic<[llvm_i32_ty], [llvm_i32_ty, llvm_i32_ty],

llvm/lib/Target/AArch64/AArch64InstrFormats.td

 
 // imm0_63 predicate - True if the immediate is in the range [0,63]
 def imm0_63 : Operand<i64>, ImmLeaf<i64, [{
   return ((uint64_t)Imm) < 64;
 }]> {
   let ParserMatchClass = Imm0_63Operand;
 }
 
+def timm0_63 : Operand<i64>, TImmLeaf<i64, [{
+  return ((uint64_t)Imm) < 64;
+}]> {
+  let ParserMatchClass = Imm0_63Operand;
+}
+
 // imm0_31 predicate - True if the immediate is in the range [0,31]
 def imm0_31 : Operand<i64>, ImmLeaf<i64, [{
   return ((uint64_t)Imm) < 32;
 }]> {
   let ParserMatchClass = Imm0_31Operand;
 }
 
 // timm0_31 predicate - same ass imm0_31, but use TargetConstant (TimmLeaf)
class BaseCryptoV82<dag oops, dag iops, string asm, string asmops, string cst,
   bits<5> Vd;
   bits<5> Vn;
   let Inst{31-25} = 0b1100111;
   let Inst{9-5}   = Vn;
   let Inst{4-0}   = Vd;
 }
 
 class CryptoRRTied<bits<1>op0, bits<2>op1, string asm, string asmops>
-  : BaseCryptoV82<(outs V128:$Vd), (ins V128:$Vn, V128:$Vm), asm, asmops,
-                  "$Vm = $Vd", []> {
+  : BaseCryptoV82<(outs V128:$Vdst), (ins V128:$Vd, V128:$Vn), asm, asmops,
+                  "$Vd = $Vdst", []> {
   let Inst{31-25} = 0b1100111;
   let Inst{24-21} = 0b0110;
   let Inst{20-15} = 0b000001;
   let Inst{14}    = op0;
   let Inst{13-12} = 0b00;
   let Inst{11-10} = op1;
 }
 class CryptoRRTied_2D<bits<1>op0, bits<2>op1, string asm>

llvm/lib/Target/AArch64/AArch64InstrInfo.td

 def SHA512H   : CryptoRRRTied<0b0, 0b00, "sha512h">;
 def SHA512H2  : CryptoRRRTied<0b0, 0b01, "sha512h2">;
 def SHA512SU0 : CryptoRRTied_2D<0b0, 0b00, "sha512su0">;
 def SHA512SU1 : CryptoRRRTied_2D<0b0, 0b10, "sha512su1">;
 def RAX1      : CryptoRRR_2D<0b0,0b11, "rax1">;
 def EOR3      : CryptoRRRR_16B<0b00, "eor3">;
 def BCAX      : CryptoRRRR_16B<0b01, "bcax">;
 def XAR       : CryptoRRRi6<"xar">;
+
+class SHA512H_pattern<Instruction INST, Intrinsic OpNode>
+  : Pat<(v2i64 (OpNode (v2i64 V128:$Vd), (v2i64 V128:$Vn), (v2i64 V128:$Vm))),
+        (INST (v2i64 FPR128:$Vd), (v2i64 FPR128:$Vn), (v2i64 V128:$Vm))>;
+
+class SHA3_pattern<Instruction INST, Intrinsic OpNode, ValueType VecTy>
+  : Pat<(VecTy (OpNode (VecTy V128:$Vd), (VecTy V128:$Vn), (VecTy V128:$Vm))),
+        (INST (VecTy V128:$Vd), (VecTy V128:$Vn), (VecTy V128:$Vm))>;
+
+def : Pat<(v2i64 (int_aarch64_crypto_sha512su0 (v2i64 V128:$Vn), (v2i64 V128:$Vm))),
+          (SHA512SU0 (v2i64 V128:$Vn), (v2i64 V128:$Vm))>;
+
+def : SHA3_pattern<SHA512H, int_aarch64_crypto_sha512h, v2i64>;
+def : SHA3_pattern<SHA512H2, int_aarch64_crypto_sha512h2, v2i64>;
+def : SHA3_pattern<SHA512SU1, int_aarch64_crypto_sha512su1, v2i64>;
+
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3u, v16i8>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3u, v8i16>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3u, v4i32>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3u, v2i64>;
+
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxu, v16i8>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxu, v8i16>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxu, v4i32>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxu, v2i64>;
+
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3s, v16i8>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3s, v8i16>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3s, v4i32>;
+def : SHA3_pattern<EOR3, int_aarch64_crypto_eor3s, v2i64>;
+
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxs, v16i8>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxs, v8i16>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxs, v4i32>;
+def : SHA3_pattern<BCAX, int_aarch64_crypto_bcaxs, v2i64>;
+
+def : Pat<(v2i64 (int_aarch64_crypto_rax1 (v2i64 V128:$Vn), (v2i64 V128:$Vm))),
+          (RAX1 (v2i64 V128:$Vn), (v2i64 V128:$Vm))>;
+
+def : Pat<(v2i64 (int_aarch64_crypto_xar (v2i64 V128:$Vn), (v2i64 V128:$Vm), (i64 timm0_63:$imm))),
+          (XAR (v2i64 V128:$Vn), (v2i64 V128:$Vm), (timm0_63:$imm))>;
+
+
 } // HasSHA3
 
 let Predicates = [HasSM4] in {
 def SM3TT1A   : CryptoRRRi2Tied<0b0, 0b00, "sm3tt1a">;
 def SM3TT1B   : CryptoRRRi2Tied<0b0, 0b01, "sm3tt1b">;
 def SM3TT2A   : CryptoRRRi2Tied<0b0, 0b10, "sm3tt2a">;
 def SM3TT2B   : CryptoRRRi2Tied<0b0, 0b11, "sm3tt2b">;
 def SM3SS1    : CryptoRRRR_4S<0b10, "sm3ss1">;

llvm/test/CodeGen/AArch64/neon-sha3.ll

+; RUN: llc %s -mtriple=aarch64 -mattr=+v8.3a,+sha3 -o - | FileCheck %s
+
+define <2 x i64> @test_vsha512h(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
+; CHECK-LABEL: test_vsha512h:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    sha512h q0, q1, v2.2d
+; CHECK-NEXT:    ret
+entry:
+  %vsha512h.i = tail call <2 x i64> @llvm.aarch64.crypto.sha512h(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c)
+  ret <2 x i64> %vsha512h.i
+}
+
+define <2 x i64> @test_vsha512h2(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
+; CHECK-LABEL: test_vsha512h2:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    sha512h2 q0, q1, v2.2d
+; CHECK-NEXT:    ret
+entry:
+  %vsha512h2.i = tail call <2 x i64> @llvm.aarch64.crypto.sha512h2(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c)
+  ret <2 x i64> %vsha512h2.i
+}
+
+define <2 x i64> @test_vsha512su0(<2 x i64> %a, <2 x i64> %b) {
+; CHECK-LABEL: test_vsha512su0:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    sha512su0 v0.2d, v1.2d
+; CHECK-NEXT:    ret
+entry:
+  %vsha512su0.i = tail call <2 x i64> @llvm.aarch64.crypto.sha512su0(<2 x i64> %a, <2 x i64> %b)
+  ret <2 x i64> %vsha512su0.i
+}
+
+define <2 x i64> @test_vsha512su1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
+; CHECK-LABEL: test_vsha512su1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    sha512su1 v0.2d, v1.2d, v2.2d
+; CHECK-NEXT:    ret
+entry:
+  %vsha512su1.i = tail call <2 x i64> @llvm.aarch64.crypto.sha512su1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c)
+  ret <2 x i64> %vsha512su1.i
+}
+
+define <2 x i64> @test_vrax1(<2 x i64> %a, <2 x i64> %b) {
+; CHECK-LABEL: test_vrax1:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    rax1 v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    ret
+entry:
+  %vrax1.i = tail call <2 x i64> @llvm.aarch64.crypto.rax1(<2 x i64> %a, <2 x i64> %b)
+  ret <2 x i64> %vrax1.i
+}
+
+define <2 x i64> @test_vxar(<2 x i64> %a,  <2 x i64> %b) {
+; CHECK-LABEL: test_vxar:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    xar v0.2d, v0.2d, v1.2d, #1
+; CHECK-NEXT:    ret
+entry:
+  %vxar.i = tail call  <2 x i64> @llvm.aarch64.crypto.xar(<2 x i64> %a, <2 x i64> %b, i64 1)
+  ret <2 x i64> %vxar.i
+}
+
+define <16 x i8> @test_bcax_8(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c) {
+; CHECK-LABEL: test_bcax_8:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    bcax v0.16b, v0.16b, v1.16b, v2.16b
+; CHECK-NEXT:    ret
+entry:
+  %vbcax_8.i = tail call <16 x i8> @llvm.aarch64.crypto.bcaxu.v16i8(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c)
+  ret <16 x i8> %vbcax_8.i
+}
+
+define <16 x i8> @test_eor3_8(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c) {
+; CHECK-LABEL: test_eor3_8:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    eor3 v0.16b, v0.16b, v1.16b, v2.16b
+; CHECK-NEXT:    ret
+entry:
+  %veor3_8.i = tail call <16 x i8> @llvm.aarch64.crypto.eor3u.v16i8(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c)
+  ret <16 x i8> %veor3_8.i
+}
+
+declare <2 x i64> @llvm.aarch64.crypto.sha512h(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <2 x i64> @llvm.aarch64.crypto.sha512h2(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <2 x i64> @llvm.aarch64.crypto.sha512su0(<2 x i64>, <2 x i64>)
+declare <2 x i64> @llvm.aarch64.crypto.sha512su1(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <2 x i64> @llvm.aarch64.crypto.rax1(<2 x i64>, <2 x i64>)
+declare <2 x i64> @llvm.aarch64.crypto.xar(<2 x i64>, <2 x i64>, i64 immarg)
+declare <16 x i8> @llvm.aarch64.crypto.bcaxu.v16i8(<16 x i8>, <16 x i8>, <16 x i8>)
+declare <8 x i16> @llvm.aarch64.crypto.bcaxu.v8i16(<8 x i16>, <8 x i16>, <8 x i16>)
+declare <4 x i32> @llvm.aarch64.crypto.bcaxu.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
+declare <2 x i64> @llvm.aarch64.crypto.bcaxu.v2i64(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <16 x i8> @llvm.aarch64.crypto.bcaxs.v16i8(<16 x i8>, <16 x i8>, <16 x i8>)
+declare <8 x i16> @llvm.aarch64.crypto.bcaxs.v8i16(<8 x i16>, <8 x i16>, <8 x i16>)
+declare <4 x i32> @llvm.aarch64.crypto.bcaxs.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
+declare <2 x i64> @llvm.aarch64.crypto.bcaxs.v2i64(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <16 x i8> @llvm.aarch64.crypto.eor3u.v16i8(<16 x i8>, <16 x i8>, <16 x i8>)
+declare <8 x i16> @llvm.aarch64.crypto.eor3u.v8i16(<8 x i16>, <8 x i16>, <8 x i16>)
+declare <4 x i32> @llvm.aarch64.crypto.eor3u.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
+declare <2 x i64> @llvm.aarch64.crypto.eor3u.v2i64(<2 x i64>, <2 x i64>, <2 x i64>)
+declare <16 x i8> @llvm.aarch64.crypto.eor3s.v16i8(<16 x i8>, <16 x i8>, <16 x i8>)
+declare <8 x i16> @llvm.aarch64.crypto.eor3s.v8i16(<8 x i16>, <8 x i16>, <8 x i16>)
+declare <4 x i32> @llvm.aarch64.crypto.eor3s.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
+declare <2 x i64> @llvm.aarch64.crypto.eor3s.v2i64(<2 x i64>, <2 x i64>, <2 x i64>)
+