[AArch64] Add missing intrinsics for vrnd

[ktkachov]

clang/include/clang/Basic/arm_neon.td

1213

Clang should use the same feature guard as GCC here, as it is mandated by ACLE: __ARM_FEATURE_FRINT
See the relevant section in https://developer.arm.com/documentation/101028/0012/8--Data-processing-intrinsics

[jaykang10]

Updated feature guard

[jaykang10]

clang/include/clang/Basic/arm_neon.td

1213

Thanks for letting know!!! That was what I want to know.

[ktkachov]

clang/lib/Basic/Targets/AArch64.cpp

532–533

I don't think this flag should exist as it's inconsistent with GCC. In GCC these instructions are enabled through -march=armv8.5-a . Generally, these intrinsics should use the same target as the assembly instructions?

[jaykang10]

clang/lib/Basic/Targets/AArch64.cpp

532–533

You are right! I will update it.

[jaykang10]

Removed 'frint3264' flag

[SjoerdMeijer]

clang/lib/Basic/Targets/AArch64.cpp

185

I think we need to add a test new macro in:

Preprocessor/aarch64-target-features.c

[SjoerdMeijer]

clang/lib/Basic/Targets/AArch64.cpp

185

Sorry, wanted to say that we need to add a test for this new macro.

[jaykang10]

clang/lib/Basic/Targets/AArch64.cpp

185

The macro is used in "arm_neon.h" which is generated at building time of clang as below so I thought we do not need to add the test for it.

#if __ARM_ARCH >= 8 && defined(__aarch64__) && defined(__ARM_FEATURE_FRINT)
...
#ifdef __LITTLE_ENDIAN__
__ai float32x2_t vrnd32x_f32(float32x2_t __p0) {
  float32x2_t __ret;
  __ret = (float32x2_t) __builtin_neon_vrnd32x_v((int8x8_t)__p0, 9);
  return __ret;
}
#else
__ai float32x2_t vrnd32x_f32(float32x2_t __p0) {
  float32x2_t __rev0;  __rev0 = __builtin_shufflevector(__p0, __p0, 1, 0);
  float32x2_t __ret;
  __ret = (float32x2_t) __builtin_neon_vrnd32x_v((int8x8_t)__rev0, 9);
  __ret = __builtin_shufflevector(__ret, __ret, 1, 0);
  return __ret;
}
#endif
...

If it is not enough, I will add a test for it.

[SjoerdMeijer]

clang/lib/Basic/Targets/AArch64.cpp

185

We let the preprocessor emit a new macro, so that's what we need to test, which I believe we need to do in that file I quoted earlier. The fact that it is used elsewhere is an indirect test.

[jaykang10]

clang/lib/Basic/Targets/AArch64.cpp

185

Yep, I will add it.

[SjoerdMeijer]

llvm/include/llvm/IR/IntrinsicsAArch64.td

484

Nit and unrelated: but perhaps you can fix the indent here.

llvm/test/CodeGen/AArch64/v8.5a-neon-frint3264-intrinsic.ll

2

Nit: fptoint seems to be implied by v8.5a, so don't need to specify it here?

[jaykang10]

llvm/include/llvm/IR/IntrinsicsAArch64.td

484

Yep, I will update it.

llvm/test/CodeGen/AArch64/v8.5a-neon-frint3264-intrinsic.ll

2

Oops! You are right! I will update it.

[jaykang10]

Removed redundant command line option from test.

[SjoerdMeijer]

clang/test/Preprocessor/aarch64-target-features.c

61 ↗

(On Diff #328454)

would be good to have a negative test too, i.e. a CHECK-NOT of this for e.g. v8.4a.

[jaykang10]

clang/test/Preprocessor/aarch64-target-features.c

61 ↗

(On Diff #328454)

Yep, I will add it.

[jaykang10]

Added a test case for frint macro.

[SjoerdMeijer]

Sorry, one more question. This implements the vector variants. How about the scalar ones? Have they been implemented already?

[jaykang10]

In D97949#2606101, @SjoerdMeijer wrote:

Sorry, one more question. This implements the vector variants. How about the scalar ones? Have they been implemented already?

um... I have not checked the scalar intrinsics for fp rounding. I have checked 'vrnd32/64x' which are neon vector intrinsics for v8.5-a. If the scalar one is not implemented yet, it could be other commit. Let me have a look.

[SjoerdMeijer]

Yeah, that's fine, if they are missing, we could that separately.

Thanks, this LGTM.

[jaykang10]

Oops, I missed to add Differential Revision. It is merged with 9b302513f6d82f0ca989b3bb1f5ffc592ed866b7.

clang/include/clang/Basic/arm_neon.td

 def FRINTA_S64 : SInst<"vrnda", "..", "dQd">;
 def FRINTP_S64 : SInst<"vrndp", "..", "dQd">;
 def FRINTM_S64 : SInst<"vrndm", "..", "dQd">;
 def FRINTX_S64 : SInst<"vrndx", "..", "dQd">;
 def FRINTZ_S64 : SInst<"vrnd", "..", "dQd">;
 def FRINTI_S64 : SInst<"vrndi", "..", "dQd">;
 }
 
+let ArchGuard = "__ARM_ARCH >= 8 && defined(__aarch64__) && defined(__ARM_FEATURE_FRINT)" in {
+def FRINT32X_S32 : SInst<"vrnd32x", "..", "fQf">;
+def FRINT32Z_S32 : SInst<"vrnd32z", "..", "fQf">;
+def FRINT64X_S32 : SInst<"vrnd64x", "..", "fQf">;
+def FRINT64Z_S32 : SInst<"vrnd64z", "..", "fQf">;
+}
+
 ////////////////////////////////////////////////////////////////////////////////
 // MaxNum/MinNum Floating Point
 
 let ArchGuard = "__ARM_ARCH >= 8 && defined(__ARM_FEATURE_NUMERIC_MAXMIN)" in {
 def FMAXNM_S32 : SInst<"vmaxnm", "...", "fQf">;
 def FMINNM_S32 : SInst<"vminnm", "...", "fQf">;
 }
 

clang/lib/Basic/Targets/AArch64.h

class LLVM_LIBRARY_VISIBILITY AArch64TargetInfo : public TargetInfo {
   bool HasSVE2AES;
   bool HasSVE2SHA3;
   bool HasSVE2SM4;
   bool HasSVE2BitPerm;
   bool HasMatmulFP64;
   bool HasMatmulFP32;
   bool HasLSE;
   bool HasFlagM;
+  bool HasFRInt3264;
 
   llvm::AArch64::ArchKind ArchKind;
 
   static const Builtin::Info BuiltinInfo[];
 
   std::string ABI;
 
 public:

clang/lib/Basic/Targets/AArch64.cpp

 void AArch64TargetInfo::getTargetDefinesARMV84A(const LangOptions &Opts,
                                                 MacroBuilder &Builder) const {
   // Also include the Armv8.3 defines
   getTargetDefinesARMV83A(Opts, Builder);
 }
 
 void AArch64TargetInfo::getTargetDefinesARMV85A(const LangOptions &Opts,
                                                 MacroBuilder &Builder) const {
   // Also include the Armv8.4 defines
   getTargetDefinesARMV84A(Opts, Builder);
 }
 
 void AArch64TargetInfo::getTargetDefinesARMV86A(const LangOptions &Opts,
                                                 MacroBuilder &Builder) const {
   // Also include the Armv8.5 defines
   // FIXME: Armv8.6 makes the following extensions mandatory:
   // - __ARM_FEATURE_BF16
if (HasTME)
     Builder.defineMacro("__ARM_FEATURE_TME", "1");
 
   if (HasMatMul)
     Builder.defineMacro("__ARM_FEATURE_MATMUL_INT8", "1");
 
   if (HasLSE)
     Builder.defineMacro("__ARM_FEATURE_ATOMICS", "1");
 
+  if (HasFRInt3264)
+    Builder.defineMacro("__ARM_FEATURE_FRINT", "1");
+
   if (HasBFloat16) {
     Builder.defineMacro("__ARM_FEATURE_BF16", "1");
     Builder.defineMacro("__ARM_FEATURE_BF16_VECTOR_ARITHMETIC", "1");
     Builder.defineMacro("__ARM_BF16_FORMAT_ALTERNATIVE", "1");
     Builder.defineMacro("__ARM_FEATURE_BF16_SCALAR_ARITHMETIC", "1");
   }
 
   if ((FPU & SveMode) && HasBFloat16) {
bool AArch64TargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
   HasSVE2 = false;
   HasSVE2AES = false;
   HasSVE2SHA3 = false;
   HasSVE2SM4 = false;
   HasSVE2BitPerm = false;
   HasMatmulFP64 = false;
   HasMatmulFP32 = false;
   HasLSE = false;
+  HasFRInt3264 = false;
 
   ArchKind = llvm::AArch64::ArchKind::ARMV8A;
 
   for (const auto &Feature : Features) {
     if (Feature == "+neon")
       FPU |= NeonMode;
     if (Feature == "+sve") {
       FPU |= SveMode;
for (const auto &Feature : Features) {
     if (Feature == "+bf16")
       HasBFloat16 = true;
     if (Feature == "+lse")
       HasLSE = true;
     if (Feature == "+ls64")
       HasLS64 = true;
     if (Feature == "+flagm")
       HasFlagM = true;
+    if (Feature == "+frint3264")
+      HasFRInt3264 = true;
   }
 
   setDataLayout();
 
   return true;
 }
 
 TargetInfo::CallingConvCheckResult

clang/lib/CodeGen/CGBuiltin.cpp

static const ARMVectorIntrinsicInfo ARMSIMDIntrinsicMap [] = {
   NEONMAP1(vst1q_x2_v, arm_neon_vst1x2, 0),
   NEONMAP1(vst1q_x3_v, arm_neon_vst1x3, 0),
   NEONMAP1(vst1q_x4_v, arm_neon_vst1x4, 0),
   NEONMAP1(vst2_lane_v, arm_neon_vst2lane, 0),
   NEONMAP1(vst2_v, arm_neon_vst2, 0),
   NEONMAP1(vst2q_lane_v, arm_neon_vst2lane, 0),
   NEONMAP1(vst2q_v, arm_neon_vst2, 0),
   NEONMAP1(vst3_lane_v, arm_neon_vst3lane, 0),
   NEONMAP1(vst3_v, arm_neon_vst3, 0),
   NEONMAP1(vst3q_lane_v, arm_neon_vst3lane, 0),
   NEONMAP1(vst3q_v, arm_neon_vst3, 0),
   NEONMAP1(vst4_lane_v, arm_neon_vst4lane, 0),
   NEONMAP1(vst4_v, arm_neon_vst4, 0),
   NEONMAP1(vst4q_lane_v, arm_neon_vst4lane, 0),
   NEONMAP1(vst4q_v, arm_neon_vst4, 0),
   NEONMAP0(vsubhn_v),
   NEONMAP0(vtrn_v),
static const ARMVectorIntrinsicInfo AArch64SIMDIntrinsicMap[] = {
   NEONMAP1(vraddhn_v, aarch64_neon_raddhn, Add1ArgType),
   NEONMAP1(vrax1q_v, aarch64_crypto_rax1, 0),
   NEONMAP2(vrecpe_v, aarch64_neon_frecpe, aarch64_neon_urecpe, 0),
   NEONMAP2(vrecpeq_v, aarch64_neon_frecpe, aarch64_neon_urecpe, 0),
   NEONMAP1(vrecps_v, aarch64_neon_frecps, Add1ArgType),
   NEONMAP1(vrecpsq_v, aarch64_neon_frecps, Add1ArgType),
   NEONMAP2(vrhadd_v, aarch64_neon_urhadd, aarch64_neon_srhadd, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrhaddq_v, aarch64_neon_urhadd, aarch64_neon_srhadd, Add1ArgType | UnsignedAlts),
+  NEONMAP1(vrnd32x_v, aarch64_neon_frint32x, Add1ArgType),
+  NEONMAP1(vrnd32xq_v, aarch64_neon_frint32x, Add1ArgType),
+  NEONMAP1(vrnd32z_v, aarch64_neon_frint32z, Add1ArgType),
+  NEONMAP1(vrnd32zq_v, aarch64_neon_frint32z, Add1ArgType),
+  NEONMAP1(vrnd64x_v, aarch64_neon_frint64x, Add1ArgType),
+  NEONMAP1(vrnd64xq_v, aarch64_neon_frint64x, Add1ArgType),
+  NEONMAP1(vrnd64z_v, aarch64_neon_frint64z, Add1ArgType),
+  NEONMAP1(vrnd64zq_v, aarch64_neon_frint64z, Add1ArgType),
   NEONMAP0(vrndi_v),
   NEONMAP0(vrndiq_v),
   NEONMAP2(vrshl_v, aarch64_neon_urshl, aarch64_neon_srshl, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrshlq_v, aarch64_neon_urshl, aarch64_neon_srshl, Add1ArgType | UnsignedAlts),
   NEONMAP2(vrshr_n_v, aarch64_neon_urshl, aarch64_neon_srshl, UnsignedAlts),
   NEONMAP2(vrshrq_n_v, aarch64_neon_urshl, aarch64_neon_srshl, UnsignedAlts),
   NEONMAP2(vrsqrte_v, aarch64_neon_frsqrte, aarch64_neon_ursqrte, 0),
   NEONMAP2(vrsqrteq_v, aarch64_neon_frsqrte, aarch64_neon_ursqrte, 0),

   }
   case NEON::BI__builtin_neon_vrndh_f16: {
     Ops.push_back(EmitScalarExpr(E->getArg(0)));
     Int = Builder.getIsFPConstrained()
               ? Intrinsic::experimental_constrained_trunc
               : Intrinsic::trunc;
     return EmitNeonCall(CGM.getIntrinsic(Int, HalfTy), Ops, "vrndz");
   }
+  case NEON::BI__builtin_neon_vrnd32x_v:
+  case NEON::BI__builtin_neon_vrnd32xq_v: {
+    Ops.push_back(EmitScalarExpr(E->getArg(0)));
+    Int = Intrinsic::aarch64_neon_frint32x;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrnd32x");
+  }
+  case NEON::BI__builtin_neon_vrnd32z_v:
+  case NEON::BI__builtin_neon_vrnd32zq_v: {
+    Ops.push_back(EmitScalarExpr(E->getArg(0)));
+    Int = Intrinsic::aarch64_neon_frint32z;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrnd32z");
+  }
+  case NEON::BI__builtin_neon_vrnd64x_v:
+  case NEON::BI__builtin_neon_vrnd64xq_v: {
+    Ops.push_back(EmitScalarExpr(E->getArg(0)));
+    Int = Intrinsic::aarch64_neon_frint64x;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrnd64x");
+  }
+  case NEON::BI__builtin_neon_vrnd64z_v:
+  case NEON::BI__builtin_neon_vrnd64zq_v: {
+    Ops.push_back(EmitScalarExpr(E->getArg(0)));
+    Int = Intrinsic::aarch64_neon_frint64z;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrnd64z");
+  }
   case NEON::BI__builtin_neon_vrnd_v:
   case NEON::BI__builtin_neon_vrndq_v: {
     Int = Builder.getIsFPConstrained()
               ? Intrinsic::experimental_constrained_trunc
               : Intrinsic::trunc;
     return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vrndz");
   }
   case NEON::BI__builtin_neon_vcvt_f64_v:

clang/test/CodeGen/aarch64-v8.5a-neon-frint3264-intrinsic.c

+// RUN: %clang_cc1 -triple arm64-none-linux-gnu -target-feature +neon -target-feature +frint3264 -target-feature +v8.5a\
+// RUN: -flax-vector-conversions=none -S -disable-O0-optnone -emit-llvm -o - %s \
+// RUN: | opt -S -mem2reg \
+// RUN: | FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+#include <arm_neon.h>
+
+// CHECK-LABEL: test_vrnd32x_f32
+// CHECK:  [[RND:%.*]] =  call <2 x float> @llvm.aarch64.neon.frint32x.v2f32(<2 x float> %a)
+// CHECK:  ret <2 x float> [[RND]]
+float32x2_t test_vrnd32x_f32(float32x2_t a) {
+  return vrnd32x_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd32xq_f32
+// CHECK:  [[RND:%.*]] =  call <4 x float> @llvm.aarch64.neon.frint32x.v4f32(<4 x float> %a)
+// CHECK:  ret <4 x float> [[RND]]
+float32x4_t test_vrnd32xq_f32(float32x4_t a) {
+  return vrnd32xq_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd32z_f32
+// CHECK:  [[RND:%.*]] =  call <2 x float> @llvm.aarch64.neon.frint32z.v2f32(<2 x float> %a)
+// CHECK:  ret <2 x float> [[RND]]
+float32x2_t test_vrnd32z_f32(float32x2_t a) {
+  return vrnd32z_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd32zq_f32
+// CHECK:  [[RND:%.*]] =  call <4 x float> @llvm.aarch64.neon.frint32z.v4f32(<4 x float> %a)
+// CHECK:  ret <4 x float> [[RND]]
+float32x4_t test_vrnd32zq_f32(float32x4_t a) {
+  return vrnd32zq_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd64x_f32
+// CHECK:  [[RND:%.*]] =  call <2 x float> @llvm.aarch64.neon.frint64x.v2f32(<2 x float> %a)
+// CHECK:  ret <2 x float> [[RND]]
+float32x2_t test_vrnd64x_f32(float32x2_t a) {
+  return vrnd64x_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd64xq_f32
+// CHECK:  [[RND:%.*]] =  call <4 x float> @llvm.aarch64.neon.frint64x.v4f32(<4 x float> %a)
+// CHECK:  ret <4 x float> [[RND]]
+float32x4_t test_vrnd64xq_f32(float32x4_t a) {
+  return vrnd64xq_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd64z_f32
+// CHECK:  [[RND:%.*]] =  call <2 x float> @llvm.aarch64.neon.frint64z.v2f32(<2 x float> %a)
+// CHECK:  ret <2 x float> [[RND]]
+float32x2_t test_vrnd64z_f32(float32x2_t a) {
+  return vrnd64z_f32(a);
+}
+
+// CHECK-LABEL: test_vrnd64zq_f32
+// CHECK:  [[RND:%.*]] =  call <4 x float> @llvm.aarch64.neon.frint64z.v4f32(<4 x float> %a)
+// CHECK:  ret <4 x float> [[RND]]
+float32x4_t test_vrnd64zq_f32(float32x4_t a) {
+  return vrnd64zq_f32(a);
+}

llvm/include/llvm/IR/IntrinsicsAArch64.td

let TargetPrefix = "aarch64", IntrProperties = [IntrNoMem] in {
   def int_aarch64_neon_fcvtpu : AdvSIMD_FPToIntRounding_Intrinsic;
   def int_aarch64_neon_fcvtzs : AdvSIMD_FPToIntRounding_Intrinsic;
   def int_aarch64_neon_fcvtzu : AdvSIMD_FPToIntRounding_Intrinsic;
 
   // Vector FP Rounding: only ties to even is unrepresented by a normal
   // intrinsic.
   def int_aarch64_neon_frintn : AdvSIMD_1FloatArg_Intrinsic;
 
+  // v8.5-A Vector FP Rounding
+  def int_aarch64_neon_frint32x : AdvSIMD_1FloatArg_Intrinsic;
+  def int_aarch64_neon_frint32z : AdvSIMD_1FloatArg_Intrinsic;
+  def int_aarch64_neon_frint64x : AdvSIMD_1FloatArg_Intrinsic;
+  def int_aarch64_neon_frint64z : AdvSIMD_1FloatArg_Intrinsic;
+
   // Scalar FP->Int conversions
 
   // Vector FP Inexact Narrowing
   def int_aarch64_neon_fcvtxn : AdvSIMD_1VectorArg_Expand_Intrinsic;
 
   // Scalar FP Inexact Narrowing
   def int_aarch64_sisd_fcvtxn : DefaultAttrsIntrinsic<[llvm_float_ty], [llvm_double_ty],
                                         [IntrNoMem]>;
 
   // v8.2-A Dot Product
   def int_aarch64_neon_udot : AdvSIMD_Dot_Intrinsic;
   def int_aarch64_neon_sdot : AdvSIMD_Dot_Intrinsic;
 
 // v8.6-A Matrix Multiply Intrinsics
   def int_aarch64_neon_ummla : AdvSIMD_MatMul_Intrinsic;
   def int_aarch64_neon_smmla : AdvSIMD_MatMul_Intrinsic;
   def int_aarch64_neon_usmmla : AdvSIMD_MatMul_Intrinsic;
   def int_aarch64_neon_usdot : AdvSIMD_Dot_Intrinsic;
   def int_aarch64_neon_bfdot : AdvSIMD_Dot_Intrinsic;
   def int_aarch64_neon_bfmmla
     : DefaultAttrsIntrinsic<[llvm_v4f32_ty],
                 [llvm_v4f32_ty, llvm_v8bf16_ty, llvm_v8bf16_ty],

llvm/lib/Target/AArch64/AArch64InstrInfo.td

 defm FRINTI : SIMDTwoVectorFP<1, 1, 0b11001, "frinti", fnearbyint>;
 defm FRINTM : SIMDTwoVectorFP<0, 0, 0b11001, "frintm", ffloor>;
 defm FRINTN : SIMDTwoVectorFP<0, 0, 0b11000, "frintn", int_aarch64_neon_frintn>;
 defm FRINTP : SIMDTwoVectorFP<0, 1, 0b11000, "frintp", fceil>;
 defm FRINTX : SIMDTwoVectorFP<1, 0, 0b11001, "frintx", frint>;
 defm FRINTZ : SIMDTwoVectorFP<0, 1, 0b11001, "frintz", ftrunc>;
 
 let Predicates = [HasFRInt3264] in {
-  defm FRINT32Z : FRIntNNTVector<0, 0, "frint32z">;
-  defm FRINT64Z : FRIntNNTVector<0, 1, "frint64z">;
-  defm FRINT32X : FRIntNNTVector<1, 0, "frint32x">;
-  defm FRINT64X : FRIntNNTVector<1, 1, "frint64x">;
+  defm FRINT32Z : FRIntNNTVector<0, 0, "frint32z", int_aarch64_neon_frint32z>;
+  defm FRINT64Z : FRIntNNTVector<0, 1, "frint64z", int_aarch64_neon_frint64z>;
+  defm FRINT32X : FRIntNNTVector<1, 0, "frint32x", int_aarch64_neon_frint32x>;
+  defm FRINT64X : FRIntNNTVector<1, 1, "frint64x", int_aarch64_neon_frint64x>;
 } // HasFRInt3264
 
 defm FRSQRTE: SIMDTwoVectorFP<1, 1, 0b11101, "frsqrte", int_aarch64_neon_frsqrte>;
 defm FSQRT  : SIMDTwoVectorFP<1, 1, 0b11111, "fsqrt", fsqrt>;
 defm NEG    : SIMDTwoVectorBHSD<1, 0b01011, "neg",
                                UnOpFrag<(sub immAllZerosV, node:$LHS)> >;
 defm NOT    : SIMDTwoVectorB<1, 0b00, 0b00101, "not", vnot>;
 // Aliases for MVN -> NOT.

llvm/test/CodeGen/AArch64/v8.5a-neon-frint3264-intrinsic.ll

+; RUN: llc < %s -mtriple=aarch64-eabi -mattr=+v8.5a,+fptoint  | FileCheck %s
+
+declare <2 x float> @llvm.aarch64.neon.frint32x.v2f32(<2 x float>)
+declare <4 x float> @llvm.aarch64.neon.frint32x.v4f32(<4 x float>)
+declare <2 x float> @llvm.aarch64.neon.frint32z.v2f32(<2 x float>)
+declare <4 x float> @llvm.aarch64.neon.frint32z.v4f32(<4 x float>)
+
+define dso_local <2 x float> @t_vrnd32x_f32(<2 x float> %a) {
+; CHECK-LABEL: t_vrnd32x_f32:
+; CHECK:         frint32x v0.2s, v0.2s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <2 x float> @llvm.aarch64.neon.frint32x.v2f32(<2 x float> %a)
+  ret <2 x float> %val
+}
+
+define dso_local <4 x float> @t_vrnd32xq_f32(<4 x float> %a) {
+; CHECK-LABEL: t_vrnd32xq_f32:
+; CHECK:         frint32x v0.4s, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <4 x float> @llvm.aarch64.neon.frint32x.v4f32(<4 x float> %a)
+  ret <4 x float> %val
+}
+
+define dso_local <2 x float> @t_vrnd32z_f32(<2 x float> %a) {
+; CHECK-LABEL: t_vrnd32z_f32:
+; CHECK:         frint32z v0.2s, v0.2s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <2 x float> @llvm.aarch64.neon.frint32z.v2f32(<2 x float> %a)
+  ret <2 x float> %val
+}
+
+define dso_local <4 x float> @t_vrnd32zq_f32(<4 x float> %a) {
+; CHECK-LABEL: t_vrnd32zq_f32:
+; CHECK:         frint32z v0.4s, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <4 x float> @llvm.aarch64.neon.frint32z.v4f32(<4 x float> %a)
+  ret <4 x float> %val
+}
+
+declare <2 x float> @llvm.aarch64.neon.frint64x.v2f32(<2 x float>)
+declare <4 x float> @llvm.aarch64.neon.frint64x.v4f32(<4 x float>)
+declare <2 x float> @llvm.aarch64.neon.frint64z.v2f32(<2 x float>)
+declare <4 x float> @llvm.aarch64.neon.frint64z.v4f32(<4 x float>)
+
+define dso_local <2 x float> @t_vrnd64x_f32(<2 x float> %a) {
+; CHECK-LABEL: t_vrnd64x_f32:
+; CHECK:         frint64x v0.2s, v0.2s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <2 x float> @llvm.aarch64.neon.frint64x.v2f32(<2 x float> %a)
+  ret <2 x float> %val
+}
+
+define dso_local <4 x float> @t_vrnd64xq_f32(<4 x float> %a) {
+; CHECK-LABEL: t_vrnd64xq_f32:
+; CHECK:         frint64x v0.4s, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <4 x float> @llvm.aarch64.neon.frint64x.v4f32(<4 x float> %a)
+  ret <4 x float> %val
+}
+
+define dso_local <2 x float> @t_vrnd64z_f32(<2 x float> %a) {
+; CHECK-LABEL: t_vrnd64z_f32:
+; CHECK:         frint64z v0.2s, v0.2s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <2 x float> @llvm.aarch64.neon.frint64z.v2f32(<2 x float> %a)
+  ret <2 x float> %val
+}
+
+define dso_local <4 x float> @t_vrnd64zq_f32(<4 x float> %a) {
+; CHECK-LABEL: t_vrnd64zq_f32:
+; CHECK:         frint64z v0.4s, v0.4s
+; CHECK-NEXT:    ret
+entry:
+  %val = tail call <4 x float> @llvm.aarch64.neon.frint64z.v4f32(<4 x float> %a)
+  ret <4 x float> %val
+}