[SveEmitter] Add more immediate operand checks.

[sdesmalen]

Summary

This patch adds a number of intrinsics that take immediates with
varying ranges based on the element size one of the operands.

svext:   immediate ranging 0 to (2048/sizeinbits(elt) - 1)
svasrd:  immediate ranging 1..sizeinbits(elt)
svqshlu: immediate ranging 1..sizeinbits(elt)/2
ftmad:   immediate ranging 0..(sizeinbits(elt) - 1)

[SjoerdMeijer]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

Just curious about the -fallow-half-arguments-and-returns, do you need that here?

And if not here, why do you need it elsewhere (looks enabled on all tests)?

[sdesmalen]

(sorry, I wrote the comments earlier but forgot to click 'submit' :) )

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

It's not needed for this test, but we've generated most of our tests from the ACLE spec and the tests that use a scalar float16_t (== __fp16) will need this, such as the ACLE intrinsic:

svfloat16_t svadd_m(svbool_t, svfloat16_t, float16_t);

If you feel strongly about it, I could remove it from the other RUN lines.

[SjoerdMeijer]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

Well, I think this is my surprise then. Thinking out loud: we're talking SVE here, which always implies FP16. That's why I am surprised that we bother with a storage-type only type. Looking at the SVE ACLE I indeed see:

float16_t equivalent to __fp16

where I was probably expecting:

float16_t equivalent to _Float16

and with that everything would be sorted I guess, then we also don't need the hack^W workaround that is -fallow-half-arguments-and-returns. But maybe there is a good reason to use/choose __fp16 that I don't see here. Probably worth a quick question for the ARM SVE ACLE, would you mind quickly checking?

[sdesmalen]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

As just checked with @rsandifo-arm, the reason is that the definition of float16_t has to be compatible with arm_neon.h, which uses __fp16 for both Clang and GCC.

[SjoerdMeijer]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

I was suspecting it was compatability reasons, but perhaps not with arm_neon.h. So what exactly does it mean to be compatible with arm_neon.h? I mean, put simply and naively, if you target SVE, you include arm_sve.h, and go from there. How does that interact with arm_neon.h and why can float16_t not be a proper half type?

[sdesmalen]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

If you target SVE, you can still use Neon instructions, so it's still possible to include arm_neon.h as well. If those have differing definitions of float16_t, that may give trouble when using builtins from both the Neon and SVE header files.

[SjoerdMeijer]

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

2

ok, thank, got it. So we are supporting this case:

#include <arm_neon.h>
#include <arm_svh.h>
void foo () {
   neon_intrinsic();
   sve_intrinsic();
}

Well, I find this very unfortunate, because it could have been so beautiful, and now we're still have this storage-only type while even the ACLE discourages its use. The use of -fallow-half-arguments-and-returns is just a minor annoyance, but point is it shouldn't have been necessary.

Now I am wondering why the ARM SVE ACLE is using float16_t, and not just _Float16. Do you have any insights in that too perhaps?

[SjoerdMeijer]

Just to clarify my last sentence:

Now I am wondering why the ARM SVE ACLE is using float16_t, and not just _Float16. Do you have any insights in that too perhaps?

What I meant to say is why SVE intrinsics are not using _Float16?

[SjoerdMeijer]

I think the float16 discussion is an interesting one, but doesn't necessarily need to be done here. I am asking some questions offline, but if we ever come to a different opinion on it, then we can follow up so it's somewhat orthogonal to this change, and so this looks fine to me.

[sdesmalen]

In D76679#1957399, @SjoerdMeijer wrote:

I think the float16 discussion is an interesting one, but doesn't necessarily need to be done here. I am asking some questions offline, but if we ever come to a different opinion on it, then we can follow up so it's somewhat orthogonal to this change, and so this looks fine to me.

Thanks for reviewing the patch Sjoerd!

clang/include/clang/Basic/arm_sve.td

 // ------------------
 // P: boolean
 // U: unsigned
 
 // Prototype modifiers
 // -------------------
 // prototype: return (arg, arg, ...)
 //
+// u: vector of unsigned integers
 // d: default
 // c: const pointer type
 // P: predicate type
+// h: 1/2 width elements, 2x element count
 //
 // i: constant uint64_t
 //
 // I: Predicate Pattern (sv_pattern)
 
 // l: int64_t
 
 // A: pointer to int8_t

 def IsZExtReturn              : FlagType<0x00080000>; // Return value is sign-extend by default
 
 // These must be kept in sync with the flags in include/clang/Basic/TargetBuiltins.h
 class ImmCheckType<int val> {
   int Value = val;
 }
 def ImmCheck0_31                : ImmCheckType<0>;  // 0..31 (used for e.g. predicate patterns)
 def ImmCheck1_16                : ImmCheckType<1>;  // 1..16
+def ImmCheckExtract             : ImmCheckType<2>;  // 0..(2048/sizeinbits(elt) - 1)
+def ImmCheckShiftRight          : ImmCheckType<3>;  // 1..sizeinbits(elt)
+def ImmCheckShiftRightNarrow    : ImmCheckType<4>;  // 1..sizeinbits(elt)/2
+def ImmCheckShiftLeft           : ImmCheckType<5>;  // 0..(sizeinbits(elt) - 1)
+def ImmCheck0_7                 : ImmCheckType<6>;  // 0..7
 
 class ImmCheck<int arg, ImmCheckType kind, int eltSizeArg = -1> {
   int Arg = arg;
-  int EltSizeArg = eltSizeArg;
+  int EltSizeArg = eltSizeArg;
   ImmCheckType Kind = kind;
 }
 
-// Every intrinsic subclasses Inst.
 class Inst<string n, string p, string t, MergeType mt, string i,
            list<FlagType> ft, list<ImmCheck> ch, MemEltType met> {
   string Name = n;
   string Prototype = p;
   string Types = t;
   string ArchGuard = "";
   int Merge = mt.Value;
   string MergeSuffix = mt.Suffix;

 
 // Store one vector, with no truncation, non-temporal (scalar base)
 def SVSTNT1 : MInst<"svstnt1[_{d}]", "vPpd", "csilUcUsUiUlhfd", [IsStore], MemEltTyDefault, "aarch64_sve_stnt1">;
 
 // Store one vector, with no truncation, non-temporal (scalar base, VL displacement)
 def SVSTNT1_VNUM : MInst<"svstnt1_vnum[_{d}]", "vPpld", "csilUcUsUiUlhfd", [IsStore], MemEltTyDefault, "aarch64_sve_stnt1">;
 
 ////////////////////////////////////////////////////////////////////////////////
+// Permutations and selection
+def SVEXT        : SInst<"svext[_{d}]",       "dddi", "csilUcUsUiUlhfd", MergeNone, "aarch64_sve_ext", [], [ImmCheck<2, ImmCheckExtract, 1>]>;
+
+////////////////////////////////////////////////////////////////////////////////
+// Shifts
+def SVASRD_M : SInst<"svasrd[_n_{d}]", "dPdi", "csil",            MergeOp1,  "aarch64_sve_asrd", [], [ImmCheck<2, ImmCheckShiftRight, 1>]>;
+
+////////////////////////////////////////////////////////////////////////////////
+// SVE2 - Narrowing DSP operations
+let ArchGuard = "defined(__ARM_FEATURE_SVE2)" in {
+def SVSHRNB      : SInst<"svshrnb[_n_{d}]",    "hdi",  "silUsUiUl", MergeNone, "aarch64_sve_shrnb",     [], [ImmCheck<1, ImmCheckShiftRightNarrow, 0>]>;
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// SVE2 - Uniform DSP operations
+let ArchGuard = "defined(__ARM_FEATURE_SVE2)" in {
+def SVQSHLU_M  : SInst<"svqshlu[_n_{d}]", "uPdi", "csil",         MergeOp1,  "aarch64_sve_sqshlu", [], [ImmCheck<2, ImmCheckShiftLeft,  1>]>;
+}
+
+////////////////////////////////////////////////////////////////////////////////
+// Floating-point arithmetic
+def SVTMAD  : SInst<"svtmad[_{d}]",  "dddi", "hfd", MergeNone, "aarch64_sve_ftmad_x", [], [ImmCheck<2, ImmCheck0_7>]>;
+
+////////////////////////////////////////////////////////////////////////////////
 // Saturating scalar arithmetic
 def SVQDECH_S : SInst<"svqdech_pat[_{d}]",   "ddIi", "s", MergeNone, "aarch64_sve_sqdech", [], [ImmCheck<2, ImmCheck1_16>]>;
 def SVQDECH_U : SInst<"svqdech_pat[_{d}]",   "ddIi", "Us", MergeNone, "aarch64_sve_uqdech", [], [ImmCheck<2, ImmCheck1_16>]>;

clang/lib/CodeGen/CGBuiltin.cpp

Value *CodeGenFunction::EmitAArch64SVEBuiltinExpr(unsigned BuiltinID,
   if (TypeFlags.isLoad())
     return EmitSVEMaskedLoad(E, Ty, Ops, Builtin->LLVMIntrinsic,
                              TypeFlags.isZExtReturn());
   else if (TypeFlags.isStore())
     return EmitSVEMaskedStore(E, Ops, Builtin->LLVMIntrinsic);
   else if (Builtin->LLVMIntrinsic != 0) {
     llvm::Type* OverloadedTy = getSVEType(TypeFlags);
 
+    // Predicates must match the main datatype.
+    for (unsigned i = 0, e = Ops.size(); i != e; ++i) {
+      if (auto PredTy = dyn_cast<llvm::VectorType>(Ops[i]->getType()))
+        if (PredTy->getScalarType()->isIntegerTy(1)) {
+          auto NewPredTy = cast<llvm::VectorType>(OverloadedTy);
+          Ops[i] = EmitSVEPredicateCast(Ops[i], NewPredTy);
+        }
+    }
+
     Function *F = CGM.getIntrinsic(Builtin->LLVMIntrinsic, OverloadedTy);
     Value *Call = Builder.CreateCall(F, Ops);
     return Call;
   }
 
   /// Should not happen
   return nullptr;
 }

clang/lib/Sema/SemaChecking.cpp

for (auto &I : ImmChecks) {
     case SVETypeFlags::ImmCheck0_31:
       if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 0, 31))
         HasError = true;
       break;
     case SVETypeFlags::ImmCheck1_16:
       if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 1, 16))
         HasError = true;
       break;
+    case SVETypeFlags::ImmCheck0_7:
+      if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 0, 7))
+        HasError = true;
+      break;
+    case SVETypeFlags::ImmCheckExtract:
+      if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 0,
+                                      (2048 / ElementSizeInBits) - 1))
+        HasError = true;
+      break;
+    case SVETypeFlags::ImmCheckShiftRight:
+      if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 1, ElementSizeInBits))
+        HasError = true;
+      break;
+    case SVETypeFlags::ImmCheckShiftRightNarrow:
+      if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 1,
+                                      ElementSizeInBits / 2))
+        HasError = true;
+      break;
+    case SVETypeFlags::ImmCheckShiftLeft:
+      if (SemaBuiltinConstantArgRange(TheCall, ArgNum, 0,
+                                      ElementSizeInBits - 1))
+        HasError = true;
+      break;
     }
   }
 
   return HasError;
 }
 
 bool Sema::CheckNeonBuiltinFunctionCall(unsigned BuiltinID, CallExpr *TheCall) {
   llvm::APSInt Result;

clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_asrd.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svint8_t test_svasrd_n_s8_m(svbool_t pg, svint8_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s8_m
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.asrd.nxv16i8(<vscale x 16 x i1> %pg, <vscale x 16 x i8> %op1, i32 1)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s8,_m,)(pg, op1, 1);
+}
+
+svint16_t test_svasrd_n_s16_m(svbool_t pg, svint16_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s16_m
+  // CHECK: %[[P0:.*]] = call <vscale x 8 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv8i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.asrd.nxv8i16(<vscale x 8 x i1> %[[P0]], <vscale x 8 x i16> %op1, i32 1)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s16,_m,)(pg, op1, 1);
+}
+
+svint32_t test_svasrd_n_s32_m(svbool_t pg, svint32_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s32_m
+  // CHECK: %[[P0:.*]] = call <vscale x 4 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.asrd.nxv4i32(<vscale x 4 x i1> %[[P0]], <vscale x 4 x i32> %op1, i32 1)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s32,_m,)(pg, op1, 1);
+}
+
+svint64_t test_svasrd_n_s64_m(svbool_t pg, svint64_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s64_m
+  // CHECK: %[[P0:.*]] = call <vscale x 2 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.asrd.nxv2i64(<vscale x 2 x i1> %[[P0]], <vscale x 2 x i64> %op1, i32 1)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s64,_m,)(pg, op1, 1);
+}
+
+svint8_t test_svasrd_n_s8_max_m(svbool_t pg, svint8_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s8_max_m
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.asrd.nxv16i8(<vscale x 16 x i1> %pg, <vscale x 16 x i8> %op1, i32 8)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s8,_m,)(pg, op1, 8);
+}
+
+svint16_t test_svasrd_n_s16_max_m(svbool_t pg, svint16_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s16_max_m
+  // CHECK: %[[P0:.*]] = call <vscale x 8 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv8i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.asrd.nxv8i16(<vscale x 8 x i1> %[[P0]], <vscale x 8 x i16> %op1, i32 16)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s16,_m,)(pg, op1, 16);
+}
+
+svint32_t test_svasrd_n_s32_max_m(svbool_t pg, svint32_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s32_max_m
+  // CHECK: %[[P0:.*]] = call <vscale x 4 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.asrd.nxv4i32(<vscale x 4 x i1> %[[P0]], <vscale x 4 x i32> %op1, i32 32)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s32,_m,)(pg, op1, 32);
+}
+
+svint64_t test_svasrd_n_s64_max_m(svbool_t pg, svint64_t op1)
+{
+  // CHECK-LABEL: test_svasrd_n_s64_max_m
+  // CHECK: %[[P0:.*]] = call <vscale x 2 x i1>  @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.asrd.nxv2i64(<vscale x 2 x i1> %[[P0]], <vscale x 2 x i64> %op1, i32 64)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svasrd,_n_s64,_m,)(pg, op1, 64);
+}

clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_ext.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svint8_t test_svext_s8(svint8_t op1, svint8_t op2)
+{
+  // CHECK-LABEL: test_svext_s8
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.ext.nxv16i8(<vscale x 16 x i8> %op1, <vscale x 16 x i8> %op2, i32 0)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s8,,)(op1, op2, 0);
+}
+
+svint8_t test_svext_s8_1(svint8_t op1, svint8_t op2)
+{
+  // CHECK-LABEL: test_svext_s8_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.ext.nxv16i8(<vscale x 16 x i8> %op1, <vscale x 16 x i8> %op2, i32 255)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s8,,)(op1, op2, 255);
+}
+
+svint16_t test_svext_s16(svint16_t op1, svint16_t op2)
+{
+  // CHECK-LABEL: test_svext_s16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.ext.nxv8i16(<vscale x 8 x i16> %op1, <vscale x 8 x i16> %op2, i32 0)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s16,,)(op1, op2, 0);
+}
+
+svint16_t test_svext_s16_1(svint16_t op1, svint16_t op2)
+{
+  // CHECK-LABEL: test_svext_s16_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.ext.nxv8i16(<vscale x 8 x i16> %op1, <vscale x 8 x i16> %op2, i32 127)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s16,,)(op1, op2, 127);
+}
+
+svint32_t test_svext_s32(svint32_t op1, svint32_t op2)
+{
+  // CHECK-LABEL: test_svext_s32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.ext.nxv4i32(<vscale x 4 x i32> %op1, <vscale x 4 x i32> %op2, i32 0)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s32,,)(op1, op2, 0);
+}
+
+svint32_t test_svext_s32_1(svint32_t op1, svint32_t op2)
+{
+  // CHECK-LABEL: test_svext_s32_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.ext.nxv4i32(<vscale x 4 x i32> %op1, <vscale x 4 x i32> %op2, i32 63)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s32,,)(op1, op2, 63);
+}
+
+svint64_t test_svext_s64(svint64_t op1, svint64_t op2)
+{
+  // CHECK-LABEL: test_svext_s64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.ext.nxv2i64(<vscale x 2 x i64> %op1, <vscale x 2 x i64> %op2, i32 0)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s64,,)(op1, op2, 0);
+}
+
+svint64_t test_svext_s64_1(svint64_t op1, svint64_t op2)
+{
+  // CHECK-LABEL: test_svext_s64_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.ext.nxv2i64(<vscale x 2 x i64> %op1, <vscale x 2 x i64> %op2, i32 31)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_s64,,)(op1, op2, 31);
+}
+
+svuint8_t test_svext_u8(svuint8_t op1, svuint8_t op2)
+{
+  // CHECK-LABEL: test_svext_u8
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.ext.nxv16i8(<vscale x 16 x i8> %op1, <vscale x 16 x i8> %op2, i32 255)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_u8,,)(op1, op2, 255);
+}
+
+svuint16_t test_svext_u16(svuint16_t op1, svuint16_t op2)
+{
+  // CHECK-LABEL: test_svext_u16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.ext.nxv8i16(<vscale x 8 x i16> %op1, <vscale x 8 x i16> %op2, i32 127)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_u16,,)(op1, op2, 127);
+}
+
+svuint32_t test_svext_u32(svuint32_t op1, svuint32_t op2)
+{
+  // CHECK-LABEL: test_svext_u32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.ext.nxv4i32(<vscale x 4 x i32> %op1, <vscale x 4 x i32> %op2, i32 63)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_u32,,)(op1, op2, 63);
+}
+
+svuint64_t test_svext_u64(svuint64_t op1, svuint64_t op2)
+{
+  // CHECK-LABEL: test_svext_u64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.ext.nxv2i64(<vscale x 2 x i64> %op1, <vscale x 2 x i64> %op2, i32 31)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_u64,,)(op1, op2, 31);
+}
+
+svfloat16_t test_svext_f16(svfloat16_t op1, svfloat16_t op2)
+{
+  // CHECK-LABEL: test_svext_f16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x half> @llvm.aarch64.sve.ext.nxv8f16(<vscale x 8 x half> %op1, <vscale x 8 x half> %op2, i32 127)
+  // CHECK: ret <vscale x 8 x half> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_f16,,)(op1, op2, 127);
+}
+
+svfloat32_t test_svext_f32(svfloat32_t op1, svfloat32_t op2)
+{
+  // CHECK-LABEL: test_svext_f32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x float> @llvm.aarch64.sve.ext.nxv4f32(<vscale x 4 x float> %op1, <vscale x 4 x float> %op2, i32 63)
+  // CHECK: ret <vscale x 4 x float> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_f32,,)(op1, op2, 63);
+}
+
+svfloat64_t test_svext_f64(svfloat64_t op1, svfloat64_t op2)
+{
+  // CHECK-LABEL: test_svext_f64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x double> @llvm.aarch64.sve.ext.nxv2f64(<vscale x 2 x double> %op1, <vscale x 2 x double> %op2, i32 31)
+  // CHECK: ret <vscale x 2 x double> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svext,_f64,,)(op1, op2, 31);
+}

clang/test/CodeGen/aarch64-sve-intrinsics/acle_sve_tmad.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svfloat16_t test_svtmad_f16(svfloat16_t op1, svfloat16_t op2)
+{
+  // CHECK-LABEL: test_svtmad_f16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x half> @llvm.aarch64.sve.ftmad.x.nxv8f16(<vscale x 8 x half> %op1, <vscale x 8 x half> %op2, i32 0)
+  // CHECK: ret <vscale x 8 x half> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svtmad,_f16,,)(op1, op2, 0);
+}
+
+svfloat16_t test_svtmad_f16_1(svfloat16_t op1, svfloat16_t op2)
+{
+  // CHECK-LABEL: test_svtmad_f16_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x half> @llvm.aarch64.sve.ftmad.x.nxv8f16(<vscale x 8 x half> %op1, <vscale x 8 x half> %op2, i32 7)
+  // CHECK: ret <vscale x 8 x half> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svtmad,_f16,,)(op1, op2, 7);
+}
+
+svfloat32_t test_svtmad_f32(svfloat32_t op1, svfloat32_t op2)
+{
+  // CHECK-LABEL: test_svtmad_f32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x float> @llvm.aarch64.sve.ftmad.x.nxv4f32(<vscale x 4 x float> %op1, <vscale x 4 x float> %op2, i32 0)
+  // CHECK: ret <vscale x 4 x float> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svtmad,_f32,,)(op1, op2, 0);
+}
+
+svfloat64_t test_svtmad_f64(svfloat64_t op1, svfloat64_t op2)
+{
+  // CHECK-LABEL: test_svtmad_f64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x double> @llvm.aarch64.sve.ftmad.x.nxv2f64(<vscale x 2 x double> %op1, <vscale x 2 x double> %op2, i32 0)
+  // CHECK: ret <vscale x 2 x double> %[[INTRINSIC]]
+  return SVE_ACLE_FUNC(svtmad,_f64,,)(op1, op2, 0);
+}

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_asrd.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+#include <arm_sve.h>
+
+svint8_t test_svasrd_n_s8_m(svbool_t pg, svint8_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 8]}}
+  return SVE_ACLE_FUNC(svasrd,_n_s8,_m,)(pg, op1, 0);
+}
+
+svint16_t test_svasrd_n_s16_m(svbool_t pg, svint16_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 16]}}
+  return SVE_ACLE_FUNC(svasrd,_n_s16,_m,)(pg, op1, 17);
+}
+
+svint32_t test_svasrd_n_s32_m(svbool_t pg, svint32_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 32]}}
+  return SVE_ACLE_FUNC(svasrd,_n_s32,_m,)(pg, op1, 0);
+}
+
+svint64_t test_svasrd_n_s64_m(svbool_t pg, svint64_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 64]}}
+  return SVE_ACLE_FUNC(svasrd,_n_s64,_m,)(pg, op1, 65);
+}

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_ext.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+#include <arm_sve.h>
+
+svint8_t test_svext_s8(svint8_t op1, svint8_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 255]}}
+  return SVE_ACLE_FUNC(svext,_s8,,)(op1, op2, -1);
+}
+
+svint8_t test_svext_s8_1(svint8_t op1, svint8_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 255]}}
+  return SVE_ACLE_FUNC(svext,_s8,,)(op1, op2, 256);
+}
+
+svint16_t test_svext_s16(svint16_t op1, svint16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 127]}}
+  return SVE_ACLE_FUNC(svext,_s16,,)(op1, op2, -1);
+}
+
+svint16_t test_svext_s16_1(svint16_t op1, svint16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 127]}}
+  return SVE_ACLE_FUNC(svext,_s16,,)(op1, op2, 128);
+}
+
+svint32_t test_svext_s32(svint32_t op1, svint32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 63]}}
+  return SVE_ACLE_FUNC(svext,_s32,,)(op1, op2, -1);
+}
+
+svint32_t test_svext_s32_1(svint32_t op1, svint32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 63]}}
+  return SVE_ACLE_FUNC(svext,_s32,,)(op1, op2, 64);
+}
+
+svint64_t test_svext_s64(svint64_t op1, svint64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 31]}}
+  return SVE_ACLE_FUNC(svext,_s64,,)(op1, op2, -1);
+}
+
+svint64_t test_svext_s64_1(svint64_t op1, svint64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 31]}}
+  return SVE_ACLE_FUNC(svext,_s64,,)(op1, op2, 32);
+}
+
+svuint8_t test_svext_u8(svuint8_t op1, svuint8_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 255]}}
+  return SVE_ACLE_FUNC(svext,_u8,,)(op1, op2, -1);
+}
+
+svuint16_t test_svext_u16(svuint16_t op1, svuint16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 127]}}
+  return SVE_ACLE_FUNC(svext,_u16,,)(op1, op2, 128);
+}
+
+svuint32_t test_svext_u32(svuint32_t op1, svuint32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 63]}}
+  return SVE_ACLE_FUNC(svext,_u32,,)(op1, op2, -1);
+}
+
+svuint64_t test_svext_u64(svuint64_t op1, svuint64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 31]}}
+  return SVE_ACLE_FUNC(svext,_u64,,)(op1, op2, 32);
+}
+
+svfloat16_t test_svext_f16(svfloat16_t op1, svfloat16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 127]}}
+  return SVE_ACLE_FUNC(svext,_f16,,)(op1, op2, -1);
+}
+
+svfloat32_t test_svext_f32(svfloat32_t op1, svfloat32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 63]}}
+  return SVE_ACLE_FUNC(svext,_f32,,)(op1, op2, 64);
+}
+
+svfloat64_t test_svext_f64(svfloat64_t op1, svfloat64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 31]}}
+  return SVE_ACLE_FUNC(svext,_f64,,)(op1, op2, -1);
+}

clang/test/CodeGen/aarch64-sve-intrinsics/negative/acle_sve_tmad.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+#include <arm_sve.h>
+
+svfloat16_t test_svtmad_f16(svfloat16_t op1, svfloat16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f16,,)(op1, op2, -1);
+}
+
+svfloat16_t test_svtmad_f16_1(svfloat16_t op1, svfloat16_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f16,,)(op1, op2, 8);
+}
+
+svfloat32_t test_svtmad_f32(svfloat32_t op1, svfloat32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f32,,)(op1, op2, -1);
+}
+
+svfloat32_t test_svtmad_f32_1(svfloat32_t op1, svfloat32_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f32,,)(op1, op2, 8);
+}
+
+svfloat64_t test_svtmad_f64(svfloat64_t op1, svfloat64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f64,,)(op1, op2, -1);
+}
+
+svfloat64_t test_svtmad_f64_1(svfloat64_t op1, svfloat64_t op2)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svtmad,_f64,,)(op1, op2, 8);
+}

clang/test/CodeGen/aarch64-sve2-intrinsics/acle_sve2_qshlu.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify -verify-ignore-unexpected=error %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify=overload -verify-ignore-unexpected=error %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svuint8_t test_svqshlu_n_s8_m(svbool_t pg, svint8_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s8_m
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.sqshlu.nxv16i8(<vscale x 16 x i1> %pg, <vscale x 16 x i8> %op1, i32 0)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s8_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s8,_m,)(pg, op1, 0);
+}
+
+svuint8_t test_svqshlu_n_s8_m_1(svbool_t pg, svint8_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s8_m_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.sqshlu.nxv16i8(<vscale x 16 x i1> %pg, <vscale x 16 x i8> %op1, i32 7)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s8_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s8,_m,)(pg, op1, 7);
+}
+
+svuint16_t test_svqshlu_n_s16_m(svbool_t pg, svint16_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s16_m
+  // CHECK: %[[PG:.*]] = call <vscale x 8 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv8i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.sqshlu.nxv8i16(<vscale x 8 x i1> %[[PG]], <vscale x 8 x i16> %op1, i32 0)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s16_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s16,_m,)(pg, op1, 0);
+}
+
+svuint16_t test_svqshlu_n_s16_m_1(svbool_t pg, svint16_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s16_m_1
+  // CHECK: %[[PG:.*]] = call <vscale x 8 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv8i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.sqshlu.nxv8i16(<vscale x 8 x i1> %[[PG]], <vscale x 8 x i16> %op1, i32 15)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s16_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s16,_m,)(pg, op1, 15);
+}
+
+svuint32_t test_svqshlu_n_s32_m(svbool_t pg, svint32_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s32_m
+  // CHECK: %[[PG:.*]] = call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.sqshlu.nxv4i32(<vscale x 4 x i1> %[[PG]], <vscale x 4 x i32> %op1, i32 0)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s32_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s32,_m,)(pg, op1, 0);
+}
+
+svuint32_t test_svqshlu_n_s32_m_1(svbool_t pg, svint32_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s32_m_1
+  // CHECK: %[[PG:.*]] = call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.sqshlu.nxv4i32(<vscale x 4 x i1> %[[PG]], <vscale x 4 x i32> %op1, i32 31)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s32_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s32,_m,)(pg, op1, 31);
+}
+
+svuint64_t test_svqshlu_n_s64_m(svbool_t pg, svint64_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s64_m
+  // CHECK: %[[PG:.*]] = call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.sqshlu.nxv2i64(<vscale x 2 x i1> %[[PG]], <vscale x 2 x i64> %op1, i32 0)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s64_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s64,_m,)(pg, op1, 0);
+}
+
+svuint64_t test_svqshlu_n_s64_m_1(svbool_t pg, svint64_t op1)
+{
+  // CHECK-LABEL: test_svqshlu_n_s64_m_1
+  // CHECK: %[[PG:.*]] = call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> %pg)
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 2 x i64> @llvm.aarch64.sve.sqshlu.nxv2i64(<vscale x 2 x i1> %[[PG]], <vscale x 2 x i64> %op1, i32 63)
+  // CHECK: ret <vscale x 2 x i64> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svqshlu_m'}}
+  // expected-warning@+1 {{implicit declaration of function 'svqshlu_n_s64_m'}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s64,_m,)(pg, op1, 63);
+}

clang/test/CodeGen/aarch64-sve2-intrinsics/acle_sve2_shrnb.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -S -O1 -Werror -Wall -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify -verify-ignore-unexpected=error %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve -fallow-half-arguments-and-returns -fsyntax-only -verify=overload -verify-ignore-unexpected=error %s
+
+#include <arm_sve.h>
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+svint8_t test_svshrnb_n_s16(svint16_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.shrnb.nxv8i16(<vscale x 8 x i16> %op1, i32 1)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s16'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s16,,)(op1, 1);
+}
+
+svint8_t test_svshrnb_n_s16_1(svint16_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s16_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.shrnb.nxv8i16(<vscale x 8 x i16> %op1, i32 8)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s16'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s16,,)(op1, 8);
+}
+
+svint16_t test_svshrnb_n_s32(svint32_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.shrnb.nxv4i32(<vscale x 4 x i32> %op1, i32 1)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s32'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s32,,)(op1, 1);
+}
+
+svint16_t test_svshrnb_n_s32_1(svint32_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s32_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.shrnb.nxv4i32(<vscale x 4 x i32> %op1, i32 16)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s32'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s32,,)(op1, 16);
+}
+
+svint32_t test_svshrnb_n_s64(svint64_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.shrnb.nxv2i64(<vscale x 2 x i64> %op1, i32 1)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s64'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s64,,)(op1, 1);
+}
+
+svint32_t test_svshrnb_n_s64_1(svint64_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_s64_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.shrnb.nxv2i64(<vscale x 2 x i64> %op1, i32 32)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_s64'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s64,,)(op1, 32);
+}
+
+svuint8_t test_svshrnb_n_u16(svuint16_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u16
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.shrnb.nxv8i16(<vscale x 8 x i16> %op1, i32 1)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u16'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u16,,)(op1, 1);
+}
+
+svuint8_t test_svshrnb_n_u16_1(svuint16_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u16_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 16 x i8> @llvm.aarch64.sve.shrnb.nxv8i16(<vscale x 8 x i16> %op1, i32 8)
+  // CHECK: ret <vscale x 16 x i8> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u16'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u16,,)(op1, 8);
+}
+
+svuint16_t test_svshrnb_n_u32(svuint32_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u32
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.shrnb.nxv4i32(<vscale x 4 x i32> %op1, i32 1)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u32'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u32,,)(op1, 1);
+}
+
+svuint16_t test_svshrnb_n_u32_1(svuint32_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u32_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 8 x i16> @llvm.aarch64.sve.shrnb.nxv4i32(<vscale x 4 x i32> %op1, i32 16)
+  // CHECK: ret <vscale x 8 x i16> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u32'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u32,,)(op1, 16);
+}
+
+svuint32_t test_svshrnb_n_u64(svuint64_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u64
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.shrnb.nxv2i64(<vscale x 2 x i64> %op1, i32 1)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u64'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u64,,)(op1, 1);
+}
+
+svuint32_t test_svshrnb_n_u64_1(svuint64_t op1)
+{
+  // CHECK-LABEL: test_svshrnb_n_u64_1
+  // CHECK: %[[INTRINSIC:.*]] = call <vscale x 4 x i32> @llvm.aarch64.sve.shrnb.nxv2i64(<vscale x 2 x i64> %op1, i32 32)
+  // CHECK: ret <vscale x 4 x i32> %[[INTRINSIC]]
+  // overload-warning@+2 {{implicit declaration of function 'svshrnb'}}
+  // expected-warning@+1 {{implicit declaration of function 'svshrnb_n_u64'}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u64,,)(op1, 32);
+}

clang/test/CodeGen/aarch64-sve2-intrinsics/negative/acle_sve2_qshlu.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+#include <arm_sve.h>
+
+svuint8_t test_svqshlu_n_s8_m(svbool_t pg, svint8_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 7]}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s8,_m,)(pg, op1, -1);
+}
+
+svuint16_t test_svqshlu_n_s16_m(svbool_t pg, svint16_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 15]}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s16,_m,)(pg, op1, -1);
+}
+
+svuint32_t test_svqshlu_n_s32_m(svbool_t pg, svint32_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 31]}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s32,_m,)(pg, op1, -1);
+}
+
+svuint64_t test_svqshlu_n_s64_m(svbool_t pg, svint64_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [0, 63]}}
+  return SVE_ACLE_FUNC(svqshlu,_n_s64,_m,)(pg, op1, -1);
+}

clang/test/CodeGen/aarch64-sve2-intrinsics/negative/acle_sve2_shrnb.c

+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+// RUN: %clang_cc1 -D__ARM_FEATURE_SVE -D__ARM_FEATURE_SVE2 -DSVE_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sve2 -fallow-half-arguments-and-returns -fsyntax-only -verify %s
+
+#ifdef SVE_OVERLOADED_FORMS
+// A simple used,unused... macro, long enough to represent any SVE builtin.
+#define SVE_ACLE_FUNC(A1,A2_UNUSED,A3,A4_UNUSED) A1##A3
+#else
+#define SVE_ACLE_FUNC(A1,A2,A3,A4) A1##A2##A3##A4
+#endif
+
+#include <arm_sve.h>
+
+svint8_t test_svshrnb_n_s16(svint16_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 8]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s16,,)(op1, 0);
+}
+
+svint16_t test_svshrnb_n_s32(svint32_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 16]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s32,,)(op1, 0);
+}
+
+svint32_t test_svshrnb_n_s64(svint64_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 32]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_s64,,)(op1, 0);
+}
+
+svuint8_t test_svshrnb_n_u16(svuint16_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 8]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u16,,)(op1, 0);
+}
+
+svuint16_t test_svshrnb_n_u32(svuint32_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 16]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u32,,)(op1, 0);
+}
+
+svuint32_t test_svshrnb_n_u64(svuint64_t op1)
+{
+  // expected-error-re@+1 {{argument value {{[0-9]+}} is outside the valid range [1, 32]}}
+  return SVE_ACLE_FUNC(svshrnb,_n_u64,,)(op1, 0);
+}

clang/utils/TableGen/SveEmitter.cpp

void SVEType::applyModifier(char Mod) {
   case 'c':
     Constant = true;
     LLVM_FALLTHROUGH;
   case 'p':
     Pointer = true;
     Bitwidth = ElementBitwidth;
     NumVectors = 0;
     break;
+  case 'h':
+    ElementBitwidth /= 2;
+    break;
   case 'P':
     Signed = true;
     Float = false;
     Predicate = true;
     Bitwidth = 16;
     ElementBitwidth = 1;
     break;
+  case 'u':
+    Predicate = false;
+    Signed = false;
+    Float = false;
+    break;
   case 'i':
     Predicate = false;
     Float = false;
     ElementBitwidth = Bitwidth = 64;
     NumVectors = 0;
     Signed = false;
     Immediate = true;
     break;