This is an archive of the discontinued LLVM Phabricator instance.

Differential D134680

[Clang][AArch64][SME] Add intrinsics for adding vector elements to ZA tile
ClosedPublic

Authored by bryanpkc on Sep 26 2022, 2:52 PM.

Details

Reviewers
sdesmalen
david-arm
rsandifo-arm
sagarkulkarni19
Commits
rG4ae900c06363: [Clang][AArch64][SME] Add intrinsics for adding vector elements to ZA tile
Summary

This patch adds support for the following SME ACLE intrinsics (as defined
in https://arm-software.github.io/acle/main/acle.html):

  • svaddha_za32[_u32]_m // also for s32
  • svaddva_za32[_u32]_m // also for s32
  • svaddha_za64[_u64]_m // also for s64
  • svaddva_za64[_u64]_m // also for s64

The _za64 versions are available only when the sme-i16i64 feature is enabled.

Co-authored-by: Sagar Kulkarni <sagar.kulkarni1@huawei.com>

Diff Detail

Event Timeline

sagarkulkarni19 created this revision.Sep 26 2022, 2:52 PM
Herald added a project: Restricted Project. · View Herald TranscriptSep 26 2022, 2:52 PM
Herald added a subscriber: kristof.beyls. · View Herald Transcript
sagarkulkarni19 requested review of this revision.Sep 26 2022, 2:52 PM
Herald added a subscriber: cfe-commits. · View Herald TranscriptSep 26 2022, 2:52 PM
Harbormaster completed remote builds in B188796: Diff 463031.Sep 26 2022, 2:52 PM
sagarkulkarni19 added a child revision: D134681: [Clang][AArch64][SME] Add outer product intrinsics.Sep 26 2022, 2:57 PM
sagarkulkarni19 added parent revisions: D134679: [Clang][AArch64][SME] Add intrinsics for reading streaming vector length, D134678: [Clang][AArch64][SME] Add intrinsics for ZA array load/store (LDR/STR), D134677: [Clang][AArch64][SME] Add ZA zeroing intrinsics, D128648: [Clang][AArch64][SME] Add vector read/write (mova) intrinsics , D127910: [Clang][AArch64][SME] Add vector load/store (ld1/st1) intrinsics.
bryanpkc commandeered this revision.Jan 21 2023, 2:23 PM
bryanpkc added a reviewer: sagarkulkarni19.
bryanpkc updated this revision to Diff 492682.Jan 27 2023, 2:51 AM
bryanpkc retitled this revision from [Clang][AArch64] Add SME svaddha and svaddva intrinsics to [Clang][AArch64][SME] Add intrinsics for adding vector elements to ZA tile.
bryanpkc edited the summary of this revision. (Show Details)

Rebased and cleaned up the patch.

Harbormaster completed remote builds in B210304: Diff 492682.Jan 27 2023, 2:52 AM
bryanpkc updated this revision to Diff 492692.Jan 27 2023, 3:37 AM
bryanpkc edited the summary of this revision. (Show Details)

Added _m suffix to the intrinsics as required by the amendment in https://github.com/ARM-software/acle/pull/218.

Harbormaster completed remote builds in B210312: Diff 492692.Jan 27 2023, 3:37 AM
bryanpkc updated this revision to Diff 526621.May 30 2023, 7:56 AM
Harbormaster completed remote builds in B235295: Diff 526621.May 30 2023, 7:57 AM
bryanpkc updated this revision to Diff 526683.May 30 2023, 9:56 AM
Harbormaster completed remote builds in B235344: Diff 526683.May 30 2023, 9:57 AM
bryanpkc removed parent revisions: D127910: [Clang][AArch64][SME] Add vector load/store (ld1/st1) intrinsics, D128648: [Clang][AArch64][SME] Add vector read/write (mova) intrinsics , D134677: [Clang][AArch64][SME] Add ZA zeroing intrinsics, D134678: [Clang][AArch64][SME] Add intrinsics for ZA array load/store (LDR/STR).Jun 4 2023, 11:18 PM
bryanpkc updated this revision to Diff 540840.Jul 16 2023, 5:54 PM

Removed the attribute macro from tests, as @sdesmalen suggested.

Harbormaster completed remote builds in B245707: Diff 540840.Jul 16 2023, 5:56 PM
sdesmalen accepted this revision.Jul 17 2023, 6:44 AM
This revision is now accepted and ready to land.Jul 17 2023, 6:44 AM
This revision was landed with ongoing or failed builds.Jul 20 2023, 2:58 AM
Closed by commit rG4ae900c06363: [Clang][AArch64][SME] Add intrinsics for adding vector elements to ZA tile (authored by bryanpkc). · Explain Why
This revision was automatically updated to reflect the committed changes.
bryanpkc added a commit: rG4ae900c06363: [Clang][AArch64][SME] Add intrinsics for adding vector elements to ZA tile.

Revision Contents

PathSize
clang/
include/
clang/
Basic/
20 lines
lib/
CodeGen/
6 lines
test/
CodeGen/
aarch64-sme-intrinsics/
116 lines
116 lines

Diff 526621

clang/include/clang/Basic/arm_sme.td

Show First 20 Lines • Show All 145 Lines • ▼ Show 20 Lines def NAME : SInst<"sv" # n_suffix, "nv", "", MergeNone,
[IsOverloadNone, IsStreamingCompatible, IsPreservesZA]>; [IsOverloadNone, IsStreamingCompatible, IsPreservesZA]>;
} }
} }
defm SVCNTSB : ZACount<"cntsb">; defm SVCNTSB : ZACount<"cntsb">;
defm SVCNTSH : ZACount<"cntsh">; defm SVCNTSH : ZACount<"cntsh">;
defm SVCNTSW : ZACount<"cntsw">; defm SVCNTSW : ZACount<"cntsw">;
defm SVCNTSD : ZACount<"cntsd">; defm SVCNTSD : ZACount<"cntsd">;
////////////////////////////////////////////////////////////////////////////////
// SME - ADDHA/ADDVA
multiclass ZAAdd<string n_suffix> {
let TargetGuard = "sme" in {
def NAME # _ZA32: SInst<"sv" # n_suffix # "_za32[_{d}]", "viPPd", "iUi", MergeOp1,
"aarch64_sme_" # n_suffix, [IsStreaming, IsSharedZA],
[ImmCheck<0, ImmCheck0_3>]>;
}
let TargetGuard = "sme-i16i64" in {
def NAME # _ZA64: SInst<"sv" # n_suffix # "_za64[_{d}]", "viPPd", "lUl", MergeOp1,
"aarch64_sme_" # n_suffix, [IsStreaming, IsSharedZA],
[ImmCheck<0, ImmCheck0_7>]>;
}
}
defm SVADDHA : ZAAdd<"addha">;
defm SVADDVA : ZAAdd<"addva">;

clang/lib/CodeGen/CGBuiltin.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 9,949 Lines • ▼ Show 20 Lines if (TypeFlags.isLoad() || TypeFlags.isStore())
return EmitSMELd1St1(TypeFlags, Ops, Builtin->LLVMIntrinsic); return EmitSMELd1St1(TypeFlags, Ops, Builtin->LLVMIntrinsic);
else if (TypeFlags.isRead() || TypeFlags.isWrite()) else if (TypeFlags.isRead() || TypeFlags.isWrite())
return EmitSMEReadWrite(TypeFlags, Ops, Builtin->LLVMIntrinsic); return EmitSMEReadWrite(TypeFlags, Ops, Builtin->LLVMIntrinsic);
else if (TypeFlags.isZero()) else if (TypeFlags.isZero())
return EmitSMEZero(TypeFlags, Ops, Builtin->LLVMIntrinsic); return EmitSMEZero(TypeFlags, Ops, Builtin->LLVMIntrinsic);
else if (TypeFlags.isLoadReg() || TypeFlags.isStoreReg()) else if (TypeFlags.isLoadReg() || TypeFlags.isStoreReg())
return EmitSMELdrStr(TypeFlags, Ops, Builtin->LLVMIntrinsic); return EmitSMELdrStr(TypeFlags, Ops, Builtin->LLVMIntrinsic);
else if (Builtin->LLVMIntrinsic != 0) { else if (Builtin->LLVMIntrinsic != 0) {
// Predicates must match the main datatype.
for (unsigned i = 0, e = Ops.size(); i != e; ++i)
if (auto PredTy = dyn_cast<llvm::VectorType>(Ops[i]->getType()))
if (PredTy->getElementType()->isIntegerTy(1))
Ops[i] = EmitSVEPredicateCast(Ops[i], getSVEType(TypeFlags));
Function *F = CGM.getIntrinsic(Builtin->LLVMIntrinsic, Function *F = CGM.getIntrinsic(Builtin->LLVMIntrinsic,
getSVEOverloadTypes(TypeFlags, Ty, Ops)); getSVEOverloadTypes(TypeFlags, Ty, Ops));
Value *Call = Builder.CreateCall(F, Ops); Value *Call = Builder.CreateCall(F, Ops);
return Call; return Call;
} }
/// Should not happen /// Should not happen
return nullptr; return nullptr;
▲ Show 20 Lines • Show All 10,483 Lines • Show Last 20 Lines

clang/test/CodeGen/aarch64-sme-intrinsics/acle_sme_add-i32.c

  • This file was added.
// REQUIRES: aarch64-registered-target
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme -target-feature +sve -S -O1 -Werror -emit-llvm -o - %s | FileCheck %s -check-prefixes=CHECK,CHECK-C
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme -target-feature +sve -S -O1 -Werror -emit-llvm -o - -x c++ %s | FileCheck %s -check-prefixes=CHECK,CHECK-CXX
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -DSME_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sme -target-feature +sve -S -O1 -Werror -emit-llvm -o - %s | FileCheck %s -check-prefixes=CHECK,CHECK-C
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -DSME_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sme -target-feature +sve -S -O1 -Werror -emit-llvm -o - -x c++ %s | FileCheck %s -check-prefixes=CHECK,CHECK-CXX
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme -target-feature +sve -S -O1 -Werror -o /dev/null %s
#include <arm_sme_draft_spec_subject_to_change.h>
#ifdef DISABLE_SME_ATTRIBUTES
#define ARM_STREAMING_ATTR
#else
#define ARM_STREAMING_ATTR __attribute__((arm_streaming))
#endif
#ifdef SME_OVERLOADED_FORMS
#define SME_ACLE_FUNC(A1,A2_UNUSED,A3) A1##A3
#else
#define SME_ACLE_FUNC(A1,A2,A3) A1##A2##A3
#endif
// CHECK-C-LABEL: @test_svaddha_za32_u32(
// CHECK-CXX-LABEL: @_Z21test_svaddha_za32_u32u10__SVBool_tu10__SVBool_tu12__SVUint32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv4i32(i32 0, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za32_u32(svbool_t pn, svbool_t pm, svuint32_t zn) {
SME_ACLE_FUNC(svaddha_za32, _u32, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za32_u32_1(
// CHECK-CXX-LABEL: @_Z23test_svaddha_za32_u32_1u10__SVBool_tu10__SVBool_tu12__SVUint32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv4i32(i32 3, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za32_u32_1(svbool_t pn, svbool_t pm, svuint32_t zn) {
SME_ACLE_FUNC(svaddha_za32, _u32, _m)(3, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za32_s32(
// CHECK-CXX-LABEL: @_Z21test_svaddha_za32_s32u10__SVBool_tu10__SVBool_tu11__SVInt32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv4i32(i32 0, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za32_s32(svbool_t pn, svbool_t pm, svint32_t zn) {
SME_ACLE_FUNC(svaddha_za32, _s32, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za32_s32_1(
// CHECK-CXX-LABEL: @_Z23test_svaddha_za32_s32_1u10__SVBool_tu10__SVBool_tu11__SVInt32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv4i32(i32 3, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za32_s32_1(svbool_t pn, svbool_t pm, svint32_t zn) {
SME_ACLE_FUNC(svaddha_za32, _s32, _m)(3, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za32_u32(
// CHECK-CXX-LABEL: @_Z21test_svaddva_za32_u32u10__SVBool_tu10__SVBool_tu12__SVUint32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv4i32(i32 0, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za32_u32(svbool_t pn, svbool_t pm, svuint32_t zn) {
SME_ACLE_FUNC(svaddva_za32, _u32, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za32_u32_1(
// CHECK-CXX-LABEL: @_Z23test_svaddva_za32_u32_1u10__SVBool_tu10__SVBool_tu12__SVUint32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv4i32(i32 3, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za32_u32_1(svbool_t pn, svbool_t pm, svuint32_t zn) {
SME_ACLE_FUNC(svaddva_za32, _u32, _m)(3, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za32_s32(
// CHECK-CXX-LABEL: @_Z21test_svaddva_za32_s32u10__SVBool_tu10__SVBool_tu11__SVInt32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv4i32(i32 0, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za32_s32(svbool_t pn, svbool_t pm, svint32_t zn) {
SME_ACLE_FUNC(svaddva_za32, _s32, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za32_s32_1(
// CHECK-CXX-LABEL: @_Z23test_svaddva_za32_s32_1u10__SVBool_tu10__SVBool_tu11__SVInt32_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 4 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv4i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv4i32(i32 3, <vscale x 4 x i1> [[TMP0]], <vscale x 4 x i1> [[TMP1]], <vscale x 4 x i32> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za32_s32_1(svbool_t pn, svbool_t pm, svint32_t zn) {
SME_ACLE_FUNC(svaddva_za32, _s32, _m)(3, pn, pm, zn);
}

clang/test/CodeGen/aarch64-sme-intrinsics/acle_sme_add-i64.c

  • This file was added.
// REQUIRES: aarch64-registered-target
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme-i16i64 -target-feature +sve -S -O1 -Werror -emit-llvm -o - %s | FileCheck %s -check-prefixes=CHECK,CHECK-C
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme-i16i64 -target-feature +sve -S -O1 -Werror -emit-llvm -o - -x c++ %s | FileCheck %s -check-prefixes=CHECK,CHECK-CXX
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -DSME_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sme-i16i64 -target-feature +sve -S -O1 -Werror -emit-llvm -o - %s | FileCheck %s -check-prefixes=CHECK,CHECK-C
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -DSME_OVERLOADED_FORMS -triple aarch64-none-linux-gnu -target-feature +sme-i16i64 -target-feature +sve -S -O1 -Werror -emit-llvm -o - -x c++ %s | FileCheck %s -check-prefixes=CHECK,CHECK-CXX
// RUN: %clang_cc1 -DDISABLE_SME_ATTRIBUTES -triple aarch64-none-linux-gnu -target-feature +sme-i16i64 -target-feature +sve -S -O1 -Werror -o /dev/null %s
#include <arm_sme_draft_spec_subject_to_change.h>
#ifdef DISABLE_SME_ATTRIBUTES
#define ARM_STREAMING_ATTR
#else
#define ARM_STREAMING_ATTR __attribute__((arm_streaming))
#endif
#ifdef SME_OVERLOADED_FORMS
#define SME_ACLE_FUNC(A1,A2_UNUSED,A3) A1##A3
#else
#define SME_ACLE_FUNC(A1,A2,A3) A1##A2##A3
#endif
// CHECK-C-LABEL: @test_svaddha_za64_u64(
// CHECK-CXX-LABEL: @_Z21test_svaddha_za64_u64u10__SVBool_tu10__SVBool_tu12__SVUint64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv2i64(i32 0, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za64_u64(svbool_t pn, svbool_t pm, svuint64_t zn) {
SME_ACLE_FUNC(svaddha_za64, _u64, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za64_u64_1(
// CHECK-CXX-LABEL: @_Z23test_svaddha_za64_u64_1u10__SVBool_tu10__SVBool_tu12__SVUint64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv2i64(i32 7, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za64_u64_1(svbool_t pn, svbool_t pm, svuint64_t zn) {
SME_ACLE_FUNC(svaddha_za64, _u64, _m)(7, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za64_s64(
// CHECK-CXX-LABEL: @_Z21test_svaddha_za64_s64u10__SVBool_tu10__SVBool_tu11__SVInt64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv2i64(i32 0, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za64_s64(svbool_t pn, svbool_t pm, svint64_t zn) {
SME_ACLE_FUNC(svaddha_za64, _s64, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddha_za64_s64_1(
// CHECK-CXX-LABEL: @_Z23test_svaddha_za64_s64_1u10__SVBool_tu10__SVBool_tu11__SVInt64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addha.nxv2i64(i32 7, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddha_za64_s64_1(svbool_t pn, svbool_t pm, svint64_t zn) {
SME_ACLE_FUNC(svaddha_za64, _s64, _m)(7, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za64_u64(
// CHECK-CXX-LABEL: @_Z21test_svaddva_za64_u64u10__SVBool_tu10__SVBool_tu12__SVUint64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv2i64(i32 0, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za64_u64(svbool_t pn, svbool_t pm, svuint64_t zn) {
SME_ACLE_FUNC(svaddva_za64, _u64, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za64_u64_1(
// CHECK-CXX-LABEL: @_Z23test_svaddva_za64_u64_1u10__SVBool_tu10__SVBool_tu12__SVUint64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv2i64(i32 7, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za64_u64_1(svbool_t pn, svbool_t pm, svuint64_t zn) {
SME_ACLE_FUNC(svaddva_za64, _u64, _m)(7, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za64_s64(
// CHECK-CXX-LABEL: @_Z21test_svaddva_za64_s64u10__SVBool_tu10__SVBool_tu11__SVInt64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv2i64(i32 0, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za64_s64(svbool_t pn, svbool_t pm, svint64_t zn) {
SME_ACLE_FUNC(svaddva_za64, _s64, _m)(0, pn, pm, zn);
}
// CHECK-C-LABEL: @test_svaddva_za64_s64_1(
// CHECK-CXX-LABEL: @_Z23test_svaddva_za64_s64_1u10__SVBool_tu10__SVBool_tu11__SVInt64_t(
// CHECK-NEXT: entry:
// CHECK-NEXT: [[TMP0:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PN:%.*]])
// CHECK-NEXT: [[TMP1:%.*]] = tail call <vscale x 2 x i1> @llvm.aarch64.sve.convert.from.svbool.nxv2i1(<vscale x 16 x i1> [[PM:%.*]])
// CHECK-NEXT: tail call void @llvm.aarch64.sme.addva.nxv2i64(i32 7, <vscale x 2 x i1> [[TMP0]], <vscale x 2 x i1> [[TMP1]], <vscale x 2 x i64> [[ZN:%.*]])
// CHECK-NEXT: ret void
//
ARM_STREAMING_ATTR void test_svaddva_za64_s64_1(svbool_t pn, svbool_t pm, svint64_t zn) {
SME_ACLE_FUNC(svaddva_za64, _s64, _m)(7, pn, pm, zn);
}