This is an archive of the discontinued LLVM Phabricator instance.

Differential D73633

[AArch64][SVE] Add remaining SVE2 mla indexed intrinsics.
ClosedPublic

Authored by dancgr on Jan 29 2020, 8:25 AM.

Details

Reviewers
efriedma
sdesmalen
cameron.mcinally
c-rhodes
rengolin
kmclaughlin
Commits
rG0610637aac9c: [AArch64][SVE] Add remaining SVE2 mla indexed intrinsics.
Summary

Add remaining SVE2 mla indexed intrinsics:

  • sqdmlalb, sqdmlalt, sqdmlslb, sqdmlslt

Add suffix _lane and switch immediate types to i32 for all mla indexed intrinsics to align with ACLE builtin definitions.

Diff Detail

Event Timeline

dancgr created this revision.Jan 29 2020, 8:25 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 29 2020, 8:25 AM
Herald added subscribers: llvm-commits, psnobl, arphaman and 4 others. · View Herald Transcript
Harbormaster completed remote builds in B45249: Diff 241168.Jan 29 2020, 8:26 AM
dancgr added a comment.Jan 29 2020, 8:27 AM

@sdesmalen Hopefully all comments from D73576 were addressed with this patch.

dancgr edited the summary of this revision. (Show Details)Jan 29 2020, 1:37 PM
sdesmalen accepted this revision.Jan 30 2020, 7:13 AM

LGTM! Thanks for fixing up the naming and i32 immediates @dancgr!

This revision is now accepted and ready to land.Jan 30 2020, 7:13 AM
Closed by commit rG0610637aac9c: [AArch64][SVE] Add remaining SVE2 mla indexed intrinsics. (authored by dancgr). · Explain WhyJan 30 2020, 10:30 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
llvm/
include/
llvm/
IR/
22 lines
lib/
Target/
AArch64/
24 lines
8 lines
test/
CodeGen/
AArch64/
516 lines

Diff 241506

llvm/include/llvm/IR/IntrinsicsAArch64.td

Show First 20 Lines • Show All 1,094 Lines • ▼ Show 20 Lines class SVE2_2VectorArg_Imm_Narrowing_Intrinsic
llvm_i32_ty], llvm_i32_ty],
[IntrNoMem, ImmArg<2>]>; [IntrNoMem, ImmArg<2>]>;
class SVE2_3VectorArg_Indexed_Intrinsic class SVE2_3VectorArg_Indexed_Intrinsic
: Intrinsic<[llvm_anyvector_ty], : Intrinsic<[llvm_anyvector_ty],
[LLVMMatchType<0>, [LLVMMatchType<0>,
LLVMSubdivide2VectorType<0>, LLVMSubdivide2VectorType<0>,
LLVMSubdivide2VectorType<0>, LLVMSubdivide2VectorType<0>,
llvm_i64_ty], llvm_i32_ty],
[IntrNoMem, ImmArg<3>]>; [IntrNoMem, ImmArg<3>]>;
// NOTE: There is no relationship between these intrinsics beyond an attempt // NOTE: There is no relationship between these intrinsics beyond an attempt
// to reuse currently identical class definitions. // to reuse currently identical class definitions.
class AdvSIMD_SVE_LOGB_Intrinsic : AdvSIMD_SVE_CNT_Intrinsic; class AdvSIMD_SVE_LOGB_Intrinsic : AdvSIMD_SVE_CNT_Intrinsic;
// This class of intrinsics are not intended to be useful within LLVM IR but // This class of intrinsics are not intended to be useful within LLVM IR but
// are instead here to support some of the more regid parts of the ACLE. // are instead here to support some of the more regid parts of the ACLE.
▲ Show 20 Lines • Show All 674 Lines • ▼ Show 20 Lines
// Saturating shift right - signed input, unsigned output // Saturating shift right - signed input, unsigned output
def int_aarch64_sve_sqshrunb : SVE2_1VectorArg_Imm_Narrowing_Intrinsic; def int_aarch64_sve_sqshrunb : SVE2_1VectorArg_Imm_Narrowing_Intrinsic;
def int_aarch64_sve_sqshrunt : SVE2_2VectorArg_Imm_Narrowing_Intrinsic; def int_aarch64_sve_sqshrunt : SVE2_2VectorArg_Imm_Narrowing_Intrinsic;
def int_aarch64_sve_sqrshrunb : SVE2_1VectorArg_Imm_Narrowing_Intrinsic; def int_aarch64_sve_sqrshrunb : SVE2_1VectorArg_Imm_Narrowing_Intrinsic;
def int_aarch64_sve_sqrshrunt : SVE2_2VectorArg_Imm_Narrowing_Intrinsic; def int_aarch64_sve_sqrshrunt : SVE2_2VectorArg_Imm_Narrowing_Intrinsic;
def int_aarch64_sve_smlalb : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_smlalb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_smlalt : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_smlalt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_umlalb : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_umlalb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_umlalt : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_umlalt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_smlslb : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_smlslb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_smlslt : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_smlslt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_umlslb : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_umlslb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_umlslt : SVE2_3VectorArg_Indexed_Intrinsic; def int_aarch64_sve_umlslt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_sqdmlalb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_sqdmlalt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_sqdmlslb_lane : SVE2_3VectorArg_Indexed_Intrinsic;
def int_aarch64_sve_sqdmlslt_lane : SVE2_3VectorArg_Indexed_Intrinsic;
} }

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

Show First 20 Lines • Show All 1,461 Lines • ▼ Show 20 Lineslet Predicates = [HasSVE2] in {
defm UMULLB_ZZZI : sve2_int_mul_long_by_indexed_elem<0b010, "umullb">; defm UMULLB_ZZZI : sve2_int_mul_long_by_indexed_elem<0b010, "umullb">;
defm UMULLT_ZZZI : sve2_int_mul_long_by_indexed_elem<0b011, "umullt">; defm UMULLT_ZZZI : sve2_int_mul_long_by_indexed_elem<0b011, "umullt">;
// SVE2 saturating multiply (indexed) // SVE2 saturating multiply (indexed)
defm SQDMULLB_ZZZI : sve2_int_mul_long_by_indexed_elem<0b100, "sqdmullb">; defm SQDMULLB_ZZZI : sve2_int_mul_long_by_indexed_elem<0b100, "sqdmullb">;
defm SQDMULLT_ZZZI : sve2_int_mul_long_by_indexed_elem<0b101, "sqdmullt">; defm SQDMULLT_ZZZI : sve2_int_mul_long_by_indexed_elem<0b101, "sqdmullt">;
// SVE2 integer multiply-add long (indexed) // SVE2 integer multiply-add long (indexed)
defm SMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1000, "smlalb", int_aarch64_sve_smlalb>; defm SMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1000, "smlalb", int_aarch64_sve_smlalb_lane>;
defm SMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1001, "smlalt", int_aarch64_sve_smlalt>; defm SMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1001, "smlalt", int_aarch64_sve_smlalt_lane>;
defm UMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1010, "umlalb", int_aarch64_sve_umlalb>; defm UMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1010, "umlalb", int_aarch64_sve_umlalb_lane>;
defm UMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1011, "umlalt", int_aarch64_sve_umlalt>; defm UMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1011, "umlalt", int_aarch64_sve_umlalt_lane>;
defm SMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1100, "smlslb", int_aarch64_sve_smlslb>; defm SMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1100, "smlslb", int_aarch64_sve_smlslb_lane>;
defm SMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1101, "smlslt", int_aarch64_sve_smlslt>; defm SMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1101, "smlslt", int_aarch64_sve_smlslt_lane>;
defm UMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1110, "umlslb", int_aarch64_sve_umlslb>; defm UMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1110, "umlslb", int_aarch64_sve_umlslb_lane>;
defm UMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1111, "umlslt", int_aarch64_sve_umlslt>; defm UMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b1111, "umlslt", int_aarch64_sve_umlslt_lane>;
// SVE2 integer multiply-add long (vectors, unpredicated) // SVE2 integer multiply-add long (vectors, unpredicated)
defm SMLALB_ZZZ : sve2_int_mla_long<0b10000, "smlalb">; defm SMLALB_ZZZ : sve2_int_mla_long<0b10000, "smlalb">;
defm SMLALT_ZZZ : sve2_int_mla_long<0b10001, "smlalt">; defm SMLALT_ZZZ : sve2_int_mla_long<0b10001, "smlalt">;
defm UMLALB_ZZZ : sve2_int_mla_long<0b10010, "umlalb">; defm UMLALB_ZZZ : sve2_int_mla_long<0b10010, "umlalb">;
defm UMLALT_ZZZ : sve2_int_mla_long<0b10011, "umlalt">; defm UMLALT_ZZZ : sve2_int_mla_long<0b10011, "umlalt">;
defm SMLSLB_ZZZ : sve2_int_mla_long<0b10100, "smlslb">; defm SMLSLB_ZZZ : sve2_int_mla_long<0b10100, "smlslb">;
defm SMLSLT_ZZZ : sve2_int_mla_long<0b10101, "smlslt">; defm SMLSLT_ZZZ : sve2_int_mla_long<0b10101, "smlslt">;
defm UMLSLB_ZZZ : sve2_int_mla_long<0b10110, "umlslb">; defm UMLSLB_ZZZ : sve2_int_mla_long<0b10110, "umlslb">;
defm UMLSLT_ZZZ : sve2_int_mla_long<0b10111, "umlslt">; defm UMLSLT_ZZZ : sve2_int_mla_long<0b10111, "umlslt">;
// SVE2 saturating multiply-add long (indexed) // SVE2 saturating multiply-add long (indexed)
defm SQDMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0100, "sqdmlalb", null_frag>; defm SQDMLALB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0100, "sqdmlalb", int_aarch64_sve_sqdmlalb_lane>;
defm SQDMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0101, "sqdmlalt", null_frag>; defm SQDMLALT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0101, "sqdmlalt", int_aarch64_sve_sqdmlalt_lane>;
defm SQDMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0110, "sqdmlslb", null_frag>; defm SQDMLSLB_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0110, "sqdmlslb", int_aarch64_sve_sqdmlslb_lane>;
defm SQDMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0111, "sqdmlslt", null_frag>; defm SQDMLSLT_ZZZI : sve2_int_mla_long_by_indexed_elem<0b0111, "sqdmlslt", int_aarch64_sve_sqdmlslt_lane>;
// SVE2 saturating multiply-add long (vectors, unpredicated) // SVE2 saturating multiply-add long (vectors, unpredicated)
defm SQDMLALB_ZZZ : sve2_int_mla_long<0b11000, "sqdmlalb">; defm SQDMLALB_ZZZ : sve2_int_mla_long<0b11000, "sqdmlalb">;
defm SQDMLALT_ZZZ : sve2_int_mla_long<0b11001, "sqdmlalt">; defm SQDMLALT_ZZZ : sve2_int_mla_long<0b11001, "sqdmlalt">;
defm SQDMLSLB_ZZZ : sve2_int_mla_long<0b11010, "sqdmlslb">; defm SQDMLSLB_ZZZ : sve2_int_mla_long<0b11010, "sqdmlslb">;
defm SQDMLSLT_ZZZ : sve2_int_mla_long<0b11011, "sqdmlslt">; defm SQDMLSLT_ZZZ : sve2_int_mla_long<0b11011, "sqdmlslt">;
// SVE2 saturating multiply-add interleaved long // SVE2 saturating multiply-add interleaved long
▲ Show 20 Lines • Show All 318 Lines • Show Last 20 Lines

llvm/lib/Target/AArch64/SVEInstrFormats.td

Show First 20 Lines • Show All 2,408 Lines • ▼ Show 20 Lines
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SVE2 Integer Multiply-Add Long - Indexed Group // SVE2 Integer Multiply-Add Long - Indexed Group
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
multiclass sve2_int_mla_long_by_indexed_elem<bits<4> opc, string asm, SDPatternOperator op> { multiclass sve2_int_mla_long_by_indexed_elem<bits<4> opc, string asm, SDPatternOperator op> {
def _S : sve2_int_mla_by_indexed_elem<0b10, { opc{3}, 0b0, opc{2-1}, ?, opc{0} }, def _S : sve2_int_mla_by_indexed_elem<0b10, { opc{3}, 0b0, opc{2-1}, ?, opc{0} },
asm, ZPR32, ZPR16, ZPR3b16, VectorIndexH> { asm, ZPR32, ZPR16, ZPR3b16, VectorIndexH32b> {
bits<3> Zm; bits<3> Zm;
bits<3> iop; bits<3> iop;
let Inst{20-19} = iop{2-1}; let Inst{20-19} = iop{2-1};
let Inst{18-16} = Zm; let Inst{18-16} = Zm;
let Inst{11} = iop{0}; let Inst{11} = iop{0};
} }
def _D : sve2_int_mla_by_indexed_elem<0b11, { opc{3}, 0b0, opc{2-1}, ?, opc{0} }, def _D : sve2_int_mla_by_indexed_elem<0b11, { opc{3}, 0b0, opc{2-1}, ?, opc{0} },
asm, ZPR64, ZPR32, ZPR4b32, VectorIndexS> { asm, ZPR64, ZPR32, ZPR4b32, VectorIndexS32b> {
bits<4> Zm; bits<4> Zm;
bits<2> iop; bits<2> iop;
let Inst{20} = iop{1}; let Inst{20} = iop{1};
let Inst{19-16} = Zm; let Inst{19-16} = Zm;
let Inst{11} = iop{0}; let Inst{11} = iop{0};
} }
def : SVE_4_Op_Imm_Pat<nxv4i32, op, nxv4i32, nxv8i16, nxv8i16, i64, VectorIndexH_timm, !cast<Instruction>(NAME # _S)>; def : SVE_4_Op_Imm_Pat<nxv4i32, op, nxv4i32, nxv8i16, nxv8i16, i32, VectorIndexH32b_timm, !cast<Instruction>(NAME # _S)>;
def : SVE_4_Op_Imm_Pat<nxv2i64, op, nxv2i64, nxv4i32, nxv4i32, i64, VectorIndexS_timm, !cast<Instruction>(NAME # _D)>; def : SVE_4_Op_Imm_Pat<nxv2i64, op, nxv2i64, nxv4i32, nxv4i32, i32, VectorIndexS32b_timm, !cast<Instruction>(NAME # _D)>;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SVE Integer Dot Product Group // SVE Integer Dot Product Group
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
class sve_intx_dot<bit sz, bit U, string asm, ZPRRegOp zprty1, class sve_intx_dot<bit sz, bit U, string asm, ZPRRegOp zprty1,
ZPRRegOp zprty2> ZPRRegOp zprty2>
▲ Show 20 Lines • Show All 4,442 Lines • Show Last 20 Lines

llvm/test/CodeGen/AArch64/sve2-mla-indexed.ll

; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sve2 < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sve2 < %s | FileCheck %s
; ;
; SMLALB ; SMLALB
; ;
define <vscale x 4 x i32> @smlalb_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlalb_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlalb_i32 ; CHECK-LABEL: smlalb_i32
; CHECK: smlalb z0.s, z1.h, z2.h[1] ; CHECK: smlalb z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @smlalb_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlalb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlalb_i32_2 ; CHECK-LABEL: smlalb_i32_2
; CHECK: smlalb z0.s, z1.h, z2.h[7] ; CHECK: smlalb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @smlalb_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlalb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlalb_i64 ; CHECK-LABEL: smlalb_i64
; CHECK: smlalb z0.d, z1.s, z2.s[0] ; CHECK: smlalb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @smlalb_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlalb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlalb_i64_2 ; CHECK-LABEL: smlalb_i64_2
; CHECK: smlalb z0.d, z1.s, z2.s[3] ; CHECK: smlalb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; SMLALT ; SMLALT
; ;
define <vscale x 4 x i32> @smlalt_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlalt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlalt_i32 ; CHECK-LABEL: smlalt_i32
; CHECK: smlalt z0.s, z1.h, z2.h[1] ; CHECK: smlalt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @smlalt_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlalt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlalt_i32_2 ; CHECK-LABEL: smlalt_i32_2
; CHECK: smlalt z0.s, z1.h, z2.h[7] ; CHECK: smlalt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @smlalt_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlalt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlalt_i64 ; CHECK-LABEL: smlalt_i64
; CHECK: smlalt z0.d, z1.s, z2.s[0] ; CHECK: smlalt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @smlalt_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlalt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlalt_i64_2 ; CHECK-LABEL: smlalt_i64_2
; CHECK: smlalt z0.d, z1.s, z2.s[3] ; CHECK: smlalt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; UMLALB ; UMLALB
; ;
define <vscale x 4 x i32> @umlalb_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlalb_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlalb_i32 ; CHECK-LABEL: umlalb_i32
; CHECK: umlalb z0.s, z1.h, z2.h[1] ; CHECK: umlalb z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @umlalb_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlalb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlalb_i32_2 ; CHECK-LABEL: umlalb_i32_2
; CHECK: umlalb z0.s, z1.h, z2.h[7] ; CHECK: umlalb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @umlalb_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlalb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlalb_i64 ; CHECK-LABEL: umlalb_i64
; CHECK: umlalb z0.d, z1.s, z2.s[0] ; CHECK: umlalb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @umlalb_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlalb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlalb_i64_2 ; CHECK-LABEL: umlalb_i64_2
; CHECK: umlalb z0.d, z1.s, z2.s[3] ; CHECK: umlalb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; UMLALT ; UMLALT
; ;
define <vscale x 4 x i32> @umlalt_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlalt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlalt_i32 ; CHECK-LABEL: umlalt_i32
; CHECK: umlalt z0.s, z1.h, z2.h[1] ; CHECK: umlalt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @umlalt_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlalt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlalt_i32_2 ; CHECK-LABEL: umlalt_i32_2
; CHECK: umlalt z0.s, z1.h, z2.h[7] ; CHECK: umlalt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @umlalt_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlalt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlalt_i64 ; CHECK-LABEL: umlalt_i64
; CHECK: umlalt z0.d, z1.s, z2.s[0] ; CHECK: umlalt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @umlalt_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlalt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlalt_i64_2 ; CHECK-LABEL: umlalt_i64_2
; CHECK: umlalt z0.d, z1.s, z2.s[3] ; CHECK: umlalt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; SMLSLB ; SMLSLB
; ;
define <vscale x 4 x i32> @smlslb_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlslb_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlslb_i32 ; CHECK-LABEL: smlslb_i32
; CHECK: smlslb z0.s, z1.h, z2.h[1] ; CHECK: smlslb z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @smlslb_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlslb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlslb_i32_2 ; CHECK-LABEL: smlslb_i32_2
; CHECK: smlslb z0.s, z1.h, z2.h[7] ; CHECK: smlslb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @smlslb_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlslb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlslb_i64 ; CHECK-LABEL: smlslb_i64
; CHECK: smlslb z0.d, z1.s, z2.s[0] ; CHECK: smlslb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @smlslb_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlslb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlslb_i64_2 ; CHECK-LABEL: smlslb_i64_2
; CHECK: smlslb z0.d, z1.s, z2.s[3] ; CHECK: smlslb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; SMLSLT ; SMLSLT
; ;
define <vscale x 4 x i32> @smlslt_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlslt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlslt_i32 ; CHECK-LABEL: smlslt_i32
; CHECK: smlslt z0.s, z1.h, z2.h[1] ; CHECK: smlslt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @smlslt_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @smlslt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: smlslt_i32_2 ; CHECK-LABEL: smlslt_i32_2
; CHECK: smlslt z0.s, z1.h, z2.h[7] ; CHECK: smlslt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @smlslt_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlslt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlslt_i64 ; CHECK-LABEL: smlslt_i64
; CHECK: smlslt z0.d, z1.s, z2.s[0] ; CHECK: smlslt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @smlslt_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @smlslt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: smlslt_i64_2 ; CHECK-LABEL: smlslt_i64_2
; CHECK: smlslt z0.d, z1.s, z2.s[3] ; CHECK: smlslt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; UMLSLB ; UMLSLB
; ;
define <vscale x 4 x i32> @umlslb_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlslb_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlslb_i32 ; CHECK-LABEL: umlslb_i32
; CHECK: umlslb z0.s, z1.h, z2.h[1] ; CHECK: umlslb z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @umlslb_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlslb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlslb_i32_2 ; CHECK-LABEL: umlslb_i32_2
; CHECK: umlslb z0.s, z1.h, z2.h[7] ; CHECK: umlslb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @umlslb_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlslb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlslb_i64 ; CHECK-LABEL: umlslb_i64
; CHECK: umlslb z0.d, z1.s, z2.s[0] ; CHECK: umlslb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @umlslb_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlslb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlslb_i64_2 ; CHECK-LABEL: umlslb_i64_2
; CHECK: umlslb z0.d, z1.s, z2.s[3] ; CHECK: umlslb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
; ;
; UMLSLT ; UMLSLT
; ;
define <vscale x 4 x i32> @umlslt_i32(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlslt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlslt_i32 ; CHECK-LABEL: umlslt_i32
; CHECK: umlslt z0.s, z1.h, z2.h[1] ; CHECK: umlslt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 1) i32 1)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 4 x i32> @umlslt_i32_2(<vscale x 4 x i32> %a, define <vscale x 4 x i32> @umlslt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) { <vscale x 8 x i16> %c) {
; CHECK-LABEL: umlslt_i32_2 ; CHECK-LABEL: umlslt_i32_2
; CHECK: umlslt z0.s, z1.h, z2.h[7] ; CHECK: umlslt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.nxv4i32(<vscale x 4 x i32> %a, %res = call <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b, <vscale x 8 x i16> %b,
<vscale x 8 x i16> %c, <vscale x 8 x i16> %c,
i64 7) i32 7)
ret <vscale x 4 x i32> %res ret <vscale x 4 x i32> %res
} }
define <vscale x 2 x i64> @umlslt_i64(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlslt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlslt_i64 ; CHECK-LABEL: umlslt_i64
; CHECK: umlslt z0.d, z1.s, z2.s[0] ; CHECK: umlslt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 0) i32 0)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
define <vscale x 2 x i64> @umlslt_i64_2(<vscale x 2 x i64> %a, define <vscale x 2 x i64> @umlslt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) { <vscale x 4 x i32> %c) {
; CHECK-LABEL: umlslt_i64_2 ; CHECK-LABEL: umlslt_i64_2
; CHECK: umlslt z0.d, z1.s, z2.s[3] ; CHECK: umlslt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.nxv2i64(<vscale x 2 x i64> %a, %res = call <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b, <vscale x 4 x i32> %b,
<vscale x 4 x i32> %c, <vscale x 4 x i32> %c,
i64 3) i32 3)
ret <vscale x 2 x i64> %res ret <vscale x 2 x i64> %res
} }
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) ;
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) ; SQDMLALB
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) ;
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) define <vscale x 4 x i32> @sqdmlalb_i32(<vscale x 4 x i32> %a,
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) <vscale x 8 x i16> %b,
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) <vscale x 8 x i16> %c) {
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) ; CHECK-LABEL: sqdmlalb_i32
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) ; CHECK: sqdmlalb z0.s, z1.h, z2.h[1]
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) ; CHECK-NEXT: ret
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) %res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) <vscale x 8 x i16> %b,
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) <vscale x 8 x i16> %c,
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) i32 1)
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64) ret <vscale x 4 x i32> %res
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i64) }
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i64)
define <vscale x 4 x i32> @sqdmlalb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlalb_i32_2
; CHECK: sqdmlalb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 7)
ret <vscale x 4 x i32> %res
}
define <vscale x 2 x i64> @sqdmlalb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlalb_i64
; CHECK: sqdmlalb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 0)
ret <vscale x 2 x i64> %res
}
define <vscale x 2 x i64> @sqdmlalb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlalb_i64_2
; CHECK: sqdmlalb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 3)
ret <vscale x 2 x i64> %res
}
;
; SQDMLALT
;
define <vscale x 4 x i32> @sqdmlalt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlalt_i32
; CHECK: sqdmlalt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 1)
ret <vscale x 4 x i32> %res
}
define <vscale x 4 x i32> @sqdmlalt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlalt_i32_2
; CHECK: sqdmlalt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 7)
ret <vscale x 4 x i32> %res
}
define <vscale x 2 x i64> @sqdmlalt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlalt_i64
; CHECK: sqdmlalt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 0)
ret <vscale x 2 x i64> %res
}
define <vscale x 2 x i64> @sqdmlalt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlalt_i64_2
; CHECK: sqdmlalt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 3)
ret <vscale x 2 x i64> %res
}
;
; SQDMLSLB
;
define <vscale x 4 x i32> @sqdmlslb_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlslb_i32
; CHECK: sqdmlslb z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 1)
ret <vscale x 4 x i32> %res
}
define <vscale x 4 x i32> @sqdmlslb_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlslb_i32_2
; CHECK: sqdmlslb z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslb.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 7)
ret <vscale x 4 x i32> %res
}
define <vscale x 2 x i64> @sqdmlslb_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlslb_i64
; CHECK: sqdmlslb z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 0)
ret <vscale x 2 x i64> %res
}
define <vscale x 2 x i64> @sqdmlslb_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlslb_i64_2
; CHECK: sqdmlslb z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslb.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 3)
ret <vscale x 2 x i64> %res
}
;
; SQDMLSLT
;
define <vscale x 4 x i32> @sqdmlslt_i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlslt_i32
; CHECK: sqdmlslt z0.s, z1.h, z2.h[1]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 1)
ret <vscale x 4 x i32> %res
}
define <vscale x 4 x i32> @sqdmlslt_i32_2(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c) {
; CHECK-LABEL: sqdmlslt_i32_2
; CHECK: sqdmlslt z0.s, z1.h, z2.h[7]
; CHECK-NEXT: ret
%res = call <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslt.lane.nxv4i32(<vscale x 4 x i32> %a,
<vscale x 8 x i16> %b,
<vscale x 8 x i16> %c,
i32 7)
ret <vscale x 4 x i32> %res
}
define <vscale x 2 x i64> @sqdmlslt_i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlslt_i64
; CHECK: sqdmlslt z0.d, z1.s, z2.s[0]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 0)
ret <vscale x 2 x i64> %res
}
define <vscale x 2 x i64> @sqdmlslt_i64_2(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c) {
; CHECK-LABEL: sqdmlslt_i64_2
; CHECK: sqdmlslt z0.d, z1.s, z2.s[3]
; CHECK-NEXT: ret
%res = call <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslt.lane.nxv2i64(<vscale x 2 x i64> %a,
<vscale x 4 x i32> %b,
<vscale x 4 x i32> %c,
i32 3)
ret <vscale x 2 x i64> %res
}
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlalb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlalb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlalt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlalt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlalb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlalb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlalt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlalt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlslb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlslb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.smlslt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.smlslt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlslb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlslb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.umlslt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.umlslt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlalt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlalt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslb.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslb.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)
declare <vscale x 4 x i32> @llvm.aarch64.sve.sqdmlslt.lane.nxv4i32(<vscale x 4 x i32>, <vscale x 8 x i16>, <vscale x 8 x i16>, i32)
declare <vscale x 2 x i64> @llvm.aarch64.sve.sqdmlslt.lane.nxv2i64(<vscale x 2 x i64>, <vscale x 4 x i32>, <vscale x 4 x i32>, i32)