This is an archive of the discontinued LLVM Phabricator instance.

Differential D106272

[AArch64][SME] Introduce feature for streaming mode
ClosedPublic

Authored by c-rhodes on Jul 19 2021, 6:15 AM.

Details

Reviewers
sdesmalen
david-arm
kmclaughlin
CarolineConcatto
paulwalker-arm
dmgreen
Commits
rG3a349d22692c: [AArch64][SME] Introduce feature for streaming mode
Summary

The Scalable Matrix Extension (SME) introduces a new execution mode
called Streaming SVE mode. In streaming mode a substantial subset of the
SVE and SVE2 instruction set is available, along with new outer product,
load, store, extract and insert instructions that operate on the new
architectural register state for the matrix.

To support streaming mode this patch introduces a new subtarget feature
+streaming-sve. If enabled, the subset of SVE(2) instructions are
available. The existing behaviour for SVE(2) remains unchanged, the
subset of instructions that are legal in streaming mode are enabled if
either +sve[2] or +streaming-sve is specified. Instructions that are
illegal in streaming mode remain predicated on +sve[2].

The SME target feature has been updated to imply +streaming-sve rather
than +sve.

The following changes are made to the SVE(2) tests:

  • For instructions that are legal in streaming mode:
    • added RUN line to verify +streaming-sve enables the instruction.
    • updated diagnostic to 'instruction requires: streaming-sve or sve'.
  • For instructions that are illegal in streaming-mode:
    • added RUN line to verify +streaming-sve does not enable the instruction.

SVE(2) instructions that are legal in streaming mode have:

if !HaveSVE[2]() && !HaveSME() then UNDEFINED;

at the top of the pseudocode in the XML.

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2021-06/SVE-Instructions

Diff Detail

Event Timeline

c-rhodes created this revision.Jul 19 2021, 6:15 AM
Herald added subscribers: danielkiss, arphaman, hiraditya, kristof.beyls. · View Herald TranscriptJul 19 2021, 6:15 AM
c-rhodes requested review of this revision.Jul 19 2021, 6:15 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 19 2021, 6:15 AM
Harbormaster completed remote builds in B114845: Diff 359769.Jul 19 2021, 7:25 AM
Matt added a subscriber: Matt.Jul 19 2021, 1:24 PM
sdesmalen added inline comments.Jul 21 2021, 2:52 AM
llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
618

Instead of moving these around, can you do:

let Predicates = [HasSVEOrStreamingSVE] in {
  ...
  defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>;
}

let Predicates = [HasSVE] in {
  defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>;
}

let Predicates = [HasSVEOrStreamingSVE] in {
  defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
  defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
  ...
}

That makes the patch easier to review, and we could possibly follow this up with a separate NFC change that shuffles these around in the file for readability (although I'm not sure yet if that's necessarily what we want).

tschuett added a subscriber: tschuett.Jul 21 2021, 3:00 AM
c-rhodes updated this revision to Diff 360737.Jul 22 2021, 2:21 AM

Address comments.

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
618

Instead of moving these around, can you do:

let Predicates = [HasSVEOrStreamingSVE] in {
  ...
  defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>;
}

let Predicates = [HasSVE] in {
  defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>;
}

let Predicates = [HasSVEOrStreamingSVE] in {
  defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
  defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
  ...
}

That makes the patch easier to review, and we could possibly follow this up with a separate NFC change that shuffles these around in the file for readability (although I'm not sure yet if that's necessarily what we want).

that's a better idea, done :)

Harbormaster completed remote builds in B115517: Diff 360737.Jul 22 2021, 3:04 AM
david-arm accepted this revision.Jul 23 2021, 6:23 AM

LGTM!

This revision is now accepted and ready to land.Jul 23 2021, 6:23 AM
sdesmalen added a comment.Jul 27 2021, 7:53 AM

Some things that are missing from this patch, but which can probably be followed up in separate patches:

  • A small subset of the AdvSimd instructions can be used under Streaming SVE, so we'll need something like hasNeonOrStreamingSVE as well.
  • We'll need to change calls to Subtarget->hasSVE() in e.g. AArch64ISelLowering.cpp or AArch64TargetTransformInfo.cpp to use Subtarget->hasSVEOrStreamingSVE() where appropriate.
llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
1604–1607

the BFMMLA instructions are illegal under streaming SVE mode.

2202

Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because the instructions they target otherwise don't exist.

c-rhodes added inline comments.Jul 27 2021, 8:10 AM
llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td
1604–1607

the BFMMLA instructions are illegal under streaming SVE mode.

BFMMLA is: https://developer.arm.com/documentation/ddi0602/2021-06/SVE-Instructions/BFMMLA--BFloat16-floating-point-matrix-multiply-accumulate-

but these variants aren't, e.g. https://developer.arm.com/documentation/ddi0602/2021-06/SVE-Instructions/BFMLALB--indexed---BFloat16-floating-point-multiply-add-long-to-single-precision--bottom--indexed--

2202

Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because the instructions they target otherwise don't exist.

Ah good spot, I'll fix this cheers.

c-rhodes updated this revision to Diff 362321.Jul 28 2021, 3:48 AM
c-rhodes set the repository for this revision to rG LLVM Github Monorepo.
  • Update let Predicates = [HasSVE, UseExperimentalZeroingPseudos] ... { for streaming mode.
  • Fix predicates for patterns selecting instructions that are illegal in streaming mode (contiguous non-faulting / first faulting loads).
c-rhodes marked an inline comment as done.Jul 28 2021, 3:52 AM
In D106272#2907429, @sdesmalen wrote:

Some things that are missing from this patch, but which can probably be followed up in separate patches:

  • A small subset of the AdvSimd instructions can be used under Streaming SVE, so we'll need something like hasNeonOrStreamingSVE as well.
  • We'll need to change calls to Subtarget->hasSVE() in e.g. AArch64ISelLowering.cpp or AArch64TargetTransformInfo.cpp to use Subtarget->hasSVEOrStreamingSVE() where appropriate.

I'll follow up with patches for those

Harbormaster completed remote builds in B116648: Diff 362321.Jul 28 2021, 4:42 AM
sdesmalen accepted this revision.Jul 29 2021, 2:53 AM

Cheers @c-rhodes

Herald added a subscriber: ctetreau. · View Herald TranscriptJul 29 2021, 2:53 AM
Closed by commit rG3a349d22692c: [AArch64][SME] Introduce feature for streaming mode (authored by c-rhodes). · Explain WhyJul 30 2021, 1:00 AM
This revision was automatically updated to reflect the committed changes.
c-rhodes added a commit: rG3a349d22692c: [AArch64][SME] Introduce feature for streaming mode.
c-rhodes mentioned this in D107807: [AArch64][SME] Support ptrue(s) in streaming mode.Aug 10 2021, 2:34 AM
c-rhodes mentioned this in rG1fe0e6a38045: [AArch64][SME] Support ptrue(s) in streaming mode.Aug 11 2021, 12:50 AM
c-rhodes mentioned this in D107903: [AArch64][SME] Support NEON vector to GPR integer moves in streaming mode.Aug 11 2021, 7:18 AM
c-rhodes mentioned this in rGdc5dd77ac700: [AArch64][SME] Support NEON vector to GPR integer moves in streaming mode.Sep 3 2021, 12:59 AM

Large Diff

This large diff affects 566 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
11 lines
12 lines
203 lines
3 lines
2 lines
test/
MC/
AArch64/
SME/
2 lines
8 lines
8 lines
SVE/
26 lines
106 lines
10 lines
10 lines
54 lines
8 lines
10 lines
70 lines
26 lines
18 lines
12 lines
12 lines
20 lines
62 lines
2 lines
52 lines
6 lines
6 lines
4 lines
6 lines
4 lines
6 lines
6 lines
6 lines
6 lines
6 lines
6 lines
30 lines
30 lines
18 lines
18 lines
32 lines
32 lines
32 lines
32 lines
32 lines
32 lines
32 lines
32 lines
32 lines
32 lines
18 lines
18 lines
14 lines
14 lines
14 lines
10 lines
14 lines
2 lines
120 lines
10 lines
10 lines
42 lines
42 lines
42 lines
34 lines
42 lines
86 lines
22 lines
22 lines
54 lines
8 lines
10 lines
10 lines
16 lines
16 lines
8 lines
8 lines
8 lines
8 lines
46 lines
2 lines
8 lines
22 lines
14 lines
14 lines
14 lines
46 lines
14 lines
14 lines
14 lines
8 lines
526 lines
22 lines
24 lines
24 lines
16 lines
16 lines
518 lines
2 lines
16 lines
38 lines
40 lines
8 lines
8 lines
40 lines
40 lines
8 lines
8 lines
26 lines
26 lines
540 lines
16 lines
56 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
8 lines
8 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
8 lines
8 lines
16 lines
16 lines
46 lines
40 lines
2 lines
2 lines
2 lines
68 lines
62 lines
62 lines
34 lines
62 lines
66 lines
34 lines
18 lines
18 lines
66 lines
100 lines
38 lines
62 lines
98 lines
118 lines
18 lines
6 lines
14 lines
12 lines
12 lines
12 lines
12 lines
14 lines
10 lines
6 lines
10 lines
60 lines
90 lines
68 lines
108 lines
38 lines
62 lines
68 lines
108 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
12 lines
12 lines
12 lines
12 lines
14 lines
70 lines
18 lines
70 lines
18 lines
18 lines
6 lines
80 lines
26 lines
18 lines
18 lines
260 lines
16 lines
10 lines
18 lines
38 lines
6 lines
6 lines
26 lines
6 lines
6 lines
22 lines
6 lines
26 lines
6 lines
60 lines
8 lines
10 lines
4 lines
6 lines
12 lines
69 lines
116 lines
69 lines
116 lines
69 lines
116 lines
69 lines
116 lines
6 lines
2 lines
2 lines
6 lines
6 lines
18 lines
2 lines
2 lines
10 lines
10 lines
16 lines
14 lines
12 lines
18 lines
8 lines
24 lines
14 lines
14 lines
18 lines
22 lines
2 lines
38 lines
10 lines
38 lines
10 lines
18 lines
14 lines
42 lines
80 lines
104 lines
104 lines
34 lines
104 lines
80 lines
104 lines
104 lines
34 lines
104 lines
42 lines
78 lines
108 lines
32 lines
66 lines
92 lines
128 lines
62 lines
118 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
12 lines
14 lines
106 lines
50 lines
8 lines
8 lines
22 lines
18 lines
14 lines
18 lines
18 lines
18 lines
18 lines
10 lines
24 lines
14 lines
14 lines
18 lines
38 lines
10 lines
38 lines
10 lines
18 lines
42 lines
80 lines
104 lines
104 lines
34 lines
104 lines
80 lines
104 lines
104 lines
34 lines
104 lines
42 lines
8 lines
8 lines
22 lines
18 lines
14 lines
18 lines
18 lines
22 lines
22 lines
22 lines
22 lines
2 lines
34 lines
34 lines
SVE2/
10 lines
10 lines
8 lines
8 lines
18 lines
2 lines
2 lines
2 lines
2 lines
14 lines
2 lines
2 lines
2 lines
8 lines
8 lines
8 lines
22 lines
30 lines
50 lines
2 lines
2 lines
2 lines
14 lines
14 lines
14 lines
6 lines
16 lines
6 lines
6 lines
14 lines
6 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
16 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
12 lines
12 lines
16 lines
8 lines
2 lines
6 lines
2 lines
6 lines
2 lines
6 lines
8 lines
8 lines
2 lines
14 lines
14 lines
8 lines
8 lines
14 lines
12 lines
12 lines
8 lines
8 lines
16 lines
8 lines
8 lines
8 lines
8 lines
8 lines
10 lines
10 lines
18 lines
14 lines
14 lines
18 lines
18 lines
18 lines
2 lines
2 lines
18 lines
18 lines
20 lines
20 lines
20 lines
20 lines
10 lines
12 lines
12 lines
10 lines
18 lines
18 lines
22 lines
20 lines
12 lines
20 lines
20 lines
12 lines
20 lines
16 lines
12 lines
12 lines
18 lines
50 lines
24 lines
24 lines
16 lines
18 lines
18 lines
14 lines
14 lines
14 lines
14 lines
42 lines
18 lines
26 lines
14 lines
14 lines
14 lines
14 lines
18 lines
18 lines
8 lines
8 lines
8 lines
8 lines
18 lines
18 lines
18 lines
18 lines
26 lines
22 lines
14 lines
14 lines
22 lines
8 lines
8 lines
8 lines
8 lines
8 lines
8 lines
2 lines
2 lines
2 lines
2 lines
8 lines
8 lines
18 lines
10 lines
10 lines
14 lines
12 lines
12 lines
8 lines
8 lines
16 lines
8 lines
8 lines
8 lines
8 lines
18 lines
18 lines
18 lines
18 lines
18 lines
20 lines
20 lines
20 lines
20 lines
10 lines
12 lines
12 lines
18 lines
18 lines
18 lines
14 lines
14 lines
42 lines
18 lines
14 lines
14 lines
18 lines
18 lines
8 lines
8 lines
12 lines
18 lines
18 lines
18 lines
26 lines
12 lines
22 lines
14 lines
14 lines
18 lines
22 lines
8 lines
8 lines
8 lines
8 lines
22 lines
22 lines
22 lines
22 lines
10 lines
10 lines
22 lines

Diff 362983

llvm/lib/Target/AArch64/AArch64.td

llvm/lib/Target/AArch64/AArch64InstrInfo.td

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

Show First 20 Lines • Show All 280 Lines • ▼ Show 20 Lines
def AArch64bic : SDNode<"AArch64ISD::BIC", SDT_AArch64Arith_Unpred>; def AArch64bic : SDNode<"AArch64ISD::BIC", SDT_AArch64Arith_Unpred>;
let Predicates = [HasSVE] in { let Predicates = [HasSVE] in {
defm RDFFR_PPz : sve_int_rdffr_pred<0b0, "rdffr", int_aarch64_sve_rdffr_z>; defm RDFFR_PPz : sve_int_rdffr_pred<0b0, "rdffr", int_aarch64_sve_rdffr_z>;
def RDFFRS_PPz : sve_int_rdffr_pred<0b1, "rdffrs">; def RDFFRS_PPz : sve_int_rdffr_pred<0b1, "rdffrs">;
defm RDFFR_P : sve_int_rdffr_unpred<"rdffr", int_aarch64_sve_rdffr>; defm RDFFR_P : sve_int_rdffr_unpred<"rdffr", int_aarch64_sve_rdffr>;
def SETFFR : sve_int_setffr<"setffr", int_aarch64_sve_setffr>; def SETFFR : sve_int_setffr<"setffr", int_aarch64_sve_setffr>;
def WRFFR : sve_int_wrffr<"wrffr", int_aarch64_sve_wrffr>; def WRFFR : sve_int_wrffr<"wrffr", int_aarch64_sve_wrffr>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm ADD_ZZZ : sve_int_bin_cons_arit_0<0b000, "add", add>; defm ADD_ZZZ : sve_int_bin_cons_arit_0<0b000, "add", add>;
defm SUB_ZZZ : sve_int_bin_cons_arit_0<0b001, "sub", sub>; defm SUB_ZZZ : sve_int_bin_cons_arit_0<0b001, "sub", sub>;
defm SQADD_ZZZ : sve_int_bin_cons_arit_0<0b100, "sqadd", saddsat>; defm SQADD_ZZZ : sve_int_bin_cons_arit_0<0b100, "sqadd", saddsat>;
defm UQADD_ZZZ : sve_int_bin_cons_arit_0<0b101, "uqadd", uaddsat>; defm UQADD_ZZZ : sve_int_bin_cons_arit_0<0b101, "uqadd", uaddsat>;
defm SQSUB_ZZZ : sve_int_bin_cons_arit_0<0b110, "sqsub", ssubsat>; defm SQSUB_ZZZ : sve_int_bin_cons_arit_0<0b110, "sqsub", ssubsat>;
defm UQSUB_ZZZ : sve_int_bin_cons_arit_0<0b111, "uqsub", usubsat>; defm UQSUB_ZZZ : sve_int_bin_cons_arit_0<0b111, "uqsub", usubsat>;
defm AND_ZZZ : sve_int_bin_cons_log<0b00, "and", and>; defm AND_ZZZ : sve_int_bin_cons_log<0b00, "and", and>;
defm ORR_ZZZ : sve_int_bin_cons_log<0b01, "orr", or>; defm ORR_ZZZ : sve_int_bin_cons_log<0b01, "orr", or>;
defm EOR_ZZZ : sve_int_bin_cons_log<0b10, "eor", xor>; defm EOR_ZZZ : sve_int_bin_cons_log<0b10, "eor", xor>;
defm BIC_ZZZ : sve_int_bin_cons_log<0b11, "bic", AArch64bic>; defm BIC_ZZZ : sve_int_bin_cons_log<0b11, "bic", AArch64bic>;
defm ADD_ZPmZ : sve_int_bin_pred_arit_0<0b000, "add", "ADD_ZPZZ", int_aarch64_sve_add, DestructiveBinaryComm>; defm ADD_ZPmZ : sve_int_bin_pred_arit_0<0b000, "add", "ADD_ZPZZ", int_aarch64_sve_add, DestructiveBinaryComm>;
defm SUB_ZPmZ : sve_int_bin_pred_arit_0<0b001, "sub", "SUB_ZPZZ", int_aarch64_sve_sub, DestructiveBinaryCommWithRev, "SUBR_ZPmZ">; defm SUB_ZPmZ : sve_int_bin_pred_arit_0<0b001, "sub", "SUB_ZPZZ", int_aarch64_sve_sub, DestructiveBinaryCommWithRev, "SUBR_ZPmZ">;
defm SUBR_ZPmZ : sve_int_bin_pred_arit_0<0b011, "subr", "SUBR_ZPZZ", int_aarch64_sve_subr, DestructiveBinaryCommWithRev, "SUB_ZPmZ", /*isReverseInstr*/ 1>; defm SUBR_ZPmZ : sve_int_bin_pred_arit_0<0b011, "subr", "SUBR_ZPZZ", int_aarch64_sve_subr, DestructiveBinaryCommWithRev, "SUB_ZPmZ", /*isReverseInstr*/ 1>;
defm ADD_ZPZZ : sve_int_bin_pred_bhsd<AArch64add_p>; defm ADD_ZPZZ : sve_int_bin_pred_bhsd<AArch64add_p>;
defm SUB_ZPZZ : sve_int_bin_pred_bhsd<AArch64sub_p>; defm SUB_ZPZZ : sve_int_bin_pred_bhsd<AArch64sub_p>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE, UseExperimentalZeroingPseudos] in { let Predicates = [HasSVEorStreamingSVE, UseExperimentalZeroingPseudos] in {
defm ADD_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_add>; defm ADD_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_add>;
defm SUB_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_sub>; defm SUB_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_sub>;
defm SUBR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_subr>; defm SUBR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_subr>;
} } // End HasSVEorStreamingSVE, UseExperimentalZeroingPseudos
let Predicates = [HasSVEorStreamingSVE] in {
defm ORR_ZPmZ : sve_int_bin_pred_log<0b000, "orr", int_aarch64_sve_orr>; defm ORR_ZPmZ : sve_int_bin_pred_log<0b000, "orr", int_aarch64_sve_orr>;
defm EOR_ZPmZ : sve_int_bin_pred_log<0b001, "eor", int_aarch64_sve_eor>; defm EOR_ZPmZ : sve_int_bin_pred_log<0b001, "eor", int_aarch64_sve_eor>;
defm AND_ZPmZ : sve_int_bin_pred_log<0b010, "and", int_aarch64_sve_and>; defm AND_ZPmZ : sve_int_bin_pred_log<0b010, "and", int_aarch64_sve_and>;
defm BIC_ZPmZ : sve_int_bin_pred_log<0b011, "bic", int_aarch64_sve_bic>; defm BIC_ZPmZ : sve_int_bin_pred_log<0b011, "bic", int_aarch64_sve_bic>;
defm ADD_ZI : sve_int_arith_imm0<0b000, "add", add>; defm ADD_ZI : sve_int_arith_imm0<0b000, "add", add>;
defm SUB_ZI : sve_int_arith_imm0<0b001, "sub", sub>; defm SUB_ZI : sve_int_arith_imm0<0b001, "sub", sub>;
defm SUBR_ZI : sve_int_arith_imm0_subr<0b011, "subr", sub>; defm SUBR_ZI : sve_int_arith_imm0_subr<0b011, "subr", sub>;
▲ Show 20 Lines • Show All 109 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm FADD_ZPZZ : sve_fp_bin_pred_hfd<AArch64fadd_p>; defm FADD_ZPZZ : sve_fp_bin_pred_hfd<AArch64fadd_p>;
defm FSUB_ZPZZ : sve_fp_bin_pred_hfd<AArch64fsub_p>; defm FSUB_ZPZZ : sve_fp_bin_pred_hfd<AArch64fsub_p>;
defm FMUL_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmul_p>; defm FMUL_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmul_p>;
defm FMAXNM_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmaxnm_p>; defm FMAXNM_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmaxnm_p>;
defm FMINNM_ZPZZ : sve_fp_bin_pred_hfd<AArch64fminnm_p>; defm FMINNM_ZPZZ : sve_fp_bin_pred_hfd<AArch64fminnm_p>;
defm FMAX_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmax_p>; defm FMAX_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmax_p>;
defm FMIN_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmin_p>; defm FMIN_ZPZZ : sve_fp_bin_pred_hfd<AArch64fmin_p>;
defm FDIV_ZPZZ : sve_fp_bin_pred_hfd<AArch64fdiv_p>; defm FDIV_ZPZZ : sve_fp_bin_pred_hfd<AArch64fdiv_p>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE, UseExperimentalZeroingPseudos] in { let Predicates = [HasSVEorStreamingSVE, UseExperimentalZeroingPseudos] in {
defm FADD_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fadd>; defm FADD_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fadd>;
defm FSUB_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fsub>; defm FSUB_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fsub>;
defm FMUL_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmul>; defm FMUL_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmul>;
defm FSUBR_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fsubr>; defm FSUBR_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fsubr>;
defm FMAXNM_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmaxnm>; defm FMAXNM_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmaxnm>;
defm FMINNM_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fminnm>; defm FMINNM_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fminnm>;
defm FMAX_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmax>; defm FMAX_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmax>;
defm FMIN_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmin>; defm FMIN_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmin>;
defm FABD_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fabd>; defm FABD_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fabd>;
defm FMULX_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmulx>; defm FMULX_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fmulx>;
defm FDIVR_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fdivr>; defm FDIVR_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fdivr>;
defm FDIV_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fdiv>; defm FDIV_ZPZZ : sve_fp_2op_p_zds_zeroing_hsd<int_aarch64_sve_fdiv>;
} } // End HasSVEorStreamingSVE, UseExperimentalZeroingPseudos
let Predicates = [HasSVEorStreamingSVE] in {
defm FADD_ZZZ : sve_fp_3op_u_zd<0b000, "fadd", fadd, AArch64fadd_p>; defm FADD_ZZZ : sve_fp_3op_u_zd<0b000, "fadd", fadd, AArch64fadd_p>;
defm FSUB_ZZZ : sve_fp_3op_u_zd<0b001, "fsub", fsub, AArch64fsub_p>; defm FSUB_ZZZ : sve_fp_3op_u_zd<0b001, "fsub", fsub, AArch64fsub_p>;
defm FMUL_ZZZ : sve_fp_3op_u_zd<0b010, "fmul", fmul, AArch64fmul_p>; defm FMUL_ZZZ : sve_fp_3op_u_zd<0b010, "fmul", fmul, AArch64fmul_p>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
defm FTSMUL_ZZZ : sve_fp_3op_u_zd_ftsmul<0b011, "ftsmul", int_aarch64_sve_ftsmul_x>; defm FTSMUL_ZZZ : sve_fp_3op_u_zd_ftsmul<0b011, "ftsmul", int_aarch64_sve_ftsmul_x>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm FRECPS_ZZZ : sve_fp_3op_u_zd<0b110, "frecps", int_aarch64_sve_frecps_x>; defm FRECPS_ZZZ : sve_fp_3op_u_zd<0b110, "frecps", int_aarch64_sve_frecps_x>;
defm FRSQRTS_ZZZ : sve_fp_3op_u_zd<0b111, "frsqrts", int_aarch64_sve_frsqrts_x>; defm FRSQRTS_ZZZ : sve_fp_3op_u_zd<0b111, "frsqrts", int_aarch64_sve_frsqrts_x>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
defm FTSSEL_ZZZ : sve_int_bin_cons_misc_0_b<"ftssel", int_aarch64_sve_ftssel_x>; defm FTSSEL_ZZZ : sve_int_bin_cons_misc_0_b<"ftssel", int_aarch64_sve_ftssel_x>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm FCADD_ZPmZ : sve_fp_fcadd<"fcadd", int_aarch64_sve_fcadd>; defm FCADD_ZPmZ : sve_fp_fcadd<"fcadd", int_aarch64_sve_fcadd>;
defm FCMLA_ZPmZZ : sve_fp_fcmla<"fcmla", int_aarch64_sve_fcmla>; defm FCMLA_ZPmZZ : sve_fp_fcmla<"fcmla", int_aarch64_sve_fcmla>;
defm FMLA_ZPmZZ : sve_fp_3op_p_zds_a<0b00, "fmla", "FMLA_ZPZZZ", int_aarch64_sve_fmla, "FMAD_ZPmZZ">; defm FMLA_ZPmZZ : sve_fp_3op_p_zds_a<0b00, "fmla", "FMLA_ZPZZZ", int_aarch64_sve_fmla, "FMAD_ZPmZZ">;
defm FMLS_ZPmZZ : sve_fp_3op_p_zds_a<0b01, "fmls", "FMLS_ZPZZZ", int_aarch64_sve_fmls, "FMSB_ZPmZZ">; defm FMLS_ZPmZZ : sve_fp_3op_p_zds_a<0b01, "fmls", "FMLS_ZPZZZ", int_aarch64_sve_fmls, "FMSB_ZPmZZ">;
defm FNMLA_ZPmZZ : sve_fp_3op_p_zds_a<0b10, "fnmla", "FNMLA_ZPZZZ", int_aarch64_sve_fnmla, "FNMAD_ZPmZZ">; defm FNMLA_ZPmZZ : sve_fp_3op_p_zds_a<0b10, "fnmla", "FNMLA_ZPZZZ", int_aarch64_sve_fnmla, "FNMAD_ZPmZZ">;
defm FNMLS_ZPmZZ : sve_fp_3op_p_zds_a<0b11, "fnmls", "FNMLS_ZPZZZ", int_aarch64_sve_fnmls, "FNMSB_ZPmZZ">; defm FNMLS_ZPmZZ : sve_fp_3op_p_zds_a<0b11, "fnmls", "FNMLS_ZPZZZ", int_aarch64_sve_fnmls, "FNMSB_ZPmZZ">;
Show All 38 Lineslet Predicates = [HasSVEorStreamingSVE] in {
} }
defm : fma<nxv8f16, nxv8i1, "H">; defm : fma<nxv8f16, nxv8i1, "H">;
defm : fma<nxv4f16, nxv4i1, "H">; defm : fma<nxv4f16, nxv4i1, "H">;
defm : fma<nxv2f16, nxv2i1, "H">; defm : fma<nxv2f16, nxv2i1, "H">;
defm : fma<nxv4f32, nxv4i1, "S">; defm : fma<nxv4f32, nxv4i1, "S">;
defm : fma<nxv2f32, nxv2i1, "S">; defm : fma<nxv2f32, nxv2i1, "S">;
defm : fma<nxv2f64, nxv2i1, "D">; defm : fma<nxv2f64, nxv2i1, "D">;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
defm FTMAD_ZZI : sve_fp_ftmad<"ftmad", int_aarch64_sve_ftmad_x>; defm FTMAD_ZZI : sve_fp_ftmad<"ftmad", int_aarch64_sve_ftmad_x>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm FMLA_ZZZI : sve_fp_fma_by_indexed_elem<0b0, "fmla", int_aarch64_sve_fmla_lane>; defm FMLA_ZZZI : sve_fp_fma_by_indexed_elem<0b0, "fmla", int_aarch64_sve_fmla_lane>;
defm FMLS_ZZZI : sve_fp_fma_by_indexed_elem<0b1, "fmls", int_aarch64_sve_fmls_lane>; defm FMLS_ZZZI : sve_fp_fma_by_indexed_elem<0b1, "fmls", int_aarch64_sve_fmls_lane>;
defm FCMLA_ZZZI : sve_fp_fcmla_by_indexed_elem<"fcmla", int_aarch64_sve_fcmla_lane>; defm FCMLA_ZZZI : sve_fp_fcmla_by_indexed_elem<"fcmla", int_aarch64_sve_fcmla_lane>;
defm FMUL_ZZZI : sve_fp_fmul_by_indexed_elem<"fmul", int_aarch64_sve_fmul_lane>; defm FMUL_ZZZI : sve_fp_fmul_by_indexed_elem<"fmul", int_aarch64_sve_fmul_lane>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
// SVE floating point reductions. // SVE floating point reductions.
defm FADDA_VPZ : sve_fp_2op_p_vd<0b000, "fadda", AArch64fadda_p>; defm FADDA_VPZ : sve_fp_2op_p_vd<0b000, "fadda", AArch64fadda_p>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm FADDV_VPZ : sve_fp_fast_red<0b000, "faddv", AArch64faddv_p>; defm FADDV_VPZ : sve_fp_fast_red<0b000, "faddv", AArch64faddv_p>;
defm FMAXNMV_VPZ : sve_fp_fast_red<0b100, "fmaxnmv", AArch64fmaxnmv_p>; defm FMAXNMV_VPZ : sve_fp_fast_red<0b100, "fmaxnmv", AArch64fmaxnmv_p>;
defm FMINNMV_VPZ : sve_fp_fast_red<0b101, "fminnmv", AArch64fminnmv_p>; defm FMINNMV_VPZ : sve_fp_fast_red<0b101, "fminnmv", AArch64fminnmv_p>;
defm FMAXV_VPZ : sve_fp_fast_red<0b110, "fmaxv", AArch64fmaxv_p>; defm FMAXV_VPZ : sve_fp_fast_red<0b110, "fmaxv", AArch64fmaxv_p>;
defm FMINV_VPZ : sve_fp_fast_red<0b111, "fminv", AArch64fminv_p>; defm FMINV_VPZ : sve_fp_fast_red<0b111, "fminv", AArch64fminv_p>;
// Splat immediate (unpredicated) // Splat immediate (unpredicated)
defm DUP_ZI : sve_int_dup_imm<"dup">; defm DUP_ZI : sve_int_dup_imm<"dup">;
▲ Show 20 Lines • Show All 71 Lines • ▼ Show 20 Lines let AddedComplexity = 2 in {
def : Pat<(nxv2f64 (AArch64dup fpimm64:$imm8)), def : Pat<(nxv2f64 (AArch64dup fpimm64:$imm8)),
(FDUP_ZI_D fpimm64:$imm8)>; (FDUP_ZI_D fpimm64:$imm8)>;
} }
// Select elements from either vector (predicated) // Select elements from either vector (predicated)
defm SEL_ZPZZ : sve_int_sel_vvv<"sel", vselect>; defm SEL_ZPZZ : sve_int_sel_vvv<"sel", vselect>;
defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>; defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>; defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>;
sdesmalenUnsubmitted
Not Done
ReplyInline Actions

Instead of moving these around, can you do:

let Predicates = [HasSVEOrStreamingSVE] in {
  ...
  defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>;
}

let Predicates = [HasSVE] in {
  defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>;
}

let Predicates = [HasSVEOrStreamingSVE] in {
  defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
  defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
  ...
}

That makes the patch easier to review, and we could possibly follow this up with a separate NFC change that shuffles these around in the file for readability (although I'm not sure yet if that's necessarily what we want).

sdesmalen: Instead of moving these around, can you do: let Predicates = [HasSVEOrStreamingSVE] in { .
c-rhodesAuthorUnsubmitted
Done
ReplyInline Actions

Instead of moving these around, can you do:

let Predicates = [HasSVEOrStreamingSVE] in {
  ...
  defm SPLICE_ZPZ : sve_int_perm_splice<"splice", AArch64splice>;
}

let Predicates = [HasSVE] in {
  defm COMPACT_ZPZ : sve_int_perm_compact<"compact", int_aarch64_sve_compact>;
}

let Predicates = [HasSVEOrStreamingSVE] in {
  defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
  defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
  defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
  ...
}

That makes the patch easier to review, and we could possibly follow this up with a separate NFC change that shuffles these around in the file for readability (although I'm not sure yet if that's necessarily what we want).

that's a better idea, done :)

c-rhodes: > Instead of moving these around, can you do: > let Predicates = [HasSVEOrStreamingSVE] in {…
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>; defm INSR_ZR : sve_int_perm_insrs<"insr", AArch64insr>;
defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>; defm INSR_ZV : sve_int_perm_insrv<"insr", AArch64insr>;
defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>; defm EXT_ZZI : sve_int_perm_extract_i<"ext", AArch64ext>;
defm RBIT_ZPmZ : sve_int_perm_rev_rbit<"rbit", AArch64rbit_mt>; defm RBIT_ZPmZ : sve_int_perm_rev_rbit<"rbit", AArch64rbit_mt>;
defm REVB_ZPmZ : sve_int_perm_rev_revb<"revb", AArch64revb_mt>; defm REVB_ZPmZ : sve_int_perm_rev_revb<"revb", AArch64revb_mt>;
defm REVH_ZPmZ : sve_int_perm_rev_revh<"revh", int_aarch64_sve_revh>; defm REVH_ZPmZ : sve_int_perm_rev_revh<"revh", int_aarch64_sve_revh>;
defm REVW_ZPmZ : sve_int_perm_rev_revw<"revw", int_aarch64_sve_revw>; defm REVW_ZPmZ : sve_int_perm_rev_revw<"revw", int_aarch64_sve_revw>;
defm REV_PP : sve_int_perm_reverse_p<"rev", vector_reverse>; defm REV_PP : sve_int_perm_reverse_p<"rev", vector_reverse>;
defm REV_ZZ : sve_int_perm_reverse_z<"rev", vector_reverse>; defm REV_ZZ : sve_int_perm_reverse_z<"rev", vector_reverse>;
defm SUNPKLO_ZZ : sve_int_perm_unpk<0b00, "sunpklo", AArch64sunpklo>; defm SUNPKLO_ZZ : sve_int_perm_unpk<0b00, "sunpklo", AArch64sunpklo>;
defm SUNPKHI_ZZ : sve_int_perm_unpk<0b01, "sunpkhi", AArch64sunpkhi>; defm SUNPKHI_ZZ : sve_int_perm_unpk<0b01, "sunpkhi", AArch64sunpkhi>;
defm UUNPKLO_ZZ : sve_int_perm_unpk<0b10, "uunpklo", AArch64uunpklo>; defm UUNPKLO_ZZ : sve_int_perm_unpk<0b10, "uunpklo", AArch64uunpklo>;
defm UUNPKHI_ZZ : sve_int_perm_unpk<0b11, "uunpkhi", AArch64uunpkhi>; defm UUNPKHI_ZZ : sve_int_perm_unpk<0b11, "uunpkhi", AArch64uunpkhi>;
defm PUNPKLO_PP : sve_int_perm_punpk<0b0, "punpklo", int_aarch64_sve_punpklo>; defm PUNPKLO_PP : sve_int_perm_punpk<0b0, "punpklo", int_aarch64_sve_punpklo>;
defm PUNPKHI_PP : sve_int_perm_punpk<0b1, "punpkhi", int_aarch64_sve_punpkhi>; defm PUNPKHI_PP : sve_int_perm_punpk<0b1, "punpkhi", int_aarch64_sve_punpkhi>;
defm MOVPRFX_ZPzZ : sve_int_movprfx_pred_zero<0b000, "movprfx">; defm MOVPRFX_ZPzZ : sve_int_movprfx_pred_zero<0b000, "movprfx">;
defm MOVPRFX_ZPmZ : sve_int_movprfx_pred_merge<0b001, "movprfx">; defm MOVPRFX_ZPmZ : sve_int_movprfx_pred_merge<0b001, "movprfx">;
def MOVPRFX_ZZ : sve_int_bin_cons_misc_0_c<0b00000001, "movprfx", ZPRAny>; def MOVPRFX_ZZ : sve_int_bin_cons_misc_0_c<0b00000001, "movprfx", ZPRAny>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
defm FEXPA_ZZ : sve_int_bin_cons_misc_0_c_fexpa<"fexpa", int_aarch64_sve_fexpa_x>; defm FEXPA_ZZ : sve_int_bin_cons_misc_0_c_fexpa<"fexpa", int_aarch64_sve_fexpa_x>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm BRKPA_PPzPP : sve_int_brkp<0b00, "brkpa", int_aarch64_sve_brkpa_z>; defm BRKPA_PPzPP : sve_int_brkp<0b00, "brkpa", int_aarch64_sve_brkpa_z>;
defm BRKPAS_PPzPP : sve_int_brkp<0b10, "brkpas", null_frag>; defm BRKPAS_PPzPP : sve_int_brkp<0b10, "brkpas", null_frag>;
defm BRKPB_PPzPP : sve_int_brkp<0b01, "brkpb", int_aarch64_sve_brkpb_z>; defm BRKPB_PPzPP : sve_int_brkp<0b01, "brkpb", int_aarch64_sve_brkpb_z>;
defm BRKPBS_PPzPP : sve_int_brkp<0b11, "brkpbs", null_frag>; defm BRKPBS_PPzPP : sve_int_brkp<0b11, "brkpbs", null_frag>;
defm BRKN_PPzP : sve_int_brkn<0b0, "brkn", int_aarch64_sve_brkn_z>; defm BRKN_PPzP : sve_int_brkn<0b0, "brkn", int_aarch64_sve_brkn_z>;
defm BRKNS_PPzP : sve_int_brkn<0b1, "brkns", null_frag>; defm BRKNS_PPzP : sve_int_brkn<0b1, "brkns", null_frag>;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm LD1SH_D : sve_mem_cld_ss<0b1000, "ld1sh", Z_d, ZPR64, GPR64NoXZRshifted16>; defm LD1SH_D : sve_mem_cld_ss<0b1000, "ld1sh", Z_d, ZPR64, GPR64NoXZRshifted16>;
defm LD1SH_S : sve_mem_cld_ss<0b1001, "ld1sh", Z_s, ZPR32, GPR64NoXZRshifted16>; defm LD1SH_S : sve_mem_cld_ss<0b1001, "ld1sh", Z_s, ZPR32, GPR64NoXZRshifted16>;
defm LD1W : sve_mem_cld_ss<0b1010, "ld1w", Z_s, ZPR32, GPR64NoXZRshifted32>; defm LD1W : sve_mem_cld_ss<0b1010, "ld1w", Z_s, ZPR32, GPR64NoXZRshifted32>;
defm LD1W_D : sve_mem_cld_ss<0b1011, "ld1w", Z_d, ZPR64, GPR64NoXZRshifted32>; defm LD1W_D : sve_mem_cld_ss<0b1011, "ld1w", Z_d, ZPR64, GPR64NoXZRshifted32>;
defm LD1SB_D : sve_mem_cld_ss<0b1100, "ld1sb", Z_d, ZPR64, GPR64NoXZRshifted8>; defm LD1SB_D : sve_mem_cld_ss<0b1100, "ld1sb", Z_d, ZPR64, GPR64NoXZRshifted8>;
defm LD1SB_S : sve_mem_cld_ss<0b1101, "ld1sb", Z_s, ZPR32, GPR64NoXZRshifted8>; defm LD1SB_S : sve_mem_cld_ss<0b1101, "ld1sb", Z_s, ZPR32, GPR64NoXZRshifted8>;
defm LD1SB_H : sve_mem_cld_ss<0b1110, "ld1sb", Z_h, ZPR16, GPR64NoXZRshifted8>; defm LD1SB_H : sve_mem_cld_ss<0b1110, "ld1sb", Z_h, ZPR16, GPR64NoXZRshifted8>;
defm LD1D : sve_mem_cld_ss<0b1111, "ld1d", Z_d, ZPR64, GPR64NoXZRshifted64>; defm LD1D : sve_mem_cld_ss<0b1111, "ld1d", Z_d, ZPR64, GPR64NoXZRshifted64>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
// non-faulting continuous load with reg+immediate // non-faulting continuous load with reg+immediate
defm LDNF1B_IMM : sve_mem_cldnf_si<0b0000, "ldnf1b", Z_b, ZPR8>; defm LDNF1B_IMM : sve_mem_cldnf_si<0b0000, "ldnf1b", Z_b, ZPR8>;
defm LDNF1B_H_IMM : sve_mem_cldnf_si<0b0001, "ldnf1b", Z_h, ZPR16>; defm LDNF1B_H_IMM : sve_mem_cldnf_si<0b0001, "ldnf1b", Z_h, ZPR16>;
defm LDNF1B_S_IMM : sve_mem_cldnf_si<0b0010, "ldnf1b", Z_s, ZPR32>; defm LDNF1B_S_IMM : sve_mem_cldnf_si<0b0010, "ldnf1b", Z_s, ZPR32>;
defm LDNF1B_D_IMM : sve_mem_cldnf_si<0b0011, "ldnf1b", Z_d, ZPR64>; defm LDNF1B_D_IMM : sve_mem_cldnf_si<0b0011, "ldnf1b", Z_d, ZPR64>;
defm LDNF1SW_D_IMM : sve_mem_cldnf_si<0b0100, "ldnf1sw", Z_d, ZPR64>; defm LDNF1SW_D_IMM : sve_mem_cldnf_si<0b0100, "ldnf1sw", Z_d, ZPR64>;
defm LDNF1H_IMM : sve_mem_cldnf_si<0b0101, "ldnf1h", Z_h, ZPR16>; defm LDNF1H_IMM : sve_mem_cldnf_si<0b0101, "ldnf1h", Z_h, ZPR16>;
defm LDNF1H_S_IMM : sve_mem_cldnf_si<0b0110, "ldnf1h", Z_s, ZPR32>; defm LDNF1H_S_IMM : sve_mem_cldnf_si<0b0110, "ldnf1h", Z_s, ZPR32>;
Show All 19 Lineslet Predicates = [HasSVE] in {
defm LDFF1SH_D : sve_mem_cldff_ss<0b1000, "ldff1sh", Z_d, ZPR64, GPR64shifted16>; defm LDFF1SH_D : sve_mem_cldff_ss<0b1000, "ldff1sh", Z_d, ZPR64, GPR64shifted16>;
defm LDFF1SH_S : sve_mem_cldff_ss<0b1001, "ldff1sh", Z_s, ZPR32, GPR64shifted16>; defm LDFF1SH_S : sve_mem_cldff_ss<0b1001, "ldff1sh", Z_s, ZPR32, GPR64shifted16>;
defm LDFF1W : sve_mem_cldff_ss<0b1010, "ldff1w", Z_s, ZPR32, GPR64shifted32>; defm LDFF1W : sve_mem_cldff_ss<0b1010, "ldff1w", Z_s, ZPR32, GPR64shifted32>;
defm LDFF1W_D : sve_mem_cldff_ss<0b1011, "ldff1w", Z_d, ZPR64, GPR64shifted32>; defm LDFF1W_D : sve_mem_cldff_ss<0b1011, "ldff1w", Z_d, ZPR64, GPR64shifted32>;
defm LDFF1SB_D : sve_mem_cldff_ss<0b1100, "ldff1sb", Z_d, ZPR64, GPR64shifted8>; defm LDFF1SB_D : sve_mem_cldff_ss<0b1100, "ldff1sb", Z_d, ZPR64, GPR64shifted8>;
defm LDFF1SB_S : sve_mem_cldff_ss<0b1101, "ldff1sb", Z_s, ZPR32, GPR64shifted8>; defm LDFF1SB_S : sve_mem_cldff_ss<0b1101, "ldff1sb", Z_s, ZPR32, GPR64shifted8>;
defm LDFF1SB_H : sve_mem_cldff_ss<0b1110, "ldff1sb", Z_h, ZPR16, GPR64shifted8>; defm LDFF1SB_H : sve_mem_cldff_ss<0b1110, "ldff1sb", Z_h, ZPR16, GPR64shifted8>;
defm LDFF1D : sve_mem_cldff_ss<0b1111, "ldff1d", Z_d, ZPR64, GPR64shifted64>; defm LDFF1D : sve_mem_cldff_ss<0b1111, "ldff1d", Z_d, ZPR64, GPR64shifted64>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
// LD(2|3|4) structured loads with reg+immediate // LD(2|3|4) structured loads with reg+immediate
defm LD2B_IMM : sve_mem_eld_si<0b00, 0b01, ZZ_b, "ld2b", simm4s2>; defm LD2B_IMM : sve_mem_eld_si<0b00, 0b01, ZZ_b, "ld2b", simm4s2>;
defm LD3B_IMM : sve_mem_eld_si<0b00, 0b10, ZZZ_b, "ld3b", simm4s3>; defm LD3B_IMM : sve_mem_eld_si<0b00, 0b10, ZZZ_b, "ld3b", simm4s3>;
defm LD4B_IMM : sve_mem_eld_si<0b00, 0b11, ZZZZ_b, "ld4b", simm4s4>; defm LD4B_IMM : sve_mem_eld_si<0b00, 0b11, ZZZZ_b, "ld4b", simm4s4>;
defm LD2H_IMM : sve_mem_eld_si<0b01, 0b01, ZZ_h, "ld2h", simm4s2>; defm LD2H_IMM : sve_mem_eld_si<0b01, 0b01, ZZ_h, "ld2h", simm4s2>;
defm LD3H_IMM : sve_mem_eld_si<0b01, 0b10, ZZZ_h, "ld3h", simm4s3>; defm LD3H_IMM : sve_mem_eld_si<0b01, 0b10, ZZZ_h, "ld3h", simm4s3>;
defm LD4H_IMM : sve_mem_eld_si<0b01, 0b11, ZZZZ_h, "ld4h", simm4s4>; defm LD4H_IMM : sve_mem_eld_si<0b01, 0b11, ZZZZ_h, "ld4h", simm4s4>;
defm LD2W_IMM : sve_mem_eld_si<0b10, 0b01, ZZ_s, "ld2w", simm4s2>; defm LD2W_IMM : sve_mem_eld_si<0b10, 0b01, ZZ_s, "ld2w", simm4s2>;
Show All 11 Lineslet Predicates = [HasSVEorStreamingSVE] in {
def LD3H : sve_mem_eld_ss<0b01, 0b10, ZZZ_h, "ld3h", GPR64NoXZRshifted16>; def LD3H : sve_mem_eld_ss<0b01, 0b10, ZZZ_h, "ld3h", GPR64NoXZRshifted16>;
def LD4H : sve_mem_eld_ss<0b01, 0b11, ZZZZ_h, "ld4h", GPR64NoXZRshifted16>; def LD4H : sve_mem_eld_ss<0b01, 0b11, ZZZZ_h, "ld4h", GPR64NoXZRshifted16>;
def LD2W : sve_mem_eld_ss<0b10, 0b01, ZZ_s, "ld2w", GPR64NoXZRshifted32>; def LD2W : sve_mem_eld_ss<0b10, 0b01, ZZ_s, "ld2w", GPR64NoXZRshifted32>;
def LD3W : sve_mem_eld_ss<0b10, 0b10, ZZZ_s, "ld3w", GPR64NoXZRshifted32>; def LD3W : sve_mem_eld_ss<0b10, 0b10, ZZZ_s, "ld3w", GPR64NoXZRshifted32>;
def LD4W : sve_mem_eld_ss<0b10, 0b11, ZZZZ_s, "ld4w", GPR64NoXZRshifted32>; def LD4W : sve_mem_eld_ss<0b10, 0b11, ZZZZ_s, "ld4w", GPR64NoXZRshifted32>;
def LD2D : sve_mem_eld_ss<0b11, 0b01, ZZ_d, "ld2d", GPR64NoXZRshifted64>; def LD2D : sve_mem_eld_ss<0b11, 0b01, ZZ_d, "ld2d", GPR64NoXZRshifted64>;
def LD3D : sve_mem_eld_ss<0b11, 0b10, ZZZ_d, "ld3d", GPR64NoXZRshifted64>; def LD3D : sve_mem_eld_ss<0b11, 0b10, ZZZ_d, "ld3d", GPR64NoXZRshifted64>;
def LD4D : sve_mem_eld_ss<0b11, 0b11, ZZZZ_d, "ld4d", GPR64NoXZRshifted64>; def LD4D : sve_mem_eld_ss<0b11, 0b11, ZZZZ_d, "ld4d", GPR64NoXZRshifted64>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
// Gathers using unscaled 32-bit offsets, e.g. // Gathers using unscaled 32-bit offsets, e.g.
// ld1h z0.s, p0/z, [x0, z0.s, uxtw] // ld1h z0.s, p0/z, [x0, z0.s, uxtw]
defm GLD1SB_S : sve_mem_32b_gld_vs_32_unscaled<0b0000, "ld1sb", AArch64ld1s_gather_sxtw_z, AArch64ld1s_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>; defm GLD1SB_S : sve_mem_32b_gld_vs_32_unscaled<0b0000, "ld1sb", AArch64ld1s_gather_sxtw_z, AArch64ld1s_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>;
defm GLDFF1SB_S : sve_mem_32b_gld_vs_32_unscaled<0b0001, "ldff1sb", AArch64ldff1s_gather_sxtw_z, AArch64ldff1s_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>; defm GLDFF1SB_S : sve_mem_32b_gld_vs_32_unscaled<0b0001, "ldff1sb", AArch64ldff1s_gather_sxtw_z, AArch64ldff1s_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>;
defm GLD1B_S : sve_mem_32b_gld_vs_32_unscaled<0b0010, "ld1b", AArch64ld1_gather_sxtw_z, AArch64ld1_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>; defm GLD1B_S : sve_mem_32b_gld_vs_32_unscaled<0b0010, "ld1b", AArch64ld1_gather_sxtw_z, AArch64ld1_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>;
defm GLDFF1B_S : sve_mem_32b_gld_vs_32_unscaled<0b0011, "ldff1b", AArch64ldff1_gather_sxtw_z, AArch64ldff1_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>; defm GLDFF1B_S : sve_mem_32b_gld_vs_32_unscaled<0b0011, "ldff1b", AArch64ldff1_gather_sxtw_z, AArch64ldff1_gather_uxtw_z, ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, nxv4i8>;
defm GLD1SH_S : sve_mem_32b_gld_vs_32_unscaled<0b0100, "ld1sh", AArch64ld1s_gather_sxtw_z, AArch64ld1s_gather_uxtw_z, ZPR32ExtSXTW8, ZPR32ExtUXTW8, nxv4i16>; defm GLD1SH_S : sve_mem_32b_gld_vs_32_unscaled<0b0100, "ld1sh", AArch64ld1s_gather_sxtw_z, AArch64ld1s_gather_uxtw_z, ZPR32ExtSXTW8, ZPR32ExtUXTW8, nxv4i16>;
defm GLDFF1SH_S : sve_mem_32b_gld_vs_32_unscaled<0b0101, "ldff1sh", AArch64ldff1s_gather_sxtw_z, AArch64ldff1s_gather_uxtw_z, ZPR32ExtSXTW8, ZPR32ExtUXTW8, nxv4i16>; defm GLDFF1SH_S : sve_mem_32b_gld_vs_32_unscaled<0b0101, "ldff1sh", AArch64ldff1s_gather_sxtw_z, AArch64ldff1s_gather_uxtw_z, ZPR32ExtSXTW8, ZPR32ExtUXTW8, nxv4i16>;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lineslet Predicates = [HasSVE] in {
defm GLD1H_D : sve_mem_64b_gld_sv_32_scaled<0b0110, "ld1h", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW16, ZPR64ExtUXTW16, nxv2i16>; defm GLD1H_D : sve_mem_64b_gld_sv_32_scaled<0b0110, "ld1h", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW16, ZPR64ExtUXTW16, nxv2i16>;
defm GLDFF1H_D : sve_mem_64b_gld_sv_32_scaled<0b0111, "ldff1h", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW16, ZPR64ExtUXTW16, nxv2i16>; defm GLDFF1H_D : sve_mem_64b_gld_sv_32_scaled<0b0111, "ldff1h", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW16, ZPR64ExtUXTW16, nxv2i16>;
defm GLD1SW_D : sve_mem_64b_gld_sv_32_scaled<0b1000, "ld1sw", AArch64ld1s_gather_sxtw_scaled_z, AArch64ld1s_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>; defm GLD1SW_D : sve_mem_64b_gld_sv_32_scaled<0b1000, "ld1sw", AArch64ld1s_gather_sxtw_scaled_z, AArch64ld1s_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>;
defm GLDFF1SW_D : sve_mem_64b_gld_sv_32_scaled<0b1001, "ldff1sw", AArch64ldff1s_gather_sxtw_scaled_z, AArch64ldff1s_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>; defm GLDFF1SW_D : sve_mem_64b_gld_sv_32_scaled<0b1001, "ldff1sw", AArch64ldff1s_gather_sxtw_scaled_z, AArch64ldff1s_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>;
defm GLD1W_D : sve_mem_64b_gld_sv_32_scaled<0b1010, "ld1w", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>; defm GLD1W_D : sve_mem_64b_gld_sv_32_scaled<0b1010, "ld1w", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>;
defm GLDFF1W_D : sve_mem_64b_gld_sv_32_scaled<0b1011, "ldff1w", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>; defm GLDFF1W_D : sve_mem_64b_gld_sv_32_scaled<0b1011, "ldff1w", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW32, ZPR64ExtUXTW32, nxv2i32>;
defm GLD1D : sve_mem_64b_gld_sv_32_scaled<0b1110, "ld1d", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW64, ZPR64ExtUXTW64, nxv2i64>; defm GLD1D : sve_mem_64b_gld_sv_32_scaled<0b1110, "ld1d", AArch64ld1_gather_sxtw_scaled_z, AArch64ld1_gather_uxtw_scaled_z, ZPR64ExtSXTW64, ZPR64ExtUXTW64, nxv2i64>;
defm GLDFF1D : sve_mem_64b_gld_sv_32_scaled<0b1111, "ldff1d", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW64, ZPR64ExtUXTW64, nxv2i64>; defm GLDFF1D : sve_mem_64b_gld_sv_32_scaled<0b1111, "ldff1d", AArch64ldff1_gather_sxtw_scaled_z, AArch64ldff1_gather_uxtw_scaled_z, ZPR64ExtSXTW64, ZPR64ExtUXTW64, nxv2i64>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
// Non-temporal contiguous loads (register + immediate) // Non-temporal contiguous loads (register + immediate)
defm LDNT1B_ZRI : sve_mem_cldnt_si<0b00, "ldnt1b", Z_b, ZPR8>; defm LDNT1B_ZRI : sve_mem_cldnt_si<0b00, "ldnt1b", Z_b, ZPR8>;
defm LDNT1H_ZRI : sve_mem_cldnt_si<0b01, "ldnt1h", Z_h, ZPR16>; defm LDNT1H_ZRI : sve_mem_cldnt_si<0b01, "ldnt1h", Z_h, ZPR16>;
defm LDNT1W_ZRI : sve_mem_cldnt_si<0b10, "ldnt1w", Z_s, ZPR32>; defm LDNT1W_ZRI : sve_mem_cldnt_si<0b10, "ldnt1w", Z_s, ZPR32>;
defm LDNT1D_ZRI : sve_mem_cldnt_si<0b11, "ldnt1d", Z_d, ZPR64>; defm LDNT1D_ZRI : sve_mem_cldnt_si<0b11, "ldnt1d", Z_d, ZPR64>;
// Non-temporal contiguous loads (register + register) // Non-temporal contiguous loads (register + register)
defm LDNT1B_ZRR : sve_mem_cldnt_ss<0b00, "ldnt1b", Z_b, ZPR8, GPR64NoXZRshifted8>; defm LDNT1B_ZRR : sve_mem_cldnt_ss<0b00, "ldnt1b", Z_b, ZPR8, GPR64NoXZRshifted8>;
Show All 19 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm ST1B_S : sve_mem_cst_ss<0b0010, "st1b", Z_s, ZPR32, GPR64NoXZRshifted8>; defm ST1B_S : sve_mem_cst_ss<0b0010, "st1b", Z_s, ZPR32, GPR64NoXZRshifted8>;
defm ST1B_D : sve_mem_cst_ss<0b0011, "st1b", Z_d, ZPR64, GPR64NoXZRshifted8>; defm ST1B_D : sve_mem_cst_ss<0b0011, "st1b", Z_d, ZPR64, GPR64NoXZRshifted8>;
defm ST1H : sve_mem_cst_ss<0b0101, "st1h", Z_h, ZPR16, GPR64NoXZRshifted16>; defm ST1H : sve_mem_cst_ss<0b0101, "st1h", Z_h, ZPR16, GPR64NoXZRshifted16>;
defm ST1H_S : sve_mem_cst_ss<0b0110, "st1h", Z_s, ZPR32, GPR64NoXZRshifted16>; defm ST1H_S : sve_mem_cst_ss<0b0110, "st1h", Z_s, ZPR32, GPR64NoXZRshifted16>;
defm ST1H_D : sve_mem_cst_ss<0b0111, "st1h", Z_d, ZPR64, GPR64NoXZRshifted16>; defm ST1H_D : sve_mem_cst_ss<0b0111, "st1h", Z_d, ZPR64, GPR64NoXZRshifted16>;
defm ST1W : sve_mem_cst_ss<0b1010, "st1w", Z_s, ZPR32, GPR64NoXZRshifted32>; defm ST1W : sve_mem_cst_ss<0b1010, "st1w", Z_s, ZPR32, GPR64NoXZRshifted32>;
defm ST1W_D : sve_mem_cst_ss<0b1011, "st1w", Z_d, ZPR64, GPR64NoXZRshifted32>; defm ST1W_D : sve_mem_cst_ss<0b1011, "st1w", Z_d, ZPR64, GPR64NoXZRshifted32>;
defm ST1D : sve_mem_cst_ss<0b1111, "st1d", Z_d, ZPR64, GPR64NoXZRshifted64>; defm ST1D : sve_mem_cst_ss<0b1111, "st1d", Z_d, ZPR64, GPR64NoXZRshifted64>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
// Scatters using unpacked, unscaled 32-bit offsets, e.g. // Scatters using unpacked, unscaled 32-bit offsets, e.g.
// st1h z0.d, p0, [x0, z0.d, uxtw] // st1h z0.d, p0, [x0, z0.d, uxtw]
defm SST1B_D : sve_mem_64b_sst_sv_32_unscaled<0b000, "st1b", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8Only, ZPR64ExtUXTW8Only, nxv2i8>; defm SST1B_D : sve_mem_64b_sst_sv_32_unscaled<0b000, "st1b", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8Only, ZPR64ExtUXTW8Only, nxv2i8>;
defm SST1H_D : sve_mem_64b_sst_sv_32_unscaled<0b010, "st1h", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i16>; defm SST1H_D : sve_mem_64b_sst_sv_32_unscaled<0b010, "st1h", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i16>;
defm SST1W_D : sve_mem_64b_sst_sv_32_unscaled<0b100, "st1w", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i32>; defm SST1W_D : sve_mem_64b_sst_sv_32_unscaled<0b100, "st1w", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i32>;
defm SST1D : sve_mem_64b_sst_sv_32_unscaled<0b110, "st1d", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i64>; defm SST1D : sve_mem_64b_sst_sv_32_unscaled<0b110, "st1d", AArch64st1_scatter_sxtw, AArch64st1_scatter_uxtw, ZPR64ExtSXTW8, ZPR64ExtUXTW8, nxv2i64>;
// Scatters using packed, unscaled 32-bit offsets, e.g. // Scatters using packed, unscaled 32-bit offsets, e.g.
Show All 33 Lineslet Predicates = [HasSVE] in {
defm SST1W_D : sve_mem_sst_sv_64_unscaled<0b10, "st1w", AArch64st1_scatter, nxv2i32>; defm SST1W_D : sve_mem_sst_sv_64_unscaled<0b10, "st1w", AArch64st1_scatter, nxv2i32>;
defm SST1D : sve_mem_sst_sv_64_unscaled<0b11, "st1d", AArch64st1_scatter, nxv2i64>; defm SST1D : sve_mem_sst_sv_64_unscaled<0b11, "st1d", AArch64st1_scatter, nxv2i64>;
// Scatters using scaled 64-bit offsets, e.g. // Scatters using scaled 64-bit offsets, e.g.
// st1h z0.d, p0, [x0, z0.d, lsl #1] // st1h z0.d, p0, [x0, z0.d, lsl #1]
defm SST1H_D_SCALED : sve_mem_sst_sv_64_scaled<0b01, "st1h", AArch64st1_scatter_scaled, ZPR64ExtLSL16, nxv2i16>; defm SST1H_D_SCALED : sve_mem_sst_sv_64_scaled<0b01, "st1h", AArch64st1_scatter_scaled, ZPR64ExtLSL16, nxv2i16>;
defm SST1W_D_SCALED : sve_mem_sst_sv_64_scaled<0b10, "st1w", AArch64st1_scatter_scaled, ZPR64ExtLSL32, nxv2i32>; defm SST1W_D_SCALED : sve_mem_sst_sv_64_scaled<0b10, "st1w", AArch64st1_scatter_scaled, ZPR64ExtLSL32, nxv2i32>;
defm SST1D_SCALED : sve_mem_sst_sv_64_scaled<0b11, "st1d", AArch64st1_scatter_scaled, ZPR64ExtLSL64, nxv2i64>; defm SST1D_SCALED : sve_mem_sst_sv_64_scaled<0b11, "st1d", AArch64st1_scatter_scaled, ZPR64ExtLSL64, nxv2i64>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
// ST(2|3|4) structured stores (register + immediate) // ST(2|3|4) structured stores (register + immediate)
defm ST2B_IMM : sve_mem_est_si<0b00, 0b01, ZZ_b, "st2b", simm4s2>; defm ST2B_IMM : sve_mem_est_si<0b00, 0b01, ZZ_b, "st2b", simm4s2>;
defm ST3B_IMM : sve_mem_est_si<0b00, 0b10, ZZZ_b, "st3b", simm4s3>; defm ST3B_IMM : sve_mem_est_si<0b00, 0b10, ZZZ_b, "st3b", simm4s3>;
defm ST4B_IMM : sve_mem_est_si<0b00, 0b11, ZZZZ_b, "st4b", simm4s4>; defm ST4B_IMM : sve_mem_est_si<0b00, 0b11, ZZZZ_b, "st4b", simm4s4>;
defm ST2H_IMM : sve_mem_est_si<0b01, 0b01, ZZ_h, "st2h", simm4s2>; defm ST2H_IMM : sve_mem_est_si<0b01, 0b01, ZZ_h, "st2h", simm4s2>;
defm ST3H_IMM : sve_mem_est_si<0b01, 0b10, ZZZ_h, "st3h", simm4s3>; defm ST3H_IMM : sve_mem_est_si<0b01, 0b10, ZZZ_h, "st3h", simm4s3>;
defm ST4H_IMM : sve_mem_est_si<0b01, 0b11, ZZZZ_h, "st4h", simm4s4>; defm ST4H_IMM : sve_mem_est_si<0b01, 0b11, ZZZZ_h, "st4h", simm4s4>;
defm ST2W_IMM : sve_mem_est_si<0b10, 0b01, ZZ_s, "st2w", simm4s2>; defm ST2W_IMM : sve_mem_est_si<0b10, 0b01, ZZ_s, "st2w", simm4s2>;
▲ Show 20 Lines • Show All 64 Lines • ▼ Show 20 Lines multiclass sve_prefetch<SDPatternOperator prefetch, ValueType PredTy, Instruction RegImmInst, Instruction RegRegInst, int scale, ComplexPattern AddrCP> {
def _default : Pat<(prefetch (PredTy PPR_3b:$gp), GPR64:$base, (i32 sve_prfop:$prfop)), def _default : Pat<(prefetch (PredTy PPR_3b:$gp), GPR64:$base, (i32 sve_prfop:$prfop)),
(RegImmInst sve_prfop:$prfop, PPR_3b:$gp, GPR64:$base, (i64 0))>; (RegImmInst sve_prfop:$prfop, PPR_3b:$gp, GPR64:$base, (i64 0))>;
} }
defm : sve_prefetch<int_aarch64_sve_prf, nxv16i1, PRFB_PRI, PRFB_PRR, 0, am_sve_regreg_lsl0>; defm : sve_prefetch<int_aarch64_sve_prf, nxv16i1, PRFB_PRI, PRFB_PRR, 0, am_sve_regreg_lsl0>;
defm : sve_prefetch<int_aarch64_sve_prf, nxv8i1, PRFH_PRI, PRFH_PRR, 1, am_sve_regreg_lsl1>; defm : sve_prefetch<int_aarch64_sve_prf, nxv8i1, PRFH_PRI, PRFH_PRR, 1, am_sve_regreg_lsl1>;
defm : sve_prefetch<int_aarch64_sve_prf, nxv4i1, PRFW_PRI, PRFS_PRR, 2, am_sve_regreg_lsl2>; defm : sve_prefetch<int_aarch64_sve_prf, nxv4i1, PRFW_PRI, PRFS_PRR, 2, am_sve_regreg_lsl2>;
defm : sve_prefetch<int_aarch64_sve_prf, nxv2i1, PRFD_PRI, PRFD_PRR, 3, am_sve_regreg_lsl3>; defm : sve_prefetch<int_aarch64_sve_prf, nxv2i1, PRFD_PRI, PRFD_PRR, 3, am_sve_regreg_lsl3>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
// Gather prefetch using scaled 32-bit offsets, e.g. // Gather prefetch using scaled 32-bit offsets, e.g.
// prfh pldl1keep, p0, [x0, z0.s, uxtw #1] // prfh pldl1keep, p0, [x0, z0.s, uxtw #1]
defm PRFB_S : sve_mem_32b_prfm_sv_scaled<0b00, "prfb", ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, int_aarch64_sve_prfb_gather_sxtw_index, int_aarch64_sve_prfb_gather_uxtw_index>; defm PRFB_S : sve_mem_32b_prfm_sv_scaled<0b00, "prfb", ZPR32ExtSXTW8Only, ZPR32ExtUXTW8Only, int_aarch64_sve_prfb_gather_sxtw_index, int_aarch64_sve_prfb_gather_uxtw_index>;
defm PRFH_S : sve_mem_32b_prfm_sv_scaled<0b01, "prfh", ZPR32ExtSXTW16, ZPR32ExtUXTW16, int_aarch64_sve_prfh_gather_sxtw_index, int_aarch64_sve_prfh_gather_uxtw_index>; defm PRFH_S : sve_mem_32b_prfm_sv_scaled<0b01, "prfh", ZPR32ExtSXTW16, ZPR32ExtUXTW16, int_aarch64_sve_prfh_gather_sxtw_index, int_aarch64_sve_prfh_gather_uxtw_index>;
defm PRFW_S : sve_mem_32b_prfm_sv_scaled<0b10, "prfw", ZPR32ExtSXTW32, ZPR32ExtUXTW32, int_aarch64_sve_prfw_gather_sxtw_index, int_aarch64_sve_prfw_gather_uxtw_index>; defm PRFW_S : sve_mem_32b_prfm_sv_scaled<0b10, "prfw", ZPR32ExtSXTW32, ZPR32ExtUXTW32, int_aarch64_sve_prfw_gather_sxtw_index, int_aarch64_sve_prfw_gather_uxtw_index>;
defm PRFD_S : sve_mem_32b_prfm_sv_scaled<0b11, "prfd", ZPR32ExtSXTW64, ZPR32ExtUXTW64, int_aarch64_sve_prfd_gather_sxtw_index, int_aarch64_sve_prfd_gather_uxtw_index>; defm PRFD_S : sve_mem_32b_prfm_sv_scaled<0b11, "prfd", ZPR32ExtSXTW64, ZPR32ExtUXTW64, int_aarch64_sve_prfd_gather_sxtw_index, int_aarch64_sve_prfd_gather_uxtw_index>;
// Gather prefetch using unpacked, scaled 32-bit offsets, e.g. // Gather prefetch using unpacked, scaled 32-bit offsets, e.g.
Show All 40 Lineslet Predicates = [HasSVE] in {
def : Pat<(nxv2i64 (int_aarch64_sve_adrb nxv2i64:$Op1, nxv2i64:$Op2)), def : Pat<(nxv2i64 (int_aarch64_sve_adrb nxv2i64:$Op1, nxv2i64:$Op2)),
(ADR_LSL_ZZZ_D_0 $Op1, $Op2)>; (ADR_LSL_ZZZ_D_0 $Op1, $Op2)>;
def : Pat<(nxv2i64 (int_aarch64_sve_adrh nxv2i64:$Op1, nxv2i64:$Op2)), def : Pat<(nxv2i64 (int_aarch64_sve_adrh nxv2i64:$Op1, nxv2i64:$Op2)),
(ADR_LSL_ZZZ_D_1 $Op1, $Op2)>; (ADR_LSL_ZZZ_D_1 $Op1, $Op2)>;
def : Pat<(nxv2i64 (int_aarch64_sve_adrw nxv2i64:$Op1, nxv2i64:$Op2)), def : Pat<(nxv2i64 (int_aarch64_sve_adrw nxv2i64:$Op1, nxv2i64:$Op2)),
(ADR_LSL_ZZZ_D_2 $Op1, $Op2)>; (ADR_LSL_ZZZ_D_2 $Op1, $Op2)>;
def : Pat<(nxv2i64 (int_aarch64_sve_adrd nxv2i64:$Op1, nxv2i64:$Op2)), def : Pat<(nxv2i64 (int_aarch64_sve_adrd nxv2i64:$Op1, nxv2i64:$Op2)),
(ADR_LSL_ZZZ_D_3 $Op1, $Op2)>; (ADR_LSL_ZZZ_D_3 $Op1, $Op2)>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
defm TBL_ZZZ : sve_int_perm_tbl<"tbl", AArch64tbl>; defm TBL_ZZZ : sve_int_perm_tbl<"tbl", AArch64tbl>;
defm ZIP1_ZZZ : sve_int_perm_bin_perm_zz<0b000, "zip1", AArch64zip1>; defm ZIP1_ZZZ : sve_int_perm_bin_perm_zz<0b000, "zip1", AArch64zip1>;
defm ZIP2_ZZZ : sve_int_perm_bin_perm_zz<0b001, "zip2", AArch64zip2>; defm ZIP2_ZZZ : sve_int_perm_bin_perm_zz<0b001, "zip2", AArch64zip2>;
defm UZP1_ZZZ : sve_int_perm_bin_perm_zz<0b010, "uzp1", AArch64uzp1>; defm UZP1_ZZZ : sve_int_perm_bin_perm_zz<0b010, "uzp1", AArch64uzp1>;
defm UZP2_ZZZ : sve_int_perm_bin_perm_zz<0b011, "uzp2", AArch64uzp2>; defm UZP2_ZZZ : sve_int_perm_bin_perm_zz<0b011, "uzp2", AArch64uzp2>;
defm TRN1_ZZZ : sve_int_perm_bin_perm_zz<0b100, "trn1", AArch64trn1>; defm TRN1_ZZZ : sve_int_perm_bin_perm_zz<0b100, "trn1", AArch64trn1>;
defm TRN2_ZZZ : sve_int_perm_bin_perm_zz<0b101, "trn2", AArch64trn2>; defm TRN2_ZZZ : sve_int_perm_bin_perm_zz<0b101, "trn2", AArch64trn2>;
▲ Show 20 Lines • Show All 245 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm ASR_ZPmI : sve_int_bin_pred_shift_imm_right_dup<0b0000, "asr", "ASR_ZPZI", int_aarch64_sve_asr>; defm ASR_ZPmI : sve_int_bin_pred_shift_imm_right_dup<0b0000, "asr", "ASR_ZPZI", int_aarch64_sve_asr>;
defm LSR_ZPmI : sve_int_bin_pred_shift_imm_right_dup<0b0001, "lsr", "LSR_ZPZI", int_aarch64_sve_lsr>; defm LSR_ZPmI : sve_int_bin_pred_shift_imm_right_dup<0b0001, "lsr", "LSR_ZPZI", int_aarch64_sve_lsr>;
defm LSL_ZPmI : sve_int_bin_pred_shift_imm_left_dup< 0b0011, "lsl", "LSL_ZPZI", int_aarch64_sve_lsl>; defm LSL_ZPmI : sve_int_bin_pred_shift_imm_left_dup< 0b0011, "lsl", "LSL_ZPZI", int_aarch64_sve_lsl>;
defm ASRD_ZPmI : sve_int_bin_pred_shift_imm_right< 0b0100, "asrd", "ASRD_ZPZI", int_aarch64_sve_asrd>; defm ASRD_ZPmI : sve_int_bin_pred_shift_imm_right< 0b0100, "asrd", "ASRD_ZPZI", int_aarch64_sve_asrd>;
defm ASR_ZPZI : sve_int_shift_pred_bhsd<AArch64asr_p, SVEShiftImmR8, SVEShiftImmR16, SVEShiftImmR32, SVEShiftImmR64>; defm ASR_ZPZI : sve_int_shift_pred_bhsd<AArch64asr_p, SVEShiftImmR8, SVEShiftImmR16, SVEShiftImmR32, SVEShiftImmR64>;
defm LSR_ZPZI : sve_int_shift_pred_bhsd<AArch64lsr_p, SVEShiftImmR8, SVEShiftImmR16, SVEShiftImmR32, SVEShiftImmR64>; defm LSR_ZPZI : sve_int_shift_pred_bhsd<AArch64lsr_p, SVEShiftImmR8, SVEShiftImmR16, SVEShiftImmR32, SVEShiftImmR64>;
defm LSL_ZPZI : sve_int_shift_pred_bhsd<AArch64lsl_p, SVEShiftImmL8, SVEShiftImmL16, SVEShiftImmL32, SVEShiftImmL64>; defm LSL_ZPZI : sve_int_shift_pred_bhsd<AArch64lsl_p, SVEShiftImmL8, SVEShiftImmL16, SVEShiftImmL32, SVEShiftImmL64>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE, UseExperimentalZeroingPseudos] in { let Predicates = [HasSVEorStreamingSVE, UseExperimentalZeroingPseudos] in {
defm ASR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_asr>; defm ASR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_asr>;
defm LSR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_lsr>; defm LSR_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_lsr>;
defm LSL_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_lsl>; defm LSL_ZPZZ : sve_int_bin_pred_zeroing_bhsd<int_aarch64_sve_lsl>;
defm ASRD_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_asrd>; defm ASRD_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_asrd>;
} } // End HasSVEorStreamingSVE, UseExperimentalZeroingPseudos
let Predicates = [HasSVEorStreamingSVE] in {
defm ASR_ZPmZ : sve_int_bin_pred_shift<0b000, "asr", "ASR_ZPZZ", int_aarch64_sve_asr, "ASRR_ZPmZ">; defm ASR_ZPmZ : sve_int_bin_pred_shift<0b000, "asr", "ASR_ZPZZ", int_aarch64_sve_asr, "ASRR_ZPmZ">;
defm LSR_ZPmZ : sve_int_bin_pred_shift<0b001, "lsr", "LSR_ZPZZ", int_aarch64_sve_lsr, "LSRR_ZPmZ">; defm LSR_ZPmZ : sve_int_bin_pred_shift<0b001, "lsr", "LSR_ZPZZ", int_aarch64_sve_lsr, "LSRR_ZPmZ">;
defm LSL_ZPmZ : sve_int_bin_pred_shift<0b011, "lsl", "LSL_ZPZZ", int_aarch64_sve_lsl, "LSLR_ZPmZ">; defm LSL_ZPmZ : sve_int_bin_pred_shift<0b011, "lsl", "LSL_ZPZZ", int_aarch64_sve_lsl, "LSLR_ZPmZ">;
defm ASRR_ZPmZ : sve_int_bin_pred_shift<0b100, "asrr", "ASRR_ZPZZ", null_frag, "ASR_ZPmZ", /*isReverseInstr*/ 1>; defm ASRR_ZPmZ : sve_int_bin_pred_shift<0b100, "asrr", "ASRR_ZPZZ", null_frag, "ASR_ZPmZ", /*isReverseInstr*/ 1>;
defm LSRR_ZPmZ : sve_int_bin_pred_shift<0b101, "lsrr", "LSRR_ZPZZ", null_frag, "LSR_ZPmZ", /*isReverseInstr*/ 1>; defm LSRR_ZPmZ : sve_int_bin_pred_shift<0b101, "lsrr", "LSRR_ZPZZ", null_frag, "LSR_ZPmZ", /*isReverseInstr*/ 1>;
defm LSLR_ZPmZ : sve_int_bin_pred_shift<0b111, "lslr", "LSLR_ZPZZ", null_frag, "LSL_ZPmZ", /*isReverseInstr*/ 1>; defm LSLR_ZPmZ : sve_int_bin_pred_shift<0b111, "lslr", "LSLR_ZPZZ", null_frag, "LSL_ZPmZ", /*isReverseInstr*/ 1>;
defm ASR_ZPZZ : sve_int_bin_pred_bhsd<AArch64asr_p>; defm ASR_ZPZZ : sve_int_bin_pred_bhsd<AArch64asr_p>;
▲ Show 20 Lines • Show All 98 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm FRINTP_ZPmZ : sve_fp_2op_p_zd_HSD<0b00001, "frintp", AArch64frintp_mt>; defm FRINTP_ZPmZ : sve_fp_2op_p_zd_HSD<0b00001, "frintp", AArch64frintp_mt>;
defm FRINTM_ZPmZ : sve_fp_2op_p_zd_HSD<0b00010, "frintm", AArch64frintm_mt>; defm FRINTM_ZPmZ : sve_fp_2op_p_zd_HSD<0b00010, "frintm", AArch64frintm_mt>;
defm FRINTZ_ZPmZ : sve_fp_2op_p_zd_HSD<0b00011, "frintz", AArch64frintz_mt>; defm FRINTZ_ZPmZ : sve_fp_2op_p_zd_HSD<0b00011, "frintz", AArch64frintz_mt>;
defm FRINTA_ZPmZ : sve_fp_2op_p_zd_HSD<0b00100, "frinta", AArch64frinta_mt>; defm FRINTA_ZPmZ : sve_fp_2op_p_zd_HSD<0b00100, "frinta", AArch64frinta_mt>;
defm FRINTX_ZPmZ : sve_fp_2op_p_zd_HSD<0b00110, "frintx", AArch64frintx_mt>; defm FRINTX_ZPmZ : sve_fp_2op_p_zd_HSD<0b00110, "frintx", AArch64frintx_mt>;
defm FRINTI_ZPmZ : sve_fp_2op_p_zd_HSD<0b00111, "frinti", AArch64frinti_mt>; defm FRINTI_ZPmZ : sve_fp_2op_p_zd_HSD<0b00111, "frinti", AArch64frinti_mt>;
defm FRECPX_ZPmZ : sve_fp_2op_p_zd_HSD<0b01100, "frecpx", AArch64frecpx_mt>; defm FRECPX_ZPmZ : sve_fp_2op_p_zd_HSD<0b01100, "frecpx", AArch64frecpx_mt>;
defm FSQRT_ZPmZ : sve_fp_2op_p_zd_HSD<0b01101, "fsqrt", AArch64fsqrt_mt>; defm FSQRT_ZPmZ : sve_fp_2op_p_zd_HSD<0b01101, "fsqrt", AArch64fsqrt_mt>;
} // End HasSVEorStreamingSVE
let Predicates = [HasBF16, HasSVE] in { let Predicates = [HasBF16, HasSVEorStreamingSVE] in {
defm BFDOT_ZZZ : sve_bfloat_dot<"bfdot", int_aarch64_sve_bfdot>; defm BFDOT_ZZZ : sve_bfloat_dot<"bfdot", int_aarch64_sve_bfdot>;
defm BFDOT_ZZI : sve_bfloat_dot_indexed<"bfdot", int_aarch64_sve_bfdot_lane>; defm BFDOT_ZZI : sve_bfloat_dot_indexed<"bfdot", int_aarch64_sve_bfdot_lane>;
} // End HasBF16, HasSVEorStreamingSVE
let Predicates = [HasBF16, HasSVE] in {
defm BFMMLA_ZZZ : sve_bfloat_matmul<"bfmmla", int_aarch64_sve_bfmmla>; defm BFMMLA_ZZZ : sve_bfloat_matmul<"bfmmla", int_aarch64_sve_bfmmla>;
} // End HasBF16, HasSVE
let Predicates = [HasBF16, HasSVEorStreamingSVE] in {
defm BFMMLA_B_ZZZ : sve_bfloat_matmul_longvecl<0b0, "bfmlalb", int_aarch64_sve_bfmlalb>; defm BFMMLA_B_ZZZ : sve_bfloat_matmul_longvecl<0b0, "bfmlalb", int_aarch64_sve_bfmlalb>;
defm BFMMLA_T_ZZZ : sve_bfloat_matmul_longvecl<0b1, "bfmlalt", int_aarch64_sve_bfmlalt>; defm BFMMLA_T_ZZZ : sve_bfloat_matmul_longvecl<0b1, "bfmlalt", int_aarch64_sve_bfmlalt>;
defm BFMMLA_B_ZZI : sve_bfloat_matmul_longvecl_idx<0b0, "bfmlalb", int_aarch64_sve_bfmlalb_lane>; defm BFMMLA_B_ZZI : sve_bfloat_matmul_longvecl_idx<0b0, "bfmlalb", int_aarch64_sve_bfmlalb_lane>;
defm BFMMLA_T_ZZI : sve_bfloat_matmul_longvecl_idx<0b1, "bfmlalt", int_aarch64_sve_bfmlalt_lane>; defm BFMMLA_T_ZZI : sve_bfloat_matmul_longvecl_idx<0b1, "bfmlalt", int_aarch64_sve_bfmlalt_lane>;
sdesmalenUnsubmitted
Not Done
ReplyInline Actions

the BFMMLA instructions are illegal under streaming SVE mode.

sdesmalen: the BFMMLA instructions are illegal under streaming SVE mode.
c-rhodesAuthorUnsubmitted
Done
ReplyInline Actions

the BFMMLA instructions are illegal under streaming SVE mode.

BFMMLA is: https://developer.arm.com/documentation/ddi0602/2021-06/SVE-Instructions/BFMMLA--BFloat16-floating-point-matrix-multiply-accumulate-

but these variants aren't, e.g. https://developer.arm.com/documentation/ddi0602/2021-06/SVE-Instructions/BFMLALB--indexed---BFloat16-floating-point-multiply-add-long-to-single-precision--bottom--indexed--

c-rhodes: > the BFMMLA instructions are illegal under streaming SVE mode. BFMMLA is: https://developer.
defm BFCVT_ZPmZ : sve_bfloat_convert<0b1, "bfcvt", int_aarch64_sve_fcvt_bf16f32>; defm BFCVT_ZPmZ : sve_bfloat_convert<0b1, "bfcvt", int_aarch64_sve_fcvt_bf16f32>;
defm BFCVTNT_ZPmZ : sve_bfloat_convert<0b0, "bfcvtnt", int_aarch64_sve_fcvtnt_bf16f32>; defm BFCVTNT_ZPmZ : sve_bfloat_convert<0b0, "bfcvtnt", int_aarch64_sve_fcvtnt_bf16f32>;
} } // End HasBF16, HasSVEorStreamingSVE
let Predicates = [HasSVEorStreamingSVE] in {
// InstAliases // InstAliases
def : InstAlias<"mov $Zd, $Zn", def : InstAlias<"mov $Zd, $Zn",
(ORR_ZZZ ZPR64:$Zd, ZPR64:$Zn, ZPR64:$Zn), 1>; (ORR_ZZZ ZPR64:$Zd, ZPR64:$Zn, ZPR64:$Zn), 1>;
def : InstAlias<"mov $Pd, $Pg/m, $Pn", def : InstAlias<"mov $Pd, $Pg/m, $Pn",
(SEL_PPPP PPR8:$Pd, PPRAny:$Pg, PPR8:$Pn, PPR8:$Pd), 1>; (SEL_PPPP PPR8:$Pd, PPRAny:$Pg, PPR8:$Pn, PPR8:$Pd), 1>;
def : InstAlias<"mov $Pd, $Pn", def : InstAlias<"mov $Pd, $Pn",
(ORR_PPzPP PPR8:$Pd, PPR8:$Pn, PPR8:$Pn, PPR8:$Pn), 1>; (ORR_PPzPP PPR8:$Pd, PPR8:$Pn, PPR8:$Pn, PPR8:$Pn), 1>;
def : InstAlias<"mov $Pd, $Pg/z, $Pn", def : InstAlias<"mov $Pd, $Pg/z, $Pn",
▲ Show 20 Lines • Show All 557 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
defm : ld1<LD1B_H, LD1B_H_IMM, nxv8i16, AArch64ld1_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>; defm : ld1<LD1B_H, LD1B_H_IMM, nxv8i16, AArch64ld1_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>;
defm : ld1<LD1SB_H, LD1SB_H_IMM, nxv8i16, AArch64ld1s_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>; defm : ld1<LD1SB_H, LD1SB_H_IMM, nxv8i16, AArch64ld1s_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>;
defm : ld1<LD1H, LD1H_IMM, nxv8i16, AArch64ld1_z, nxv8i1, nxv8i16, am_sve_regreg_lsl1>; defm : ld1<LD1H, LD1H_IMM, nxv8i16, AArch64ld1_z, nxv8i1, nxv8i16, am_sve_regreg_lsl1>;
defm : ld1<LD1H, LD1H_IMM, nxv8f16, AArch64ld1_z, nxv8i1, nxv8f16, am_sve_regreg_lsl1>; defm : ld1<LD1H, LD1H_IMM, nxv8f16, AArch64ld1_z, nxv8i1, nxv8f16, am_sve_regreg_lsl1>;
defm : ld1<LD1H, LD1H_IMM, nxv8bf16, AArch64ld1_z, nxv8i1, nxv8bf16, am_sve_regreg_lsl1>; defm : ld1<LD1H, LD1H_IMM, nxv8bf16, AArch64ld1_z, nxv8i1, nxv8bf16, am_sve_regreg_lsl1>;
// 16-element contiguous loads // 16-element contiguous loads
defm : ld1<LD1B, LD1B_IMM, nxv16i8, AArch64ld1_z, nxv16i1, nxv16i8, am_sve_regreg_lsl0>; defm : ld1<LD1B, LD1B_IMM, nxv16i8, AArch64ld1_z, nxv16i1, nxv16i8, am_sve_regreg_lsl0>;
} // End HasSVEorStreamingSVE
let Predicates = [HasSVE] in {
multiclass ldnf1<Instruction I, ValueType Ty, SDPatternOperator Load, ValueType PredTy, ValueType MemVT> { multiclass ldnf1<Instruction I, ValueType Ty, SDPatternOperator Load, ValueType PredTy, ValueType MemVT> {
// scalar + immediate (mul vl) // scalar + immediate (mul vl)
let AddedComplexity = 1 in { let AddedComplexity = 1 in {
def : Pat<(Ty (Load (PredTy PPR:$gp), (am_sve_indexed_s4 GPR64sp:$base, simm4s1:$offset), MemVT)), def : Pat<(Ty (Load (PredTy PPR:$gp), (am_sve_indexed_s4 GPR64sp:$base, simm4s1:$offset), MemVT)),
(I PPR:$gp, GPR64sp:$base, simm4s1:$offset)>; (I PPR:$gp, GPR64sp:$base, simm4s1:$offset)>;
} }
// base // base
def : Pat<(Ty (Load (PredTy PPR:$gp), GPR64:$base, MemVT)), def : Pat<(Ty (Load (PredTy PPR:$gp), GPR64:$base, MemVT)),
(I PPR:$gp, GPR64sp:$base, (i64 0))>; (I PPR:$gp, GPR64sp:$base, (i64 0))>;
} }
// 2-element contiguous non-faulting loads // 2-element contiguous non-faulting loads
defm : ldnf1<LDNF1B_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i8>; defm : ldnf1<LDNF1B_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i8>;
sdesmalenUnsubmitted
Done
ReplyInline Actions

Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because the instructions they target otherwise don't exist.

sdesmalen: Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because…
c-rhodesAuthorUnsubmitted
Not Done
ReplyInline Actions

Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because the instructions they target otherwise don't exist.

Ah good spot, I'll fix this cheers.

c-rhodes: > Patterns must be similarly guarded by HasSVE. They're not valid under HasStreamingSVE because…
defm : ldnf1<LDNF1SB_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i8>; defm : ldnf1<LDNF1SB_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i8>;
defm : ldnf1<LDNF1H_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i16>; defm : ldnf1<LDNF1H_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i16>;
defm : ldnf1<LDNF1SH_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i16>; defm : ldnf1<LDNF1SH_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i16>;
defm : ldnf1<LDNF1W_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i32>; defm : ldnf1<LDNF1W_D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i32>;
defm : ldnf1<LDNF1SW_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i32>; defm : ldnf1<LDNF1SW_D_IMM, nxv2i64, AArch64ldnf1s_z, nxv2i1, nxv2i32>;
defm : ldnf1<LDNF1D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i64>; defm : ldnf1<LDNF1D_IMM, nxv2i64, AArch64ldnf1_z, nxv2i1, nxv2i64>;
defm : ldnf1<LDNF1D_IMM, nxv2f64, AArch64ldnf1_z, nxv2i1, nxv2f64>; defm : ldnf1<LDNF1D_IMM, nxv2f64, AArch64ldnf1_z, nxv2i1, nxv2f64>;
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lineslet Predicates = [HasSVE] in {
defm : ldff1<LDFF1B_H, nxv8i16, AArch64ldff1_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>; defm : ldff1<LDFF1B_H, nxv8i16, AArch64ldff1_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>;
defm : ldff1<LDFF1SB_H, nxv8i16, AArch64ldff1s_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>; defm : ldff1<LDFF1SB_H, nxv8i16, AArch64ldff1s_z, nxv8i1, nxv8i8, am_sve_regreg_lsl0>;
defm : ldff1<LDFF1H, nxv8i16, AArch64ldff1_z, nxv8i1, nxv8i16, am_sve_regreg_lsl1>; defm : ldff1<LDFF1H, nxv8i16, AArch64ldff1_z, nxv8i1, nxv8i16, am_sve_regreg_lsl1>;
defm : ldff1<LDFF1H, nxv8f16, AArch64ldff1_z, nxv8i1, nxv8f16, am_sve_regreg_lsl1>; defm : ldff1<LDFF1H, nxv8f16, AArch64ldff1_z, nxv8i1, nxv8f16, am_sve_regreg_lsl1>;
defm : ldff1<LDFF1H, nxv8bf16, AArch64ldff1_z, nxv8i1, nxv8bf16, am_sve_regreg_lsl1>; defm : ldff1<LDFF1H, nxv8bf16, AArch64ldff1_z, nxv8i1, nxv8bf16, am_sve_regreg_lsl1>;
// 16-element contiguous first faulting loads // 16-element contiguous first faulting loads
defm : ldff1<LDFF1B, nxv16i8, AArch64ldff1_z, nxv16i1, nxv16i8, am_sve_regreg_lsl0>; defm : ldff1<LDFF1B, nxv16i8, AArch64ldff1_z, nxv16i1, nxv16i8, am_sve_regreg_lsl0>;
} // End HasSVE
let Predicates = [HasSVEorStreamingSVE] in {
multiclass st1<Instruction RegRegInst, Instruction RegImmInst, ValueType Ty, multiclass st1<Instruction RegRegInst, Instruction RegImmInst, ValueType Ty,
SDPatternOperator Store, ValueType PredTy, ValueType MemVT, ComplexPattern AddrCP> { SDPatternOperator Store, ValueType PredTy, ValueType MemVT, ComplexPattern AddrCP> {
// reg + reg // reg + reg
let AddedComplexity = 1 in { let AddedComplexity = 1 in {
def : Pat<(Store (Ty ZPR:$vec), (AddrCP GPR64:$base, GPR64:$offset), (PredTy PPR:$gp), MemVT), def : Pat<(Store (Ty ZPR:$vec), (AddrCP GPR64:$base, GPR64:$offset), (PredTy PPR:$gp), MemVT),
(RegRegInst ZPR:$vec, PPR:$gp, GPR64sp:$base, GPR64:$offset)>; (RegRegInst ZPR:$vec, PPR:$gp, GPR64sp:$base, GPR64:$offset)>;
} }
▲ Show 20 Lines • Show All 213 Lines • ▼ Show 20 Lineslet Predicates = [HasSVEorStreamingSVE] in {
def : Pat<(nxv16i8 (vector_splice (nxv16i8 ZPR:$Z1), (nxv16i8 ZPR:$Z2), (i64 (sve_ext_imm_0_15 i32:$index)))), def : Pat<(nxv16i8 (vector_splice (nxv16i8 ZPR:$Z1), (nxv16i8 ZPR:$Z2), (i64 (sve_ext_imm_0_15 i32:$index)))),
(EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_15:$index)>; (EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_15:$index)>;
def : Pat<(nxv8i16 (vector_splice (nxv8i16 ZPR:$Z1), (nxv8i16 ZPR:$Z2), (i64 (sve_ext_imm_0_7 i32:$index)))), def : Pat<(nxv8i16 (vector_splice (nxv8i16 ZPR:$Z1), (nxv8i16 ZPR:$Z2), (i64 (sve_ext_imm_0_7 i32:$index)))),
(EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_7:$index)>; (EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_7:$index)>;
def : Pat<(nxv4i32 (vector_splice (nxv4i32 ZPR:$Z1), (nxv4i32 ZPR:$Z2), (i64 (sve_ext_imm_0_3 i32:$index)))), def : Pat<(nxv4i32 (vector_splice (nxv4i32 ZPR:$Z1), (nxv4i32 ZPR:$Z2), (i64 (sve_ext_imm_0_3 i32:$index)))),
(EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_3:$index)>; (EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_3:$index)>;
def : Pat<(nxv2i64 (vector_splice (nxv2i64 ZPR:$Z1), (nxv2i64 ZPR:$Z2), (i64 (sve_ext_imm_0_1 i32:$index)))), def : Pat<(nxv2i64 (vector_splice (nxv2i64 ZPR:$Z1), (nxv2i64 ZPR:$Z2), (i64 (sve_ext_imm_0_1 i32:$index)))),
(EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_1:$index)>; (EXT_ZZI ZPR:$Z1, ZPR:$Z2, sve_ext_imm_0_1:$index)>;
} } // End HasSVEorStreamingSVE
let Predicates = [HasSVE, HasMatMulInt8] in { let Predicates = [HasSVE, HasMatMulInt8] in {
defm SMMLA_ZZZ : sve_int_matmul<0b00, "smmla", int_aarch64_sve_smmla>; defm SMMLA_ZZZ : sve_int_matmul<0b00, "smmla", int_aarch64_sve_smmla>;
defm UMMLA_ZZZ : sve_int_matmul<0b11, "ummla", int_aarch64_sve_ummla>; defm UMMLA_ZZZ : sve_int_matmul<0b11, "ummla", int_aarch64_sve_ummla>;
defm USMMLA_ZZZ : sve_int_matmul<0b10, "usmmla", int_aarch64_sve_usmmla>; defm USMMLA_ZZZ : sve_int_matmul<0b10, "usmmla", int_aarch64_sve_usmmla>;
} // End HasSVE, HasMatMulInt8
let Predicates = [HasSVEorStreamingSVE, HasMatMulInt8] in {
defm USDOT_ZZZ : sve_int_dot_mixed<"usdot", int_aarch64_sve_usdot>; defm USDOT_ZZZ : sve_int_dot_mixed<"usdot", int_aarch64_sve_usdot>;
defm USDOT_ZZZI : sve_int_dot_mixed_indexed<0, "usdot", int_aarch64_sve_usdot_lane>; defm USDOT_ZZZI : sve_int_dot_mixed_indexed<0, "usdot", int_aarch64_sve_usdot_lane>;
defm SUDOT_ZZZI : sve_int_dot_mixed_indexed<1, "sudot", int_aarch64_sve_sudot_lane>; defm SUDOT_ZZZI : sve_int_dot_mixed_indexed<1, "sudot", int_aarch64_sve_sudot_lane>;
} } // End HasSVEorStreamingSVE, HasMatMulInt8
let Predicates = [HasSVE, HasMatMulFP32] in { let Predicates = [HasSVE, HasMatMulFP32] in {
defm FMMLA_ZZZ_S : sve_fp_matrix_mla<0, "fmmla", ZPR32, int_aarch64_sve_fmmla, nxv4f32>; defm FMMLA_ZZZ_S : sve_fp_matrix_mla<0, "fmmla", ZPR32, int_aarch64_sve_fmmla, nxv4f32>;
} } // End HasSVE, HasMatMulFP32
let Predicates = [HasSVE, HasMatMulFP64] in { let Predicates = [HasSVE, HasMatMulFP64] in {
defm FMMLA_ZZZ_D : sve_fp_matrix_mla<1, "fmmla", ZPR64, int_aarch64_sve_fmmla, nxv2f64>; defm FMMLA_ZZZ_D : sve_fp_matrix_mla<1, "fmmla", ZPR64, int_aarch64_sve_fmmla, nxv2f64>;
defm LD1RO_B_IMM : sve_mem_ldor_si<0b00, "ld1rob", Z_b, ZPR8, nxv16i8, nxv16i1, AArch64ld1ro_z>; defm LD1RO_B_IMM : sve_mem_ldor_si<0b00, "ld1rob", Z_b, ZPR8, nxv16i8, nxv16i1, AArch64ld1ro_z>;
defm LD1RO_H_IMM : sve_mem_ldor_si<0b01, "ld1roh", Z_h, ZPR16, nxv8i16, nxv8i1, AArch64ld1ro_z>; defm LD1RO_H_IMM : sve_mem_ldor_si<0b01, "ld1roh", Z_h, ZPR16, nxv8i16, nxv8i1, AArch64ld1ro_z>;
defm LD1RO_W_IMM : sve_mem_ldor_si<0b10, "ld1row", Z_s, ZPR32, nxv4i32, nxv4i1, AArch64ld1ro_z>; defm LD1RO_W_IMM : sve_mem_ldor_si<0b10, "ld1row", Z_s, ZPR32, nxv4i32, nxv4i1, AArch64ld1ro_z>;
defm LD1RO_D_IMM : sve_mem_ldor_si<0b11, "ld1rod", Z_d, ZPR64, nxv2i64, nxv2i1, AArch64ld1ro_z>; defm LD1RO_D_IMM : sve_mem_ldor_si<0b11, "ld1rod", Z_d, ZPR64, nxv2i64, nxv2i1, AArch64ld1ro_z>;
defm LD1RO_B : sve_mem_ldor_ss<0b00, "ld1rob", Z_b, ZPR8, GPR64NoXZRshifted8, nxv16i8, nxv16i1, AArch64ld1ro_z, am_sve_regreg_lsl0>; defm LD1RO_B : sve_mem_ldor_ss<0b00, "ld1rob", Z_b, ZPR8, GPR64NoXZRshifted8, nxv16i8, nxv16i1, AArch64ld1ro_z, am_sve_regreg_lsl0>;
defm LD1RO_H : sve_mem_ldor_ss<0b01, "ld1roh", Z_h, ZPR16, GPR64NoXZRshifted16, nxv8i16, nxv8i1, AArch64ld1ro_z, am_sve_regreg_lsl1>; defm LD1RO_H : sve_mem_ldor_ss<0b01, "ld1roh", Z_h, ZPR16, GPR64NoXZRshifted16, nxv8i16, nxv8i1, AArch64ld1ro_z, am_sve_regreg_lsl1>;
defm LD1RO_W : sve_mem_ldor_ss<0b10, "ld1row", Z_s, ZPR32, GPR64NoXZRshifted32, nxv4i32, nxv4i1, AArch64ld1ro_z, am_sve_regreg_lsl2>; defm LD1RO_W : sve_mem_ldor_ss<0b10, "ld1row", Z_s, ZPR32, GPR64NoXZRshifted32, nxv4i32, nxv4i1, AArch64ld1ro_z, am_sve_regreg_lsl2>;
defm LD1RO_D : sve_mem_ldor_ss<0b11, "ld1rod", Z_d, ZPR64, GPR64NoXZRshifted64, nxv2i64, nxv2i1, AArch64ld1ro_z, am_sve_regreg_lsl3>; defm LD1RO_D : sve_mem_ldor_ss<0b11, "ld1rod", Z_d, ZPR64, GPR64NoXZRshifted64, nxv2i64, nxv2i1, AArch64ld1ro_z, am_sve_regreg_lsl3>;
} // End HasSVE, HasMatMulFP64
let Predicates = [HasSVEorStreamingSVE, HasMatMulFP64] in {
defm ZIP1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b00, 0, "zip1", int_aarch64_sve_zip1q>; defm ZIP1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b00, 0, "zip1", int_aarch64_sve_zip1q>;
defm ZIP2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b00, 1, "zip2", int_aarch64_sve_zip2q>; defm ZIP2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b00, 1, "zip2", int_aarch64_sve_zip2q>;
defm UZP1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b01, 0, "uzp1", int_aarch64_sve_uzp1q>; defm UZP1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b01, 0, "uzp1", int_aarch64_sve_uzp1q>;
defm UZP2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b01, 1, "uzp2", int_aarch64_sve_uzp2q>; defm UZP2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b01, 1, "uzp2", int_aarch64_sve_uzp2q>;
defm TRN1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b11, 0, "trn1", int_aarch64_sve_trn1q>; defm TRN1_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b11, 0, "trn1", int_aarch64_sve_trn1q>;
defm TRN2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b11, 1, "trn2", int_aarch64_sve_trn2q>; defm TRN2_ZZZ_Q : sve_int_perm_bin_perm_128_zz<0b11, 1, "trn2", int_aarch64_sve_trn2q>;
} } // End HasSVEorStreamingSVE, HasMatMulFP64
let Predicates = [HasSVE2] in { let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 integer multiply-add (indexed) // SVE2 integer multiply-add (indexed)
defm MLA_ZZZI : sve2_int_mla_by_indexed_elem<0b01, 0b0, "mla", int_aarch64_sve_mla_lane>; defm MLA_ZZZI : sve2_int_mla_by_indexed_elem<0b01, 0b0, "mla", int_aarch64_sve_mla_lane>;
defm MLS_ZZZI : sve2_int_mla_by_indexed_elem<0b01, 0b1, "mls", int_aarch64_sve_mls_lane>; defm MLS_ZZZI : sve2_int_mla_by_indexed_elem<0b01, 0b1, "mls", int_aarch64_sve_mls_lane>;
// SVE2 saturating multiply-add high (indexed) // SVE2 saturating multiply-add high (indexed)
defm SQRDMLAH_ZZZI : sve2_int_mla_by_indexed_elem<0b10, 0b0, "sqrdmlah", int_aarch64_sve_sqrdmlah_lane>; defm SQRDMLAH_ZZZI : sve2_int_mla_by_indexed_elem<0b10, 0b0, "sqrdmlah", int_aarch64_sve_sqrdmlah_lane>;
defm SQRDMLSH_ZZZI : sve2_int_mla_by_indexed_elem<0b10, 0b1, "sqrdmlsh", int_aarch64_sve_sqrdmlsh_lane>; defm SQRDMLSH_ZZZI : sve2_int_mla_by_indexed_elem<0b10, 0b1, "sqrdmlsh", int_aarch64_sve_sqrdmlsh_lane>;
▲ Show 20 Lines • Show All 131 Lines • ▼ Show 20 Lineslet Predicates = [HasSVE2orStreamingSVE] in {
defm UQRSHLR_ZPmZ : sve2_int_arith_pred<0b011110, "uqrshlr", null_frag, "UQRSHLR_ZPZZ", DestructiveBinaryCommWithRev, "UQRSHL_ZPmZ", /*isReverseInstr*/ 1>; defm UQRSHLR_ZPmZ : sve2_int_arith_pred<0b011110, "uqrshlr", null_frag, "UQRSHLR_ZPZZ", DestructiveBinaryCommWithRev, "UQRSHL_ZPmZ", /*isReverseInstr*/ 1>;
defm SRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_srshl>; defm SRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_srshl>;
defm URSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_urshl>; defm URSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_urshl>;
defm SQSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_sqshl>; defm SQSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_sqshl>;
defm UQSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_uqshl>; defm UQSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_uqshl>;
defm SQRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_sqrshl>; defm SQRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_sqrshl>;
defm UQRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_uqrshl>; defm UQRSHL_ZPZZ : sve_int_bin_pred_all_active_bhsd<int_aarch64_sve_uqrshl>;
} // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2, UseExperimentalZeroingPseudos] in { let Predicates = [HasSVE2orStreamingSVE, UseExperimentalZeroingPseudos] in {
defm SQSHL_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<null_frag>; defm SQSHL_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<null_frag>;
defm UQSHL_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<null_frag>; defm UQSHL_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<null_frag>;
defm SRSHR_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_srshr>; defm SRSHR_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_srshr>;
defm URSHR_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_urshr>; defm URSHR_ZPZI : sve_int_bin_pred_shift_imm_right_zeroing_bhsd<int_aarch64_sve_urshr>;
defm SQSHLU_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<int_aarch64_sve_sqshlu>; defm SQSHLU_ZPZI : sve_int_bin_pred_shift_imm_left_zeroing_bhsd<int_aarch64_sve_sqshlu>;
} } // End HasSVE2orStreamingSVE, UseExperimentalZeroingPseudos
let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 predicated shifts // SVE2 predicated shifts
defm SQSHL_ZPmI : sve_int_bin_pred_shift_imm_left_dup<0b0110, "sqshl", "SQSHL_ZPZI", int_aarch64_sve_sqshl>; defm SQSHL_ZPmI : sve_int_bin_pred_shift_imm_left_dup<0b0110, "sqshl", "SQSHL_ZPZI", int_aarch64_sve_sqshl>;
defm UQSHL_ZPmI : sve_int_bin_pred_shift_imm_left_dup<0b0111, "uqshl", "UQSHL_ZPZI", int_aarch64_sve_uqshl>; defm UQSHL_ZPmI : sve_int_bin_pred_shift_imm_left_dup<0b0111, "uqshl", "UQSHL_ZPZI", int_aarch64_sve_uqshl>;
defm SRSHR_ZPmI : sve_int_bin_pred_shift_imm_right< 0b1100, "srshr", "SRSHR_ZPZI", int_aarch64_sve_srshr>; defm SRSHR_ZPmI : sve_int_bin_pred_shift_imm_right< 0b1100, "srshr", "SRSHR_ZPZI", int_aarch64_sve_srshr>;
defm URSHR_ZPmI : sve_int_bin_pred_shift_imm_right< 0b1101, "urshr", "URSHR_ZPZI", int_aarch64_sve_urshr>; defm URSHR_ZPmI : sve_int_bin_pred_shift_imm_right< 0b1101, "urshr", "URSHR_ZPZI", int_aarch64_sve_urshr>;
defm SQSHLU_ZPmI : sve_int_bin_pred_shift_imm_left< 0b1111, "sqshlu", "SQSHLU_ZPZI", int_aarch64_sve_sqshlu>; defm SQSHLU_ZPmI : sve_int_bin_pred_shift_imm_left< 0b1111, "sqshlu", "SQSHLU_ZPZI", int_aarch64_sve_sqshlu>;
// SVE2 integer add/subtract long // SVE2 integer add/subtract long
▲ Show 20 Lines • Show All 96 Lines • ▼ Show 20 Lineslet Predicates = [HasSVE2orStreamingSVE] in {
defm SQXTNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b00, "sqxtnb", int_aarch64_sve_sqxtnb>; defm SQXTNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b00, "sqxtnb", int_aarch64_sve_sqxtnb>;
defm UQXTNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b01, "uqxtnb", int_aarch64_sve_uqxtnb>; defm UQXTNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b01, "uqxtnb", int_aarch64_sve_uqxtnb>;
defm SQXTUNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b10, "sqxtunb", int_aarch64_sve_sqxtunb>; defm SQXTUNB_ZZ : sve2_int_sat_extract_narrow_bottom<0b10, "sqxtunb", int_aarch64_sve_sqxtunb>;
// SVE2 saturating extract narrow (top) // SVE2 saturating extract narrow (top)
defm SQXTNT_ZZ : sve2_int_sat_extract_narrow_top<0b00, "sqxtnt", int_aarch64_sve_sqxtnt>; defm SQXTNT_ZZ : sve2_int_sat_extract_narrow_top<0b00, "sqxtnt", int_aarch64_sve_sqxtnt>;
defm UQXTNT_ZZ : sve2_int_sat_extract_narrow_top<0b01, "uqxtnt", int_aarch64_sve_uqxtnt>; defm UQXTNT_ZZ : sve2_int_sat_extract_narrow_top<0b01, "uqxtnt", int_aarch64_sve_uqxtnt>;
defm SQXTUNT_ZZ : sve2_int_sat_extract_narrow_top<0b10, "sqxtunt", int_aarch64_sve_sqxtunt>; defm SQXTUNT_ZZ : sve2_int_sat_extract_narrow_top<0b10, "sqxtunt", int_aarch64_sve_sqxtunt>;
} // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2] in {
// SVE2 character match // SVE2 character match
defm MATCH_PPzZZ : sve2_char_match<0b0, "match", int_aarch64_sve_match>; defm MATCH_PPzZZ : sve2_char_match<0b0, "match", int_aarch64_sve_match>;
defm NMATCH_PPzZZ : sve2_char_match<0b1, "nmatch", int_aarch64_sve_nmatch>; defm NMATCH_PPzZZ : sve2_char_match<0b1, "nmatch", int_aarch64_sve_nmatch>;
} // End HasSVE2
let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 bitwise exclusive-or interleaved // SVE2 bitwise exclusive-or interleaved
defm EORBT_ZZZ : sve2_bitwise_xor_interleaved<0b0, "eorbt", int_aarch64_sve_eorbt>; defm EORBT_ZZZ : sve2_bitwise_xor_interleaved<0b0, "eorbt", int_aarch64_sve_eorbt>;
defm EORTB_ZZZ : sve2_bitwise_xor_interleaved<0b1, "eortb", int_aarch64_sve_eortb>; defm EORTB_ZZZ : sve2_bitwise_xor_interleaved<0b1, "eortb", int_aarch64_sve_eortb>;
// SVE2 bitwise shift left long // SVE2 bitwise shift left long
defm SSHLLB_ZZI : sve2_bitwise_shift_left_long<0b00, "sshllb", int_aarch64_sve_sshllb>; defm SSHLLB_ZZI : sve2_bitwise_shift_left_long<0b00, "sshllb", int_aarch64_sve_sshllb>;
defm SSHLLT_ZZI : sve2_bitwise_shift_left_long<0b01, "sshllt", int_aarch64_sve_sshllt>; defm SSHLLT_ZZI : sve2_bitwise_shift_left_long<0b01, "sshllt", int_aarch64_sve_sshllt>;
defm USHLLB_ZZI : sve2_bitwise_shift_left_long<0b10, "ushllb", int_aarch64_sve_ushllb>; defm USHLLB_ZZI : sve2_bitwise_shift_left_long<0b10, "ushllb", int_aarch64_sve_ushllb>;
defm USHLLT_ZZI : sve2_bitwise_shift_left_long<0b11, "ushllt", int_aarch64_sve_ushllt>; defm USHLLT_ZZI : sve2_bitwise_shift_left_long<0b11, "ushllt", int_aarch64_sve_ushllt>;
// SVE2 integer add/subtract interleaved long // SVE2 integer add/subtract interleaved long
defm SADDLBT_ZZZ : sve2_misc_int_addsub_long_interleaved<0b00, "saddlbt", int_aarch64_sve_saddlbt>; defm SADDLBT_ZZZ : sve2_misc_int_addsub_long_interleaved<0b00, "saddlbt", int_aarch64_sve_saddlbt>;
defm SSUBLBT_ZZZ : sve2_misc_int_addsub_long_interleaved<0b10, "ssublbt", int_aarch64_sve_ssublbt>; defm SSUBLBT_ZZZ : sve2_misc_int_addsub_long_interleaved<0b10, "ssublbt", int_aarch64_sve_ssublbt>;
defm SSUBLTB_ZZZ : sve2_misc_int_addsub_long_interleaved<0b11, "ssubltb", int_aarch64_sve_ssubltb>; defm SSUBLTB_ZZZ : sve2_misc_int_addsub_long_interleaved<0b11, "ssubltb", int_aarch64_sve_ssubltb>;
} // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2] in {
// SVE2 histogram generation (segment) // SVE2 histogram generation (segment)
def HISTSEG_ZZZ : sve2_hist_gen_segment<"histseg", int_aarch64_sve_histseg>; def HISTSEG_ZZZ : sve2_hist_gen_segment<"histseg", int_aarch64_sve_histseg>;
// SVE2 histogram generation (vector) // SVE2 histogram generation (vector)
defm HISTCNT_ZPzZZ : sve2_hist_gen_vector<"histcnt", int_aarch64_sve_histcnt>; defm HISTCNT_ZPzZZ : sve2_hist_gen_vector<"histcnt", int_aarch64_sve_histcnt>;
} // End HasSVE2
let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 floating-point base 2 logarithm as integer // SVE2 floating-point base 2 logarithm as integer
defm FLOGB_ZPmZ : sve2_fp_flogb<"flogb", int_aarch64_sve_flogb>; defm FLOGB_ZPmZ : sve2_fp_flogb<"flogb", int_aarch64_sve_flogb>;
// SVE2 floating-point convert precision // SVE2 floating-point convert precision
defm FCVTXNT_ZPmZ : sve2_fp_convert_down_odd_rounding_top<"fcvtxnt", "int_aarch64_sve_fcvtxnt">; defm FCVTXNT_ZPmZ : sve2_fp_convert_down_odd_rounding_top<"fcvtxnt", "int_aarch64_sve_fcvtxnt">;
defm FCVTX_ZPmZ : sve2_fp_convert_down_odd_rounding<"fcvtx", "int_aarch64_sve_fcvtx">; defm FCVTX_ZPmZ : sve2_fp_convert_down_odd_rounding<"fcvtx", "int_aarch64_sve_fcvtx">;
defm FCVTNT_ZPmZ : sve2_fp_convert_down_narrow<"fcvtnt", "int_aarch64_sve_fcvtnt">; defm FCVTNT_ZPmZ : sve2_fp_convert_down_narrow<"fcvtnt", "int_aarch64_sve_fcvtnt">;
defm FCVTLT_ZPmZ : sve2_fp_convert_up_long<"fcvtlt", "int_aarch64_sve_fcvtlt">; defm FCVTLT_ZPmZ : sve2_fp_convert_up_long<"fcvtlt", "int_aarch64_sve_fcvtlt">;
Show All 25 Lineslet Predicates = [HasSVE2orStreamingSVE] in {
defm BSL2N_ZZZZ : sve2_int_bitwise_ternary_op<0b101, "bsl2n", int_aarch64_sve_bsl2n>; defm BSL2N_ZZZZ : sve2_int_bitwise_ternary_op<0b101, "bsl2n", int_aarch64_sve_bsl2n>;
defm NBSL_ZZZZ : sve2_int_bitwise_ternary_op<0b111, "nbsl", int_aarch64_sve_nbsl>; defm NBSL_ZZZZ : sve2_int_bitwise_ternary_op<0b111, "nbsl", int_aarch64_sve_nbsl>;
// SVE2 bitwise xor and rotate right by immediate // SVE2 bitwise xor and rotate right by immediate
defm XAR_ZZZI : sve2_int_rotate_right_imm<"xar", int_aarch64_sve_xar>; defm XAR_ZZZI : sve2_int_rotate_right_imm<"xar", int_aarch64_sve_xar>;
// SVE2 extract vector (immediate offset, constructive) // SVE2 extract vector (immediate offset, constructive)
def EXT_ZZI_B : sve2_int_perm_extract_i_cons<"ext">; def EXT_ZZI_B : sve2_int_perm_extract_i_cons<"ext">;
} // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2] in {
// SVE2 non-temporal gather loads // SVE2 non-temporal gather loads
defm LDNT1SB_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00000, "ldnt1sb", AArch64ldnt1s_gather_z, nxv4i8>; defm LDNT1SB_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00000, "ldnt1sb", AArch64ldnt1s_gather_z, nxv4i8>;
defm LDNT1B_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00001, "ldnt1b", AArch64ldnt1_gather_z, nxv4i8>; defm LDNT1B_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00001, "ldnt1b", AArch64ldnt1_gather_z, nxv4i8>;
defm LDNT1SH_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00100, "ldnt1sh", AArch64ldnt1s_gather_z, nxv4i16>; defm LDNT1SH_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00100, "ldnt1sh", AArch64ldnt1s_gather_z, nxv4i16>;
defm LDNT1H_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00101, "ldnt1h", AArch64ldnt1_gather_z, nxv4i16>; defm LDNT1H_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b00101, "ldnt1h", AArch64ldnt1_gather_z, nxv4i16>;
defm LDNT1W_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b01001, "ldnt1w", AArch64ldnt1_gather_z, nxv4i32>; defm LDNT1W_ZZR_S : sve2_mem_gldnt_vs_32_ptrs<0b01001, "ldnt1w", AArch64ldnt1_gather_z, nxv4i32>;
defm LDNT1SB_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10000, "ldnt1sb", AArch64ldnt1s_gather_z, nxv2i8>; defm LDNT1SB_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10000, "ldnt1sb", AArch64ldnt1s_gather_z, nxv2i8>;
defm LDNT1B_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10010, "ldnt1b", AArch64ldnt1_gather_z, nxv2i8>; defm LDNT1B_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10010, "ldnt1b", AArch64ldnt1_gather_z, nxv2i8>;
defm LDNT1SH_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10100, "ldnt1sh", AArch64ldnt1s_gather_z, nxv2i16>; defm LDNT1SH_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10100, "ldnt1sh", AArch64ldnt1s_gather_z, nxv2i16>;
defm LDNT1H_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10110, "ldnt1h", AArch64ldnt1_gather_z, nxv2i16>; defm LDNT1H_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b10110, "ldnt1h", AArch64ldnt1_gather_z, nxv2i16>;
defm LDNT1SW_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11000, "ldnt1sw", AArch64ldnt1s_gather_z, nxv2i32>; defm LDNT1SW_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11000, "ldnt1sw", AArch64ldnt1s_gather_z, nxv2i32>;
defm LDNT1W_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11010, "ldnt1w", AArch64ldnt1_gather_z, nxv2i32>; defm LDNT1W_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11010, "ldnt1w", AArch64ldnt1_gather_z, nxv2i32>;
defm LDNT1D_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11110, "ldnt1d", AArch64ldnt1_gather_z, nxv2i64>; defm LDNT1D_ZZR_D : sve2_mem_gldnt_vs_64_ptrs<0b11110, "ldnt1d", AArch64ldnt1_gather_z, nxv2i64>;
} // End HasSVE2
let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 vector splice (constructive) // SVE2 vector splice (constructive)
defm SPLICE_ZPZZ : sve2_int_perm_splice_cons<"splice">; defm SPLICE_ZPZZ : sve2_int_perm_splice_cons<"splice">;
} // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2] in {
// SVE2 non-temporal scatter stores // SVE2 non-temporal scatter stores
defm STNT1B_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b001, "stnt1b", AArch64stnt1_scatter, nxv4i8>; defm STNT1B_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b001, "stnt1b", AArch64stnt1_scatter, nxv4i8>;
defm STNT1H_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b011, "stnt1h", AArch64stnt1_scatter, nxv4i16>; defm STNT1H_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b011, "stnt1h", AArch64stnt1_scatter, nxv4i16>;
defm STNT1W_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b101, "stnt1w", AArch64stnt1_scatter, nxv4i32>; defm STNT1W_ZZR_S : sve2_mem_sstnt_vs_32_ptrs<0b101, "stnt1w", AArch64stnt1_scatter, nxv4i32>;
defm STNT1B_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b000, "stnt1b", AArch64stnt1_scatter, nxv2i8>; defm STNT1B_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b000, "stnt1b", AArch64stnt1_scatter, nxv2i8>;
defm STNT1H_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b010, "stnt1h", AArch64stnt1_scatter, nxv2i16>; defm STNT1H_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b010, "stnt1h", AArch64stnt1_scatter, nxv2i16>;
defm STNT1W_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b100, "stnt1w", AArch64stnt1_scatter, nxv2i32>; defm STNT1W_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b100, "stnt1w", AArch64stnt1_scatter, nxv2i32>;
defm STNT1D_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b110, "stnt1d", AArch64stnt1_scatter, nxv2i64>; defm STNT1D_ZZR_D : sve2_mem_sstnt_vs_64_ptrs<0b110, "stnt1d", AArch64stnt1_scatter, nxv2i64>;
} // End HasSVE2
let Predicates = [HasSVE2orStreamingSVE] in {
// SVE2 table lookup (three sources) // SVE2 table lookup (three sources)
defm TBL_ZZZZ : sve2_int_perm_tbl<"tbl", int_aarch64_sve_tbl2>; defm TBL_ZZZZ : sve2_int_perm_tbl<"tbl", int_aarch64_sve_tbl2>;
defm TBX_ZZZ : sve2_int_perm_tbx<"tbx", int_aarch64_sve_tbx>; defm TBX_ZZZ : sve2_int_perm_tbx<"tbx", int_aarch64_sve_tbx>;
// SVE2 integer compare scalar count and limit // SVE2 integer compare scalar count and limit
defm WHILEGE_PWW : sve_int_while4_rr<0b000, "whilege", int_aarch64_sve_whilege>; defm WHILEGE_PWW : sve_int_while4_rr<0b000, "whilege", int_aarch64_sve_whilege>;
defm WHILEGT_PWW : sve_int_while4_rr<0b001, "whilegt", int_aarch64_sve_whilegt>; defm WHILEGT_PWW : sve_int_while4_rr<0b001, "whilegt", int_aarch64_sve_whilegt>;
defm WHILEHS_PWW : sve_int_while4_rr<0b100, "whilehs", int_aarch64_sve_whilehs>; defm WHILEHS_PWW : sve_int_while4_rr<0b100, "whilehs", int_aarch64_sve_whilehs>;
defm WHILEHI_PWW : sve_int_while4_rr<0b101, "whilehi", int_aarch64_sve_whilehi>; defm WHILEHI_PWW : sve_int_while4_rr<0b101, "whilehi", int_aarch64_sve_whilehi>;
defm WHILEGE_PXX : sve_int_while8_rr<0b000, "whilege", int_aarch64_sve_whilege>; defm WHILEGE_PXX : sve_int_while8_rr<0b000, "whilege", int_aarch64_sve_whilege>;
defm WHILEGT_PXX : sve_int_while8_rr<0b001, "whilegt", int_aarch64_sve_whilegt>; defm WHILEGT_PXX : sve_int_while8_rr<0b001, "whilegt", int_aarch64_sve_whilegt>;
defm WHILEHS_PXX : sve_int_while8_rr<0b100, "whilehs", int_aarch64_sve_whilehs>; defm WHILEHS_PXX : sve_int_while8_rr<0b100, "whilehs", int_aarch64_sve_whilehs>;
defm WHILEHI_PXX : sve_int_while8_rr<0b101, "whilehi", int_aarch64_sve_whilehi>; defm WHILEHI_PXX : sve_int_while8_rr<0b101, "whilehi", int_aarch64_sve_whilehi>;
// SVE2 pointer conflict compare // SVE2 pointer conflict compare
defm WHILEWR_PXX : sve2_int_while_rr<0b0, "whilewr", "int_aarch64_sve_whilewr">; defm WHILEWR_PXX : sve2_int_while_rr<0b0, "whilewr", "int_aarch64_sve_whilewr">;
defm WHILERW_PXX : sve2_int_while_rr<0b1, "whilerw", "int_aarch64_sve_whilerw">; defm WHILERW_PXX : sve2_int_while_rr<0b1, "whilerw", "int_aarch64_sve_whilerw">;
} } // End HasSVE2orStreamingSVE
let Predicates = [HasSVE2AES] in { let Predicates = [HasSVE2AES] in {
// SVE2 crypto destructive binary operations // SVE2 crypto destructive binary operations
defm AESE_ZZZ_B : sve2_crypto_des_bin_op<0b00, "aese", ZPR8, int_aarch64_sve_aese, nxv16i8>; defm AESE_ZZZ_B : sve2_crypto_des_bin_op<0b00, "aese", ZPR8, int_aarch64_sve_aese, nxv16i8>;
defm AESD_ZZZ_B : sve2_crypto_des_bin_op<0b01, "aesd", ZPR8, int_aarch64_sve_aesd, nxv16i8>; defm AESD_ZZZ_B : sve2_crypto_des_bin_op<0b01, "aesd", ZPR8, int_aarch64_sve_aesd, nxv16i8>;
// SVE2 crypto unary operations // SVE2 crypto unary operations
defm AESMC_ZZ_B : sve2_crypto_unary_op<0b0, "aesmc", int_aarch64_sve_aesmc>; defm AESMC_ZZ_B : sve2_crypto_unary_op<0b0, "aesmc", int_aarch64_sve_aesmc>;
defm AESIMC_ZZ_B : sve2_crypto_unary_op<0b1, "aesimc", int_aarch64_sve_aesimc>; defm AESIMC_ZZ_B : sve2_crypto_unary_op<0b1, "aesimc", int_aarch64_sve_aesimc>;
// PMULLB and PMULLT instructions which operate with 64-bit source and // PMULLB and PMULLT instructions which operate with 64-bit source and
// 128-bit destination elements are enabled with crypto extensions, similar // 128-bit destination elements are enabled with crypto extensions, similar
// to NEON PMULL2 instruction. // to NEON PMULL2 instruction.
defm PMULLB_ZZZ_Q : sve2_wide_int_arith_pmul<0b00, 0b11010, "pmullb", int_aarch64_sve_pmullb_pair>; defm PMULLB_ZZZ_Q : sve2_wide_int_arith_pmul<0b00, 0b11010, "pmullb", int_aarch64_sve_pmullb_pair>;
defm PMULLT_ZZZ_Q : sve2_wide_int_arith_pmul<0b00, 0b11011, "pmullt", int_aarch64_sve_pmullt_pair>; defm PMULLT_ZZZ_Q : sve2_wide_int_arith_pmul<0b00, 0b11011, "pmullt", int_aarch64_sve_pmullt_pair>;
} } // End HasSVE2AES
let Predicates = [HasSVE2SM4] in { let Predicates = [HasSVE2SM4] in {
// SVE2 crypto constructive binary operations // SVE2 crypto constructive binary operations
defm SM4EKEY_ZZZ_S : sve2_crypto_cons_bin_op<0b0, "sm4ekey", ZPR32, int_aarch64_sve_sm4ekey, nxv4i32>; defm SM4EKEY_ZZZ_S : sve2_crypto_cons_bin_op<0b0, "sm4ekey", ZPR32, int_aarch64_sve_sm4ekey, nxv4i32>;
// SVE2 crypto destructive binary operations // SVE2 crypto destructive binary operations
defm SM4E_ZZZ_S : sve2_crypto_des_bin_op<0b10, "sm4e", ZPR32, int_aarch64_sve_sm4e, nxv4i32>; defm SM4E_ZZZ_S : sve2_crypto_des_bin_op<0b10, "sm4e", ZPR32, int_aarch64_sve_sm4e, nxv4i32>;
} } // End HasSVE2SM4
let Predicates = [HasSVE2SHA3] in { let Predicates = [HasSVE2SHA3] in {
// SVE2 crypto constructive binary operations // SVE2 crypto constructive binary operations
defm RAX1_ZZZ_D : sve2_crypto_cons_bin_op<0b1, "rax1", ZPR64, int_aarch64_sve_rax1, nxv2i64>; defm RAX1_ZZZ_D : sve2_crypto_cons_bin_op<0b1, "rax1", ZPR64, int_aarch64_sve_rax1, nxv2i64>;
} } // End HasSVE2SHA3
let Predicates = [HasSVE2BitPerm] in { let Predicates = [HasSVE2BitPerm] in {
// SVE2 bitwise permute // SVE2 bitwise permute
defm BEXT_ZZZ : sve2_misc_bitwise<0b1100, "bext", int_aarch64_sve_bext_x>; defm BEXT_ZZZ : sve2_misc_bitwise<0b1100, "bext", int_aarch64_sve_bext_x>;
defm BDEP_ZZZ : sve2_misc_bitwise<0b1101, "bdep", int_aarch64_sve_bdep_x>; defm BDEP_ZZZ : sve2_misc_bitwise<0b1101, "bdep", int_aarch64_sve_bdep_x>;
defm BGRP_ZZZ : sve2_misc_bitwise<0b1110, "bgrp", int_aarch64_sve_bgrp_x>; defm BGRP_ZZZ : sve2_misc_bitwise<0b1110, "bgrp", int_aarch64_sve_bgrp_x>;
} } // End HasSVE2BitPerm

llvm/lib/Target/AArch64/AArch64SchedA64FX.td

llvm/lib/Target/AArch64/AArch64Subtarget.h

llvm/test/MC/AArch64/SME/revd.s

llvm/test/MC/AArch64/SME/sclamp.s

llvm/test/MC/AArch64/SME/uclamp.s

llvm/test/MC/AArch64/SVE/abs.s

llvm/test/MC/AArch64/SVE/add.s

llvm/test/MC/AArch64/SVE/addpl.s

llvm/test/MC/AArch64/SVE/addvl.s

llvm/test/MC/AArch64/SVE/and.s

llvm/test/MC/AArch64/SVE/ands.s

llvm/test/MC/AArch64/SVE/andv.s

llvm/test/MC/AArch64/SVE/asr.s

llvm/test/MC/AArch64/SVE/asrd.s

llvm/test/MC/AArch64/SVE/asrr.s

llvm/test/MC/AArch64/SVE/bfcvt.s

llvm/test/MC/AArch64/SVE/bfcvtnt.s

llvm/test/MC/AArch64/SVE/bfdot.s

llvm/test/MC/AArch64/SVE/bfmlal.s

llvm/test/MC/AArch64/SVE/bfmmla.s

llvm/test/MC/AArch64/SVE/bic.s

llvm/test/MC/AArch64/SVE/bics.s

llvm/test/MC/AArch64/SVE/brka.s

llvm/test/MC/AArch64/SVE/brkas.s

llvm/test/MC/AArch64/SVE/brkb.s

llvm/test/MC/AArch64/SVE/brkbs.s

llvm/test/MC/AArch64/SVE/brkn.s

llvm/test/MC/AArch64/SVE/brkns.s

llvm/test/MC/AArch64/SVE/brkpa.s

llvm/test/MC/AArch64/SVE/brkpas.s

llvm/test/MC/AArch64/SVE/brkpb.s

llvm/test/MC/AArch64/SVE/brkpbs.s

llvm/test/MC/AArch64/SVE/clasta.s

llvm/test/MC/AArch64/SVE/clastb.s

llvm/test/MC/AArch64/SVE/cls.s

llvm/test/MC/AArch64/SVE/clz.s

llvm/test/MC/AArch64/SVE/cmpeq.s

llvm/test/MC/AArch64/SVE/cmpge.s

llvm/test/MC/AArch64/SVE/cmpgt.s

llvm/test/MC/AArch64/SVE/cmphi.s

llvm/test/MC/AArch64/SVE/cmphs.s

llvm/test/MC/AArch64/SVE/cmple.s

llvm/test/MC/AArch64/SVE/cmplo.s

llvm/test/MC/AArch64/SVE/cmpls.s

llvm/test/MC/AArch64/SVE/cmplt.s

llvm/test/MC/AArch64/SVE/cmpne.s

llvm/test/MC/AArch64/SVE/cnot.s

llvm/test/MC/AArch64/SVE/cnt.s

llvm/test/MC/AArch64/SVE/cntb.s

llvm/test/MC/AArch64/SVE/cntd.s

llvm/test/MC/AArch64/SVE/cnth.s

llvm/test/MC/AArch64/SVE/cntp.s

llvm/test/MC/AArch64/SVE/cntw.s

llvm/test/MC/AArch64/SVE/compact.s

llvm/test/MC/AArch64/SVE/cpy.s

llvm/test/MC/AArch64/SVE/ctermeq.s

llvm/test/MC/AArch64/SVE/ctermne.s

llvm/test/MC/AArch64/SVE/decb.s

llvm/test/MC/AArch64/SVE/decd.s

llvm/test/MC/AArch64/SVE/dech.s

llvm/test/MC/AArch64/SVE/decp.s

llvm/test/MC/AArch64/SVE/decw.s

llvm/test/MC/AArch64/SVE/dup.s

llvm/test/MC/AArch64/SVE/dupm.s

llvm/test/MC/AArch64/SVE/eon.s

llvm/test/MC/AArch64/SVE/eor.s

llvm/test/MC/AArch64/SVE/eors.s

llvm/test/MC/AArch64/SVE/eorv.s

llvm/test/MC/AArch64/SVE/ext.s

llvm/test/MC/AArch64/SVE/fabd.s

llvm/test/MC/AArch64/SVE/fabs.s

llvm/test/MC/AArch64/SVE/facge.s

llvm/test/MC/AArch64/SVE/facgt.s

llvm/test/MC/AArch64/SVE/facle.s

llvm/test/MC/AArch64/SVE/faclt.s

llvm/test/MC/AArch64/SVE/fadd.s

llvm/test/MC/AArch64/SVE/fadda.s

llvm/test/MC/AArch64/SVE/faddv.s

llvm/test/MC/AArch64/SVE/fcadd.s

llvm/test/MC/AArch64/SVE/fcmeq.s

llvm/test/MC/AArch64/SVE/fcmge.s

llvm/test/MC/AArch64/SVE/fcmgt.s

llvm/test/MC/AArch64/SVE/fcmla.s

llvm/test/MC/AArch64/SVE/fcmle.s

llvm/test/MC/AArch64/SVE/fcmlt.s

llvm/test/MC/AArch64/SVE/fcmne.s

llvm/test/MC/AArch64/SVE/fcmuo.s

llvm/test/MC/AArch64/SVE/fcpy.s

llvm/test/MC/AArch64/SVE/fcvt.s

llvm/test/MC/AArch64/SVE/fcvtzs.s

llvm/test/MC/AArch64/SVE/fcvtzu.s

llvm/test/MC/AArch64/SVE/fdiv.s

llvm/test/MC/AArch64/SVE/fdivr.s

llvm/test/MC/AArch64/SVE/fdup.s

llvm/test/MC/AArch64/SVE/fexpa.s

llvm/test/MC/AArch64/SVE/fmad.s

llvm/test/MC/AArch64/SVE/fmax.s

llvm/test/MC/AArch64/SVE/fmaxnm.s

llvm/test/MC/AArch64/SVE/fmaxnmv.s

llvm/test/MC/AArch64/SVE/fmaxv.s

llvm/test/MC/AArch64/SVE/fmin.s

llvm/test/MC/AArch64/SVE/fminnm.s

llvm/test/MC/AArch64/SVE/fminnmv.s

llvm/test/MC/AArch64/SVE/fminv.s

llvm/test/MC/AArch64/SVE/fmla.s

llvm/test/MC/AArch64/SVE/fmls.s

llvm/test/MC/AArch64/SVE/fmov.s

llvm/test/MC/AArch64/SVE/fmsb.s

llvm/test/MC/AArch64/SVE/fmul.s

llvm/test/MC/AArch64/SVE/fmulx.s

llvm/test/MC/AArch64/SVE/fneg.s

llvm/test/MC/AArch64/SVE/fnmad.s

llvm/test/MC/AArch64/SVE/fnmla.s

llvm/test/MC/AArch64/SVE/fnmls.s

llvm/test/MC/AArch64/SVE/fnmsb.s

llvm/test/MC/AArch64/SVE/frecpe.s

llvm/test/MC/AArch64/SVE/frecps.s

llvm/test/MC/AArch64/SVE/frecpx.s

llvm/test/MC/AArch64/SVE/frinta.s

llvm/test/MC/AArch64/SVE/frinti.s

llvm/test/MC/AArch64/SVE/frintm.s

llvm/test/MC/AArch64/SVE/frintn.s

llvm/test/MC/AArch64/SVE/frintp.s

llvm/test/MC/AArch64/SVE/frintx.s

llvm/test/MC/AArch64/SVE/frintz.s

llvm/test/MC/AArch64/SVE/frsqrte.s

llvm/test/MC/AArch64/SVE/frsqrts.s

llvm/test/MC/AArch64/SVE/fscale.s

llvm/test/MC/AArch64/SVE/fsqrt.s

llvm/test/MC/AArch64/SVE/fsub.s

llvm/test/MC/AArch64/SVE/fsubr.s

llvm/test/MC/AArch64/SVE/ftmad.s

llvm/test/MC/AArch64/SVE/ftsmul.s

llvm/test/MC/AArch64/SVE/ftssel.s

llvm/test/MC/AArch64/SVE/incb.s

llvm/test/MC/AArch64/SVE/incd.s

llvm/test/MC/AArch64/SVE/inch.s

llvm/test/MC/AArch64/SVE/incp.s

llvm/test/MC/AArch64/SVE/incw.s

llvm/test/MC/AArch64/SVE/index.s

llvm/test/MC/AArch64/SVE/insr.s

llvm/test/MC/AArch64/SVE/lasta.s

llvm/test/MC/AArch64/SVE/lastb.s

llvm/test/MC/AArch64/SVE/ld1b-sve-only.s

llvm/test/MC/AArch64/SVE/ld1b.s

llvm/test/MC/AArch64/SVE/ld1d-sve-only.s

llvm/test/MC/AArch64/SVE/ld1d.s

llvm/test/MC/AArch64/SVE/ld1h-sve-only.s

llvm/test/MC/AArch64/SVE/ld1h.s

llvm/test/MC/AArch64/SVE/ld1rb.s

llvm/test/MC/AArch64/SVE/ld1rd.s

llvm/test/MC/AArch64/SVE/ld1rh.s

llvm/test/MC/AArch64/SVE/ld1rqb.s

llvm/test/MC/AArch64/SVE/ld1rqd.s

llvm/test/MC/AArch64/SVE/ld1rqh.s

llvm/test/MC/AArch64/SVE/ld1rqw.s

llvm/test/MC/AArch64/SVE/ld1rsb.s

llvm/test/MC/AArch64/SVE/ld1rsh.s

llvm/test/MC/AArch64/SVE/ld1rsw.s

llvm/test/MC/AArch64/SVE/ld1rw.s

llvm/test/MC/AArch64/SVE/ld1sb-sve-only.s

llvm/test/MC/AArch64/SVE/ld1sb.s

llvm/test/MC/AArch64/SVE/ld1sh-sve-only.s

llvm/test/MC/AArch64/SVE/ld1sh.s

llvm/test/MC/AArch64/SVE/ld1sw-sve-only.s

llvm/test/MC/AArch64/SVE/ld1sw.s

llvm/test/MC/AArch64/SVE/ld1w-sve-only.s

llvm/test/MC/AArch64/SVE/ld1w.s

llvm/test/MC/AArch64/SVE/ld2b.s

llvm/test/MC/AArch64/SVE/ld2d.s

llvm/test/MC/AArch64/SVE/ld2h.s

llvm/test/MC/AArch64/SVE/ld2w.s

llvm/test/MC/AArch64/SVE/ld3b.s

llvm/test/MC/AArch64/SVE/ld3d.s

llvm/test/MC/AArch64/SVE/ld3h.s

llvm/test/MC/AArch64/SVE/ld3w.s

llvm/test/MC/AArch64/SVE/ld4b.s

llvm/test/MC/AArch64/SVE/ld4d.s

llvm/test/MC/AArch64/SVE/ld4h.s

llvm/test/MC/AArch64/SVE/ld4w.s

llvm/test/MC/AArch64/SVE/ldff1b.s

llvm/test/MC/AArch64/SVE/ldff1d.s

llvm/test/MC/AArch64/SVE/ldff1h.s

llvm/test/MC/AArch64/SVE/ldff1sb.s

llvm/test/MC/AArch64/SVE/ldff1sh.s

llvm/test/MC/AArch64/SVE/ldff1sw.s

llvm/test/MC/AArch64/SVE/ldff1w.s

llvm/test/MC/AArch64/SVE/ldnf1b.s

llvm/test/MC/AArch64/SVE/ldnf1d.s

llvm/test/MC/AArch64/SVE/ldnf1h.s

llvm/test/MC/AArch64/SVE/ldnf1sb.s

llvm/test/MC/AArch64/SVE/ldnf1sh.s

llvm/test/MC/AArch64/SVE/ldnf1sw.s

llvm/test/MC/AArch64/SVE/ldnf1w.s

llvm/test/MC/AArch64/SVE/ldnt1b.s

llvm/test/MC/AArch64/SVE/ldnt1d.s

llvm/test/MC/AArch64/SVE/ldnt1h.s

llvm/test/MC/AArch64/SVE/ldnt1w.s

llvm/test/MC/AArch64/SVE/ldr.s

llvm/test/MC/AArch64/SVE/lsl.s

llvm/test/MC/AArch64/SVE/lslr.s

llvm/test/MC/AArch64/SVE/lsr.s

llvm/test/MC/AArch64/SVE/lsrr.s

llvm/test/MC/AArch64/SVE/mad.s

llvm/test/MC/AArch64/SVE/matrix-multiply-fp32.s

llvm/test/MC/AArch64/SVE/matrix-multiply-fp64.s

llvm/test/MC/AArch64/SVE/matrix-multiply-int8.s

llvm/test/MC/AArch64/SVE/mla.s

llvm/test/MC/AArch64/SVE/mls.s

llvm/test/MC/AArch64/SVE/mov.s

llvm/test/MC/AArch64/SVE/movprfx.s

llvm/test/MC/AArch64/SVE/movs.s

llvm/test/MC/AArch64/SVE/msb.s

llvm/test/MC/AArch64/SVE/mul.s

llvm/test/MC/AArch64/SVE/nand.s

llvm/test/MC/AArch64/SVE/nands.s

llvm/test/MC/AArch64/SVE/neg.s

llvm/test/MC/AArch64/SVE/nor.s

llvm/test/MC/AArch64/SVE/nors.s

llvm/test/MC/AArch64/SVE/not.s

llvm/test/MC/AArch64/SVE/nots.s

llvm/test/MC/AArch64/SVE/orn.s

llvm/test/MC/AArch64/SVE/orns.s

llvm/test/MC/AArch64/SVE/orr.s

llvm/test/MC/AArch64/SVE/orrs.s

llvm/test/MC/AArch64/SVE/orv.s

llvm/test/MC/AArch64/SVE/pfalse.s

llvm/test/MC/AArch64/SVE/pfirst.s

llvm/test/MC/AArch64/SVE/pnext.s

llvm/test/MC/AArch64/SVE/prfb-sve-only.s

llvm/test/MC/AArch64/SVE/prfb.s

llvm/test/MC/AArch64/SVE/prfd-sve-only.s

llvm/test/MC/AArch64/SVE/prfd.s

llvm/test/MC/AArch64/SVE/prfh-sve-only.s

llvm/test/MC/AArch64/SVE/prfh.s

llvm/test/MC/AArch64/SVE/prfw-sve-only.s

llvm/test/MC/AArch64/SVE/prfw.s

llvm/test/MC/AArch64/SVE/ptest.s

llvm/test/MC/AArch64/SVE/ptrue.s

llvm/test/MC/AArch64/SVE/ptrues.s

llvm/test/MC/AArch64/SVE/punpkhi.s

llvm/test/MC/AArch64/SVE/punpklo.s

llvm/test/MC/AArch64/SVE/rbit.s

llvm/test/MC/AArch64/SVE/rdffr.s

llvm/test/MC/AArch64/SVE/rdffrs.s

llvm/test/MC/AArch64/SVE/rdvl.s

llvm/test/MC/AArch64/SVE/rev.s

llvm/test/MC/AArch64/SVE/revb.s

llvm/test/MC/AArch64/SVE/revh.s

llvm/test/MC/AArch64/SVE/revw.s

llvm/test/MC/AArch64/SVE/sabd.s

llvm/test/MC/AArch64/SVE/saddv.s

llvm/test/MC/AArch64/SVE/scvtf.s

llvm/test/MC/AArch64/SVE/sdiv.s

llvm/test/MC/AArch64/SVE/sdivr.s

llvm/test/MC/AArch64/SVE/sdot.s

llvm/test/MC/AArch64/SVE/sel.s

llvm/test/MC/AArch64/SVE/setffr.s

llvm/test/MC/AArch64/SVE/smax.s

llvm/test/MC/AArch64/SVE/smaxv.s

llvm/test/MC/AArch64/SVE/smin.s

llvm/test/MC/AArch64/SVE/sminv.s

llvm/test/MC/AArch64/SVE/smulh.s

llvm/test/MC/AArch64/SVE/splice.s

llvm/test/MC/AArch64/SVE/sqadd.s

llvm/test/MC/AArch64/SVE/sqdecb.s

llvm/test/MC/AArch64/SVE/sqdecd.s

llvm/test/MC/AArch64/SVE/sqdech.s

llvm/test/MC/AArch64/SVE/sqdecp.s

llvm/test/MC/AArch64/SVE/sqdecw.s

llvm/test/MC/AArch64/SVE/sqincb.s

llvm/test/MC/AArch64/SVE/sqincd.s

llvm/test/MC/AArch64/SVE/sqinch.s

llvm/test/MC/AArch64/SVE/sqincp.s

llvm/test/MC/AArch64/SVE/sqincw.s

llvm/test/MC/AArch64/SVE/sqsub.s

llvm/test/MC/AArch64/SVE/st1b-sve-only.s

llvm/test/MC/AArch64/SVE/st1b.s

llvm/test/MC/AArch64/SVE/st1d-sve-only.s

llvm/test/MC/AArch64/SVE/st1d.s

llvm/test/MC/AArch64/SVE/st1h-sve-only.s

llvm/test/MC/AArch64/SVE/st1h.s

llvm/test/MC/AArch64/SVE/st1w-sve-only.s

llvm/test/MC/AArch64/SVE/st1w.s

llvm/test/MC/AArch64/SVE/st2b.s

llvm/test/MC/AArch64/SVE/st2d.s

llvm/test/MC/AArch64/SVE/st2h.s

llvm/test/MC/AArch64/SVE/st2w.s

llvm/test/MC/AArch64/SVE/st3b.s

llvm/test/MC/AArch64/SVE/st3d.s

llvm/test/MC/AArch64/SVE/st3h.s

llvm/test/MC/AArch64/SVE/st3w.s

llvm/test/MC/AArch64/SVE/st4b.s

llvm/test/MC/AArch64/SVE/st4d.s

llvm/test/MC/AArch64/SVE/st4h.s

llvm/test/MC/AArch64/SVE/st4w.s

llvm/test/MC/AArch64/SVE/stnt1b.s

llvm/test/MC/AArch64/SVE/stnt1d.s

llvm/test/MC/AArch64/SVE/stnt1h.s

llvm/test/MC/AArch64/SVE/stnt1w.s

llvm/test/MC/AArch64/SVE/str.s

llvm/test/MC/AArch64/SVE/sub.s

llvm/test/MC/AArch64/SVE/subr.s

llvm/test/MC/AArch64/SVE/sunpkhi.s

llvm/test/MC/AArch64/SVE/sunpklo.s

llvm/test/MC/AArch64/SVE/sxtb.s

llvm/test/MC/AArch64/SVE/sxth.s

llvm/test/MC/AArch64/SVE/sxtw.s

llvm/test/MC/AArch64/SVE/tbl.s

llvm/test/MC/AArch64/SVE/trn1.s

llvm/test/MC/AArch64/SVE/trn2.s

llvm/test/MC/AArch64/SVE/uabd.s

llvm/test/MC/AArch64/SVE/uaddv.s

llvm/test/MC/AArch64/SVE/ucvtf.s

llvm/test/MC/AArch64/SVE/udiv.s

llvm/test/MC/AArch64/SVE/udivr.s

llvm/test/MC/AArch64/SVE/udot.s

llvm/test/MC/AArch64/SVE/umax.s

llvm/test/MC/AArch64/SVE/umaxv.s

llvm/test/MC/AArch64/SVE/umin.s

llvm/test/MC/AArch64/SVE/uminv.s

llvm/test/MC/AArch64/SVE/umulh.s

llvm/test/MC/AArch64/SVE/uqadd.s

llvm/test/MC/AArch64/SVE/uqdecb.s

llvm/test/MC/AArch64/SVE/uqdecd.s

llvm/test/MC/AArch64/SVE/uqdech.s

llvm/test/MC/AArch64/SVE/uqdecp.s

llvm/test/MC/AArch64/SVE/uqdecw.s

llvm/test/MC/AArch64/SVE/uqincb.s

llvm/test/MC/AArch64/SVE/uqincd.s

llvm/test/MC/AArch64/SVE/uqinch.s

llvm/test/MC/AArch64/SVE/uqincp.s

llvm/test/MC/AArch64/SVE/uqincw.s

llvm/test/MC/AArch64/SVE/uqsub.s

llvm/test/MC/AArch64/SVE/uunpkhi.s

llvm/test/MC/AArch64/SVE/uunpklo.s

llvm/test/MC/AArch64/SVE/uxtb.s

llvm/test/MC/AArch64/SVE/uxth.s

llvm/test/MC/AArch64/SVE/uxtw.s

llvm/test/MC/AArch64/SVE/uzp1.s

llvm/test/MC/AArch64/SVE/uzp2.s

llvm/test/MC/AArch64/SVE/whilele.s

llvm/test/MC/AArch64/SVE/whilelo.s

llvm/test/MC/AArch64/SVE/whilels.s

llvm/test/MC/AArch64/SVE/whilelt.s

llvm/test/MC/AArch64/SVE/wrffr.s

llvm/test/MC/AArch64/SVE/zip1.s

llvm/test/MC/AArch64/SVE/zip2.s

llvm/test/MC/AArch64/SVE2/adclb.s

llvm/test/MC/AArch64/SVE2/adclt.s

llvm/test/MC/AArch64/SVE2/addhnb.s

llvm/test/MC/AArch64/SVE2/addhnt.s

llvm/test/MC/AArch64/SVE2/addp.s

llvm/test/MC/AArch64/SVE2/aesd.s

llvm/test/MC/AArch64/SVE2/aese.s

llvm/test/MC/AArch64/SVE2/aesimc.s

llvm/test/MC/AArch64/SVE2/aesmc.s

llvm/test/MC/AArch64/SVE2/bcax.s

llvm/test/MC/AArch64/SVE2/bdep.s

llvm/test/MC/AArch64/SVE2/bext.s

llvm/test/MC/AArch64/SVE2/bgrp.s

llvm/test/MC/AArch64/SVE2/bsl.s

llvm/test/MC/AArch64/SVE2/bsl1n.s

llvm/test/MC/AArch64/SVE2/bsl2n.s

llvm/test/MC/AArch64/SVE2/cadd.s

llvm/test/MC/AArch64/SVE2/cdot.s

llvm/test/MC/AArch64/SVE2/cmla.s

llvm/test/MC/AArch64/SVE2/directive-arch-negative.s

llvm/test/MC/AArch64/SVE2/directive-arch_extension-negative.s

llvm/test/MC/AArch64/SVE2/directive-cpu-negative.s

llvm/test/MC/AArch64/SVE2/eor3.s

llvm/test/MC/AArch64/SVE2/eorbt.s

llvm/test/MC/AArch64/SVE2/eortb.s

llvm/test/MC/AArch64/SVE2/ext.s

llvm/test/MC/AArch64/SVE2/faddp.s

llvm/test/MC/AArch64/SVE2/fcvtlt.s

llvm/test/MC/AArch64/SVE2/fcvtnt.s

llvm/test/MC/AArch64/SVE2/fcvtx.s

llvm/test/MC/AArch64/SVE2/fcvtxnt.s

llvm/test/MC/AArch64/SVE2/flogb.s

llvm/test/MC/AArch64/SVE2/fmaxnmp.s

llvm/test/MC/AArch64/SVE2/fmaxp.s

llvm/test/MC/AArch64/SVE2/fminnmp.s

llvm/test/MC/AArch64/SVE2/fminp.s

llvm/test/MC/AArch64/SVE2/fmlalb.s

llvm/test/MC/AArch64/SVE2/fmlalt.s

llvm/test/MC/AArch64/SVE2/fmlslb.s

llvm/test/MC/AArch64/SVE2/fmlslt.s

llvm/test/MC/AArch64/SVE2/histcnt.s

llvm/test/MC/AArch64/SVE2/histseg.s

llvm/test/MC/AArch64/SVE2/ldnt1b.s

llvm/test/MC/AArch64/SVE2/ldnt1d.s

llvm/test/MC/AArch64/SVE2/ldnt1h.s

llvm/test/MC/AArch64/SVE2/ldnt1sb.s

llvm/test/MC/AArch64/SVE2/ldnt1sh.s

llvm/test/MC/AArch64/SVE2/ldnt1sw.s

llvm/test/MC/AArch64/SVE2/ldnt1w.s

llvm/test/MC/AArch64/SVE2/match.s

llvm/test/MC/AArch64/SVE2/mla.s

llvm/test/MC/AArch64/SVE2/mls.s

llvm/test/MC/AArch64/SVE2/mul.s

llvm/test/MC/AArch64/SVE2/nbsl.s

llvm/test/MC/AArch64/SVE2/nmatch.s

llvm/test/MC/AArch64/SVE2/pmul.s

llvm/test/MC/AArch64/SVE2/pmullb-128.s

llvm/test/MC/AArch64/SVE2/pmullb.s

llvm/test/MC/AArch64/SVE2/pmullt-128.s

llvm/test/MC/AArch64/SVE2/pmullt.s

llvm/test/MC/AArch64/SVE2/raddhnb.s

llvm/test/MC/AArch64/SVE2/raddhnt.s

llvm/test/MC/AArch64/SVE2/rax1.s

llvm/test/MC/AArch64/SVE2/rshrnb.s

llvm/test/MC/AArch64/SVE2/rshrnt.s

llvm/test/MC/AArch64/SVE2/rsubhnb.s

llvm/test/MC/AArch64/SVE2/rsubhnt.s

llvm/test/MC/AArch64/SVE2/saba.s

llvm/test/MC/AArch64/SVE2/sabalb.s

llvm/test/MC/AArch64/SVE2/sabalt.s

llvm/test/MC/AArch64/SVE2/sabdlb.s

llvm/test/MC/AArch64/SVE2/sabdlt.s

llvm/test/MC/AArch64/SVE2/sadalp.s

llvm/test/MC/AArch64/SVE2/saddlb.s

llvm/test/MC/AArch64/SVE2/saddlbt.s

llvm/test/MC/AArch64/SVE2/saddlt.s

llvm/test/MC/AArch64/SVE2/saddwb.s

llvm/test/MC/AArch64/SVE2/saddwt.s

llvm/test/MC/AArch64/SVE2/sbclb.s

llvm/test/MC/AArch64/SVE2/sbclt.s

llvm/test/MC/AArch64/SVE2/shadd.s

llvm/test/MC/AArch64/SVE2/shrnb.s

llvm/test/MC/AArch64/SVE2/shrnt.s

llvm/test/MC/AArch64/SVE2/shsub.s

llvm/test/MC/AArch64/SVE2/shsubr.s

llvm/test/MC/AArch64/SVE2/sli.s

llvm/test/MC/AArch64/SVE2/sm4e.s

llvm/test/MC/AArch64/SVE2/sm4ekey.s

llvm/test/MC/AArch64/SVE2/smaxp.s

llvm/test/MC/AArch64/SVE2/sminp.s

llvm/test/MC/AArch64/SVE2/smlalb.s

llvm/test/MC/AArch64/SVE2/smlalt.s

llvm/test/MC/AArch64/SVE2/smlslb.s

llvm/test/MC/AArch64/SVE2/smlslt.s

llvm/test/MC/AArch64/SVE2/smulh.s

llvm/test/MC/AArch64/SVE2/smullb.s

llvm/test/MC/AArch64/SVE2/smullt.s

llvm/test/MC/AArch64/SVE2/splice.s

llvm/test/MC/AArch64/SVE2/sqabs.s

llvm/test/MC/AArch64/SVE2/sqadd.s

llvm/test/MC/AArch64/SVE2/sqcadd.s

llvm/test/MC/AArch64/SVE2/sqdmlalb.s

llvm/test/MC/AArch64/SVE2/sqdmlalbt.s

llvm/test/MC/AArch64/SVE2/sqdmlalt.s

llvm/test/MC/AArch64/SVE2/sqdmlslb.s

llvm/test/MC/AArch64/SVE2/sqdmlslbt.s

llvm/test/MC/AArch64/SVE2/sqdmlslt.s

llvm/test/MC/AArch64/SVE2/sqdmulh.s

llvm/test/MC/AArch64/SVE2/sqdmullb.s

llvm/test/MC/AArch64/SVE2/sqdmullt.s

llvm/test/MC/AArch64/SVE2/sqneg.s

llvm/test/MC/AArch64/SVE2/sqrdcmlah.s

llvm/test/MC/AArch64/SVE2/sqrdmlah.s

llvm/test/MC/AArch64/SVE2/sqrdmlsh.s

llvm/test/MC/AArch64/SVE2/sqrdmulh.s

llvm/test/MC/AArch64/SVE2/sqrshl.s

llvm/test/MC/AArch64/SVE2/sqrshlr.s

llvm/test/MC/AArch64/SVE2/sqrshrnb.s

llvm/test/MC/AArch64/SVE2/sqrshrnt.s

llvm/test/MC/AArch64/SVE2/sqrshrunb.s

llvm/test/MC/AArch64/SVE2/sqrshrunt.s

llvm/test/MC/AArch64/SVE2/sqshl.s

llvm/test/MC/AArch64/SVE2/sqshlr.s

llvm/test/MC/AArch64/SVE2/sqshlu.s

llvm/test/MC/AArch64/SVE2/sqshrnb.s

llvm/test/MC/AArch64/SVE2/sqshrnt.s

llvm/test/MC/AArch64/SVE2/sqshrunb.s

llvm/test/MC/AArch64/SVE2/sqshrunt.s

llvm/test/MC/AArch64/SVE2/sqsub.s

llvm/test/MC/AArch64/SVE2/sqsubr.s

llvm/test/MC/AArch64/SVE2/sqxtnb.s

llvm/test/MC/AArch64/SVE2/sqxtnt.s

llvm/test/MC/AArch64/SVE2/sqxtunb.s

llvm/test/MC/AArch64/SVE2/sqxtunt.s

llvm/test/MC/AArch64/SVE2/srhadd.s

llvm/test/MC/AArch64/SVE2/sri.s

llvm/test/MC/AArch64/SVE2/srshl.s

llvm/test/MC/AArch64/SVE2/srshlr.s

llvm/test/MC/AArch64/SVE2/srshr.s

llvm/test/MC/AArch64/SVE2/srsra.s

llvm/test/MC/AArch64/SVE2/sshllb.s

llvm/test/MC/AArch64/SVE2/sshllt.s

llvm/test/MC/AArch64/SVE2/ssra.s

llvm/test/MC/AArch64/SVE2/ssublb.s

llvm/test/MC/AArch64/SVE2/ssublbt.s

llvm/test/MC/AArch64/SVE2/ssublt.s

llvm/test/MC/AArch64/SVE2/ssubltb.s

llvm/test/MC/AArch64/SVE2/ssubwb.s

llvm/test/MC/AArch64/SVE2/ssubwt.s

llvm/test/MC/AArch64/SVE2/stnt1b.s

llvm/test/MC/AArch64/SVE2/stnt1d.s

llvm/test/MC/AArch64/SVE2/stnt1h.s

llvm/test/MC/AArch64/SVE2/stnt1w.s

llvm/test/MC/AArch64/SVE2/subhnb.s

llvm/test/MC/AArch64/SVE2/subhnt.s

llvm/test/MC/AArch64/SVE2/suqadd.s

llvm/test/MC/AArch64/SVE2/tbl.s

llvm/test/MC/AArch64/SVE2/tbx.s

llvm/test/MC/AArch64/SVE2/uaba.s

llvm/test/MC/AArch64/SVE2/uabalb.s

llvm/test/MC/AArch64/SVE2/uabalt.s

llvm/test/MC/AArch64/SVE2/uabdlb.s

llvm/test/MC/AArch64/SVE2/uabdlt.s

llvm/test/MC/AArch64/SVE2/uadalp.s

llvm/test/MC/AArch64/SVE2/uaddlb.s

llvm/test/MC/AArch64/SVE2/uaddlt.s

llvm/test/MC/AArch64/SVE2/uaddwb.s

llvm/test/MC/AArch64/SVE2/uaddwt.s

llvm/test/MC/AArch64/SVE2/uhadd.s

llvm/test/MC/AArch64/SVE2/uhsub.s

llvm/test/MC/AArch64/SVE2/uhsubr.s

llvm/test/MC/AArch64/SVE2/umaxp.s

llvm/test/MC/AArch64/SVE2/uminp.s

llvm/test/MC/AArch64/SVE2/umlalb.s

llvm/test/MC/AArch64/SVE2/umlalt.s

llvm/test/MC/AArch64/SVE2/umlslb.s

llvm/test/MC/AArch64/SVE2/umlslt.s

llvm/test/MC/AArch64/SVE2/umulh.s

llvm/test/MC/AArch64/SVE2/umullb.s

llvm/test/MC/AArch64/SVE2/umullt.s

llvm/test/MC/AArch64/SVE2/uqadd.s

llvm/test/MC/AArch64/SVE2/uqrshl.s

llvm/test/MC/AArch64/SVE2/uqrshlr.s

llvm/test/MC/AArch64/SVE2/uqrshrnb.s

llvm/test/MC/AArch64/SVE2/uqrshrnt.s

llvm/test/MC/AArch64/SVE2/uqshl.s

llvm/test/MC/AArch64/SVE2/uqshlr.s

llvm/test/MC/AArch64/SVE2/uqshrnb.s

llvm/test/MC/AArch64/SVE2/uqshrnt.s

llvm/test/MC/AArch64/SVE2/uqsub.s

llvm/test/MC/AArch64/SVE2/uqsubr.s

llvm/test/MC/AArch64/SVE2/uqxtnb.s

llvm/test/MC/AArch64/SVE2/uqxtnt.s

llvm/test/MC/AArch64/SVE2/urecpe.s

llvm/test/MC/AArch64/SVE2/urhadd.s

llvm/test/MC/AArch64/SVE2/urshl.s

llvm/test/MC/AArch64/SVE2/urshlr.s

llvm/test/MC/AArch64/SVE2/urshr.s

llvm/test/MC/AArch64/SVE2/ursqrte.s

llvm/test/MC/AArch64/SVE2/ursra.s

llvm/test/MC/AArch64/SVE2/ushllb.s

llvm/test/MC/AArch64/SVE2/ushllt.s

llvm/test/MC/AArch64/SVE2/usqadd.s

llvm/test/MC/AArch64/SVE2/usra.s

llvm/test/MC/AArch64/SVE2/usublb.s

llvm/test/MC/AArch64/SVE2/usublt.s

llvm/test/MC/AArch64/SVE2/usubwb.s

llvm/test/MC/AArch64/SVE2/usubwt.s

llvm/test/MC/AArch64/SVE2/whilege.s

llvm/test/MC/AArch64/SVE2/whilegt.s

llvm/test/MC/AArch64/SVE2/whilehi.s

llvm/test/MC/AArch64/SVE2/whilehs.s

llvm/test/MC/AArch64/SVE2/whilerw.s

llvm/test/MC/AArch64/SVE2/whilewr.s

llvm/test/MC/AArch64/SVE2/xar.s