[AArch64]SME2 Outer Product and Accumulate instructions

[CarolineConcatto]

Summary

This patch adds the assembly/disassembly for the following instructions:

BMOPA: Bitwise exclusive NOR population count outer product and accumulate.
BMOPS: Bitwise exclusive NOR population count outer product and subtract.

SMOPA (2-way): Signed integer sum of outer products and accumulate.
SMOPS (2-way): Signed integer sum of outer products and subtract.

UMOPA (2-way): Unsigned integer sum of outer products and accumulate.
UMOPS (2-way): Signed integer sum of outer products and accumulate.

The reference can be found here:

https://developer.arm.com/documentation/ddi0602/2022-09

[CarolineConcatto]

• Rebase

[CarolineConcatto]

• Re-use sme class
• Rebase

[CarolineConcatto]

• Rebase correct(hopefully)

[CarolineConcatto]

• Remove zt

[paulwalker-arm]

I'm only really bothered about the instruction names. My other suggestions are nice to have but only if you agree they'll make things better.

llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td

554

Should this be BMOPA_MPPZZ_S because it uses ZA not ZT0?

557

The equivalent SME1 instruction is SMOPA_MPPZZ_S so I'd expect this to be named similar but much like with the DOT instructions you'll need to rename the original to reflect the difference between 4way and 2way. So the original becomes SMOPA_MPPZZ_BtoS and SMOPA_MPPZZ_HtoD and this new one becomes SMOPA_MPPZZ_HtoS. Do you agree?

llvm/lib/Target/AArch64/SMEInstrFormats.td

90

Up to you but given the change perhaps bit u0, bit u1, bit S is better expressed as a 3bit opcode? It seems weird seeing opc{2}, opc{1}, opc{0}. I'm almost tempted to suggest all the operands are just a 5 bit opcode but I'll leave it up to you if you want to make such a change.

148

Again it's up to you but this just looks like a 3bit opcode to me.

[CarolineConcatto]

• Address review commentsy

[CarolineConcatto]

llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td

554

Now, I am not sure if we should do the same for BMOPA.
In sme1 BFMOPA do not have a size, but FMOPA_MPPZZ has(but because it has a D and S size)

557

I had changed for SME1, but these instructions name are in use in AArch64ISelLowering.
So I did not changed for sme1, but did for sme2.
TBH I don't know what is more consistent, changing or not changing the names for sme2.

[paulwalker-arm]

I still want BMOPA_MPPZZ/BMOPS_MPPZZ to include an element suffix but otherwise looks good.

llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td

554

Most all instructions have a size. It is just that some only support a single element type. In such cases we should still add the size suffix so _S in this case because otherwise if a new variant of bmopa is added later we'll be forced to have to rename stuff, much like with the smopa discussion.

557

We should change the old names to be consistent. It doesn't matter that they are used by AArch64ISelLowering because it's just a textual change. It's not that the old names were wrong when originally added, but based on these new SME2 instructions the old names will lead to confusion, hence the new _HtoS style for such things.

With all that said, this assembler work is more important so feel free to defer that refactoring until the assembler support is complete.

[CarolineConcatto]

llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td

557

Thank you Paul, I will create a to do for that.

[dyung]

Hi @CarolineConcatto, your change seems to be causing a test failure, can you take a look and revert if it will take a while to resolve?
https://lab.llvm.org/buildbot/#/builders/188/builds/21487

Note, the same test also failed in the pre-commit check builds: https://reviews.llvm.org/B194938

[nathanchance]

I just bisected check-llvm failing to this change. Based on that and the above comment and the fact that it does not look like there is any progress on a forward fix that I can see (presumably due to time zones), I reverted this change in 23c50432fb25775fe0eb958bc8c2a099b0a0c286.

llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td

 def ADDVA_MPPZ_D : sme_add_vector_to_tile_u64<0b1, "addva">;
 }
 
 let Predicates = [HasSME] in {
 //===----------------------------------------------------------------------===//
 // Outer products
 //===----------------------------------------------------------------------===//
 
-defm BFMOPA_MPPZZ  : sme_bf16_outer_product<0b0, "bfmopa", int_aarch64_sme_mopa_wide>;
-defm BFMOPS_MPPZZ  : sme_bf16_outer_product<0b1, "bfmops", int_aarch64_sme_mops_wide>;
+defm BFMOPA_MPPZZ  : sme_bf16_outer_product<0b000, "bfmopa", int_aarch64_sme_mopa_wide>;
+defm BFMOPS_MPPZZ  : sme_bf16_outer_product<0b001, "bfmops", int_aarch64_sme_mops_wide>;
 
 defm FMOPA_MPPZZ_S : sme_outer_product_fp32<0b0, "fmopa", int_aarch64_sme_mopa>;
 defm FMOPS_MPPZZ_S : sme_outer_product_fp32<0b1, "fmops", int_aarch64_sme_mops>;
 }
 
 let Predicates = [HasSMEF64F64] in {
 defm FMOPA_MPPZZ_D : sme_outer_product_fp64<0b0, "fmopa", int_aarch64_sme_mopa>;
 defm FMOPS_MPPZZ_D : sme_outer_product_fp64<0b1, "fmops", int_aarch64_sme_mops>;
 }
 
 let Predicates = [HasSME] in {
-defm FMOPAL_MPPZZ  : sme_f16_outer_product<0b0, "fmopa", int_aarch64_sme_mopa_wide>;
-defm FMOPSL_MPPZZ  : sme_f16_outer_product<0b1, "fmops", int_aarch64_sme_mops_wide>;
+defm FMOPAL_MPPZZ  : sme_f16_outer_product<0b010, "fmopa", int_aarch64_sme_mopa_wide>;
+defm FMOPSL_MPPZZ  : sme_f16_outer_product<0b011, "fmops", int_aarch64_sme_mops_wide>;
 
 defm SMOPA_MPPZZ_S  : sme_int_outer_product_i32<0b000, "smopa",  int_aarch64_sme_smopa_wide>;
 defm SMOPS_MPPZZ_S  : sme_int_outer_product_i32<0b001, "smops",  int_aarch64_sme_smops_wide>;
 defm UMOPA_MPPZZ_S  : sme_int_outer_product_i32<0b110, "umopa",  int_aarch64_sme_umopa_wide>;
 defm UMOPS_MPPZZ_S  : sme_int_outer_product_i32<0b111, "umops",  int_aarch64_sme_umops_wide>;
 defm SUMOPA_MPPZZ_S : sme_int_outer_product_i32<0b010, "sumopa", int_aarch64_sme_sumopa_wide>;
 defm SUMOPS_MPPZZ_S : sme_int_outer_product_i32<0b011, "sumops", int_aarch64_sme_sumops_wide>;
 defm USMOPA_MPPZZ_S : sme_int_outer_product_i32<0b100, "usmopa", int_aarch64_sme_usmopa_wide>;

 def  UMLSLL_MZZI_BtoS      : sme2_mla_ll_array_index_32b<"umlsll", 0b110>;
 defm UMLSLL_VG2_M2ZZI_BtoS : sme2_mla_ll_array_vg2_index_32b<"umlsll", 0b011>;
 defm UMLSLL_VG4_M4ZZI_BtoS : sme2_mla_ll_array_vg4_index_32b<"umlsll", 0b011>;
 def  UMLSLL_MZZ_BtoS       : sme2_mla_ll_array_single<"umlsll", 0b0110, MatrixOp32, ZPR8, ZPR4b8>;
 defm UMLSLL_VG2_M2ZZ_BtoS  : sme2_mla_ll_array_vg24_single<"umlsll", 0b00110, MatrixOp32, ZZ_b, ZPR4b8>;
 defm UMLSLL_VG4_M4ZZ_BtoS  : sme2_mla_ll_array_vg24_single<"umlsll", 0b01110, MatrixOp32, ZZZZ_b, ZPR4b8>;
 defm UMLSLL_VG2_M2Z2Z_BtoS : sme2_mla_ll_array_vg2_multi<"umlsll", 0b0110, MatrixOp32, ZZ_b_mul_r>;
 defm UMLSLL_VG4_M4Z4Z_BtoS : sme2_mla_ll_array_vg4_multi<"umlsll", 0b0110, MatrixOp32, ZZZZ_b_mul_r>;
-}
 
+defm BMOPA_MPPZZ_S : sme2_bfp_mopx_tile<"bmopa", 0b100>;
+defm BMOPS_MPPZZ_S : sme2_bfp_mopx_tile<"bmops", 0b101>;
+
+defm SMOPA_MPPZZ_HtoS : sme2_int_mopx_tile<"smopa", 0b000>;
+defm SMOPS_MPPZZ_HtoS : sme2_int_mopx_tile<"smops", 0b001>;
+
+defm UMOPA_MPPZZ_HtoS : sme2_int_mopx_tile<"umopa", 0b100>;
+defm UMOPS_MPPZZ_HtoS : sme2_int_mopx_tile<"umops", 0b101>;
+}
 
 let Predicates = [HasSME2, HasSMEI16I64] in {
 defm ADD_VG2_M2ZZ_D  : sme2_dot_mla_add_sub_array_vg24_single<"add", 0b1011010, MatrixOp64, ZZ_d, ZPR4b64>;
 defm ADD_VG4_M4ZZ_D  : sme2_dot_mla_add_sub_array_vg24_single<"add", 0b1111010, MatrixOp64, ZZZZ_d, ZPR4b64>;
 defm ADD_VG2_M2Z2Z_D : sme2_dot_mla_add_sub_array_vg2_multi<"add", 0b111010, MatrixOp64, ZZ_d_mul_r>;
 defm ADD_VG4_M4Z4Z_D : sme2_dot_mla_add_sub_array_vg4_multi<"add", 0b111010, MatrixOp64, ZZZZ_d_mul_r>;
 
 defm SUB_VG2_M2ZZ_D  : sme2_dot_mla_add_sub_array_vg24_single<"sub", 0b1011011, MatrixOp64, ZZ_d, ZPR4b64>;

llvm/lib/Target/AArch64/SMEInstrFormats.td

multiclass sme_outer_product_fp64<bit S, string mnemonic, SDPatternOperator op> {
 
   def NAME # _PSEUDO : sme_outer_product_pseudo<ZPR64>;
 
   def : Pat<(op imm0_7:$tile, (nxv2i1 PPR3bAny:$pn), (nxv2i1 PPR3bAny:$pm),
                 (nxv2f64 ZPR64:$zn), (nxv2f64 ZPR64:$zm)),
             (!cast<Instruction>(NAME # _PSEUDO) imm0_7:$tile, $pn, $pm, $zn, $zm)>;
 }
 
-class sme_int_outer_product_inst<bit u0, bit u1, bit S, bit sz,
+class sme_int_outer_product_inst<bits<3> opc, bit sz, bit sme2,
                                  MatrixTileOperand za_ty, ZPRRegOp zpr_ty,
                                  string mnemonic>
     : I<(outs za_ty:$ZAda),
         (ins za_ty:$_ZAda, PPR3bAny:$Pn, PPR3bAny:$Pm, zpr_ty:$Zn, zpr_ty:$Zm),
         mnemonic, "\t$ZAda, $Pn/m, $Pm/m, $Zn, $Zm",
         "", []>,
       Sched<[]> {
   bits<5> Zm;
   bits<3> Pm;
   bits<3> Pn;
   bits<5> Zn;
   let Inst{31-25} = 0b1010000;
-  let Inst{24}    = u0;
+  let Inst{24}    = opc{2}; // u0
   let Inst{23}    = 0b1;
   let Inst{22}    = sz;
-  let Inst{21}    = u1;
+  let Inst{21}    = opc{1}; // u1
   let Inst{20-16} = Zm;
   let Inst{15-13} = Pm;
   let Inst{12-10} = Pn;
   let Inst{9-5}   = Zn;
-  let Inst{4}     = S;
-  let Inst{3}     = 0b0;
+  let Inst{4}     = opc{0};  //S;
+  let Inst{3}     = sme2;
 
   let Constraints = "$ZAda = $_ZAda";
 }
 
 multiclass sme_int_outer_product_i32<bits<3> opc, string mnemonic,
                                      SDPatternOperator op> {
-  def NAME : sme_int_outer_product_inst<opc{2}, opc{1}, opc{0}, 0b0, TileOp32,
+  def NAME : sme_int_outer_product_inst<opc, 0b0, 0b0,  TileOp32,
                                         ZPR8, mnemonic> {
     bits<2> ZAda;
     let Inst{1-0} = ZAda;
     let Inst{2}   = 0b0;
   }
 
   def NAME # _PSEUDO : sme_outer_product_pseudo<ZPR8>;
 
   def : Pat<(op imm0_3:$tile, (nxv16i1 PPR3bAny:$pn), (nxv16i1 PPR3bAny:$pm),
                 (nxv16i8 ZPR8:$zn), (nxv16i8 ZPR8:$zm)),
             (!cast<Instruction>(NAME # _PSEUDO) imm0_3:$tile, $pn, $pm, $zn, $zm)>;
 }
 
 multiclass sme_int_outer_product_i64<bits<3> opc, string mnemonic,
                                      SDPatternOperator op> {
-  def NAME : sme_int_outer_product_inst<opc{2}, opc{1}, opc{0}, 0b1, TileOp64,
+  def NAME : sme_int_outer_product_inst<opc, 0b1, 0b0, TileOp64,
                                         ZPR16, mnemonic> {
     bits<3> ZAda;
     let Inst{2-0} = ZAda;
   }
 
   def NAME # _PSEUDO : sme_outer_product_pseudo<ZPR16>;
 
   def : Pat<(op imm0_7:$tile, (nxv8i1 PPR3bAny:$pn), (nxv8i1 PPR3bAny:$pm),
                 (nxv8i16 ZPR16:$zn), (nxv8i16 ZPR16:$zm)),
             (!cast<Instruction>(NAME # _PSEUDO) imm0_7:$tile, $pn, $pm, $zn, $zm)>;
 }
 
-class sme_outer_product_widening_inst<bit op, bit S, string mnemonic>
+class sme_outer_product_widening_inst<bits<3> opc, ZPRRegOp zpr_ty, string mnemonic>
     : I<(outs TileOp32:$ZAda),
-        (ins  TileOp32:$_ZAda, PPR3bAny:$Pn, PPR3bAny:$Pm, ZPR16:$Zn, ZPR16:$Zm),
+        (ins  TileOp32:$_ZAda, PPR3bAny:$Pn, PPR3bAny:$Pm, zpr_ty:$Zn, zpr_ty:$Zm),
         mnemonic, "\t$ZAda, $Pn/m, $Pm/m, $Zn, $Zm",
         "", []>,
       Sched<[]> {
   bits<5> Zm;
   bits<3> Pm;
   bits<3> Pn;
   bits<5> Zn;
   bits<2> ZAda;
-  let Inst{31-22} = 0b1000000110;
-  let Inst{21}    = op;
+  let Inst{31-25} = 0b1000000;
+  let Inst{24}    = !if(opc{2}, 0, 1);
+  let Inst{23-22} = 0b10;
+  let Inst{21}    = opc{1};
   let Inst{20-16} = Zm;
   let Inst{15-13} = Pm;
   let Inst{12-10} = Pn;
   let Inst{9-5}   = Zn;
-  let Inst{4}     = S;
-  let Inst{3-2}   = 0b00;
+  let Inst{4}     = opc{0};
+  let Inst{3}     = opc{2};
+  let Inst{2}     = 0b0;
   let Inst{1-0}   = ZAda;
 
   let Constraints = "$ZAda = $_ZAda";
 }
 
-multiclass sme_bf16_outer_product<bit S, string mnemonic, SDPatternOperator op> {
-  def NAME : sme_outer_product_widening_inst<0b0, S, mnemonic>;
+multiclass sme_bf16_outer_product<bits<3> opc, string mnemonic, SDPatternOperator op> {
+  def NAME : sme_outer_product_widening_inst<opc, ZPR16, mnemonic>;
 
   def NAME # _PSEUDO : sme_outer_product_pseudo<ZPR16>;
 
   def : Pat<(op imm0_3:$tile, (nxv8i1 PPR3bAny:$pn), (nxv8i1 PPR3bAny:$pm),
                 (nxv8bf16 ZPR16:$zn), (nxv8bf16 ZPR16:$zm)),
             (!cast<Instruction>(NAME # _PSEUDO) imm0_3:$tile, $pn, $pm, $zn, $zm)>;
 }
 
-multiclass sme_f16_outer_product<bit S, string mnemonic, SDPatternOperator op> {
-  def NAME : sme_outer_product_widening_inst<0b1, S, mnemonic>;
+multiclass sme_f16_outer_product<bits<3> opc, string mnemonic, SDPatternOperator op> {
+  def NAME : sme_outer_product_widening_inst<opc, ZPR16, mnemonic>;
 
   def NAME # _PSEUDO : sme_outer_product_pseudo<ZPR16>;
 
   def : Pat<(op imm0_3:$tile, (nxv8i1 PPR3bAny:$pn), (nxv8i1 PPR3bAny:$pm),
                 (nxv8f16 ZPR16:$zn), (nxv8f16 ZPR16:$zm)),
             (!cast<Instruction>(NAME # _PSEUDO) imm0_3:$tile, $pn, $pm, $zn, $zm)>;
 }
 

 multiclass sme2_mla_ll_array_vg4_multi<string mnemonic, bits<4> op,
                                        MatrixOperand matrix_ty,
                                        RegisterOperand vector_ty> {
   def NAME : sme2_mla_ll_array_vg4_multi<op, matrix_ty, vector_ty, mnemonic>;
 
   def : InstAlias<mnemonic # "\t$ZAda[$Rv, $imm], $Zn, $Zm",
                  (!cast<Instruction>(NAME) matrix_ty:$ZAda, MatrixIndexGPR32Op8_11:$Rv, uimm1s4range:$imm, vector_ty:$Zn, vector_ty:$Zm), 0>;
 }
+
+//===----------------------------------------------------------------------===//
+// SME2 Outer Product and Accumulate
+
+multiclass sme2_int_mopx_tile<string mnemonic, bits<3> op> {
+  def NAME : sme_int_outer_product_inst<op, 0b0, 0b1, TileOp32, ZPR16, mnemonic> {
+    bits<2> ZAda;
+    let Inst{1-0} = ZAda;
+    let Inst{2}   = 0b0;
+  }
+}
+
+multiclass  sme2_bfp_mopx_tile<string mnemonic, bits<3> op> {
+  def NAME : sme_outer_product_widening_inst<op, ZPR32, mnemonic>;
+}

llvm/test/MC/AArch64/SME/smopa-diagnostics.s

 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 // ------------------------------------------------------------------------- //
 // Invalid ZPR type suffix
 //
 // expected: .s => .b, .d => .h
 
 smopa za0.s, p0/m, p0/m, z0.h, z0.b
-// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 // CHECK-NEXT: smopa za0.s, p0/m, p0/m, z0.h, z0.b
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 smopa za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
 // CHECK-NEXT: smopa za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 

llvm/test/MC/AArch64/SME/smops-diagnostics.s

 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 // ------------------------------------------------------------------------- //
 // Invalid ZPR type suffix
 //
 // expected: .s => .b, .d => .h
 
 smops za0.s, p0/m, p0/m, z0.h, z0.b
-// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 // CHECK-NEXT: smops za0.s, p0/m, p0/m, z0.h, z0.b
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 smops za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
 // CHECK-NEXT: smops za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 

llvm/test/MC/AArch64/SME/umopa-diagnostics.s

 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 // ------------------------------------------------------------------------- //
 // Invalid ZPR type suffix
 //
 // expected: .s => .b, .d => .h
 
 umopa za0.s, p0/m, p0/m, z0.h, z0.b
-// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 // CHECK-NEXT: umopa za0.s, p0/m, p0/m, z0.h, z0.b
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 umopa za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
 // CHECK-NEXT: umopa za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 

llvm/test/MC/AArch64/SME/umops-diagnostics.s

 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 // ------------------------------------------------------------------------- //
 // Invalid ZPR type suffix
 //
 // expected: .s => .b, .d => .h
 
 umops za0.s, p0/m, p0/m, z0.h, z0.b
-// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
 // CHECK-NEXT: umops za0.s, p0/m, p0/m, z0.h, z0.b
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 
 umops za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
 // CHECK-NEXT: umops za0.s, p0/m, p0/m, z0.b, z0.s
 // CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
 

llvm/test/MC/AArch64/SME2/bmopa-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+bmopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: bmopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+bmopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: bmopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+bmopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: bmopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+bmopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: bmopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+bmopa za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/bmopa.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+bmopa   za0.s, p0/m, p0/m, z0.s, z0.s  // 10000000-10000000-00000000-00001000
+// CHECK-INST: bmopa   za0.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-ENCODING: [0x08,0x00,0x80,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80800008 <unknown>
+
+bmopa   za1.s, p5/m, p2/m, z10.s, z21.s  // 10000000-10010101-01010101-01001001
+// CHECK-INST: bmopa   za1.s, p5/m, p2/m, z10.s, z21.s
+// CHECK-ENCODING: [0x49,0x55,0x95,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80955549 <unknown>
+
+bmopa   za3.s, p3/m, p7/m, z13.s, z8.s  // 10000000-10001000-11101101-10101011
+// CHECK-INST: bmopa   za3.s, p3/m, p7/m, z13.s, z8.s
+// CHECK-ENCODING: [0xab,0xed,0x88,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 8088edab <unknown>
+
+bmopa   za3.s, p7/m, p7/m, z31.s, z31.s  // 10000000-10011111-11111111-11101011
+// CHECK-INST: bmopa   za3.s, p7/m, p7/m, z31.s, z31.s
+// CHECK-ENCODING: [0xeb,0xff,0x9f,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809fffeb <unknown>
+
+bmopa   za1.s, p3/m, p0/m, z17.s, z16.s  // 10000000-10010000-00001110-00101001
+// CHECK-INST: bmopa   za1.s, p3/m, p0/m, z17.s, z16.s
+// CHECK-ENCODING: [0x29,0x0e,0x90,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80900e29 <unknown>
+
+bmopa   za1.s, p1/m, p4/m, z1.s, z30.s  // 10000000-10011110-10000100-00101001
+// CHECK-INST: bmopa   za1.s, p1/m, p4/m, z1.s, z30.s
+// CHECK-ENCODING: [0x29,0x84,0x9e,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809e8429 <unknown>
+
+bmopa   za0.s, p5/m, p2/m, z19.s, z20.s  // 10000000-10010100-01010110-01101000
+// CHECK-INST: bmopa   za0.s, p5/m, p2/m, z19.s, z20.s
+// CHECK-ENCODING: [0x68,0x56,0x94,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80945668 <unknown>
+
+bmopa   za0.s, p6/m, p0/m, z12.s, z2.s  // 10000000-10000010-00011001-10001000
+// CHECK-INST: bmopa   za0.s, p6/m, p0/m, z12.s, z2.s
+// CHECK-ENCODING: [0x88,0x19,0x82,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80821988 <unknown>
+
+bmopa   za1.s, p2/m, p6/m, z1.s, z26.s  // 10000000-10011010-11001000-00101001
+// CHECK-INST: bmopa   za1.s, p2/m, p6/m, z1.s, z26.s
+// CHECK-ENCODING: [0x29,0xc8,0x9a,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809ac829 <unknown>
+
+bmopa   za1.s, p2/m, p0/m, z22.s, z30.s  // 10000000-10011110-00001010-11001001
+// CHECK-INST: bmopa   za1.s, p2/m, p0/m, z22.s, z30.s
+// CHECK-ENCODING: [0xc9,0x0a,0x9e,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809e0ac9 <unknown>
+
+bmopa   za2.s, p5/m, p7/m, z9.s, z1.s  // 10000000-10000001-11110101-00101010
+// CHECK-INST: bmopa   za2.s, p5/m, p7/m, z9.s, z1.s
+// CHECK-ENCODING: [0x2a,0xf5,0x81,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 8081f52a <unknown>
+
+bmopa   za3.s, p2/m, p5/m, z12.s, z11.s  // 10000000-10001011-10101001-10001011
+// CHECK-INST: bmopa   za3.s, p2/m, p5/m, z12.s, z11.s
+// CHECK-ENCODING: [0x8b,0xa9,0x8b,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 808ba98b <unknown>
+

llvm/test/MC/AArch64/SME2/bmops-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+bmops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: bmops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+bmops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: bmops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+bmops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: bmops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+bmops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: bmops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+bmops za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/bmops.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+bmops   za0.s, p0/m, p0/m, z0.s, z0.s  // 10000000-10000000-00000000-00011000
+// CHECK-INST: bmops   za0.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-ENCODING: [0x18,0x00,0x80,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80800018 <unknown>
+
+bmops   za1.s, p5/m, p2/m, z10.s, z21.s  // 10000000-10010101-01010101-01011001
+// CHECK-INST: bmops   za1.s, p5/m, p2/m, z10.s, z21.s
+// CHECK-ENCODING: [0x59,0x55,0x95,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80955559 <unknown>
+
+bmops   za3.s, p3/m, p7/m, z13.s, z8.s  // 10000000-10001000-11101101-10111011
+// CHECK-INST: bmops   za3.s, p3/m, p7/m, z13.s, z8.s
+// CHECK-ENCODING: [0xbb,0xed,0x88,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 8088edbb <unknown>
+
+bmops   za3.s, p7/m, p7/m, z31.s, z31.s  // 10000000-10011111-11111111-11111011
+// CHECK-INST: bmops   za3.s, p7/m, p7/m, z31.s, z31.s
+// CHECK-ENCODING: [0xfb,0xff,0x9f,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809ffffb <unknown>
+
+bmops   za1.s, p3/m, p0/m, z17.s, z16.s  // 10000000-10010000-00001110-00111001
+// CHECK-INST: bmops   za1.s, p3/m, p0/m, z17.s, z16.s
+// CHECK-ENCODING: [0x39,0x0e,0x90,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80900e39 <unknown>
+
+bmops   za1.s, p1/m, p4/m, z1.s, z30.s  // 10000000-10011110-10000100-00111001
+// CHECK-INST: bmops   za1.s, p1/m, p4/m, z1.s, z30.s
+// CHECK-ENCODING: [0x39,0x84,0x9e,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809e8439 <unknown>
+
+bmops   za0.s, p5/m, p2/m, z19.s, z20.s  // 10000000-10010100-01010110-01111000
+// CHECK-INST: bmops   za0.s, p5/m, p2/m, z19.s, z20.s
+// CHECK-ENCODING: [0x78,0x56,0x94,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80945678 <unknown>
+
+bmops   za0.s, p6/m, p0/m, z12.s, z2.s  // 10000000-10000010-00011001-10011000
+// CHECK-INST: bmops   za0.s, p6/m, p0/m, z12.s, z2.s
+// CHECK-ENCODING: [0x98,0x19,0x82,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 80821998 <unknown>
+
+bmops   za1.s, p2/m, p6/m, z1.s, z26.s  // 10000000-10011010-11001000-00111001
+// CHECK-INST: bmops   za1.s, p2/m, p6/m, z1.s, z26.s
+// CHECK-ENCODING: [0x39,0xc8,0x9a,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809ac839 <unknown>
+
+bmops   za1.s, p2/m, p0/m, z22.s, z30.s  // 10000000-10011110-00001010-11011001
+// CHECK-INST: bmops   za1.s, p2/m, p0/m, z22.s, z30.s
+// CHECK-ENCODING: [0xd9,0x0a,0x9e,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 809e0ad9 <unknown>
+
+bmops   za2.s, p5/m, p7/m, z9.s, z1.s  // 10000000-10000001-11110101-00111010
+// CHECK-INST: bmops   za2.s, p5/m, p7/m, z9.s, z1.s
+// CHECK-ENCODING: [0x3a,0xf5,0x81,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 8081f53a <unknown>
+
+bmops   za3.s, p2/m, p5/m, z12.s, z11.s  // 10000000-10001011-10101001-10011011
+// CHECK-INST: bmops   za3.s, p2/m, p5/m, z12.s, z11.s
+// CHECK-ENCODING: [0x9b,0xa9,0x8b,0x80]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: 808ba99b <unknown>
+

llvm/test/MC/AArch64/SME2/smopa-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+smopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: smopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+smopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: smopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+smopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: smopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+smopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: smopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+smopa za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/smopa.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+smopa   za0.s, p0/m, p0/m, z0.h, z0.h  // 10100000-10000000-00000000-00001000
+// CHECK-INST: smopa   za0.s, p0/m, p0/m, z0.h, z0.h
+// CHECK-ENCODING: [0x08,0x00,0x80,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0800008 <unknown>
+
+smopa   za1.s, p5/m, p2/m, z10.h, z21.h  // 10100000-10010101-01010101-01001001
+// CHECK-INST: smopa   za1.s, p5/m, p2/m, z10.h, z21.h
+// CHECK-ENCODING: [0x49,0x55,0x95,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0955549 <unknown>
+
+smopa   za3.s, p3/m, p7/m, z13.h, z8.h  // 10100000-10001000-11101101-10101011
+// CHECK-INST: smopa   za3.s, p3/m, p7/m, z13.h, z8.h
+// CHECK-ENCODING: [0xab,0xed,0x88,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a088edab <unknown>
+
+smopa   za3.s, p7/m, p7/m, z31.h, z31.h  // 10100000-10011111-11111111-11101011
+// CHECK-INST: smopa   za3.s, p7/m, p7/m, z31.h, z31.h
+// CHECK-ENCODING: [0xeb,0xff,0x9f,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09fffeb <unknown>
+
+smopa   za1.s, p3/m, p0/m, z17.h, z16.h  // 10100000-10010000-00001110-00101001
+// CHECK-INST: smopa   za1.s, p3/m, p0/m, z17.h, z16.h
+// CHECK-ENCODING: [0x29,0x0e,0x90,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0900e29 <unknown>
+
+smopa   za1.s, p1/m, p4/m, z1.h, z30.h  // 10100000-10011110-10000100-00101001
+// CHECK-INST: smopa   za1.s, p1/m, p4/m, z1.h, z30.h
+// CHECK-ENCODING: [0x29,0x84,0x9e,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09e8429 <unknown>
+
+smopa   za0.s, p5/m, p2/m, z19.h, z20.h  // 10100000-10010100-01010110-01101000
+// CHECK-INST: smopa   za0.s, p5/m, p2/m, z19.h, z20.h
+// CHECK-ENCODING: [0x68,0x56,0x94,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0945668 <unknown>
+
+smopa   za0.s, p6/m, p0/m, z12.h, z2.h  // 10100000-10000010-00011001-10001000
+// CHECK-INST: smopa   za0.s, p6/m, p0/m, z12.h, z2.h
+// CHECK-ENCODING: [0x88,0x19,0x82,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0821988 <unknown>
+
+smopa   za1.s, p2/m, p6/m, z1.h, z26.h  // 10100000-10011010-11001000-00101001
+// CHECK-INST: smopa   za1.s, p2/m, p6/m, z1.h, z26.h
+// CHECK-ENCODING: [0x29,0xc8,0x9a,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09ac829 <unknown>
+
+smopa   za1.s, p2/m, p0/m, z22.h, z30.h  // 10100000-10011110-00001010-11001001
+// CHECK-INST: smopa   za1.s, p2/m, p0/m, z22.h, z30.h
+// CHECK-ENCODING: [0xc9,0x0a,0x9e,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09e0ac9 <unknown>
+
+smopa   za2.s, p5/m, p7/m, z9.h, z1.h  // 10100000-10000001-11110101-00101010
+// CHECK-INST: smopa   za2.s, p5/m, p7/m, z9.h, z1.h
+// CHECK-ENCODING: [0x2a,0xf5,0x81,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a081f52a <unknown>
+
+smopa   za3.s, p2/m, p5/m, z12.h, z11.h  // 10100000-10001011-10101001-10001011
+// CHECK-INST: smopa   za3.s, p2/m, p5/m, z12.h, z11.h
+// CHECK-ENCODING: [0x8b,0xa9,0x8b,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a08ba98b <unknown>
+

llvm/test/MC/AArch64/SME2/smops-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+smops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: smops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+smops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: smops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+smops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: smops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+smops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: smops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+smops za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/smops.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+smops   za0.s, p0/m, p0/m, z0.h, z0.h  // 10100000-10000000-00000000-00011000
+// CHECK-INST: smops   za0.s, p0/m, p0/m, z0.h, z0.h
+// CHECK-ENCODING: [0x18,0x00,0x80,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0800018 <unknown>
+
+smops   za1.s, p5/m, p2/m, z10.h, z21.h  // 10100000-10010101-01010101-01011001
+// CHECK-INST: smops   za1.s, p5/m, p2/m, z10.h, z21.h
+// CHECK-ENCODING: [0x59,0x55,0x95,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0955559 <unknown>
+
+smops   za3.s, p3/m, p7/m, z13.h, z8.h  // 10100000-10001000-11101101-10111011
+// CHECK-INST: smops   za3.s, p3/m, p7/m, z13.h, z8.h
+// CHECK-ENCODING: [0xbb,0xed,0x88,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a088edbb <unknown>
+
+smops   za3.s, p7/m, p7/m, z31.h, z31.h  // 10100000-10011111-11111111-11111011
+// CHECK-INST: smops   za3.s, p7/m, p7/m, z31.h, z31.h
+// CHECK-ENCODING: [0xfb,0xff,0x9f,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09ffffb <unknown>
+
+smops   za1.s, p3/m, p0/m, z17.h, z16.h  // 10100000-10010000-00001110-00111001
+// CHECK-INST: smops   za1.s, p3/m, p0/m, z17.h, z16.h
+// CHECK-ENCODING: [0x39,0x0e,0x90,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0900e39 <unknown>
+
+smops   za1.s, p1/m, p4/m, z1.h, z30.h  // 10100000-10011110-10000100-00111001
+// CHECK-INST: smops   za1.s, p1/m, p4/m, z1.h, z30.h
+// CHECK-ENCODING: [0x39,0x84,0x9e,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09e8439 <unknown>
+
+smops   za0.s, p5/m, p2/m, z19.h, z20.h  // 10100000-10010100-01010110-01111000
+// CHECK-INST: smops   za0.s, p5/m, p2/m, z19.h, z20.h
+// CHECK-ENCODING: [0x78,0x56,0x94,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0945678 <unknown>
+
+smops   za0.s, p6/m, p0/m, z12.h, z2.h  // 10100000-10000010-00011001-10011000
+// CHECK-INST: smops   za0.s, p6/m, p0/m, z12.h, z2.h
+// CHECK-ENCODING: [0x98,0x19,0x82,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a0821998 <unknown>
+
+smops   za1.s, p2/m, p6/m, z1.h, z26.h  // 10100000-10011010-11001000-00111001
+// CHECK-INST: smops   za1.s, p2/m, p6/m, z1.h, z26.h
+// CHECK-ENCODING: [0x39,0xc8,0x9a,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09ac839 <unknown>
+
+smops   za1.s, p2/m, p0/m, z22.h, z30.h  // 10100000-10011110-00001010-11011001
+// CHECK-INST: smops   za1.s, p2/m, p0/m, z22.h, z30.h
+// CHECK-ENCODING: [0xd9,0x0a,0x9e,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a09e0ad9 <unknown>
+
+smops   za2.s, p5/m, p7/m, z9.h, z1.h  // 10100000-10000001-11110101-00111010
+// CHECK-INST: smops   za2.s, p5/m, p7/m, z9.h, z1.h
+// CHECK-ENCODING: [0x3a,0xf5,0x81,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a081f53a <unknown>
+
+smops   za3.s, p2/m, p5/m, z12.h, z11.h  // 10100000-10001011-10101001-10011011
+// CHECK-INST: smops   za3.s, p2/m, p5/m, z12.h, z11.h
+// CHECK-ENCODING: [0x9b,0xa9,0x8b,0xa0]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a08ba99b <unknown>
+

llvm/test/MC/AArch64/SME2/umopa-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+umopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: umopa za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+umopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: umopa za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+umopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: umopa za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+umopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: umopa za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+umopa za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/umopa.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+umopa   za0.s, p0/m, p0/m, z0.h, z0.h  // 10100001-10000000-00000000-00001000
+// CHECK-INST: umopa   za0.s, p0/m, p0/m, z0.h, z0.h
+// CHECK-ENCODING: [0x08,0x00,0x80,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1800008 <unknown>
+
+umopa   za1.s, p5/m, p2/m, z10.h, z21.h  // 10100001-10010101-01010101-01001001
+// CHECK-INST: umopa   za1.s, p5/m, p2/m, z10.h, z21.h
+// CHECK-ENCODING: [0x49,0x55,0x95,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1955549 <unknown>
+
+umopa   za3.s, p3/m, p7/m, z13.h, z8.h  // 10100001-10001000-11101101-10101011
+// CHECK-INST: umopa   za3.s, p3/m, p7/m, z13.h, z8.h
+// CHECK-ENCODING: [0xab,0xed,0x88,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a188edab <unknown>
+
+umopa   za3.s, p7/m, p7/m, z31.h, z31.h  // 10100001-10011111-11111111-11101011
+// CHECK-INST: umopa   za3.s, p7/m, p7/m, z31.h, z31.h
+// CHECK-ENCODING: [0xeb,0xff,0x9f,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19fffeb <unknown>
+
+umopa   za1.s, p3/m, p0/m, z17.h, z16.h  // 10100001-10010000-00001110-00101001
+// CHECK-INST: umopa   za1.s, p3/m, p0/m, z17.h, z16.h
+// CHECK-ENCODING: [0x29,0x0e,0x90,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1900e29 <unknown>
+
+umopa   za1.s, p1/m, p4/m, z1.h, z30.h  // 10100001-10011110-10000100-00101001
+// CHECK-INST: umopa   za1.s, p1/m, p4/m, z1.h, z30.h
+// CHECK-ENCODING: [0x29,0x84,0x9e,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19e8429 <unknown>
+
+umopa   za0.s, p5/m, p2/m, z19.h, z20.h  // 10100001-10010100-01010110-01101000
+// CHECK-INST: umopa   za0.s, p5/m, p2/m, z19.h, z20.h
+// CHECK-ENCODING: [0x68,0x56,0x94,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1945668 <unknown>
+
+umopa   za0.s, p6/m, p0/m, z12.h, z2.h  // 10100001-10000010-00011001-10001000
+// CHECK-INST: umopa   za0.s, p6/m, p0/m, z12.h, z2.h
+// CHECK-ENCODING: [0x88,0x19,0x82,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1821988 <unknown>
+
+umopa   za1.s, p2/m, p6/m, z1.h, z26.h  // 10100001-10011010-11001000-00101001
+// CHECK-INST: umopa   za1.s, p2/m, p6/m, z1.h, z26.h
+// CHECK-ENCODING: [0x29,0xc8,0x9a,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19ac829 <unknown>
+
+umopa   za1.s, p2/m, p0/m, z22.h, z30.h  // 10100001-10011110-00001010-11001001
+// CHECK-INST: umopa   za1.s, p2/m, p0/m, z22.h, z30.h
+// CHECK-ENCODING: [0xc9,0x0a,0x9e,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19e0ac9 <unknown>
+
+umopa   za2.s, p5/m, p7/m, z9.h, z1.h  // 10100001-10000001-11110101-00101010
+// CHECK-INST: umopa   za2.s, p5/m, p7/m, z9.h, z1.h
+// CHECK-ENCODING: [0x2a,0xf5,0x81,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a181f52a <unknown>
+
+umopa   za3.s, p2/m, p5/m, z12.h, z11.h  // 10100001-10001011-10101001-10001011
+// CHECK-INST: umopa   za3.s, p2/m, p5/m, z12.h, z11.h
+// CHECK-ENCODING: [0x8b,0xa9,0x8b,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a18ba98b <unknown>
+

llvm/test/MC/AArch64/SME2/umops-diagnostics.s

+// RUN: not llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 2>&1 < %s | FileCheck %s
+
+// --------------------------------------------------------------------------//
+// Invalid tile
+
+umops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: umops za8.s, p0/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid predicate
+
+umops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT: umops za0.s, p0/z, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+umops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid restricted predicate register, expected p0..p7 (without element suffix)
+// CHECK-NEXT: umops za0.s, p15/m, p0/m, z0.s, z0.s
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+// --------------------------------------------------------------------------//
+// Invalid suffixes
+
+umops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid matrix operand, expected za[0-3].s
+// CHECK-NEXT: umops za0.d, p0/z, p0/m, z0.d, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:
+
+umops za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid element width
+// CHECK-NEXT: za0.s, p0/m, p0/m, z0.s, z0.d
+// CHECK-NOT: [[@LINE-1]]:{{[0-9]+}}:

llvm/test/MC/AArch64/SME2/umops.s

+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+// RUN: not llvm-mc -triple=aarch64 -show-encoding < %s 2>&1 \
+// RUN:        | FileCheck %s --check-prefix=CHECK-ERROR
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d --mattr=+sme2 - | FileCheck %s --check-prefix=CHECK-INST
+// RUN: llvm-mc -triple=aarch64 -filetype=obj -mattr=+sme2 < %s \
+// RUN:        | llvm-objdump -d - | FileCheck %s --check-prefix=CHECK-UNKNOWN
+// RUN: llvm-mc -triple=aarch64 -show-encoding -mattr=+sme2 < %s \
+// RUN:        | sed '/.text/d' | sed 's/.*encoding: //g' \
+// RUN:        | llvm-mc -triple=aarch64 -mattr=+sme2 -disassemble -show-encoding \
+// RUN:        | FileCheck %s --check-prefixes=CHECK-ENCODING,CHECK-INST
+
+
+umops   za0.s, p0/m, p0/m, z0.h, z0.h  // 10100001-10000000-00000000-00011000
+// CHECK-INST: umops   za0.s, p0/m, p0/m, z0.h, z0.h
+// CHECK-ENCODING: [0x18,0x00,0x80,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1800018 <unknown>
+
+umops   za1.s, p5/m, p2/m, z10.h, z21.h  // 10100001-10010101-01010101-01011001
+// CHECK-INST: umops   za1.s, p5/m, p2/m, z10.h, z21.h
+// CHECK-ENCODING: [0x59,0x55,0x95,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1955559 <unknown>
+
+umops   za3.s, p3/m, p7/m, z13.h, z8.h  // 10100001-10001000-11101101-10111011
+// CHECK-INST: umops   za3.s, p3/m, p7/m, z13.h, z8.h
+// CHECK-ENCODING: [0xbb,0xed,0x88,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a188edbb <unknown>
+
+umops   za3.s, p7/m, p7/m, z31.h, z31.h  // 10100001-10011111-11111111-11111011
+// CHECK-INST: umops   za3.s, p7/m, p7/m, z31.h, z31.h
+// CHECK-ENCODING: [0xfb,0xff,0x9f,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19ffffb <unknown>
+
+umops   za1.s, p3/m, p0/m, z17.h, z16.h  // 10100001-10010000-00001110-00111001
+// CHECK-INST: umops   za1.s, p3/m, p0/m, z17.h, z16.h
+// CHECK-ENCODING: [0x39,0x0e,0x90,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1900e39 <unknown>
+
+umops   za1.s, p1/m, p4/m, z1.h, z30.h  // 10100001-10011110-10000100-00111001
+// CHECK-INST: umops   za1.s, p1/m, p4/m, z1.h, z30.h
+// CHECK-ENCODING: [0x39,0x84,0x9e,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19e8439 <unknown>
+
+umops   za0.s, p5/m, p2/m, z19.h, z20.h  // 10100001-10010100-01010110-01111000
+// CHECK-INST: umops   za0.s, p5/m, p2/m, z19.h, z20.h
+// CHECK-ENCODING: [0x78,0x56,0x94,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1945678 <unknown>
+
+umops   za0.s, p6/m, p0/m, z12.h, z2.h  // 10100001-10000010-00011001-10011000
+// CHECK-INST: umops   za0.s, p6/m, p0/m, z12.h, z2.h
+// CHECK-ENCODING: [0x98,0x19,0x82,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a1821998 <unknown>
+
+umops   za1.s, p2/m, p6/m, z1.h, z26.h  // 10100001-10011010-11001000-00111001
+// CHECK-INST: umops   za1.s, p2/m, p6/m, z1.h, z26.h
+// CHECK-ENCODING: [0x39,0xc8,0x9a,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19ac839 <unknown>
+
+umops   za1.s, p2/m, p0/m, z22.h, z30.h  // 10100001-10011110-00001010-11011001
+// CHECK-INST: umops   za1.s, p2/m, p0/m, z22.h, z30.h
+// CHECK-ENCODING: [0xd9,0x0a,0x9e,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a19e0ad9 <unknown>
+
+umops   za2.s, p5/m, p7/m, z9.h, z1.h  // 10100001-10000001-11110101-00111010
+// CHECK-INST: umops   za2.s, p5/m, p7/m, z9.h, z1.h
+// CHECK-ENCODING: [0x3a,0xf5,0x81,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a181f53a <unknown>
+
+umops   za3.s, p2/m, p5/m, z12.h, z11.h  // 10100001-10001011-10101001-10011011
+// CHECK-INST: umops   za3.s, p2/m, p5/m, z12.h, z11.h
+// CHECK-ENCODING: [0x9b,0xa9,0x8b,0xa1]
+// CHECK-ERROR: instruction requires: sme2
+// CHECK-UNKNOWN: a18ba99b <unknown>
+