[AArch32] Armv8.6a Matrix Mul Assembly Parser Support

[LukeGeeson]

Summary

This patch upstreams support for the Armv8.6-a Matrix Multiplication
Extension. A summary of the features can be found here:

https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/arm-architecture-developments-armv8-6-a

This patch includes:

• Assembly support for AArch32 and Assembly Parsing

D77872 has already added the MC representations of the instructions so that they can be used in code gen; this patch fills in the details needed to make assembly parsing work, and adds tests for asm and disasm

This is part of a patch series, starting with BFloat16 support and
the other components in the armv8.6a extension (in previous patches
linked in phabricator)

Based on work by:

• Luke Geeson
• Oliver Stannard
• Luke Cheeseman

[LukeGeeson]

Harbourmaster builds, unit tests pass, failure down to linting

[simon_tatham]

The important part of this patch seems to be missing! It says it's adding AArch32 assembly parsing for some new instructions, but I don't see any new Tablegen work that adds the instructions. New mnemonics are added to a couple of exclusion lists in ARMAsmParser, but there's no code in here that makes the mnemonics actually exist in the first place, or that says what their operands should be or what their encodings are – just that small C++ tweak and a load of tests. Where's the rest of it?

[LukeGeeson]

In D77874#1991735, @simon_tatham wrote:

The important part of this patch seems to be missing! It says it's adding AArch32 assembly parsing for some new instructions, but I don't see any new Tablegen work that adds the instructions. New mnemonics are added to a couple of exclusion lists in ARMAsmParser, but there's no code in here that makes the mnemonics actually exist in the first place, or that says what their operands should be or what their encodings are – just that small C++ tweak and a load of tests. Where's the rest of it?

Apologies, I submitted multiple patches for review at once. The assembly support is ready to land and is found in [1], so I suppose I should rename this patch to "AArch32 Assembly Parser Support" or some variant thereof. What do you think?

[1] https://reviews.llvm.org/D77872

[simon_tatham]

I see. In that case I suppose the simplest thing to do is just to explain that in the commit message, along the lines of "D77872 has already added the MC representations of the instructions so that they can be used in code gen; this patch fills in the details needed to make assembly parsing work, and adds tests for asm and disasm'.

llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp

6335

Now I look closely, I'm slightly confused about why all of these instructions needed to be added to this if statement. I thought this if statement was supposed to be a list of exceptions to the general rule that a mnemonic ending in a known condition-code string like "eq" or "ne" or "lt" should be broken up into a shorter mnemonic and a conditional suffix. Hence, for example, "wls" is on the list so that it doesn't get misparsed as "w" with condition "ls".

Mind you, I'm also slightly confused about what some of the existing mnemonics in this list are here for. But what specific thing fails if you remove this hunk of the diff?

[LukeGeeson]

llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp

6335

Nothing fails when removing this hunk and running tests (check-all, llvm/test, clang/test).

I agree with you in principle, but I suppose here I'm appealing to the age old adage of 'we did it this way here'

I think we should keep it, and log a separate issue for refactoring this statement in a different issue

[LukeGeeson]

• Updated Commit Message
• Updated Commit Title

[simon_tatham]

llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp

6335

I'm inclined to take the opposite view: if we don't add this hunk now, then if we later find something that breaks as a result, we'll know more about why it had to be here!

[LukeGeeson]

• removed hunk

[LukeGeeson]

• uploaded the right patch this time... (hunk is removed)

[LukeGeeson]

llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp

6335

Very well, I shall remove this hunk

llvm/lib/Target/ARM/AsmParser/ARMAsmParser.cpp

if ((Mnemonic == "movs" && isThumb()) ||
       Mnemonic == "vfmal" || Mnemonic == "vfmsl" ||
       Mnemonic == "wls" || Mnemonic == "le" || Mnemonic == "dls" ||
       Mnemonic == "csel" || Mnemonic == "csinc" ||
       Mnemonic == "csinv" || Mnemonic == "csneg" || Mnemonic == "cinc" ||
       Mnemonic == "cinv" || Mnemonic == "cneg" || Mnemonic == "cset" ||
       Mnemonic == "csetm")
     return Mnemonic;
 
   // First, split out any predication code. Ignore mnemonics we know aren't
   // predicated but do have a carry-set and so weren't caught above.
   if (Mnemonic != "adcs" && Mnemonic != "bics" && Mnemonic != "movs" &&
       Mnemonic != "muls" && Mnemonic != "smlals" && Mnemonic != "smulls" &&
       Mnemonic != "umlals" && Mnemonic != "umulls" && Mnemonic != "lsls" &&
       Mnemonic != "sbcs" && Mnemonic != "rscs" &&
       !(hasMVE() &&
         (Mnemonic == "vmine" ||
          Mnemonic == "vshle" || Mnemonic == "vshlt" || Mnemonic == "vshllt" ||
if (Mnemonic == "bkpt" || Mnemonic == "cbnz" || Mnemonic == "setend" ||
       (FullInst.startswith("vmull") && FullInst.endswith(".p64")) ||
       Mnemonic == "vmovx" || Mnemonic == "vins" ||
       Mnemonic == "vudot" || Mnemonic == "vsdot" ||
       Mnemonic == "vcmla" || Mnemonic == "vcadd" ||
       Mnemonic == "vfmal" || Mnemonic == "vfmsl" ||
       Mnemonic == "vfmat" || Mnemonic == "vfmab" ||
       Mnemonic == "vdot"  || Mnemonic == "vmmla" ||
       Mnemonic == "sb"    || Mnemonic == "ssbb"  ||
-      Mnemonic == "pssbb" ||
+      Mnemonic == "pssbb" || Mnemonic == "vsmmla" ||
+      Mnemonic == "vummla" || Mnemonic == "vusmmla" ||
+      Mnemonic == "vusdot" || Mnemonic == "vsudot" ||
       Mnemonic == "bfcsel" || Mnemonic == "wls" ||
       Mnemonic == "dls" || Mnemonic == "le" || Mnemonic == "csel" ||
       Mnemonic == "csinc" || Mnemonic == "csinv" || Mnemonic == "csneg" ||
       Mnemonic == "cinc" || Mnemonic == "cinv" || Mnemonic == "cneg" ||
       Mnemonic == "cset" || Mnemonic == "csetm" ||
       Mnemonic.startswith("vpt") || Mnemonic.startswith("vpst") ||
       (hasCDE() && MS.isCDEInstr(Mnemonic) &&
        !MS.isITPredicableCDEInstr(Mnemonic)) ||

llvm/test/MC/ARM/armv8.6a-matmul-error.s

+// RUN: not llvm-mc -triple armv8a   -show-encoding -mattr=+i8mm < %s 2>&1 | FileCheck %s
+// RUN: not llvm-mc -triple thumbv8a -show-encoding -mattr=+i8mm < %s 2>&1 | FileCheck %s
+
+
+// VSMMLA, VUMMLA, VUSMMLA
+
+// Data type specifier must match instruction
+
+vsmmla.u8 q0, q1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vsmmla.u8 q0, q1, q2
+// CHECK-NEXT: {{^      \^}}
+
+vummla.s8 q0, q1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vummla.s8 q0, q1, q2
+// CHECK-NEXT: {{^      \^}}
+
+vusmmla.u8 q0, q1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vusmmla.u8 q0, q1, q2
+// CHECK-NEXT: {{^       \^}}
+
+
+// Incorrect register type
+
+vsmmla.s8 d0, q1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [q0, q15]
+// CHECK-NEXT:    vsmmla.s8 d0, q1, q2
+// CHECK-NEXT: {{^          \^}}
+
+vummla.u8 q0, d1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [q0, q15]
+// CHECK-NEXT:    vummla.u8 q0, d1, q2
+// CHECK-NEXT: {{^              \^}}
+
+vusmmla.s8 q0, q1, d2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [q0, q15]
+// CHECK-NEXT:    vusmmla.s8 q0, q1, d2
+// CHECK-NEXT: {{^                   \^}}
+
+
+// VUSDOT (vector)
+
+// Data type specifier must match instruction
+
+vusdot.u8 q0, q1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vusdot.u8 q0, q1, q2
+// CHECK-NEXT: {{^      \^}}
+
+// Mis-matched register types
+
+vusdot.s8 q0, d1, d2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [d0, d31]
+vusdot.s8 d0, q1, d2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [d0, d31]
+vusdot.s8 d0, d1, q2
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: operand must be a register in range [d0, d31]
+
+
+// VUSDOT, VSUDOT (by scalar)
+
+// Data type specifier must match instruction
+
+vusdot.u8 d0, d1, d2[0]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vusdot.u8 d0, d1, d2[0]
+// CHECK-NEXT: {{^      \^}}
+
+vsudot.s8 d0, d1, d2[0]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+// CHECK-NEXT:    vsudot.s8 d0, d1, d2[0]
+// CHECK-NEXT: {{^      \^}}
+
+// Incorrect register types
+
+vusdot.s8 q0, d1, d2[0]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid instruction, any one of the following would fix this:
+// CHECK-NEXT: vusdot.s8 q0, d1, d2[0]
+// CHECK: [[@LINE-3]]:{{[0-9]+}}: note: operand must be a register in range [d0, d31]
+// CHECK-NEXT: vusdot.s8 q0, d1, d2[0]
+// CHECK-NEXT: {{^       \^}}
+// CHECK: [[@LINE-6]]:{{[0-9]+}}: note: operand must be a register in range [q0, q15]
+// CHECK-NEXT: vusdot.s8 q0, d1, d2[0]
+// CHECK-NEXT: {{^           \^}}
+
+vusdot.s8 d0, q1, d2[0]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid instruction, any one of the following would fix this:
+// CHECK-NEXT: vusdot.s8 d0, q1, d2[0]
+// CHECK: [[@LINE-3]]:{{[0-9]+}}: note: operand must be a register in range [d0, d31]
+// CHECK-NEXT: vusdot.s8 d0, q1, d2[0]
+// CHECK-NEXT: {{^           \^}}
+// CHECK: [[@LINE-6]]:{{[0-9]+}}: note: operand must be a register in range [q0, q15]
+// CHECK-NEXT: vusdot.s8 d0, q1, d2[0]
+// CHECK-NEXT: {{^       \^}}
+
+vusdot.s8 q0, q1, q2[0]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid instruction, any one of the following would fix this:
+// CHECK-NEXT: vusdot.s8 q0, q1, q2[0]
+// CHECK: [[@LINE-3]]:{{[0-9]+}}: note: operand must be a register in range [d0, d15]
+// CHECK-NEXT: vusdot.s8 q0, q1, q2[0]
+// CHECK-NEXT: {{^               \^}}
+// CHECK: [[@LINE-6]]:{{[0-9]+}}: note: too many operands for instruction
+// CHECK-NEXT: vusdot.s8 q0, q1, q2[0]
+// CHECK-NEXT: {{^                 \^}}
+
+// Out of range lane index
+
+vusdot.s8 d0, d1, d2[2]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction
+vsudot.u8 q0, q1, d2[2]
+// CHECK: [[@LINE-1]]:{{[0-9]+}}: error: invalid operand for instruction

llvm/test/MC/ARM/armv8.6a-matmul.s

+// RUN:     llvm-mc -triple armv8a   -show-encoding -mattr=+i8mm < %s      | FileCheck %s --check-prefix=ARM
+// RUN:     llvm-mc -triple thumbv8a -show-encoding -mattr=+i8mm < %s      | FileCheck %s --check-prefix=THUMB
+// RUN: not llvm-mc -triple armv8a   -show-encoding -mattr=v8.5a < %s 2>&1 | FileCheck %s --check-prefix=NOMATMUL
+// RUN: not llvm-mc -triple thumbv8a -show-encoding -mattr=v8.5a < %s 2>&1 | FileCheck %s --check-prefix=NOMATMUL
+
+vsmmla.s8 q0, q1, q2
+// ARM: vsmmla.s8       q0, q1, q2      @ encoding: [0x44,0x0c,0x22,0xfc]
+// THUMB: vsmmla.s8       q0, q1, q2      @ encoding: [0x22,0xfc,0x44,0x0c]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vummla.u8 q0, q1, q2
+// ARM: vummla.u8       q0, q1, q2      @ encoding: [0x54,0x0c,0x22,0xfc]
+// THUMB: vummla.u8       q0, q1, q2      @ encoding: [0x22,0xfc,0x54,0x0c]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vusmmla.s8 q0, q1, q2
+// ARM: vusmmla.s8      q0, q1, q2      @ encoding: [0x44,0x0c,0xa2,0xfc]
+// THUMB: vusmmla.s8      q0, q1, q2      @ encoding: [0xa2,0xfc,0x44,0x0c]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vusdot.s8 d0, d1, d2
+// ARM: vusdot.s8       d0, d1, d2      @ encoding: [0x02,0x0d,0xa1,0xfc]
+// THUMB: vusdot.s8       d0, d1, d2      @ encoding: [0xa1,0xfc,0x02,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vusdot.s8 q0, q1, q2
+// ARM: vusdot.s8       q0, q1, q2      @ encoding: [0x44,0x0d,0xa2,0xfc]
+// THUMB: vusdot.s8       q0, q1, q2      @ encoding: [0xa2,0xfc,0x44,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vusdot.s8 d0, d1, d2[0]
+// ARM: vusdot.s8       d0, d1, d2[0]   @ encoding: [0x02,0x0d,0x81,0xfe]
+// THUMB: vusdot.s8       d0, d1, d2[0]   @ encoding: [0x81,0xfe,0x02,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vsudot.u8 d0, d1, d2[1]
+// ARM: vsudot.u8       d0, d1, d2[1]   @ encoding: [0x32,0x0d,0x81,0xfe]
+// THUMB: vsudot.u8       d0, d1, d2[1]   @ encoding: [0x81,0xfe,0x32,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vusdot.s8 q0, q1, d2[0]
+// ARM: vusdot.s8       q0, q1, d2[0]   @ encoding: [0x42,0x0d,0x82,0xfe]
+// THUMB: vusdot.s8       q0, q1, d2[0]   @ encoding: [0x82,0xfe,0x42,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply
+
+vsudot.u8 q0, q1, d2[1]
+// ARM: vsudot.u8       q0, q1, d2[1]   @ encoding: [0x72,0x0d,0x82,0xfe]
+// THUMB: vsudot.u8       q0, q1, d2[1]   @ encoding: [0x82,0xfe,0x72,0x0d]
+// NOMATMUL: [[@LINE-3]]:{{[0-9]+}}: error: instruction requires: 8-bit integer matrix multiply

llvm/test/MC/Disassembler/ARM/armv8.6a-matmul-arm.txt

+# RUN:     llvm-mc -triple=armv8 -mattr=+i8mm  -disassemble < %s      | FileCheck %s
+# RUN: not llvm-mc -triple=armv8 -mattr=+v8.4a -disassemble < %s 2>&1 | FileCheck %s --check-prefix=NOMATMUL
+
+[0x44,0x0c,0x22,0xfc]
+# CHECK: vsmmla.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x54,0x0c,0x22,0xfc]
+# CHECK: vummla.u8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x44,0x0c,0xa2,0xfc]
+# CHECK: vusmmla.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x02,0x0d,0xa1,0xfc]
+# CHECK: vusdot.s8 d0, d1, d2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x44,0x0d,0xa2,0xfc]
+# CHECK: vusdot.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x02,0x0d,0x81,0xfe]
+# CHECK: vusdot.s8 d0, d1, d2[0]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x32,0x0d,0x81,0xfe]
+# CHECK: vsudot.u8 d0, d1, d2[1]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x42,0x0d,0x82,0xfe]
+# CHECK: vusdot.s8 q0, q1, d2[0]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x72,0x0d,0x82,0xfe]
+# CHECK: vsudot.u8 q0, q1, d2[1]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding

llvm/test/MC/Disassembler/ARM/armv8.6a-matmul-thumb.txt

+# RUN:     llvm-mc -triple=thumbv8a -mattr=+i8mm  -disassemble < %s      | FileCheck %s
+# RUN: not llvm-mc -triple=thumbv8a -mattr=+v8.4a -disassemble < %s 2>&1 | FileCheck %s --check-prefix=NOMATMUL
+
+[0x22,0xfc,0x44,0x0c]
+# CHECK: vsmmla.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x22,0xfc,0x54,0x0c]
+# CHECK: vummla.u8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0xa2,0xfc,0x44,0x0c]
+# CHECK: vusmmla.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0xa1,0xfc,0x02,0x0d]
+# CHECK: vusdot.s8 d0, d1, d2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0xa2,0xfc,0x44,0x0d]
+# CHECK: vusdot.s8 q0, q1, q2
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x81,0xfe,0x02,0x0d]
+# CHECK: vusdot.s8 d0, d1, d2[0]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x81,0xfe,0x32,0x0d]
+# CHECK: vsudot.u8 d0, d1, d2[1]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x82,0xfe,0x42,0x0d]
+# CHECK: vusdot.s8 q0, q1, d2[0]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding
+
+[0x82,0xfe,0x72,0x0d]
+# CHECK: vsudot.u8 q0, q1, d2[1]
+# NOMATMUL: :[[@LINE-2]]:{{[0-9]+}}: warning: invalid instruction encoding