This is an archive of the discontinued LLVM Phabricator instance.

Differential D144508

[AArch64] Fix N2 SchedModel INS instruction latencies
ClosedPublic

Authored by SjoerdMeijer on Feb 21 2023, 11:06 AM.

Details

Reviewers
c-rhodes
dmgreen
paulwalker-arm
peterwaller-arm
andreadb
Commits
rG314e431406de: [AArch64] Fix N2 SchedModel element-to-element INS latencies
Summary

The instruction regexp "^INSv" for the insert gen-reg-to-element was also matching the element-to-element instruction, which has a latency of 2 and not 5 according to the Software Optimization Guide [1], so we were getting that wrong.

I haven't done any performance runs with this change because I don't have access to N2 hardware and also because the fix is hopefully obvious enough. My use-case with this was llvm-mca which is getting things wrong because of this.

[1] https://developer.arm.com/documentation/PJDOC-466751330-18256/latest/

Diff Detail

Event Timeline

SjoerdMeijer created this revision.Feb 21 2023, 11:06 AM
Herald added a reviewer: andreadb. · View Herald TranscriptFeb 21 2023, 11:06 AM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: gbedwell, hiraditya, kristof.beyls. · View Herald Transcript
SjoerdMeijer requested review of this revision.Feb 21 2023, 11:06 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 21 2023, 11:06 AM
Harbormaster completed remote builds in B215071: Diff 499238.Feb 21 2023, 1:40 PM
dmgreen accepted this revision.Feb 21 2023, 11:47 PM

Sounds good to me. This LGTM but you may be able to remove the extra pattern, and it may be worth quickly adding tests for each of the 4 type sizes.

llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td
1216

I don't think this line needs to be added, as it will be handled by WriteV being N2Write_2cyc_1V already. The tighter regex on INSv..gpr should be enough. I believe that is what it means above in:

// ASIMD insert, element to element
// ...
// Handled by SchedAlias<WriteV[dq], ...>
This revision is now accepted and ready to land.Feb 21 2023, 11:47 PM
SjoerdMeijer added a comment.Feb 22 2023, 12:55 AM

Thanks, and I will apply those changes before committing.

This revision was landed with ongoing or failed builds.Feb 22 2023, 3:21 AM
Closed by commit rG314e431406de: [AArch64] Fix N2 SchedModel element-to-element INS latencies (authored by SjoerdMeijer). · Explain Why
This revision was automatically updated to reflect the committed changes.
SjoerdMeijer added a commit: rG314e431406de: [AArch64] Fix N2 SchedModel element-to-element INS latencies.

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
2 lines
test/
tools/
llvm-mca/
AArch64/
Neoverse/
26 lines

Diff 499440

llvm/lib/Target/AArch64/AArch64SchedNeoverseN2.td

Show First 20 Lines • Show All 1,204 Lines • ▼ Show 20 Lines
// ASIMD table lookup extension, 3 table reg // ASIMD table lookup extension, 3 table reg
def : InstRW<[N2Write_6cyc_4V], (instrs TBXv8i8Three, TBXv16i8Three)>; def : InstRW<[N2Write_6cyc_4V], (instrs TBXv8i8Three, TBXv16i8Three)>;
// ASIMD table lookup extension, 4 table reg // ASIMD table lookup extension, 4 table reg
def : InstRW<[N2Write_6cyc_8V], (instrs TBXv8i8Four, TBXv16i8Four)>; def : InstRW<[N2Write_6cyc_8V], (instrs TBXv8i8Four, TBXv16i8Four)>;
// ASIMD transfer, gen reg to element // ASIMD transfer, gen reg to element
def : InstRW<[N2Write_5cyc_1M0_1V], (instregex "^INSv")>; def : InstRW<[N2Write_5cyc_1M0_1V], (instregex "^INSvi(8|16|32|64)gpr$")>;
// ASIMD load instructions // ASIMD load instructions
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
dmgreenUnsubmitted
Not Done
ReplyInline Actions

I don't think this line needs to be added, as it will be handled by WriteV being N2Write_2cyc_1V already. The tighter regex on INSv..gpr should be enough. I believe that is what it means above in:

// ASIMD insert, element to element
// ...
// Handled by SchedAlias<WriteV[dq], ...>
dmgreen: I don't think this line needs to be added, as it will be handled by WriteV being…
// ASIMD load, 1 element, multiple, 1 reg, D-form // ASIMD load, 1 element, multiple, 1 reg, D-form
def : InstRW<[N2Write_6cyc_1L], (instregex "^LD1Onev(8b|4h|2s|1d)$")>; def : InstRW<[N2Write_6cyc_1L], (instregex "^LD1Onev(8b|4h|2s|1d)$")>;
def : InstRW<[N2Write_6cyc_1L, WriteAdr], def : InstRW<[N2Write_6cyc_1L, WriteAdr],
(instregex "^LD1Onev(8b|4h|2s|1d)_POST$")>; (instregex "^LD1Onev(8b|4h|2s|1d)_POST$")>;
// ASIMD load, 1 element, multiple, 1 reg, Q-form // ASIMD load, 1 element, multiple, 1 reg, Q-form
def : InstRW<[N2Write_6cyc_1L], (instregex "^LD1Onev(16b|8h|4s|2d)$")>; def : InstRW<[N2Write_6cyc_1L], (instregex "^LD1Onev(16b|8h|4s|2d)$")>;
▲ Show 20 Lines • Show All 1,056 Lines • Show Last 20 Lines

llvm/test/tools/llvm-mca/AArch64/Neoverse/N2-neon-instructions.s

Show First 20 Lines • Show All 354 Lines • ▼ Show 20 Lines
ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp] ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp]
ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30 ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30
mla v0.8b, v0.8b, v0.8b mla v0.8b, v0.8b, v0.8b
mls v0.4h, v0.4h, v0.4h mls v0.4h, v0.4h, v0.4h
mov b0, v0.b[15] mov b0, v0.b[15]
mov d6, v0.d[1] mov d6, v0.d[1]
mov h2, v0.h[5] mov h2, v0.h[5]
mov s17, v0.s[2] mov s17, v0.s[2]
mov v2.b[0], v0.b[0]
mov v2.h[1], v0.h[1]
mov v2.s[2], v0.s[2]
mov v2.d[1], v0.d[1]
mov v0.b[0], w8
mov v0.h[1], w8
mov v0.s[2], w8
mov v0.d[1], x8
mov v0.16b, v0.16b mov v0.16b, v0.16b
mov v0.8b, v0.8b mov v0.8b, v0.8b
movi d15, #0xff00ff00ff00ff movi d15, #0xff00ff00ff00ff
movi v0.16b, #31 movi v0.16b, #31
movi v0.2d, #0xff0000ff0000ffff movi v0.2d, #0xff0000ff0000ffff
movi v0.2s, #8, msl #8 movi v0.2s, #8, msl #8
movi v0.4s, #255, lsl #24 movi v0.4s, #255, lsl #24
movi v0.8b, #255 movi v0.8b, #255
▲ Show 20 Lines • Show All 1,053 Lines • ▼ Show 20 Lines
# CHECK-NEXT: 7 8 2.00 * ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp] # CHECK-NEXT: 7 8 2.00 * ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp]
# CHECK-NEXT: 8 8 2.00 * ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30 # CHECK-NEXT: 8 8 2.00 * ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30
# CHECK-NEXT: 1 4 1.00 mla v0.8b, v0.8b, v0.8b # CHECK-NEXT: 1 4 1.00 mla v0.8b, v0.8b, v0.8b
# CHECK-NEXT: 1 4 1.00 mls v0.4h, v0.4h, v0.4h # CHECK-NEXT: 1 4 1.00 mls v0.4h, v0.4h, v0.4h
# CHECK-NEXT: 1 2 0.50 mov b0, v0.b[15] # CHECK-NEXT: 1 2 0.50 mov b0, v0.b[15]
# CHECK-NEXT: 1 2 0.50 mov d6, v0.d[1] # CHECK-NEXT: 1 2 0.50 mov d6, v0.d[1]
# CHECK-NEXT: 1 2 0.50 mov h2, v0.h[5] # CHECK-NEXT: 1 2 0.50 mov h2, v0.h[5]
# CHECK-NEXT: 1 2 0.50 mov s17, v0.s[2] # CHECK-NEXT: 1 2 0.50 mov s17, v0.s[2]
# CHECK-NEXT: 1 2 0.50 mov v2.b[0], v0.b[0]
# CHECK-NEXT: 1 2 0.50 mov v2.h[1], v0.h[1]
# CHECK-NEXT: 1 2 0.50 mov v2.s[2], v0.s[2]
# CHECK-NEXT: 1 2 0.50 mov v2.d[1], v0.d[1]
# CHECK-NEXT: 2 5 1.00 mov v0.b[0], w8
# CHECK-NEXT: 2 5 1.00 mov v0.h[1], w8
# CHECK-NEXT: 2 5 1.00 mov v0.s[2], w8
# CHECK-NEXT: 2 5 1.00 mov v0.d[1], x8
# CHECK-NEXT: 1 2 0.50 mov v0.16b, v0.16b # CHECK-NEXT: 1 2 0.50 mov v0.16b, v0.16b
# CHECK-NEXT: 1 2 0.50 mov v0.8b, v0.8b # CHECK-NEXT: 1 2 0.50 mov v0.8b, v0.8b
# CHECK-NEXT: 1 2 0.50 movi d15, #0xff00ff00ff00ff # CHECK-NEXT: 1 2 0.50 movi d15, #0xff00ff00ff00ff
# CHECK-NEXT: 1 2 0.50 movi v0.16b, #31 # CHECK-NEXT: 1 2 0.50 movi v0.16b, #31
# CHECK-NEXT: 1 2 0.50 movi v0.2d, #0xff0000ff0000ffff # CHECK-NEXT: 1 2 0.50 movi v0.2d, #0xff0000ff0000ffff
# CHECK-NEXT: 1 2 0.50 movi v0.2s, #8, msl #8 # CHECK-NEXT: 1 2 0.50 movi v0.2s, #8, msl #8
# CHECK-NEXT: 1 2 0.50 movi v0.4s, #255, lsl #24 # CHECK-NEXT: 1 2 0.50 movi v0.4s, #255, lsl #24
# CHECK-NEXT: 1 2 0.50 movi v0.8b, #255 # CHECK-NEXT: 1 2 0.50 movi v0.8b, #255
▲ Show 20 Lines • Show All 702 Lines • ▼ Show 20 Lines
# CHECK-NEXT: [5] - N2UnitM1 # CHECK-NEXT: [5] - N2UnitM1
# CHECK-NEXT: [6.0] - N2UnitS # CHECK-NEXT: [6.0] - N2UnitS
# CHECK-NEXT: [6.1] - N2UnitS # CHECK-NEXT: [6.1] - N2UnitS
# CHECK-NEXT: [7] - N2UnitV0 # CHECK-NEXT: [7] - N2UnitV0
# CHECK-NEXT: [8] - N2UnitV1 # CHECK-NEXT: [8] - N2UnitV1
# CHECK: Resource pressure per iteration: # CHECK: Resource pressure per iteration:
# CHECK-NEXT: [0.0] [0.1] [1.0] [1.1] [2] [3.0] [3.1] [4] [5] [6.0] [6.1] [7] [8] # CHECK-NEXT: [0.0] [0.1] [1.0] [1.1] [2] [3.0] [3.1] [4] [5] [6.0] [6.1] [7] [8]
# CHECK-NEXT: - - - - 26.67 52.17 52.17 28.75 7.75 7.75 7.75 708.00 617.00 # CHECK-NEXT: - - - - 26.67 52.17 52.17 32.75 7.75 7.75 7.75 712.00 621.00
# CHECK: Resource pressure by instruction: # CHECK: Resource pressure by instruction:
# CHECK-NEXT: [0.0] [0.1] [1.0] [1.1] [2] [3.0] [3.1] [4] [5] [6.0] [6.1] [7] [8] Instructions: # CHECK-NEXT: [0.0] [0.1] [1.0] [1.1] [2] [3.0] [3.1] [4] [5] [6.0] [6.1] [7] [8] Instructions:
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs d29, d24 # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs d29, d24
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.16b, v0.16b # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.16b, v0.16b
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.2d, v0.2d # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.2d, v0.2d
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.2s, v0.2s # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.2s, v0.2s
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.4h, v0.4h # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 abs v0.4h, v0.4h
▲ Show 20 Lines • Show All 346 Lines • ▼ Show 20 Lines
# CHECK-NEXT: - - - - 1.00 1.00 1.00 - - - - 2.00 2.00 ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp] # CHECK-NEXT: - - - - 1.00 1.00 1.00 - - - - 2.00 2.00 ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp]
# CHECK-NEXT: - - - - 1.00 1.00 1.00 0.25 0.25 0.25 0.25 2.00 2.00 ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30 # CHECK-NEXT: - - - - 1.00 1.00 1.00 0.25 0.25 0.25 0.25 2.00 2.00 ld4r { v0.2s, v1.2s, v2.2s, v3.2s }, [sp], x30
# CHECK-NEXT: - - - - - - - - - - - 1.00 - mla v0.8b, v0.8b, v0.8b # CHECK-NEXT: - - - - - - - - - - - 1.00 - mla v0.8b, v0.8b, v0.8b
# CHECK-NEXT: - - - - - - - - - - - 1.00 - mls v0.4h, v0.4h, v0.4h # CHECK-NEXT: - - - - - - - - - - - 1.00 - mls v0.4h, v0.4h, v0.4h
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov b0, v0.b[15] # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov b0, v0.b[15]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov d6, v0.d[1] # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov d6, v0.d[1]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov h2, v0.h[5] # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov h2, v0.h[5]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov s17, v0.s[2] # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov s17, v0.s[2]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v2.b[0], v0.b[0]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v2.h[1], v0.h[1]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v2.s[2], v0.s[2]
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v2.d[1], v0.d[1]
# CHECK-NEXT: - - - - - - - 1.00 - - - 0.50 0.50 mov v0.b[0], w8
# CHECK-NEXT: - - - - - - - 1.00 - - - 0.50 0.50 mov v0.h[1], w8
# CHECK-NEXT: - - - - - - - 1.00 - - - 0.50 0.50 mov v0.s[2], w8
# CHECK-NEXT: - - - - - - - 1.00 - - - 0.50 0.50 mov v0.d[1], x8
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v0.16b, v0.16b # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v0.16b, v0.16b
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v0.8b, v0.8b # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 mov v0.8b, v0.8b
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi d15, #0xff00ff00ff00ff # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi d15, #0xff00ff00ff00ff
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.16b, #31 # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.16b, #31
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.2d, #0xff0000ff0000ffff # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.2d, #0xff0000ff0000ffff
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.2s, #8, msl #8 # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.2s, #8, msl #8
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.4s, #255, lsl #24 # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.4s, #255, lsl #24
# CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.8b, #255 # CHECK-NEXT: - - - - - - - - - - - 0.50 0.50 movi v0.8b, #255
▲ Show 20 Lines • Show All 692 Lines • Show Last 20 Lines