This is an archive of the discontinued LLVM Phabricator instance.

Differential D114777

[AMDGPU] Set most sched model resource's BufferSize to one
ClosedPublic

Authored by kerbowa on Nov 30 2021, 12:26 AM.

Details

Reviewers
rampitec
arsenm
foad
vangthao
vpykhtin
Commits
rGda067ed569e0: [AMDGPU] Set most sched model resource's BufferSize to one
Summary

Using a BufferSize of one for memory ProcResources will result in better
ILP since it more accurately models the dependencies between memory ops
and their consumers on an in-order processor. After this change, the
scheduler will treat the data edges from loads as blocking so that
stalls are guaranteed when waiting for data to be retreaved from memory.
Since we don't actually track waitcnt here, this should do a better job
at modeling their behavior.

Practically, this means that the scheduler will trigger the 'STALL'
heuristic more often.

This type of change needs to be evaluated experimentally. Preliminary
results are positive.

See test: schedule-ilp.mir

Diff Detail

Event Timeline

kerbowa created this revision.Nov 30 2021, 12:26 AM
Herald added subscribers: wenlei, arphaman, hiraditya and 8 others. · View Herald TranscriptNov 30 2021, 12:26 AM
kerbowa requested review of this revision.Nov 30 2021, 12:26 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 30 2021, 12:26 AM
Herald added subscribers: llvm-commits, wdng. · View Herald Transcript
Harbormaster completed remote builds in B136620: Diff 390598.Nov 30 2021, 1:06 AM
critson added a subscriber: critson.Nov 30 2021, 1:14 AM
foad added a comment.Nov 30 2021, 2:13 AM

Would it be possible to write some llvm-mca tests that show exactly how this change affects the scheduling model? schedule-ilp.mir is better than nothing, but it is still a very indirect way of observing what's going on under the covers.

foad added a comment.Nov 30 2021, 2:33 AM

I am mostly in favour of this change just because I never understood how setting BufferSize to (say) 15 bore any relation to that fact that we can have 15 VMEM loads in flight at once. In fact I really don't understand what effect all these different ProcResources have. I think they will stop the scheduler from trying to issue two different (say) VMEM instructions in the same cycle, because they both need to use the same resource; but we have already set IssueWidth = 1, so the scheduler should already know that it can't issue any two instructions in the same cycle.

Incidentally in D87621 we discussed whether most resources should use BufferSize = 0 or 1. I am still unsure.

rampitec accepted this revision.Nov 30 2021, 9:11 AM

LGTM. As previously discussed that seem to describe our HW better. If you can add a test as suggested by Jay that will not hurt.

This revision is now accepted and ready to land.Nov 30 2021, 9:11 AM
critson added a comment.Dec 1 2021, 3:07 AM

On GFX10.1 Vulkan with our usual test cases this seems be performance agnostic.

Closed by commit rGda067ed569e0: [AMDGPU] Set most sched model resource's BufferSize to one (authored by kerbowa). · Explain WhyDec 1 2021, 10:35 PM
This revision was automatically updated to reflect the committed changes.
kerbowa added a commit: rGda067ed569e0: [AMDGPU] Set most sched model resource's BufferSize to one.
kerbowa added a comment.Dec 1 2021, 10:38 PM
In D114777#3160939, @foad wrote:

Would it be possible to write some llvm-mca tests that show exactly how this change affects the scheduling model? schedule-ilp.mir is better than nothing, but it is still a very indirect way of observing what's going on under the covers.

I will add these mca tests along with another patch to improve scheduling for latency. This fixes a few pressing perf issues so I'm committing it now.

foad mentioned this in D112839: [AMDGPU][NFC] Remove autogenerated comment for test.Dec 3 2021, 4:58 AM

Large Diff

This large diff affects 200 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
lib/
Target/
AMDGPU/
6 lines
test/
CodeGen/
AMDGPU/
GlobalISel/
272 lines
14 lines
132 lines
44 lines
62 lines
34 lines
30 lines
64 lines
122 lines
4 lines
8 lines
244 lines
324 lines
191 lines
100 lines
8 lines
93 lines
10 lines
472 lines
50 lines
26 lines
574 lines
12 lines
174 lines
30 lines
10 lines
12 lines
36 lines
234 lines
154 lines
134 lines
190 lines
8 lines
2 lines
2619 lines
22 lines
60 lines
48 lines
460 lines
2 lines
1 line
64 lines
12 lines
12 lines
2 lines
20 lines
4 lines
4 lines
414 lines
30 lines
236 lines
350 lines
617 lines
2 lines
236 lines
535 lines
1429 lines
40 lines
95 lines
115 lines
388 lines
108 lines
4 lines
8 lines
13 lines
39 lines
27 lines
64 lines
128 lines
4 lines
268 lines
70 lines
94 lines
72 lines
2 lines
2 lines
4 lines
358 lines
352 lines
10 lines
56 lines
681 lines
186 lines
351 lines
747 lines
656 lines
1818 lines
516 lines
272 lines
12 lines
2 lines
6 lines
1180 lines
424 lines
818 lines
8 lines
572 lines
6 lines
128 lines
2 lines
18 lines
2 lines
2 lines
57 lines
2 lines
2 lines
533 lines
46 lines
12 lines
4 lines
437 lines
437 lines
222 lines
46 lines
1867 lines
2141 lines
6 lines
20 lines
150 lines
121 lines
3 lines
84 lines
132 lines
70 lines
432 lines
112 lines
432 lines
432 lines
82 lines
432 lines
432 lines
3014 lines
54 lines
3056 lines
2870 lines
138 lines
3056 lines
3056 lines
64 lines
36 lines
64 lines
64 lines
28 lines
64 lines
64 lines
64 lines
74 lines
105 lines
8 lines
15 lines
9 lines
144 lines
240 lines
4 lines
2 lines
2 lines
2221 lines
2 lines
15 lines
56 lines
94 lines
63 lines
833 lines
242 lines
4 lines
126 lines
198 lines
62 lines
4 lines
28 lines
152 lines
920 lines
116 lines
24 lines
226 lines
44 lines
4 lines
570 lines
188 lines
466 lines
566 lines
248 lines
42 lines
32 lines
86 lines
2 lines
146 lines
885 lines
260 lines
9 lines
150 lines
2 lines
232 lines
3 lines
28 lines
9 lines
33 lines
12 lines
12 lines

Diff 391215

llvm/lib/Target/AMDGPU/SISchedule.td

llvm/test/CodeGen/AMDGPU/GlobalISel/cvt_f32_ubyte.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/divergent-control-flow.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i128.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.i8.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/extractelement.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/fmed3.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/fp64-atomics-gfx90a.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/insertelement.i16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/insertelement.i8.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/lds-global-non-entry-func.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/lds-global-value.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.atomic.dec.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.atomic.inc.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.div.fmas.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.div.scale.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.global.atomic.fadd.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.mfma.gfx90a.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.mov.dpp.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.sbfe.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.set.inactive.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.trig.preop.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.ubfe.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.update.dpp.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/load-constant.96.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/load-local.96.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/load-unaligned.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/localizer.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/non-entry-alloca.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/sdivrem.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/store-local.128.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/store-local.96.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/udivrem.ll

llvm/test/CodeGen/AMDGPU/abi-attribute-hints-undefined-behavior.ll

llvm/test/CodeGen/AMDGPU/add.v2i16.ll

llvm/test/CodeGen/AMDGPU/amdgpu-codegenprepare-idiv.ll

llvm/test/CodeGen/AMDGPU/and.ll

llvm/test/CodeGen/AMDGPU/anyext.ll

llvm/test/CodeGen/AMDGPU/atomic_optimizations_global_pointer.ll

llvm/test/CodeGen/AMDGPU/bitreverse.ll

llvm/test/CodeGen/AMDGPU/branch-relaxation-inst-size-gfx10.ll

llvm/test/CodeGen/AMDGPU/branch-relaxation.ll

llvm/test/CodeGen/AMDGPU/bswap.ll

llvm/test/CodeGen/AMDGPU/build_vector.ll

llvm/test/CodeGen/AMDGPU/call-argument-types.ll

llvm/test/CodeGen/AMDGPU/callee-special-input-vgprs.ll

llvm/test/CodeGen/AMDGPU/chain-hi-to-lo.ll

llvm/test/CodeGen/AMDGPU/code-object-v3.ll

llvm/test/CodeGen/AMDGPU/commute-shifts.ll

llvm/test/CodeGen/AMDGPU/copy-illegal-type.ll

llvm/test/CodeGen/AMDGPU/copy_to_scc.ll

llvm/test/CodeGen/AMDGPU/ctlz.ll

llvm/test/CodeGen/AMDGPU/ctlz_zero_undef.ll

llvm/test/CodeGen/AMDGPU/ctpop16.ll

llvm/test/CodeGen/AMDGPU/ctpop64.ll

llvm/test/CodeGen/AMDGPU/cttz.ll

llvm/test/CodeGen/AMDGPU/cttz_zero_undef.ll

llvm/test/CodeGen/AMDGPU/cvt_f32_ubyte.ll

llvm/test/CodeGen/AMDGPU/dag-divergence-atomic.ll

llvm/test/CodeGen/AMDGPU/dbg-value-ends-sched-region.mir

llvm/test/CodeGen/AMDGPU/ds-alignment.ll

llvm/test/CodeGen/AMDGPU/ds_read2.ll

llvm/test/CodeGen/AMDGPU/ds_write2.ll

llvm/test/CodeGen/AMDGPU/extract_vector_elt-i16.ll

llvm/test/CodeGen/AMDGPU/extract_vector_elt-i8.ll

llvm/test/CodeGen/AMDGPU/fabs.ll

llvm/test/CodeGen/AMDGPU/fast-unaligned-load-store.global.ll

llvm/test/CodeGen/AMDGPU/fast-unaligned-load-store.private.ll

llvm/test/CodeGen/AMDGPU/fmax_legacy.f64.ll

llvm/test/CodeGen/AMDGPU/fmin_legacy.f64.ll

llvm/test/CodeGen/AMDGPU/fminnum.f64.ll

llvm/test/CodeGen/AMDGPU/fp-min-max-atomics.ll

llvm/test/CodeGen/AMDGPU/fp64-atomics-gfx90a.ll

llvm/test/CodeGen/AMDGPU/fp_to_sint.ll

llvm/test/CodeGen/AMDGPU/fp_to_uint.ll

llvm/test/CodeGen/AMDGPU/fptosi.f16.ll

llvm/test/CodeGen/AMDGPU/fptoui.f16.ll

llvm/test/CodeGen/AMDGPU/frame-index-elimination.ll

llvm/test/CodeGen/AMDGPU/fshl.ll

llvm/test/CodeGen/AMDGPU/fshr.ll

llvm/test/CodeGen/AMDGPU/fused-bitlogic.ll

llvm/test/CodeGen/AMDGPU/gfx-callable-argument-types.ll

llvm/test/CodeGen/AMDGPU/half.ll

llvm/test/CodeGen/AMDGPU/idot2.ll

llvm/test/CodeGen/AMDGPU/idot4s.ll

llvm/test/CodeGen/AMDGPU/idot4u.ll

llvm/test/CodeGen/AMDGPU/idot8s.ll

llvm/test/CodeGen/AMDGPU/idot8u.ll

llvm/test/CodeGen/AMDGPU/imm.ll

llvm/test/CodeGen/AMDGPU/imm16.ll

llvm/test/CodeGen/AMDGPU/immv216.ll

llvm/test/CodeGen/AMDGPU/insert-subvector-unused-scratch.ll

llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll

llvm/test/CodeGen/AMDGPU/insert_vector_elt.ll

llvm/test/CodeGen/AMDGPU/insert_vector_elt.v2i16.ll

llvm/test/CodeGen/AMDGPU/kernel-args.ll

llvm/test/CodeGen/AMDGPU/kernel-argument-dag-lowering.ll

llvm/test/CodeGen/AMDGPU/lds-atomic-fmin-fmax.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.class.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.cvt.pkrtz.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.fmad.ftz.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.image.dim.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.dim.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.raw.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.set.inactive.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.struct.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.ubfe.ll

llvm/test/CodeGen/AMDGPU/llvm.cos.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.fma.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.fmuladd.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.maxnum.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.minnum.f16.ll

llvm/test/CodeGen/AMDGPU/llvm.round.f64.ll

llvm/test/CodeGen/AMDGPU/llvm.sin.f16.ll

llvm/test/CodeGen/AMDGPU/load-constant-i16.ll

llvm/test/CodeGen/AMDGPU/load-global-i16.ll

llvm/test/CodeGen/AMDGPU/load-hi16.ll

llvm/test/CodeGen/AMDGPU/load-lo16.ll

llvm/test/CodeGen/AMDGPU/load-local.128.ll

llvm/test/CodeGen/AMDGPU/load-local.96.ll

llvm/test/CodeGen/AMDGPU/local-memory.amdgcn.ll

llvm/test/CodeGen/AMDGPU/lshl64-to-32.ll

llvm/test/CodeGen/AMDGPU/lshr.v2i16.ll

llvm/test/CodeGen/AMDGPU/max.i16.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-agent.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-nontemporal.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-singlethread.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-system.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-volatile.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-wavefront.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-flat-workgroup.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-agent.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-nontemporal.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-singlethread.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-system.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-volatile.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-wavefront.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-global-workgroup.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-agent.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-nontemporal.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-singlethread.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-system.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-volatile.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-wavefront.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-local-workgroup.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-private-nontemporal.ll

llvm/test/CodeGen/AMDGPU/memory-legalizer-private-volatile.ll

llvm/test/CodeGen/AMDGPU/memory_clause.ll

llvm/test/CodeGen/AMDGPU/min.ll

llvm/test/CodeGen/AMDGPU/missing-store.ll

llvm/test/CodeGen/AMDGPU/move-addr64-rsrc-dead-subreg-writes.ll

llvm/test/CodeGen/AMDGPU/mul_int24.ll

llvm/test/CodeGen/AMDGPU/mul_uint24-amdgcn.ll

llvm/test/CodeGen/AMDGPU/operand-spacing.ll

llvm/test/CodeGen/AMDGPU/packed-op-sel.ll

llvm/test/CodeGen/AMDGPU/pr51516.mir

llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll

llvm/test/CodeGen/AMDGPU/saddo.ll

llvm/test/CodeGen/AMDGPU/salu-to-valu.ll

llvm/test/CodeGen/AMDGPU/scalar_to_vector.ll

llvm/test/CodeGen/AMDGPU/sched-assert-onlydbg-value-empty-region.mir

llvm/test/CodeGen/AMDGPU/schedule-ilp.mir

llvm/test/CodeGen/AMDGPU/sdiv.ll

llvm/test/CodeGen/AMDGPU/sdiv64.ll

llvm/test/CodeGen/AMDGPU/sdwa-peephole.ll

llvm/test/CodeGen/AMDGPU/select.f16.ll

llvm/test/CodeGen/AMDGPU/select64.ll

llvm/test/CodeGen/AMDGPU/sext-divergence-driven-isel.ll

llvm/test/CodeGen/AMDGPU/sgpr-control-flow.ll

llvm/test/CodeGen/AMDGPU/shift-and-i128-ubfe.ll

llvm/test/CodeGen/AMDGPU/shift-i128.ll

llvm/test/CodeGen/AMDGPU/shl.ll

llvm/test/CodeGen/AMDGPU/shl.v2i16.ll

llvm/test/CodeGen/AMDGPU/shl_add_ptr_global.ll

llvm/test/CodeGen/AMDGPU/shrink-add-sub-constant.ll

llvm/test/CodeGen/AMDGPU/si-annotate-cf.ll

llvm/test/CodeGen/AMDGPU/si-triv-disjoint-mem-access.ll

llvm/test/CodeGen/AMDGPU/sign_extend.ll

llvm/test/CodeGen/AMDGPU/sint_to_fp.i64.ll

llvm/test/CodeGen/AMDGPU/sra.ll

llvm/test/CodeGen/AMDGPU/srem64.ll

llvm/test/CodeGen/AMDGPU/srl.ll

llvm/test/CodeGen/AMDGPU/store-local.128.ll

llvm/test/CodeGen/AMDGPU/store-local.96.ll

llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll

llvm/test/CodeGen/AMDGPU/sub.ll

llvm/test/CodeGen/AMDGPU/sub.v2i16.ll

llvm/test/CodeGen/AMDGPU/udiv.ll

llvm/test/CodeGen/AMDGPU/udiv64.ll

llvm/test/CodeGen/AMDGPU/udivrem.ll

llvm/test/CodeGen/AMDGPU/uint_to_fp.i64.ll

llvm/test/CodeGen/AMDGPU/uniform-cfg.ll

llvm/test/CodeGen/AMDGPU/urem64.ll

llvm/test/CodeGen/AMDGPU/use-sgpr-multiple-times.ll

llvm/test/CodeGen/AMDGPU/v_madak_f16.ll

llvm/test/CodeGen/AMDGPU/vector-extract-insert.ll

llvm/test/CodeGen/AMDGPU/vector_shuffle.packed.ll

llvm/test/CodeGen/AMDGPU/widen-smrd-loads.ll

llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll