This is an archive of the discontinued LLVM Phabricator instance.

Differential D110708

[X86][Costmodel] Load/store i8 Stride=2 VF=16 interleaving costs
ClosedPublic

Authored by lebedev.ri on Sep 29 2021, 6:02 AM.

Details

Reviewers
RKSimon
Commits
rGbac60c55e0ff: [X86][Costmodel] Load/store i8 Stride=2 VF=16 interleaving costs
Summary

The only sched models that for cpu's that support avx2
but not avx512 are: haswell, broadwell, skylake, zen1-3

For load we have:
https://godbolt.org/z/a9hv4z47v - for intels Block RThroughput: =4.0; for ryzens, Block RThroughput: =2.0
So pick cost of 4.

For store we have:
https://godbolt.org/z/6GfPn1b79 - for intels Block RThroughput: =3.0; for ryzens, Block RThroughput: <=2.0
So pick cost of 3.

I'm directly using the shuffling asm the llc produced,
without any manual fixups that may be needed
to ensure sequential execution.

Diff Detail

Event Timeline

lebedev.ri created this revision.Sep 29 2021, 6:02 AM
Herald added subscribers: pengfei, hiraditya. · View Herald TranscriptSep 29 2021, 6:02 AM
lebedev.ri requested review of this revision.Sep 29 2021, 6:02 AM
lebedev.ri added a parent revision: D110705: [X86][Costmodel] Load/store i8 Stride=2 VF=8 interleaving costs.
lebedev.ri added a child revision: D110709: [X86][Costmodel] Load/store i8 Stride=2 VF=32 interleaving costs .Sep 29 2021, 6:07 AM
Harbormaster completed remote builds in B126303: Diff 375851.Sep 29 2021, 6:25 AM
lebedev.ri mentioned this in D110709: [X86][Costmodel] Load/store i8 Stride=2 VF=32 interleaving costs .Sep 29 2021, 10:50 AM
RKSimon accepted this revision.Sep 29 2021, 11:38 AM

LGTM - sorry I missed this one!

This revision is now accepted and ready to land.Sep 29 2021, 11:38 AM
lebedev.ri added a comment.Sep 29 2021, 11:40 AM
In D110708#3031212, @RKSimon wrote:

LGTM - sorry I missed this one!

No problem, thank you for the reviews!

This revision was landed with ongoing or failed builds.Sep 29 2021, 11:53 AM
Closed by commit rGbac60c55e0ff: [X86][Costmodel] Load/store i8 Stride=2 VF=16 interleaving costs (authored by lebedev.ri). · Explain Why
This revision was automatically updated to reflect the committed changes.
lebedev.ri added a commit: rGbac60c55e0ff: [X86][Costmodel] Load/store i8 Stride=2 VF=16 interleaving costs.

Revision Contents

PathSize
llvm/
lib/
Target/
X86/
2 lines
test/
Analysis/
CostModel/
X86/
2 lines
2 lines

Diff 375984

llvm/lib/Target/X86/X86TargetTransformInfo.cpp

Show First 20 Lines • Show All 5,060 Lines • ▼ Show 20 LinesInstructionCost X86TTIImpl::getInterleavedMemoryOpCostAVX2(
// Factor (stride) and VectorType=VFxiN. // Factor (stride) and VectorType=VFxiN.
// The Cost accounts only for the shuffle sequence; // The Cost accounts only for the shuffle sequence;
// The cost of the loads/stores is accounted for separately. // The cost of the loads/stores is accounted for separately.
// //
static const CostTblEntry AVX2InterleavedLoadTbl[] = { static const CostTblEntry AVX2InterleavedLoadTbl[] = {
{2, MVT::v2i8, 2}, // (load 4i8 and) deinterleave into 2 x 2i8 {2, MVT::v2i8, 2}, // (load 4i8 and) deinterleave into 2 x 2i8
{2, MVT::v4i8, 2}, // (load 8i8 and) deinterleave into 2 x 4i8 {2, MVT::v4i8, 2}, // (load 8i8 and) deinterleave into 2 x 4i8
{2, MVT::v8i8, 2}, // (load 16i8 and) deinterleave into 2 x 8i8 {2, MVT::v8i8, 2}, // (load 16i8 and) deinterleave into 2 x 8i8
{2, MVT::v16i8, 4}, // (load 32i8 and) deinterleave into 2 x 16i8
{2, MVT::v2i16, 2}, // (load 4i16 and) deinterleave into 2 x 2i16 {2, MVT::v2i16, 2}, // (load 4i16 and) deinterleave into 2 x 2i16
{2, MVT::v4i16, 2}, // (load 8i16 and) deinterleave into 2 x 4i16 {2, MVT::v4i16, 2}, // (load 8i16 and) deinterleave into 2 x 4i16
{2, MVT::v8i16, 6}, // (load 16i16 and) deinterleave into 2 x 8i16 {2, MVT::v8i16, 6}, // (load 16i16 and) deinterleave into 2 x 8i16
{2, MVT::v16i16, 9}, // (load 32i16 and) deinterleave into 2 x 16i16 {2, MVT::v16i16, 9}, // (load 32i16 and) deinterleave into 2 x 16i16
{2, MVT::v32i16, 18}, // (load 64i16 and) deinterleave into 2 x 32i16 {2, MVT::v32i16, 18}, // (load 64i16 and) deinterleave into 2 x 32i16
{2, MVT::v4i64, 6}, // (load 8i64 and) deinterleave into 2 x 4i64 {2, MVT::v4i64, 6}, // (load 8i64 and) deinterleave into 2 x 4i64
Show All 25 Lines static const CostTblEntry AVX2InterleavedLoadTbl[] = {
{8, MVT::v8i32, 40} // (load 64i32 and) deinterleave into 8 x 8i32 {8, MVT::v8i32, 40} // (load 64i32 and) deinterleave into 8 x 8i32
}; };
static const CostTblEntry AVX2InterleavedStoreTbl[] = { static const CostTblEntry AVX2InterleavedStoreTbl[] = {
{2, MVT::v2i8, 1}, // interleave 2 x 2i8 into 4i8 (and store) {2, MVT::v2i8, 1}, // interleave 2 x 2i8 into 4i8 (and store)
{2, MVT::v4i8, 1}, // interleave 2 x 4i8 into 8i8 (and store) {2, MVT::v4i8, 1}, // interleave 2 x 4i8 into 8i8 (and store)
{2, MVT::v8i8, 1}, // interleave 2 x 8i8 into 16i8 (and store) {2, MVT::v8i8, 1}, // interleave 2 x 8i8 into 16i8 (and store)
{2, MVT::v16i8, 3}, // interleave 2 x 16i8 into 32i8 (and store)
{2, MVT::v2i16, 1}, // interleave 2 x 2i16 into 4i16 (and store) {2, MVT::v2i16, 1}, // interleave 2 x 2i16 into 4i16 (and store)
{2, MVT::v4i16, 1}, // interleave 2 x 4i16 into 8i16 (and store) {2, MVT::v4i16, 1}, // interleave 2 x 4i16 into 8i16 (and store)
{2, MVT::v8i16, 3}, // interleave 2 x 8i16 into 16i16 (and store) {2, MVT::v8i16, 3}, // interleave 2 x 8i16 into 16i16 (and store)
{2, MVT::v16i16, 4}, // interleave 2 x 16i16 into 32i16 (and store) {2, MVT::v16i16, 4}, // interleave 2 x 16i16 into 32i16 (and store)
{2, MVT::v32i16, 8}, // interleave 2 x 32i16 into 64i16 (and store) {2, MVT::v32i16, 8}, // interleave 2 x 32i16 into 64i16 (and store)
{2, MVT::v4i64, 6}, // interleave 2 x 4i64 into 8i64 (and store) {2, MVT::v4i64, 6}, // interleave 2 x 4i64 into 8i64 (and store)
▲ Show 20 Lines • Show All 195 Lines • Show Last 20 Lines

llvm/test/Analysis/CostModel/X86/interleaved-load-i8-stride-2.ll

Show All 23 Lines
; AVX1: LV: Found an estimated cost of 33 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX1: LV: Found an estimated cost of 33 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX1: LV: Found an estimated cost of 81 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX1: LV: Found an estimated cost of 81 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX1: LV: Found an estimated cost of 166 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX1: LV: Found an estimated cost of 166 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1
; ;
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX2: LV: Found an estimated cost of 3 for VF 2 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 3 for VF 2 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX2: LV: Found an estimated cost of 3 for VF 4 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 3 for VF 4 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX2: LV: Found an estimated cost of 3 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 3 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX2: LV: Found an estimated cost of 81 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 5 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX2: LV: Found an estimated cost of 166 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX2: LV: Found an estimated cost of 166 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1
; ;
; AVX512: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 1 for VF 1 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX512: LV: Found an estimated cost of 3 for VF 2 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 3 for VF 2 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX512: LV: Found an estimated cost of 3 for VF 4 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 3 for VF 4 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX512: LV: Found an estimated cost of 3 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 3 for VF 8 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX512: LV: Found an estimated cost of 9 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 9 for VF 16 For instruction: %v0 = load i8, i8* %in0, align 1
; AVX512: LV: Found an estimated cost of 17 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1 ; AVX512: LV: Found an estimated cost of 17 for VF 32 For instruction: %v0 = load i8, i8* %in0, align 1
Show All 32 Lines

llvm/test/Analysis/CostModel/X86/interleaved-store-i8-stride-2.ll

Show All 23 Lines
; AVX1: LV: Found an estimated cost of 33 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX1: LV: Found an estimated cost of 33 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1
; AVX1: LV: Found an estimated cost of 67 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX1: LV: Found an estimated cost of 67 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1
; AVX1: LV: Found an estimated cost of 166 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX1: LV: Found an estimated cost of 166 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 1 for VF 1 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 2 for VF 2 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 2 for VF 2 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 2 for VF 4 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 2 for VF 4 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 2 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 2 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 67 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 4 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1
; AVX2: LV: Found an estimated cost of 166 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX2: LV: Found an estimated cost of 166 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1
; ;
; AVX512: LV: Found an estimated cost of 1 for VF 1 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 1 for VF 1 For instruction: store i8 %v1, i8* %out1, align 1
; AVX512: LV: Found an estimated cost of 4 for VF 2 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 4 for VF 2 For instruction: store i8 %v1, i8* %out1, align 1
; AVX512: LV: Found an estimated cost of 4 for VF 4 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 4 for VF 4 For instruction: store i8 %v1, i8* %out1, align 1
; AVX512: LV: Found an estimated cost of 4 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 4 for VF 8 For instruction: store i8 %v1, i8* %out1, align 1
; AVX512: LV: Found an estimated cost of 8 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 8 for VF 16 For instruction: store i8 %v1, i8* %out1, align 1
; AVX512: LV: Found an estimated cost of 20 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1 ; AVX512: LV: Found an estimated cost of 20 for VF 32 For instruction: store i8 %v1, i8* %out1, align 1
Show All 33 Lines