This is an archive of the discontinued LLVM Phabricator instance.

Differential D100908

[AArch64][GlobalISel] Legalize narrow type G_CTPOPs
ClosedPublic

Authored by paquette on Apr 20 2021, 4:16 PM.

Details

Reviewers
aemerson
jroelofs
t.p.northover
Commits
rG1312852040b3: [AArch64][GlobalISel] Legalize narrow type G_CTPOPs
Summary

Using clampScalar here because we ought to mark s128 as custom eventually.

(Right now, it will just fall back.)

With this legalization, we get the same code as SDAG: https://godbolt.org/z/TneoPKrKG

Diff Detail

Event Timeline

paquette created this revision.Apr 20 2021, 4:16 PM
Herald added subscribers: danielkiss, hiraditya, kristof.beyls, rovka. · View Herald TranscriptApr 20 2021, 4:16 PM
paquette requested review of this revision.Apr 20 2021, 4:16 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 20 2021, 4:16 PM
Harbormaster completed remote builds in B99837: Diff 339031.Apr 20 2021, 5:41 PM
jroelofs added inline comments.Apr 20 2021, 5:44 PM
llvm/test/CodeGen/AArch64/GlobalISel/legalize-ctpop.mir
125

Can we emit a COPY (or an extract?) instead of the uaddlv since all but one lane are guaranteed to be zero in the s8 case?

jroelofs added inline comments.Apr 20 2021, 6:20 PM
llvm/test/CodeGen/AArch64/GlobalISel/legalize-ctpop.mir
97

Could use the guaranteed zeroing I mentioned in the other case to emit uaddlp.4s here.

jroelofs accepted this revision.May 4 2021, 5:58 PM

LGTM. No need to gate getting GISel to parity on doing *better* than SDAG.

This revision is now accepted and ready to land.May 4 2021, 5:58 PM
Closed by commit rG1312852040b3: [AArch64][GlobalISel] Legalize narrow type G_CTPOPs (authored by paquette). · Explain WhyMay 7 2021, 2:52 PM
This revision was automatically updated to reflect the committed changes.
paquette added a commit: rG1312852040b3: [AArch64][GlobalISel] Legalize narrow type G_CTPOPs.

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
GISel/
5 lines
test/
CodeGen/
AArch64/
GlobalISel/
111 lines
4 lines

Diff 339031

llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp

Show First 20 Lines • Show All 713 Lines • ▼ Show 20 Lines getActionDefinitionsBuilder(G_ROTR)
return Q.Types[0].isScalar() && Q.Types[1].getScalarSizeInBits() < 64; return Q.Types[0].isScalar() && Q.Types[1].getScalarSizeInBits() < 64;
}) })
.lower(); .lower();
getActionDefinitionsBuilder(G_ROTL).lower(); getActionDefinitionsBuilder(G_ROTL).lower();
getActionDefinitionsBuilder({G_SBFX, G_UBFX}) getActionDefinitionsBuilder({G_SBFX, G_UBFX})
.customFor({{s32, s32}, {s64, s64}}); .customFor({{s32, s32}, {s64, s64}});
// TODO: s8, s16, s128 // TODO: Custom legalization for s128
// TODO: v2s64, v2s32, v4s32, v4s16, v8s16 // TODO: v2s64, v2s32, v4s32, v4s16, v8s16
// TODO: Use generic lowering when custom lowering is not possible. // TODO: Use generic lowering when custom lowering is not possible.
getActionDefinitionsBuilder(G_CTPOP) getActionDefinitionsBuilder(G_CTPOP)
.legalFor({{v8s8, v8s8}, {v16s8, v16s8}}) .legalFor({{v8s8, v8s8}, {v16s8, v16s8}})
.clampScalar(0, s32, s128)
.widenScalarToNextPow2(0)
.scalarSameSizeAs(1, 0)
.customFor({{s32, s32}, {s64, s64}}); .customFor({{s32, s32}, {s64, s64}});
computeTables(); computeTables();
verify(*ST.getInstrInfo()); verify(*ST.getInstrInfo());
} }
bool AArch64LegalizerInfo::legalizeCustom(LegalizerHelper &Helper, bool AArch64LegalizerInfo::legalizeCustom(LegalizerHelper &Helper,
MachineInstr &MI) const { MachineInstr &MI) const {
▲ Show 20 Lines • Show All 315 Lines • Show Last 20 Lines

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ctpop.mir

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Lines bb.0:
; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>) ; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>)
; CHECK: %ctpop:_(s64) = G_ZEXT [[INT]](s32) ; CHECK: %ctpop:_(s64) = G_ZEXT [[INT]](s32)
; CHECK: $x0 = COPY %ctpop(s64) ; CHECK: $x0 = COPY %ctpop(s64)
; CHECK: RET_ReallyLR implicit $x0 ; CHECK: RET_ReallyLR implicit $x0
%copy:_(s64) = COPY $x0 %copy:_(s64) = COPY $x0
%ctpop:_(s64) = G_CTPOP %copy(s64) %ctpop:_(s64) = G_CTPOP %copy(s64)
$x0 = COPY %ctpop(s64) $x0 = COPY %ctpop(s64)
RET_ReallyLR implicit $x0 RET_ReallyLR implicit $x0
...
---
name: widen_s16
tracksRegLiveness: true
body: |
bb.0:
liveins: $w0
; CHECK-LABEL: name: widen_s16
; CHECK: liveins: $w0
; CHECK: %copy:_(s32) = COPY $w0
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 65535
; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT %copy(s32)
; CHECK: [[AND:%[0-9]+]]:_(s64) = G_AND [[ANYEXT]], [[C]]
; CHECK: [[BITCAST:%[0-9]+]]:_(<8 x s8>) = G_BITCAST [[AND]](s64)
; CHECK: [[CTPOP:%[0-9]+]]:_(<8 x s8>) = G_CTPOP [[BITCAST]](<8 x s8>)
; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>)
jroelofsUnsubmitted
Not Done
ReplyInline Actions

Could use the guaranteed zeroing I mentioned in the other case to emit uaddlp.4s here.

jroelofs: Could use the guaranteed zeroing I mentioned in the other case to emit `uaddlp.4s` here.
; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY [[INT]](s32)
; CHECK: %ext:_(s32) = COPY [[COPY]](s32)
; CHECK: $w0 = COPY %ext(s32)
; CHECK: RET_ReallyLR implicit $w0
%copy:_(s32) = COPY $w0
%trunc:_(s16) = G_TRUNC %copy(s32)
%ctpop:_(s16) = G_CTPOP %trunc(s16)
%ext:_(s32) = G_ANYEXT %ctpop(s16)
$w0 = COPY %ext(s32)
RET_ReallyLR implicit $w0
...
---
name: widen_s8
tracksRegLiveness: true
body: |
bb.0:
liveins: $w0
; CHECK-LABEL: name: widen_s8
; CHECK: liveins: $w0
; CHECK: %copy:_(s32) = COPY $w0
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 255
; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT %copy(s32)
; CHECK: [[AND:%[0-9]+]]:_(s64) = G_AND [[ANYEXT]], [[C]]
; CHECK: [[BITCAST:%[0-9]+]]:_(<8 x s8>) = G_BITCAST [[AND]](s64)
; CHECK: [[CTPOP:%[0-9]+]]:_(<8 x s8>) = G_CTPOP [[BITCAST]](<8 x s8>)
; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>)
jroelofsUnsubmitted
Not Done
ReplyInline Actions

Can we emit a COPY (or an extract?) instead of the uaddlv since all but one lane are guaranteed to be zero in the s8 case?

jroelofs: Can we emit a `COPY` (or an extract?) instead of the `uaddlv` since all but one lane are…
; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY [[INT]](s32)
; CHECK: %ext:_(s32) = COPY [[COPY]](s32)
; CHECK: $w0 = COPY %ext(s32)
; CHECK: RET_ReallyLR implicit $w0
%copy:_(s32) = COPY $w0
%trunc:_(s8) = G_TRUNC %copy(s32)
%ctpop:_(s8) = G_CTPOP %trunc(s8)
%ext:_(s32) = G_ANYEXT %ctpop(s8)
$w0 = COPY %ext(s32)
RET_ReallyLR implicit $w0
...
---
name: widen_s3
tracksRegLiveness: true
body: |
bb.0:
liveins: $w0
; CHECK-LABEL: name: widen_s3
; CHECK: liveins: $w0
; CHECK: %copy:_(s32) = COPY $w0
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 7
; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT %copy(s32)
; CHECK: [[AND:%[0-9]+]]:_(s64) = G_AND [[ANYEXT]], [[C]]
; CHECK: [[BITCAST:%[0-9]+]]:_(<8 x s8>) = G_BITCAST [[AND]](s64)
; CHECK: [[CTPOP:%[0-9]+]]:_(<8 x s8>) = G_CTPOP [[BITCAST]](<8 x s8>)
; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>)
; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY [[INT]](s32)
; CHECK: %ext:_(s32) = COPY [[COPY]](s32)
; CHECK: $w0 = COPY %ext(s32)
; CHECK: RET_ReallyLR implicit $w0
%copy:_(s32) = COPY $w0
%trunc:_(s3) = G_TRUNC %copy(s32)
%ctpop:_(s3) = G_CTPOP %trunc(s3)
%ext:_(s32) = G_ANYEXT %ctpop(s3)
$w0 = COPY %ext(s32)
RET_ReallyLR implicit $w0
...
---
name: different_sizes
tracksRegLiveness: true
body: |
bb.0:
liveins: $w0
; CHECK-LABEL: name: different_sizes
; CHECK: liveins: $w0
; CHECK: %copy:_(s32) = COPY $w0
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 255
; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT %copy(s32)
; CHECK: [[AND:%[0-9]+]]:_(s64) = G_AND [[ANYEXT]], [[C]]
; CHECK: [[BITCAST:%[0-9]+]]:_(<8 x s8>) = G_BITCAST [[AND]](s64)
; CHECK: [[CTPOP:%[0-9]+]]:_(<8 x s8>) = G_CTPOP [[BITCAST]](<8 x s8>)
; CHECK: [[INT:%[0-9]+]]:_(s32) = G_INTRINSIC intrinsic(@llvm.aarch64.neon.uaddlv), [[CTPOP]](<8 x s8>)
; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY [[INT]](s32)
; CHECK: %ext:_(s32) = COPY [[COPY]](s32)
; CHECK: $w0 = COPY %ext(s32)
; CHECK: RET_ReallyLR implicit $w0
%copy:_(s32) = COPY $w0
%trunc:_(s8) = G_TRUNC %copy(s32)
%ctpop:_(s16) = G_CTPOP %trunc(s8)
%ext:_(s32) = G_ANYEXT %ctpop(s16)
$w0 = COPY %ext(s32)
RET_ReallyLR implicit $w0

llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir

Show First 20 Lines • Show All 545 Lines • ▼ Show 20 Lines
# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined # DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_CTLZ (opcode {{[0-9]+}}): 2 type indices, 0 imm indices # DEBUG-NEXT: G_CTLZ (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. the first uncovered type index: 2, OK # DEBUG-NEXT: .. the first uncovered type index: 2, OK
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK # DEBUG-NEXT: .. the first uncovered imm index: 0, OK
# DEBUG-NEXT: G_CTLZ_ZERO_UNDEF (opcode {{[0-9]+}}): 2 type indices, 0 imm indices # DEBUG-NEXT: G_CTLZ_ZERO_UNDEF (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected # DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: G_CTPOP (opcode {{[0-9]+}}): 2 type indices, 0 imm indices # DEBUG-NEXT: G_CTPOP (opcode {{[0-9]+}}): 2 type indices, 0 imm indices
# DEBUG-NEXT: .. the first uncovered type index: 2, OK # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK # DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
# DEBUG-NEXT: G_BSWAP (opcode {{[0-9]+}}): 1 type index, 0 imm indices # DEBUG-NEXT: G_BSWAP (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. the first uncovered type index: 1, OK # DEBUG-NEXT: .. the first uncovered type index: 1, OK
# DEBUG-NEXT: .. the first uncovered imm index: 0, OK # DEBUG-NEXT: .. the first uncovered imm index: 0, OK
# DEBUG-NEXT: G_BITREVERSE (opcode {{[0-9]+}}): 1 type index, 0 imm indices # DEBUG-NEXT: G_BITREVERSE (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined # DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined # DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
# DEBUG-NEXT: G_FCEIL (opcode {{[0-9]+}}): 1 type index, 0 imm indices # DEBUG-NEXT: G_FCEIL (opcode {{[0-9]+}}): 1 type index, 0 imm indices
# DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
▲ Show 20 Lines • Show All 137 Lines • Show Last 20 Lines