This is an archive of the discontinued LLVM Phabricator instance.

Differential D107508

[AArch64][GlobalISel] Legalize wide vector G_PHIs
ClosedPublic

Authored by paquette on Aug 4 2021, 3:32 PM.

Details

Reviewers
aemerson
jroelofs
Commits
rGca2e0536527f: [AArch64][GlobalISel] Legalize wide vector G_PHIs
Summary

Clamp the max number of elements when legalizing G_PHI. This allows us to legalize some common fallbacks like 4 x s64.

Here's an example: https://godbolt.org/z/6YocsEYTd

Had to add -global-isel-abort=0 to legalize-phi.mir to account for the G_EXTRACT_VECTOR_ELT from the 32 x s8 G_PHI.

Diff Detail

Event Timeline

paquette created this revision.Aug 4 2021, 3:32 PM
Herald added subscribers: danielkiss, hiraditya, kristof.beyls, rovka. · View Herald TranscriptAug 4 2021, 3:32 PM
paquette requested review of this revision.Aug 4 2021, 3:32 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 4 2021, 3:32 PM
paquette added a parent revision: D107505: [AArch64][GlobalISel] Overhaul G_EXTRACT legalization.Aug 4 2021, 3:33 PM
jroelofs added inline comments.Aug 4 2021, 3:36 PM
llvm/test/CodeGen/AArch64/GlobalISel/legalize-phi.mir
2

Why is the fallback check being turned off?

2

nvm, just saw the note.

Harbormaster completed remote builds in B118024: Diff 364273.Aug 4 2021, 3:54 PM
aemerson accepted this revision.Aug 4 2021, 4:28 PM
This revision is now accepted and ready to land.Aug 4 2021, 4:28 PM
This revision was landed with ongoing or failed builds.Aug 4 2021, 4:49 PM
Closed by commit rGca2e0536527f: [AArch64][GlobalISel] Legalize wide vector G_PHIs (authored by paquette). · Explain Why
This revision was automatically updated to reflect the committed changes.
paquette added a commit: rGca2e0536527f: [AArch64][GlobalISel] Legalize wide vector G_PHIs.

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
GISel/
11 lines
test/
CodeGen/
AArch64/
GlobalISel/
226 lines

Diff 364291

llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp

Show First 20 Lines • Show All 89 Lines • ▼ Show 20 Lines getActionDefinitionsBuilder({G_IMPLICIT_DEF, G_FREEZE})
}, },
[=](const LegalityQuery &Query) { [=](const LegalityQuery &Query) {
LLT EltTy = Query.Types[0].getElementType(); LLT EltTy = Query.Types[0].getElementType();
if (EltTy == s64) if (EltTy == s64)
return std::make_pair(0, LLT::fixed_vector(2, 64)); return std::make_pair(0, LLT::fixed_vector(2, 64));
return std::make_pair(0, EltTy); return std::make_pair(0, EltTy);
}); });
getActionDefinitionsBuilder(G_PHI).legalFor({p0, s16, s32, s64}) getActionDefinitionsBuilder(G_PHI)
.legalFor({p0, s16, s32, s64})
.legalFor(PackedVectorAllTypeList) .legalFor(PackedVectorAllTypeList)
.widenScalarToNextPow2(0) .widenScalarToNextPow2(0)
.clampScalar(0, s16, s64); .clampScalar(0, s16, s64)
// Maximum: sN * k = 128
.clampMaxNumElements(0, s8, 16)
.clampMaxNumElements(0, s16, 8)
.clampMaxNumElements(0, s32, 4)
.clampMaxNumElements(0, s64, 2)
.clampMaxNumElements(0, p0, 2);
getActionDefinitionsBuilder(G_BSWAP) getActionDefinitionsBuilder(G_BSWAP)
.legalFor({s32, s64, v4s32, v2s32, v2s64}) .legalFor({s32, s64, v4s32, v2s32, v2s64})
.clampScalar(0, s32, s64) .clampScalar(0, s32, s64)
.widenScalarToNextPow2(0) .widenScalarToNextPow2(0)
.customIf(typeIs(0, v2s16)); // custom lower as G_REV32 + G_LSHR .customIf(typeIs(0, v2s16)); // custom lower as G_REV32 + G_LSHR
getActionDefinitionsBuilder({G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR}) getActionDefinitionsBuilder({G_ADD, G_SUB, G_MUL, G_AND, G_OR, G_XOR})
▲ Show 20 Lines • Show All 1,206 Lines • Show Last 20 Lines

llvm/test/CodeGen/AArch64/GlobalISel/legalize-phi.mir

# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -O0 -mtriple=aarch64-unknown-unknown -verify-machineinstrs -global-isel-abort=1 -run-pass=legalizer %s -o - | FileCheck %s # RUN: llc -O0 -mtriple=aarch64-unknown-unknown -verify-machineinstrs -global-isel-abort=0 -run-pass=legalizer %s -o - | FileCheck %s
jroelofsUnsubmitted
Not Done
ReplyInline Actions

Why is the fallback check being turned off?

jroelofs: Why is the fallback check being turned off?
jroelofsUnsubmitted
Not Done
ReplyInline Actions

nvm, just saw the note.

jroelofs: nvm, just saw the note.
--- ---
name: legalize_phi name: legalize_phi
alignment: 4 alignment: 4
exposesReturnsTwice: false exposesReturnsTwice: false
legalized: false legalized: false
regBankSelected: false regBankSelected: false
selected: false selected: false
tracksRegLiveness: true tracksRegLiveness: true
▲ Show 20 Lines • Show All 649 Lines • ▼ Show 20 Linesbody: |
bb.2: bb.2:
successors: %bb.3(0x80000000) successors: %bb.3(0x80000000)
%imp_2:_(s88) = G_IMPLICIT_DEF %imp_2:_(s88) = G_IMPLICIT_DEF
bb.3: bb.3:
%phi:_(s88) = G_PHI %imp_1:_(s88), %bb.1, %imp_2:_(s88), %bb.2 %phi:_(s88) = G_PHI %imp_1:_(s88), %bb.1, %imp_2:_(s88), %bb.2
%trunc:_(s64) = G_TRUNC %phi %trunc:_(s64) = G_TRUNC %phi
$x0 = COPY %trunc $x0 = COPY %trunc
RET_ReallyLR implicit $x0 RET_ReallyLR implicit $x0
...
---
name: v4s64
alignment: 4
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: v4s64
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x50000000), %bb.2(0x30000000)
; CHECK: liveins: $x0, $x1
; CHECK: %ptr1:_(p0) = COPY $x1
; CHECK: %ptr2:_(p0) = COPY $x0
; CHECK: %cond:_(s1) = G_IMPLICIT_DEF
; CHECK: [[LOAD:%[0-9]+]]:_(<2 x s64>) = G_LOAD %ptr1(p0) :: (load (<2 x s64>), align 32)
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 16
; CHECK: [[PTR_ADD:%[0-9]+]]:_(p0) = G_PTR_ADD %ptr1, [[C]](s64)
; CHECK: [[LOAD1:%[0-9]+]]:_(<2 x s64>) = G_LOAD [[PTR_ADD]](p0) :: (load (<2 x s64>) from unknown-address + 16)
; CHECK: G_BRCOND %cond(s1), %bb.2
; CHECK: G_BR %bb.1
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: [[LOAD2:%[0-9]+]]:_(<2 x s64>) = G_LOAD %ptr2(p0) :: (load (<2 x s64>), align 32)
; CHECK: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 16
; CHECK: [[PTR_ADD1:%[0-9]+]]:_(p0) = G_PTR_ADD %ptr2, [[C1]](s64)
; CHECK: [[LOAD3:%[0-9]+]]:_(<2 x s64>) = G_LOAD [[PTR_ADD1]](p0) :: (load (<2 x s64>) from unknown-address + 16)
; CHECK: bb.2:
; CHECK: [[PHI:%[0-9]+]]:_(<2 x s64>) = G_PHI [[LOAD2]](<2 x s64>), %bb.1, [[LOAD]](<2 x s64>), %bb.0
; CHECK: [[PHI1:%[0-9]+]]:_(<2 x s64>) = G_PHI [[LOAD3]](<2 x s64>), %bb.1, [[LOAD1]](<2 x s64>), %bb.0
; CHECK: $q0 = COPY [[PHI]](<2 x s64>)
; CHECK: $q1 = COPY [[PHI1]](<2 x s64>)
; CHECK: RET_ReallyLR implicit $q0, implicit $q1
bb.0:
successors: %bb.1(0x50000000), %bb.2(0x30000000)
liveins: $x0, $x1
%ptr1:_(p0) = COPY $x1
%ptr2:_(p0) = COPY $x0
%cond:_(s1) = G_IMPLICIT_DEF
%val_1:_(<4 x s64>) = G_LOAD %ptr1(p0) :: (load (<4 x s64>))
G_BRCOND %cond(s1), %bb.2
G_BR %bb.1
bb.1:
%val_2:_(<4 x s64>) = G_LOAD %ptr2(p0) :: (load (<4 x s64>))
bb.2:
%phi:_(<4 x s64>) = G_PHI %val_2(<4 x s64>), %bb.1, %val_1(<4 x s64>), %bb.0
%unmerge_1:_(<2 x s64>), %unmerge_2:_(<2 x s64>) = G_UNMERGE_VALUES %phi(<4 x s64>)
$q0 = COPY %unmerge_1(<2 x s64>)
$q1 = COPY %unmerge_2(<2 x s64>)
RET_ReallyLR implicit $q0, implicit $q1
...
---
name: v8s32
alignment: 4
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: v8s32
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x50000000), %bb.2(0x30000000)
; CHECK: liveins: $x0, $x1
; CHECK: %cond:_(s1) = G_IMPLICIT_DEF
; CHECK: [[DEF:%[0-9]+]]:_(s32) = G_IMPLICIT_DEF
; CHECK: [[BUILD_VECTOR:%[0-9]+]]:_(<4 x s32>) = G_BUILD_VECTOR [[DEF]](s32), [[DEF]](s32), [[DEF]](s32), [[DEF]](s32)
; CHECK: [[BUILD_VECTOR1:%[0-9]+]]:_(<4 x s32>) = G_BUILD_VECTOR [[DEF]](s32), [[DEF]](s32), [[DEF]](s32), [[DEF]](s32)
; CHECK: G_BRCOND %cond(s1), %bb.2
; CHECK: G_BR %bb.1
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: [[DEF1:%[0-9]+]]:_(s32) = G_IMPLICIT_DEF
; CHECK: [[BUILD_VECTOR2:%[0-9]+]]:_(<4 x s32>) = G_BUILD_VECTOR [[DEF1]](s32), [[DEF1]](s32), [[DEF1]](s32), [[DEF1]](s32)
; CHECK: [[BUILD_VECTOR3:%[0-9]+]]:_(<4 x s32>) = G_BUILD_VECTOR [[DEF1]](s32), [[DEF1]](s32), [[DEF1]](s32), [[DEF1]](s32)
; CHECK: bb.2:
; CHECK: [[PHI:%[0-9]+]]:_(<4 x s32>) = G_PHI [[BUILD_VECTOR2]](<4 x s32>), %bb.1, [[BUILD_VECTOR]](<4 x s32>), %bb.0
; CHECK: [[PHI1:%[0-9]+]]:_(<4 x s32>) = G_PHI [[BUILD_VECTOR3]](<4 x s32>), %bb.1, [[BUILD_VECTOR1]](<4 x s32>), %bb.0
; CHECK: %one:_(s32) = G_CONSTANT i32 1
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 1
; CHECK: %extract:_(s32) = G_EXTRACT_VECTOR_ELT [[PHI]](<4 x s32>), [[C]](s64)
; CHECK: $w0 = COPY %extract(s32)
; CHECK: RET_ReallyLR implicit $w0
bb.0:
successors: %bb.1(0x50000000), %bb.2(0x30000000)
liveins: $x0, $x1
%cond:_(s1) = G_IMPLICIT_DEF
%val_1:_(<8 x s32>) = G_IMPLICIT_DEF
G_BRCOND %cond(s1), %bb.2
G_BR %bb.1
bb.1:
%val_2:_(<8 x s32>) = G_IMPLICIT_DEF
bb.2:
%phi:_(<8 x s32>) = G_PHI %val_2(<8 x s32>), %bb.1, %val_1(<8 x s32>), %bb.0
%one:_(s32) = G_CONSTANT i32 1
%extract:_(s32) = G_EXTRACT_VECTOR_ELT %phi(<8 x s32>), %one(s32)
$w0 = COPY %extract
RET_ReallyLR implicit $w0
...
---
name: v16s16
alignment: 4
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: v16s16
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x50000000), %bb.2(0x30000000)
; CHECK: liveins: $x0, $x1
; CHECK: %cond:_(s1) = G_IMPLICIT_DEF
; CHECK: [[DEF:%[0-9]+]]:_(s16) = G_IMPLICIT_DEF
; CHECK: [[BUILD_VECTOR:%[0-9]+]]:_(<8 x s16>) = G_BUILD_VECTOR [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16)
; CHECK: [[BUILD_VECTOR1:%[0-9]+]]:_(<8 x s16>) = G_BUILD_VECTOR [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16), [[DEF]](s16)
; CHECK: G_BRCOND %cond(s1), %bb.2
; CHECK: G_BR %bb.1
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: [[DEF1:%[0-9]+]]:_(s16) = G_IMPLICIT_DEF
; CHECK: [[BUILD_VECTOR2:%[0-9]+]]:_(<8 x s16>) = G_BUILD_VECTOR [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16)
; CHECK: [[BUILD_VECTOR3:%[0-9]+]]:_(<8 x s16>) = G_BUILD_VECTOR [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16), [[DEF1]](s16)
; CHECK: bb.2:
; CHECK: [[PHI:%[0-9]+]]:_(<8 x s16>) = G_PHI [[BUILD_VECTOR2]](<8 x s16>), %bb.1, [[BUILD_VECTOR]](<8 x s16>), %bb.0
; CHECK: [[PHI1:%[0-9]+]]:_(<8 x s16>) = G_PHI [[BUILD_VECTOR3]](<8 x s16>), %bb.1, [[BUILD_VECTOR1]](<8 x s16>), %bb.0
; CHECK: %one:_(s16) = G_CONSTANT i16 1
; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 1
; CHECK: %extract:_(s16) = G_EXTRACT_VECTOR_ELT [[PHI]](<8 x s16>), [[C]](s64)
; CHECK: $h0 = COPY %extract(s16)
; CHECK: RET_ReallyLR implicit $h0
bb.0:
successors: %bb.1(0x50000000), %bb.2(0x30000000)
liveins: $x0, $x1
%cond:_(s1) = G_IMPLICIT_DEF
%val_1:_(<16 x s16>) = G_IMPLICIT_DEF
G_BRCOND %cond(s1), %bb.2
G_BR %bb.1
bb.1:
%val_2:_(<16 x s16>) = G_IMPLICIT_DEF
bb.2:
%phi:_(<16 x s16>) = G_PHI %val_2(<16 x s16>), %bb.1, %val_1(<16 x s16>), %bb.0
%one:_(s16) = G_CONSTANT i16 1
%extract:_(s16) = G_EXTRACT_VECTOR_ELT %phi(<16 x s16>), %one(s16)
$h0 = COPY %extract
RET_ReallyLR implicit $h0
...
---
name: v32s8
alignment: 4
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: v32s8
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x50000000), %bb.2(0x30000000)
; CHECK: liveins: $x0, $x1
; CHECK: %cond:_(s1) = G_IMPLICIT_DEF
; CHECK: %val_1:_(<32 x s8>) = G_IMPLICIT_DEF
; CHECK: G_BRCOND %cond(s1), %bb.2
; CHECK: G_BR %bb.1
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: %val_2:_(<32 x s8>) = G_IMPLICIT_DEF
; CHECK: bb.2:
; CHECK: %phi:_(<32 x s8>) = G_PHI %val_2(<32 x s8>), %bb.1, %val_1(<32 x s8>), %bb.0
; CHECK: %one:_(s8) = G_CONSTANT i8 1
; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT %one(s8)
; CHECK: %extract:_(s8) = G_EXTRACT_VECTOR_ELT %phi(<32 x s8>), [[SEXT]](s64)
; CHECK: $b0 = COPY %extract(s8)
; CHECK: RET_ReallyLR implicit $b0
bb.0:
successors: %bb.1(0x50000000), %bb.2(0x30000000)
liveins: $x0, $x1
%cond:_(s1) = G_IMPLICIT_DEF
%val_1:_(<32 x s8>) = G_IMPLICIT_DEF
G_BRCOND %cond(s1), %bb.2
G_BR %bb.1
bb.1:
%val_2:_(<32 x s8>) = G_IMPLICIT_DEF
bb.2:
%phi:_(<32 x s8>) = G_PHI %val_2(<32 x s8>), %bb.1, %val_1(<32 x s8>), %bb.0
%one:_(s8) = G_CONSTANT i8 1
%extract:_(s8) = G_EXTRACT_VECTOR_ELT %phi(<32 x s8>), %one(s8)
$b0 = COPY %extract
RET_ReallyLR implicit $b0
...
---
name: v4p0
alignment: 4
tracksRegLiveness: true
body: |
; CHECK-LABEL: name: v4p0
; CHECK: bb.0:
; CHECK: successors: %bb.1(0x50000000), %bb.2(0x30000000)
; CHECK: liveins: $x0, $x1
; CHECK: %ptr1:_(p0) = COPY $x1
; CHECK: %ptr2:_(p0) = COPY $x0
; CHECK: %cond:_(s1) = G_IMPLICIT_DEF
; CHECK: %val_1:_(<4 x p0>) = G_LOAD %ptr1(p0) :: (load (<4 x p0>))
; CHECK: [[UV:%[0-9]+]]:_(<2 x p0>), [[UV1:%[0-9]+]]:_(<2 x p0>) = G_UNMERGE_VALUES %val_1(<4 x p0>)
; CHECK: G_BRCOND %cond(s1), %bb.2
; CHECK: G_BR %bb.1
; CHECK: bb.1:
; CHECK: successors: %bb.2(0x80000000)
; CHECK: [[LOAD:%[0-9]+]]:_(<4 x s64>) = G_LOAD %ptr2(p0) :: (load (<4 x s64>))
; CHECK: %val_2:_(<4 x p0>) = G_BITCAST [[LOAD]](<4 x s64>)
; CHECK: [[UV2:%[0-9]+]]:_(<2 x p0>), [[UV3:%[0-9]+]]:_(<2 x p0>) = G_UNMERGE_VALUES %val_2(<4 x p0>)
; CHECK: bb.2:
; CHECK: [[PHI:%[0-9]+]]:_(<2 x p0>) = G_PHI [[UV2]](<2 x p0>), %bb.1, [[UV]](<2 x p0>), %bb.0
; CHECK: [[PHI1:%[0-9]+]]:_(<2 x p0>) = G_PHI [[UV3]](<2 x p0>), %bb.1, [[UV1]](<2 x p0>), %bb.0
; CHECK: %phi:_(<4 x p0>) = G_CONCAT_VECTORS [[PHI]](<2 x p0>), [[PHI1]](<2 x p0>)
; CHECK: %unmerge_1:_(<2 x p0>), %unmerge_2:_(<2 x p0>) = G_UNMERGE_VALUES %phi(<4 x p0>)
; CHECK: $q0 = COPY %unmerge_1(<2 x p0>)
; CHECK: $q1 = COPY %unmerge_2(<2 x p0>)
; CHECK: RET_ReallyLR implicit $q0, implicit $q1
bb.0:
successors: %bb.1(0x50000000), %bb.2(0x30000000)
liveins: $x0, $x1
%ptr1:_(p0) = COPY $x1
%ptr2:_(p0) = COPY $x0
%cond:_(s1) = G_IMPLICIT_DEF
%val_1:_(<4 x p0>) = G_LOAD %ptr1(p0) :: (load (<4 x p0>))
G_BRCOND %cond(s1), %bb.2
G_BR %bb.1
bb.1:
%val_2:_(<4 x p0>) = G_LOAD %ptr2(p0) :: (load (<4 x p0>))
bb.2:
%phi:_(<4 x p0>) = G_PHI %val_2(<4 x p0>), %bb.1, %val_1(<4 x p0>), %bb.0
%unmerge_1:_(<2 x p0>), %unmerge_2:_(<2 x p0>) = G_UNMERGE_VALUES %phi(<4 x p0>)
$q0 = COPY %unmerge_1(<2 x p0>)
$q1 = COPY %unmerge_2(<2 x p0>)
RET_ReallyLR implicit $q0, implicit $q1