This is an archive of the discontinued LLVM Phabricator instance.

Differential D64862

AMDGPU/GlobalISel: RegBankSelect interp intrinsics
ClosedPublic

Authored by arsenm on Jul 17 2019, 6:05 AM.

Details

Reviewers
nhaehnle
tstellar
Summary

Note this assumes the future use of immediate for immarg, not the current situation which G_CONSTANT

Diff Detail

Event Timeline

arsenm created this revision.Jul 17 2019, 6:05 AM
Herald added subscribers: Petar.Avramovic, t-tye, tpr and 6 others. · View Herald TranscriptJul 17 2019, 6:05 AM
arsenm edited the summary of this revision. (Show Details)Jul 17 2019, 6:06 AM
nhaehnle added inline comments.Jul 22 2019, 2:33 AM
lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

We don't waterfall these in the non-GlobalISel path as far as I can tell? Should just use readfirstlane if necessary.

arsenm marked an inline comment as done.Jul 22 2019, 3:43 AM
arsenm added inline comments.
lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

I think this is more of a defect in SelectionDAG because handling this correctly is hard. I’m considering adding a pseudo UniformVGPR register bank for cases where readfirstlane is OK

nhaehnle added inline comments.Jul 22 2019, 4:12 AM
lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

The fact that a readfirstlane is generated instead of a waterfall loop is not a defect in SelectionDAG, at least not in general.

Frontend languages have builtins whose behavior is undefined when certain arguments are dynamically non-uniform. That is, programmers can use them in a context where the compiler cannot possibly prove that the value will be uniform, but the programmer implicitly guarantees that it will be at runtime. Emitting a readfirstlane for those when necessary is the correct behavior.

arsenm marked an inline comment as done.Jul 22 2019, 4:56 AM
arsenm added inline comments.
lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

What about the case where optimizations introduce divergence into the value?

nhaehnle added inline comments.Jul 24 2019, 1:19 AM
lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

Such transforms usually aren't optimizations, so mostly they shouldn't be done in the first place...

Okay, so admittedly there may be some cases where going the waterfall route is actually preferable. If this is really the case, then we probably need to do some more thinking about how to properly represent it, perhaps by having the base intrinsics (interp, load/store with resource descriptors) support divergence via waterfall loops, but adding some kind of "assume uniform" intrinsic whose semantics are that it passes through a value similarly to readfirstlane, except that the behavior is actually undefined if the input value is divergent.

Something like that seems like a reasonable long-term direction, but I'd say that goes beyond the scope of this change :)

arsenm updated this revision to Diff 235450.Dec 27 2019, 2:51 PM

Use readfirstlane

Herald added a project: Restricted Project. · View Herald TranscriptDec 27 2019, 2:51 PM
Herald added a subscriber: hiraditya. · View Herald Transcript
arsenm added a comment.Jan 13 2020, 5:37 AM

ping

arsenm marked an inline comment as done.Jan 20 2020, 4:35 AM

ping

lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp
1332

These could also assert uniformity by using readfirstlane themselves

Herald added a subscriber: kerbowa. · View Herald TranscriptJan 20 2020, 4:35 AM
nhaehnle accepted this revision.Jan 22 2020, 4:46 AM

LGTM

This revision is now accepted and ready to land.Jan 22 2020, 4:46 AM
arsenm closed this revision.Jan 22 2020, 6:02 AM

b94d3b9b77a0ee2e55a38133d69a458158ef4073

Revision Contents

Diff 210301

lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

Show First 20 Lines • Show All 1,315 Lines • ▼ Show 20 Lines case Intrinsic::amdgcn_writelane: {
assert(empty(OpdMapper.getVRegs(2))); assert(empty(OpdMapper.getVRegs(2)));
assert(empty(OpdMapper.getVRegs(3))); assert(empty(OpdMapper.getVRegs(3)));
substituteSimpleCopyRegs(OpdMapper, 4); // VGPR input val substituteSimpleCopyRegs(OpdMapper, 4); // VGPR input val
constrainOpWithReadfirstlane(MI, MRI, 2); // Source value constrainOpWithReadfirstlane(MI, MRI, 2); // Source value
constrainOpWithReadfirstlane(MI, MRI, 3); // Index constrainOpWithReadfirstlane(MI, MRI, 3); // Index
return; return;
} }
case Intrinsic::amdgcn_interp_p1:
case Intrinsic::amdgcn_interp_p2:
case Intrinsic::amdgcn_interp_mov:
case Intrinsic::amdgcn_interp_p1_f16:
case Intrinsic::amdgcn_interp_p2_f16: {
applyDefaultMapping(OpdMapper);
// Waterfall loop for m0 value, which is always the last operand.
executeInWaterfallLoop(MI, MRI, { MI.getNumOperands() - 1 });
nhaehnleUnsubmitted
Not Done
ReplyInline Actions

We don't waterfall these in the non-GlobalISel path as far as I can tell? Should just use readfirstlane if necessary.

nhaehnle: We don't waterfall these in the non-GlobalISel path as far as I can tell? Should just use…
arsenmAuthorUnsubmitted
Done
ReplyInline Actions

I think this is more of a defect in SelectionDAG because handling this correctly is hard. I’m considering adding a pseudo UniformVGPR register bank for cases where readfirstlane is OK

arsenm: I think this is more of a defect in SelectionDAG because handling this correctly is hard. I’m…
nhaehnleUnsubmitted
Not Done
ReplyInline Actions

The fact that a readfirstlane is generated instead of a waterfall loop is not a defect in SelectionDAG, at least not in general.

Frontend languages have builtins whose behavior is undefined when certain arguments are dynamically non-uniform. That is, programmers can use them in a context where the compiler cannot possibly prove that the value will be uniform, but the programmer implicitly guarantees that it will be at runtime. Emitting a readfirstlane for those when necessary is the correct behavior.

nhaehnle: The fact that a readfirstlane is generated instead of a waterfall loop is not a defect in…
arsenmAuthorUnsubmitted
Done
ReplyInline Actions

What about the case where optimizations introduce divergence into the value?

arsenm: What about the case where optimizations introduce divergence into the value?
nhaehnleUnsubmitted
Not Done
ReplyInline Actions

Such transforms usually aren't optimizations, so mostly they shouldn't be done in the first place...

Okay, so admittedly there may be some cases where going the waterfall route is actually preferable. If this is really the case, then we probably need to do some more thinking about how to properly represent it, perhaps by having the base intrinsics (interp, load/store with resource descriptors) support divergence via waterfall loops, but adding some kind of "assume uniform" intrinsic whose semantics are that it passes through a value similarly to readfirstlane, except that the behavior is actually undefined if the input value is divergent.

Something like that seems like a reasonable long-term direction, but I'd say that goes beyond the scope of this change :)

nhaehnle: Such transforms usually aren't optimizations, so mostly they shouldn't be done in the first…
arsenmAuthorUnsubmitted
Done
ReplyInline Actions

These could also assert uniformity by using readfirstlane themselves

arsenm: These could also assert uniformity by using readfirstlane themselves
return;
}
default: default:
break; break;
} }
break; break;
} }
case AMDGPU::G_INTRINSIC_W_SIDE_EFFECTS: { case AMDGPU::G_INTRINSIC_W_SIDE_EFFECTS: {
switch (MI.getOperand(MI.getNumExplicitDefs()).getIntrinsicID()) { switch (MI.getOperand(MI.getNumExplicitDefs()).getIntrinsicID()) {
case Intrinsic::amdgcn_buffer_load: { case Intrinsic::amdgcn_buffer_load: {
▲ Show 20 Lines • Show All 163 Lines • ▼ Show 20 Lines
} }
/// ///
/// This function must return a legal mapping, because /// This function must return a legal mapping, because
/// AMDGPURegisterBankInfo::getInstrAlternativeMappings() is not called /// AMDGPURegisterBankInfo::getInstrAlternativeMappings() is not called
/// in RegBankSelect::Mode::Fast. Any mapping that would cause a /// in RegBankSelect::Mode::Fast. Any mapping that would cause a
/// VGPR to SGPR generated is illegal. /// VGPR to SGPR generated is illegal.
/// ///
// Operands that must be SGPRs must accept potentially divergent VGPRs as
// legal. These will be dealt with in applyMappingImpl.
//
const RegisterBankInfo::InstructionMapping & const RegisterBankInfo::InstructionMapping &
AMDGPURegisterBankInfo::getInstrMapping(const MachineInstr &MI) const { AMDGPURegisterBankInfo::getInstrMapping(const MachineInstr &MI) const {
const MachineFunction &MF = *MI.getParent()->getParent(); const MachineFunction &MF = *MI.getParent()->getParent();
const MachineRegisterInfo &MRI = MF.getRegInfo(); const MachineRegisterInfo &MRI = MF.getRegInfo();
if (MI.isRegSequence()) { if (MI.isRegSequence()) {
// If any input is a VGPR, the result must be a VGPR. The default handling // If any input is a VGPR, the result must be a VGPR. The default handling
// assumes any copy between banks is legal. // assumes any copy between banks is legal.
▲ Show 20 Lines • Show All 561 Lines • ▼ Show 20 Lines case AMDGPU::G_INTRINSIC: {
} }
case Intrinsic::amdgcn_if_break: { case Intrinsic::amdgcn_if_break: {
unsigned Size = getSizeInBits(MI.getOperand(0).getReg(), MRI, *TRI); unsigned Size = getSizeInBits(MI.getOperand(0).getReg(), MRI, *TRI);
OpdsMapping[0] = AMDGPU::getValueMapping(AMDGPU::SGPRRegBankID, Size); OpdsMapping[0] = AMDGPU::getValueMapping(AMDGPU::SGPRRegBankID, Size);
OpdsMapping[2] = AMDGPU::getValueMapping(AMDGPU::VCCRegBankID, 1); OpdsMapping[2] = AMDGPU::getValueMapping(AMDGPU::VCCRegBankID, 1);
OpdsMapping[3] = AMDGPU::getValueMapping(AMDGPU::SGPRRegBankID, Size); OpdsMapping[3] = AMDGPU::getValueMapping(AMDGPU::SGPRRegBankID, Size);
break; break;
} }
case Intrinsic::amdgcn_interp_p1:
case Intrinsic::amdgcn_interp_p2:
case Intrinsic::amdgcn_interp_mov:
case Intrinsic::amdgcn_interp_p1_f16:
case Intrinsic::amdgcn_interp_p2_f16: {
const int M0Idx = MI.getNumOperands() - 1;
Register M0Reg = MI.getOperand(M0Idx).getReg();
unsigned M0Bank = getRegBankID(M0Reg, MRI, *TRI, AMDGPU::SGPRRegBankID);
unsigned DstSize = MRI.getType(MI.getOperand(0).getReg()).getSizeInBits();
OpdsMapping[0] = AMDGPU::getValueMapping(AMDGPU::VGPRRegBankID, DstSize);
for (int I = 2; I != M0Idx && MI.getOperand(I).isReg(); ++I)
OpdsMapping[I] = AMDGPU::getValueMapping(AMDGPU::VGPRRegBankID, 32);
// Must be SGPR, but we must take whatever the original bank is and fix it
// later.
OpdsMapping[M0Idx] = AMDGPU::getValueMapping(M0Bank, 32);
break;
}
} }
break; break;
} }
case AMDGPU::G_INTRINSIC_W_SIDE_EFFECTS: { case AMDGPU::G_INTRINSIC_W_SIDE_EFFECTS: {
switch (MI.getOperand(MI.getNumExplicitDefs()).getIntrinsicID()) { switch (MI.getOperand(MI.getNumExplicitDefs()).getIntrinsicID()) {
default: default:
return getInvalidInstructionMapping(); return getInvalidInstructionMapping();
case Intrinsic::amdgcn_s_getreg: case Intrinsic::amdgcn_s_getreg:
▲ Show 20 Lines • Show All 163 Lines • Show Last 20 Lines

test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.interp.mov.mir

  • This file was added.
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-fast -verify-machineinstrs -o - %s | FileCheck %s
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-greedy -verify-machineinstrs -o - %s | FileCheck %s
---
name: interp_mov_ss
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
; CHECK-LABEL: name: interp_mov_ss
; CHECK: liveins: $sgpr0, $sgpr1
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.mov), [[COPY2]](s32), 1, 1, [[COPY1]](s32)
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.mov), %0, 1, 1, %1
...
---
name: interp_mov_sv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $vgpr0
; CHECK-LABEL: name: interp_mov_sv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $sgpr0, $vgpr0
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr0
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[DEF:%[0-9]+]]:vgpr(s32) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %9, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s32) = G_PHI [[DEF]](s32), %bb.0, %2(s32), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY1]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY1]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.mov), [[COPY2]](s32), 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $vgpr0
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.mov), %0, 1, 1, %1
...

test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.interp.p1.f16.mir

  • This file was added.
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-fast -verify-machineinstrs -o - %s | FileCheck %s
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-greedy -verify-machineinstrs -o - %s | FileCheck %s
---
name: interp_p1_f16_ss
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
; CHECK-LABEL: name: interp_p1_f16_ss
; CHECK: liveins: $sgpr0, $sgpr1
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1.f16), [[COPY2]](s32), 1, 1, 1, [[COPY1]](s32)
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1.f16), %0, 1, 1, 1, %1
...
---
name: interp_p1_f16_sv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $vgpr0
; CHECK-LABEL: name: interp_p1_f16_sv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $sgpr0, $vgpr0
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr0
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[DEF:%[0-9]+]]:vgpr(s32) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %9, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s32) = G_PHI [[DEF]](s32), %bb.0, %2(s32), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY1]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY1]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1.f16), [[COPY2]](s32), 1, 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $vgpr0
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1.f16), %0, 1, 1, 1, %1
...

test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.interp.p1.mir

  • This file was added.
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-fast -verify-machineinstrs -o - %s | FileCheck %s
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-greedy -verify-machineinstrs -o - %s | FileCheck %s
---
name: interp_p1_ss
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
; CHECK-LABEL: name: interp_p1_ss
; CHECK: liveins: $sgpr0, $sgpr1
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), [[COPY2]](s32), 1, 1, [[COPY1]](s32)
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), %0, 1, 1, %1
...
---
name: interp_p1_vs
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $vgpr0, $sgpr0
; CHECK-LABEL: name: interp_p1_vs
; CHECK: liveins: $vgpr0, $sgpr0
; CHECK: [[COPY:%[0-9]+]]:vgpr(s32) = COPY $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), [[COPY]](s32), 1, 1, [[COPY1]](s32)
%0:_(s32) = COPY $vgpr0
%1:_(s32) = COPY $sgpr0
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), %0, 1, 1, %1
...
---
name: interp_p1_sv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $vgpr0
; CHECK-LABEL: name: interp_p1_sv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $sgpr0, $vgpr0
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr0
; CHECK: [[COPY2:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[DEF:%[0-9]+]]:vgpr(s32) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %9, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s32) = G_PHI [[DEF]](s32), %bb.0, %2(s32), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY1]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY1]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), [[COPY2]](s32), 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $vgpr0
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), %0, 1, 1, %1
...
---
name: interp_p1_vv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $vgpr0, $vgpr1
; CHECK-LABEL: name: interp_p1_vv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $vgpr0, $vgpr1
; CHECK: [[COPY:%[0-9]+]]:vgpr(s32) = COPY $vgpr0
; CHECK: [[COPY1:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr1
; CHECK: [[DEF:%[0-9]+]]:vgpr(s32) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %8, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s32) = G_PHI [[DEF]](s32), %bb.0, %2(s32), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY1]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY1]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), [[COPY]](s32), 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $vgpr0
%1:_(s32) = COPY $vgpr1
%2:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p1), %0, 1, 1, %1
...

test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.interp.p2.f16.mir

  • This file was added.
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect %s -verify-machineinstrs -o - -regbankselect-fast | FileCheck %s
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect %s -verify-machineinstrs -o - -regbankselect-greedy | FileCheck %s
---
name: interp_p2_f16_sss
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1, $sgpr2
; CHECK-LABEL: name: interp_p2_f16_sss
; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:sgpr(s32) = COPY $sgpr2
; CHECK: [[COPY3:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[COPY4:%[0-9]+]]:vgpr(s32) = COPY [[COPY1]](s32)
; CHECK: [[INT:%[0-9]+]]:vgpr(s16) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2.f16), [[COPY3]](s32), [[COPY4]](s32), 1, 1, 1, [[COPY2]](s32)
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = COPY $sgpr2
%3:_(s16) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2.f16), %0, %1, 1, 1, 1, %2
...
---
name: interp_p2_f16_ssv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1, $vgpr0
; CHECK-LABEL: name: interp_p2_f16_ssv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $sgpr0, $sgpr1, $vgpr0
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr0
; CHECK: [[COPY3:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[COPY4:%[0-9]+]]:vgpr(s32) = COPY [[COPY1]](s32)
; CHECK: [[DEF:%[0-9]+]]:vgpr(s16) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %11, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s16) = G_PHI [[DEF]](s16), %bb.0, %3(s16), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY2]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY2]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s16) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2.f16), [[COPY3]](s32), [[COPY4]](s32), 1, 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = COPY $vgpr0
%3:_(s16) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2.f16), %0, %1, 1, 1, 1, %2
...

test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.interp.p2.mir

  • This file was added.
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-fast -verify-machineinstrs -o - %s | FileCheck %s
# RUN: llc -march=amdgcn -mcpu=fiji -run-pass=regbankselect -regbankselect-greedy -verify-machineinstrs -o - %s | FileCheck %s
---
name: interp_p2_sss
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1, $sgpr2
; CHECK-LABEL: name: interp_p2_sss
; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:sgpr(s32) = COPY $sgpr2
; CHECK: [[COPY3:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[COPY4:%[0-9]+]]:vgpr(s32) = COPY [[COPY1]](s32)
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2), [[COPY3]](s32), [[COPY4]](s32), 1, 1, [[COPY2]](s32)
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = COPY $sgpr2
%3:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2), %0, %1, 1, 1, %2
...
---
name: interp_p2_ssv
legalized: true
tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1, $vgpr0
; CHECK-LABEL: name: interp_p2_ssv
; CHECK: successors: %bb.1(0x80000000)
; CHECK: liveins: $sgpr0, $sgpr1, $vgpr0
; CHECK: [[COPY:%[0-9]+]]:sgpr(s32) = COPY $sgpr0
; CHECK: [[COPY1:%[0-9]+]]:sgpr(s32) = COPY $sgpr1
; CHECK: [[COPY2:%[0-9]+]]:vgpr_32(s32) = COPY $vgpr0
; CHECK: [[COPY3:%[0-9]+]]:vgpr(s32) = COPY [[COPY]](s32)
; CHECK: [[COPY4:%[0-9]+]]:vgpr(s32) = COPY [[COPY1]](s32)
; CHECK: [[DEF:%[0-9]+]]:vgpr(s32) = G_IMPLICIT_DEF
; CHECK: [[DEF1:%[0-9]+]]:sreg_64_xexec = IMPLICIT_DEF
; CHECK: [[S_MOV_B64_term:%[0-9]+]]:sreg_64_xexec = S_MOV_B64_term $exec
; CHECK: .1:
; CHECK: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK: [[PHI:%[0-9]+]]:sreg_64 = PHI [[DEF1]], %bb.0, %11, %bb.1
; CHECK: [[PHI1:%[0-9]+]]:vgpr(s32) = G_PHI [[DEF]](s32), %bb.0, %3(s32), %bb.1
; CHECK: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0(s32) = V_READFIRSTLANE_B32 [[COPY2]](s32), implicit $exec
; CHECK: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64 = V_CMP_EQ_U32_e64 [[V_READFIRSTLANE_B32_]](s32), [[COPY2]](s32), implicit $exec
; CHECK: [[INT:%[0-9]+]]:vgpr(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2), [[COPY3]](s32), [[COPY4]](s32), 1, 1, [[V_READFIRSTLANE_B32_]](s32)
; CHECK: [[S_AND_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_AND_SAVEEXEC_B64 killed [[V_CMP_EQ_U32_e64_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK: $exec = S_XOR_B64_term $exec, [[S_AND_SAVEEXEC_B64_]], implicit-def $scc
; CHECK: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; CHECK: .2:
; CHECK: successors: %bb.3(0x80000000)
; CHECK: $exec = S_MOV_B64_term [[S_MOV_B64_term]]
; CHECK: .3:
%0:_(s32) = COPY $sgpr0
%1:_(s32) = COPY $sgpr1
%2:_(s32) = COPY $vgpr0
%3:_(s32) = G_INTRINSIC intrinsic(@llvm.amdgcn.interp.p2), %0, %1, 1, 1, %2
...