This is an archive of the discontinued LLVM Phabricator instance.

Differential D137783

[AMDGPU][MC] Support TFE modifiers in MUBUF loads and stores.
ClosedPublic

Authored by kosarev on Nov 10 2022, 5:33 AM.

Details

Reviewers
dp
foad
arsenm
Joe_Nash
Commits
rG1b560e6ab7fc: [AMDGPU][MC] Support TFE modifiers in MUBUF loads and stores.

Diff Detail

Event Timeline

kosarev created this revision.Nov 10 2022, 5:33 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 10 2022, 5:33 AM
Herald added subscribers: wenlei, kerbowa, arphaman and 8 others. · View Herald Transcript
kosarev requested review of this revision.Nov 10 2022, 5:33 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 10 2022, 5:33 AM
Herald added subscribers: llvm-commits, wdng. · View Herald Transcript
kosarev added a comment.Nov 10 2022, 5:34 AM

The most of the changes are for tests where the presence of the TFE operands doesn't make much sense and the code changes seem compact enough to not bother with making non-existing instructions possible and then slashing them with custom checks as we do for MIMGs. Also agrees with the existing practice of having the LDS and Offset/Index/Both variants as separate instructions.

For some reason SP3 additionally accepts TFE forms with vN VData operands where v[N:N+1] is expected, so for example buffer_load_dword v1, v0, s[4:7], s1 glc tfe and buffer_load_dword v[1:2], v0, s[4:7], s1 glc tfe are both accepted and produce same opcodes. It should be possible to support that with a separate change, if needed.

Apart from that, https://reviews.llvm.org/D19584 mentions that use of off operands agrees with SP3. I wasn't able to find any signs in SP3 behaviour confirming this.

dp added a comment.Nov 10 2022, 6:19 AM

AFAIK, tfe should be supported for loads and atomics only. I think that there is no sense to use tfe with stores.
SP3 is very forgiving and does not check many limitations.

dp added a comment.Nov 10 2022, 6:30 AM

For some reason SP3 additionally accepts TFE forms with vN VData operands where v[N:N+1] is expected, so for example buffer_load_dword v1, v0, s[4:7], s1 glc tfe and buffer_load_dword v[1:2], v0, s[4:7], s1 glc tfe are both accepted and produce same opcodes. It should be possible to support that with a separate change, if needed.

I think we do not have to support this SP3 feature.

Apart from that, https://reviews.llvm.org/D19584 mentions that use of off operands agrees with SP3. I wasn't able to find any signs in SP3 behaviour confirming this.

Unfortunately, SP3 is not consistent in accepting 'off' for unused operands. It does support off for FLAT operands, but not for MUBUF.

Harbormaster completed remote builds in B197068: Diff 474524.Nov 10 2022, 6:40 AM
kosarev added a comment.Nov 10 2022, 7:36 AM

AFAIK, tfe should be supported for loads and atomics only. I think that there is no sense to use tfe with stores.

That was my guess as well. But isn't it right that we still want to be able to disassemble TFE stores, even if functionally useless?

dp added a comment.Nov 10 2022, 7:47 AM

That was my guess as well. But isn't it right that we still want to be able to disassemble TFE stores, even if functionally useless?

You are right, that would be useful. But I suggest correcting the parser to trigger an error if a store is used with tfe.

kosarev added a comment.Nov 10 2022, 9:07 AM

You are right, that would be useful. But I suggest correcting the parser to trigger an error if a store is used with tfe.

Do you mind if we do it with a separate follow-up patch? We already have some TFE stores in tests and whatever we do with them here might be masked by the amount of other test changes.

arsenm accepted this revision.Nov 10 2022, 9:11 AM

LGTM. Parser error handling should be a separate patch anyway

This revision is now accepted and ready to land.Nov 10 2022, 9:11 AM
foad added inline comments.Nov 10 2022, 10:18 AM
llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp
7704

Remove IsLds now it is unused?

kosarev updated this revision to Diff 474712.Nov 11 2022, 3:16 AM

Removed unused function parameter and cleaned up related code (thanks Jay!).

dp added inline comments.Nov 11 2022, 3:24 AM
llvm/lib/Target/AMDGPU/BUFInstructions.td
2286–2290

Is there a reason to use both isTFE and noTFE? The expressions like !not(noTFE) are difficult to read.

kosarev added inline comments.Nov 11 2022, 3:37 AM
llvm/lib/Target/AMDGPU/BUFInstructions.td
2286–2290

noTFE means we don't want the TFE version, for which isTFE is true, so not the same thing. !not(noTFE) indeed catches the eye, but I have no better options and it looks rather minor. Would appreciate any suggestions, though.

Harbormaster completed remote builds in B197194: Diff 474712.Nov 11 2022, 4:03 AM
kosarev added a child revision: D137832: [AMDGPU][AsmParser] Forbid TFE modifiers for MBUF stores..Nov 11 2022, 5:40 AM
dp added inline comments.Nov 11 2022, 8:35 AM
llvm/lib/Target/AMDGPU/BUFInstructions.td
2443

noTFE is always 0 and may be removed.

2821

It looks like noTFE is always 0 and may be removed.

dp added inline comments.Nov 11 2022, 8:44 AM
llvm/lib/Target/AMDGPU/BUFInstructions.td
2286–2290

Maybe hasTFE or enableTFE (with default=1) would be more readable?

kosarev updated this revision to Diff 474801.Nov 11 2022, 10:32 AM

Removed unused TableGen class parameters and reworked noTFEs into hasTFEs.

Harbormaster completed remote builds in B197259: Diff 474801.Nov 11 2022, 11:11 AM
dp accepted this revision.Nov 11 2022, 11:53 AM

LGTM, thanks!

This revision was landed with ongoing or failed builds.Nov 14 2022, 7:36 AM
Closed by commit rG1b560e6ab7fc: [AMDGPU][MC] Support TFE modifiers in MUBUF loads and stores. (authored by kosarev). · Explain Why
This revision was automatically updated to reflect the committed changes.
kosarev added a commit: rG1b560e6ab7fc: [AMDGPU][MC] Support TFE modifiers in MUBUF loads and stores..
dp mentioned this in rGcc426402bed6: [AMDGPU][GFX7][DOC][NFC] Update assembler syntax description.Dec 13 2022, 2:51 AM
dp mentioned this in rG564d47db9eb0: [AMDGPU][GFX8][DOC][NFC] Update assembler syntax description.Dec 13 2022, 2:59 AM
dp mentioned this in rGd8ac03f15e81: [AMDGPU][GFX9][DOC][NFC] Update assembler syntax description.Dec 13 2022, 3:06 AM
dp mentioned this in rG37e6f8402621: [AMDGPU][GFX10][DOC][NFC] Update assembler syntax description.Dec 13 2022, 3:39 AM
dp mentioned this in rGc27eded938f7: [AMDGPU][GFX1030][DOC][NFC] Update assembler syntax description.Dec 13 2022, 3:51 AM

Large Diff

This large diff affects 154 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
lib/
Target/
AMDGPU/
AsmParser/
13 lines
349 lines
Disassembler/
7 lines
1 line
4 lines
4 lines
test/
CodeGen/
AMDGPU/
GlobalISel/
20 lines
68 lines
2 lines
112 lines
24 lines
2 lines
88 lines
2 lines
20 lines
12 lines
66 lines
44 lines
24 lines
66 lines
32 lines
18 lines
40 lines
16 lines
50 lines
300 lines
24 lines
14 lines
32 lines
20 lines
12 lines
20 lines
24 lines
14 lines
2 lines
4320 lines
68 lines
24 lines
42 lines
128 lines
24 lines
4 lines
2 lines
2 lines
6 lines
2 lines
8 lines
20 lines
4 lines
2 lines
4 lines
6 lines
4 lines
56 lines
2 lines
78 lines
24 lines
2 lines
4 lines
2 lines
498 lines
2 lines
4 lines
2 lines
2 lines
4 lines
8 lines
12 lines
6 lines
32 lines
2 lines
2 lines
28 lines
28 lines
13 lines
12 lines
2 lines
2 lines
6 lines
6 lines
6 lines
4 lines
60 lines
60 lines
58 lines
128 lines
94 lines
128 lines
40 lines
1206 lines
32 lines
10 lines
12 lines
48 lines
8 lines
1344 lines
88 lines
1232 lines
4 lines
4 lines
4 lines
4 lines
16 lines
2 lines
16 lines
2 lines
136 lines
2 lines
2 lines
40 lines
8 lines
10 lines
4 lines
8 lines
4 lines
8 lines
8 lines
104 lines
12 lines
324 lines
36 lines
4 lines
12 lines
8 lines
24 lines
36 lines
160 lines
60 lines
4 lines
20 lines
8 lines
148 lines
28 lines
30 lines
20 lines
8 lines
4 lines
2 lines
536 lines
52 lines
16 lines
2 lines
14 lines
MIR/
AMDGPU/
2 lines
4 lines
4 lines
4 lines
MC/
AMDGPU/
6 lines
6 lines
3 lines
4 lines
3 lines
12 lines
240 lines

Diff 475147

llvm/lib/Target/AMDGPU/AsmParser/AMDGPUAsmParser.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 1,359 Lines • ▼ Show 20 Linesprivate:
bool isRegister(); bool isRegister();
bool isRegister(const AsmToken &Token, const AsmToken &NextToken) const; bool isRegister(const AsmToken &Token, const AsmToken &NextToken) const;
Optional<StringRef> getGprCountSymbolName(RegisterKind RegKind); Optional<StringRef> getGprCountSymbolName(RegisterKind RegKind);
void initializeGprCountSymbol(RegisterKind RegKind); void initializeGprCountSymbol(RegisterKind RegKind);
bool updateGprCountSymbols(RegisterKind RegKind, unsigned DwordRegIndex, bool updateGprCountSymbols(RegisterKind RegKind, unsigned DwordRegIndex,
unsigned RegWidth); unsigned RegWidth);
void cvtMubufImpl(MCInst &Inst, const OperandVector &Operands, void cvtMubufImpl(MCInst &Inst, const OperandVector &Operands,
bool IsAtomic, bool IsLds = false); bool IsAtomic);
void cvtDSImpl(MCInst &Inst, const OperandVector &Operands, void cvtDSImpl(MCInst &Inst, const OperandVector &Operands,
bool IsGdsHardcoded); bool IsGdsHardcoded);
public: public:
enum AMDGPUMatchResultTy { enum AMDGPUMatchResultTy {
Match_PreferE32 = FIRST_TARGET_MATCH_RESULT_TY Match_PreferE32 = FIRST_TARGET_MATCH_RESULT_TY
}; };
enum OperandMode { enum OperandMode {
▲ Show 20 Lines • Show All 378 Lines • ▼ Show 20 Linespublic:
bool parseSwizzleSwap(int64_t &Imm); bool parseSwizzleSwap(int64_t &Imm);
bool parseSwizzleReverse(int64_t &Imm); bool parseSwizzleReverse(int64_t &Imm);
OperandMatchResultTy parseGPRIdxMode(OperandVector &Operands); OperandMatchResultTy parseGPRIdxMode(OperandVector &Operands);
int64_t parseGPRIdxMacro(); int64_t parseGPRIdxMacro();
void cvtMubuf(MCInst &Inst, const OperandVector &Operands) { cvtMubufImpl(Inst, Operands, false); } void cvtMubuf(MCInst &Inst, const OperandVector &Operands) { cvtMubufImpl(Inst, Operands, false); }
void cvtMubufAtomic(MCInst &Inst, const OperandVector &Operands) { cvtMubufImpl(Inst, Operands, true); } void cvtMubufAtomic(MCInst &Inst, const OperandVector &Operands) { cvtMubufImpl(Inst, Operands, true); }
void cvtMubufLds(MCInst &Inst, const OperandVector &Operands) { cvtMubufImpl(Inst, Operands, false, true); }
void cvtMtbuf(MCInst &Inst, const OperandVector &Operands); void cvtMtbuf(MCInst &Inst, const OperandVector &Operands);
AMDGPUOperand::Ptr defaultCPol() const; AMDGPUOperand::Ptr defaultCPol() const;
AMDGPUOperand::Ptr defaultSMRDOffset8() const; AMDGPUOperand::Ptr defaultSMRDOffset8() const;
AMDGPUOperand::Ptr defaultSMEMOffset() const; AMDGPUOperand::Ptr defaultSMEMOffset() const;
AMDGPUOperand::Ptr defaultSMRDLiteralOffset() const; AMDGPUOperand::Ptr defaultSMRDLiteralOffset() const;
AMDGPUOperand::Ptr defaultFlatOffset() const; AMDGPUOperand::Ptr defaultFlatOffset() const;
▲ Show 20 Lines • Show All 5,925 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
AMDGPUOperand::Ptr AMDGPUAsmParser::defaultCPol() const { AMDGPUOperand::Ptr AMDGPUAsmParser::defaultCPol() const {
return AMDGPUOperand::CreateImm(this, 0, SMLoc(), AMDGPUOperand::ImmTyCPol); return AMDGPUOperand::CreateImm(this, 0, SMLoc(), AMDGPUOperand::ImmTyCPol);
} }
void AMDGPUAsmParser::cvtMubufImpl(MCInst &Inst, void AMDGPUAsmParser::cvtMubufImpl(MCInst &Inst,
const OperandVector &Operands, const OperandVector &Operands,
bool IsAtomic, bool IsAtomic) {
foadUnsubmitted
Not Done
ReplyInline Actions

Remove IsLds now it is unused?

foad: Remove IsLds now it is unused?
bool IsLds) {
OptionalImmIndexMap OptionalIdx; OptionalImmIndexMap OptionalIdx;
unsigned FirstOperandIdx = 1; unsigned FirstOperandIdx = 1;
bool IsAtomicReturn = false; bool IsAtomicReturn = false;
if (IsAtomic) { if (IsAtomic) {
for (unsigned i = FirstOperandIdx, e = Operands.size(); i != e; ++i) { for (unsigned i = FirstOperandIdx, e = Operands.size(); i != e; ++i) {
AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]); AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]);
if (!Op.isCPol()) if (!Op.isCPol())
Show All 40 Lines for (unsigned i = FirstOperandIdx, e = Operands.size(); i != e; ++i) {
assert(Op.isImm()); assert(Op.isImm());
// Handle optional arguments // Handle optional arguments
OptionalIdx[Op.getImmTy()] = i; OptionalIdx[Op.getImmTy()] = i;
} }
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyOffset); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyOffset);
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyCPol, 0); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyCPol, 0);
if (!IsLds) { // tfe is not legal with lds opcodes
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyTFE);
}
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySWZ); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySWZ);
} }
void AMDGPUAsmParser::cvtMtbuf(MCInst &Inst, const OperandVector &Operands) { void AMDGPUAsmParser::cvtMtbuf(MCInst &Inst, const OperandVector &Operands) {
OptionalImmIndexMap OptionalIdx; OptionalImmIndexMap OptionalIdx;
for (unsigned i = 1, e = Operands.size(); i != e; ++i) { for (unsigned i = 1, e = Operands.size(); i != e; ++i) {
AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]); AMDGPUOperand &Op = ((AMDGPUOperand &)*Operands[i]);
Show All 20 Lines for (unsigned i = 1, e = Operands.size(); i != e; ++i) {
// Handle optional arguments // Handle optional arguments
OptionalIdx[Op.getImmTy()] = i; OptionalIdx[Op.getImmTy()] = i;
} }
addOptionalImmOperand(Inst, Operands, OptionalIdx, addOptionalImmOperand(Inst, Operands, OptionalIdx,
AMDGPUOperand::ImmTyOffset); AMDGPUOperand::ImmTyOffset);
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyFORMAT); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyFORMAT);
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyCPol, 0); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyCPol, 0);
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTyTFE);
addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySWZ); addOptionalImmOperand(Inst, Operands, OptionalIdx, AMDGPUOperand::ImmTySWZ);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// mimg // mimg
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void AMDGPUAsmParser::cvtMIMG(MCInst &Inst, const OperandVector &Operands, void AMDGPUAsmParser::cvtMIMG(MCInst &Inst, const OperandVector &Operands,
▲ Show 20 Lines • Show All 1,424 Lines • ▼ Show 20 Linesunsigned AMDGPUAsmParser::validateTargetOperandClass(MCParsedAsmOperand &Op,
case MCK_gds: case MCK_gds:
return Operand.isGDS() ? Match_Success : Match_InvalidOperand; return Operand.isGDS() ? Match_Success : Match_InvalidOperand;
case MCK_lds: case MCK_lds:
return Operand.isLDS() ? Match_Success : Match_InvalidOperand; return Operand.isLDS() ? Match_Success : Match_InvalidOperand;
case MCK_idxen: case MCK_idxen:
return Operand.isIdxen() ? Match_Success : Match_InvalidOperand; return Operand.isIdxen() ? Match_Success : Match_InvalidOperand;
case MCK_offen: case MCK_offen:
return Operand.isOffen() ? Match_Success : Match_InvalidOperand; return Operand.isOffen() ? Match_Success : Match_InvalidOperand;
case MCK_tfe:
return Operand.isTFE() ? Match_Success : Match_InvalidOperand;
case MCK_SSrcB32: case MCK_SSrcB32:
// When operands have expression values, they will return true for isToken, // When operands have expression values, they will return true for isToken,
// because it is not possible to distinguish between a token and an // because it is not possible to distinguish between a token and an
// expression at parse time. MatchInstructionImpl() will always try to // expression at parse time. MatchInstructionImpl() will always try to
// match an operand as a token, when isToken returns true, and when the // match an operand as a token, when isToken returns true, and when the
// name of the expression is not a valid token, the match will fail, // name of the expression is not a valid token, the match will fail,
// so we need to handle it here. // so we need to handle it here.
return Operand.isSSrcB32() ? Match_Success : Match_InvalidOperand; return Operand.isSSrcB32() ? Match_Success : Match_InvalidOperand;
▲ Show 20 Lines • Show All 75 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/BUFInstructions.td

Show First 20 Lines • Show All 83 Lines • ▼ Show 20 Linesclass MTBUF_Pseudo <string opName, dag outs, dag ins,
bits<1> has_glc = 1; bits<1> has_glc = 1;
bits<1> has_dlc = 1; bits<1> has_dlc = 1;
bits<1> glc_value = 0; // the value for glc if no such operand bits<1> glc_value = 0; // the value for glc if no such operand
bits<1> dlc_value = 0; // the value for dlc if no such operand bits<1> dlc_value = 0; // the value for dlc if no such operand
bits<1> has_srsrc = 1; bits<1> has_srsrc = 1;
bits<1> has_soffset = 1; bits<1> has_soffset = 1;
bits<1> has_offset = 1; bits<1> has_offset = 1;
bits<1> has_slc = 1; bits<1> has_slc = 1;
bits<1> has_tfe = 1; bits<1> tfe = ?;
bits<4> elements = 0; bits<4> elements = 0;
bits<1> has_sccb = 1; bits<1> has_sccb = 1;
bits<1> sccb_value = 0; bits<1> sccb_value = 0;
} }
class MTBUF_Real <MTBUF_Pseudo ps, string real_name = ps.Mnemonic> : class MTBUF_Real <MTBUF_Pseudo ps, string real_name = ps.Mnemonic> :
InstSI <ps.OutOperandList, ps.InOperandList, real_name # ps.AsmOperands, []> { InstSI <ps.OutOperandList, ps.InOperandList, real_name # ps.AsmOperands, []> {
Show All 18 Linesclass MTBUF_Real <MTBUF_Pseudo ps, string real_name = ps.Mnemonic> :
let IsAtomicNoRet = ps.IsAtomicNoRet; let IsAtomicNoRet = ps.IsAtomicNoRet;
bits<12> offset; bits<12> offset;
bits<5> cpol; bits<5> cpol;
bits<7> format; bits<7> format;
bits<8> vaddr; bits<8> vaddr;
bits<10> vdata; bits<10> vdata;
bits<7> srsrc; bits<7> srsrc;
bits<1> tfe;
bits<8> soffset; bits<8> soffset;
bits<4> dfmt = format{3-0}; bits<4> dfmt = format{3-0};
bits<3> nfmt = format{6-4}; bits<3> nfmt = format{6-4};
// GFX90A+ only: instruction uses AccVGPR for data // GFX90A+ only: instruction uses AccVGPR for data
// Bit supersedes tfe. // Bit supersedes tfe.
bits<1> acc = !if(ps.has_vdata, vdata{9}, 0); bits<1> acc = !if(ps.has_vdata, vdata{9}, 0);
} }
class getMTBUFInsDA<list<RegisterClass> vdataList, class getMTBUFInsDA<list<RegisterClass> vdataList,
list<RegisterClass> vaddrList=[]> { list<RegisterClass> vaddrList=[]> {
RegisterClass vdataClass = !if(!empty(vdataList), ?, !head(vdataList)); RegisterClass vdataClass = !if(!empty(vdataList), ?, !head(vdataList));
RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList)); RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList));
RegisterOperand vdata_op = getLdStRegisterOperand<vdataClass>.ret; RegisterOperand vdata_op = getLdStRegisterOperand<vdataClass>.ret;
dag InsNoData = !if(!empty(vaddrList), dag InsNoData = !if(!empty(vaddrList),
(ins SReg_128:$srsrc, SCSrc_b32:$soffset, (ins SReg_128:$srsrc, SCSrc_b32:$soffset,
offset:$offset, FORMAT:$format, CPol:$cpol, TFE:$tfe, SWZ:$swz), offset:$offset, FORMAT:$format, CPol:$cpol, SWZ:$swz),
(ins vaddrClass:$vaddr, SReg_128:$srsrc, SCSrc_b32:$soffset, (ins vaddrClass:$vaddr, SReg_128:$srsrc, SCSrc_b32:$soffset,
offset:$offset, FORMAT:$format, CPol:$cpol, TFE:$tfe, SWZ:$swz) offset:$offset, FORMAT:$format, CPol:$cpol, SWZ:$swz)
); );
dag InsData = !if(!empty(vaddrList), dag InsData = !if(!empty(vaddrList),
(ins vdata_op:$vdata, SReg_128:$srsrc, (ins vdata_op:$vdata, SReg_128:$srsrc,
SCSrc_b32:$soffset, offset:$offset, FORMAT:$format, CPol:$cpol, SCSrc_b32:$soffset, offset:$offset, FORMAT:$format, CPol:$cpol,
TFE:$tfe, SWZ:$swz), SWZ:$swz),
(ins vdata_op:$vdata, vaddrClass:$vaddr, SReg_128:$srsrc, (ins vdata_op:$vdata, vaddrClass:$vaddr, SReg_128:$srsrc,
SCSrc_b32:$soffset, offset:$offset, FORMAT:$format, CPol:$cpol, SCSrc_b32:$soffset, offset:$offset, FORMAT:$format, CPol:$cpol,
TFE:$tfe, SWZ:$swz) SWZ:$swz)
); );
dag ret = !if(!empty(vdataList), InsNoData, InsData); dag ret = !if(!empty(vdataList), InsNoData, InsData);
} }
class getMTBUFIns<int addrKind, list<RegisterClass> vdataList=[]> { class getMTBUFIns<int addrKind, list<RegisterClass> vdataList=[]> {
dag ret = dag ret =
!if(!eq(addrKind, BUFAddrKind.Offset), getMTBUFInsDA<vdataList>.ret, !if(!eq(addrKind, BUFAddrKind.Offset), getMTBUFInsDA<vdataList>.ret,
!if(!eq(addrKind, BUFAddrKind.OffEn), getMTBUFInsDA<vdataList, [VGPR_32]>.ret, !if(!eq(addrKind, BUFAddrKind.OffEn), getMTBUFInsDA<vdataList, [VGPR_32]>.ret,
Show All 35 Linesclass MTBUF_Load_Pseudo <string opName,
RegisterClass vdataClass, RegisterClass vdataClass,
int elems, int elems,
list<dag> pattern=[], list<dag> pattern=[],
// Workaround bug bz30254 // Workaround bug bz30254
int addrKindCopy = addrKind> int addrKindCopy = addrKind>
: MTBUF_Pseudo<opName, : MTBUF_Pseudo<opName,
(outs getLdStRegisterOperand<vdataClass>.ret:$vdata), (outs getLdStRegisterOperand<vdataClass>.ret:$vdata),
getMTBUFIns<addrKindCopy>.ret, getMTBUFIns<addrKindCopy>.ret,
" $vdata, " # getMTBUFAsmOps<addrKindCopy>.ret # "$cpol$tfe$swz", " $vdata, " # getMTBUFAsmOps<addrKindCopy>.ret # "$cpol$swz",
pattern>, pattern>,
MTBUF_SetupAddr<addrKindCopy> { MTBUF_SetupAddr<addrKindCopy> {
let PseudoInstr = opName # "_" # getAddrName<addrKindCopy>.ret; let PseudoInstr = opName # "_" # getAddrName<addrKindCopy>.ret;
let mayLoad = 1; let mayLoad = 1;
let mayStore = 0; let mayStore = 0;
let elements = elems; let elements = elems;
} }
Show All 24 Linesclass MTBUF_Store_Pseudo <string opName,
int elems, int elems,
list<dag> pattern=[], list<dag> pattern=[],
// Workaround bug bz30254 // Workaround bug bz30254
int addrKindCopy = addrKind, int addrKindCopy = addrKind,
RegisterClass vdataClassCopy = vdataClass> RegisterClass vdataClassCopy = vdataClass>
: MTBUF_Pseudo<opName, : MTBUF_Pseudo<opName,
(outs), (outs),
getMTBUFIns<addrKindCopy, [vdataClassCopy]>.ret, getMTBUFIns<addrKindCopy, [vdataClassCopy]>.ret,
" $vdata, " # getMTBUFAsmOps<addrKindCopy>.ret # "$cpol$tfe$swz", " $vdata, " # getMTBUFAsmOps<addrKindCopy>.ret # "$cpol$swz",
pattern>, pattern>,
MTBUF_SetupAddr<addrKindCopy> { MTBUF_SetupAddr<addrKindCopy> {
let PseudoInstr = opName # "_" # getAddrName<addrKindCopy>.ret; let PseudoInstr = opName # "_" # getAddrName<addrKindCopy>.ret;
let mayLoad = 0; let mayLoad = 0;
let mayStore = 1; let mayStore = 1;
let elements = elems; let elements = elems;
} }
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Linesclass MUBUF_Pseudo <string opName, dag outs, dag ins,
bits<1> has_glc = 1; bits<1> has_glc = 1;
bits<1> has_dlc = 1; bits<1> has_dlc = 1;
bits<1> glc_value = 0; // the value for glc if no such operand bits<1> glc_value = 0; // the value for glc if no such operand
bits<1> dlc_value = 0; // the value for dlc if no such operand bits<1> dlc_value = 0; // the value for dlc if no such operand
bits<1> has_srsrc = 1; bits<1> has_srsrc = 1;
bits<1> has_soffset = 1; bits<1> has_soffset = 1;
bits<1> has_offset = 1; bits<1> has_offset = 1;
bits<1> has_slc = 1; bits<1> has_slc = 1;
bits<1> has_tfe = 1; bits<1> tfe = ?;
bits<4> elements = 0; bits<4> elements = 0;
bits<1> has_sccb = 1; bits<1> has_sccb = 1;
bits<1> sccb_value = 0; bits<1> sccb_value = 0;
bits<1> IsBufferInv = 0; bits<1> IsBufferInv = 0;
} }
class MUBUF_Real <MUBUF_Pseudo ps, string real_name = ps.Mnemonic> : class MUBUF_Real <MUBUF_Pseudo ps, string real_name = ps.Mnemonic> :
InstSI <ps.OutOperandList, ps.InOperandList, real_name # ps.AsmOperands, []> { InstSI <ps.OutOperandList, ps.InOperandList, real_name # ps.AsmOperands, []> {
Show All 21 Linesclass MUBUF_Real <MUBUF_Pseudo ps, string real_name = ps.Mnemonic> :
let VALU = ps.VALU; let VALU = ps.VALU;
let LGKM_CNT = ps.LGKM_CNT; let LGKM_CNT = ps.LGKM_CNT;
bits<12> offset; bits<12> offset;
bits<5> cpol; bits<5> cpol;
bits<8> vaddr; bits<8> vaddr;
bits<10> vdata; bits<10> vdata;
bits<7> srsrc; bits<7> srsrc;
bits<1> tfe;
bits<8> soffset; bits<8> soffset;
// GFX90A+ only: instruction uses AccVGPR for data // GFX90A+ only: instruction uses AccVGPR for data
// Bit supersedes tfe. // Bit supersedes tfe.
bits<1> acc = !if(ps.has_vdata, vdata{9}, !if(ps.lds, ?, 0)); bits<1> acc = !if(ps.has_vdata, vdata{9}, !if(ps.lds, ?, 0));
} }
Show All 17 Linesclass MUBUF_Invalidate <string opName, SDPatternOperator node = null_frag> :
let has_glc = 0; let has_glc = 0;
let has_dlc = 0; let has_dlc = 0;
let glc_value = 0; let glc_value = 0;
let dlc_value = 0; let dlc_value = 0;
let has_srsrc = 0; let has_srsrc = 0;
let has_soffset = 0; let has_soffset = 0;
let has_offset = 0; let has_offset = 0;
let has_slc = 0; let has_slc = 0;
let has_tfe = 0;
let has_sccb = 0; let has_sccb = 0;
let sccb_value = 0; let sccb_value = 0;
} }
class getLdStVDataRegisterOperand<RegisterClass RC, bit isTFE> {
RegisterOperand tfeVDataOp =
!if(!eq(RC.Size, 32), AVLdSt_64,
!if(!eq(RC.Size, 64), AVLdSt_96,
!if(!eq(RC.Size, 96), AVLdSt_128,
!if(!eq(RC.Size, 128), AVLdSt_160,
RegisterOperand<VReg_1> // Invalid register.
))));
RegisterOperand ret = !if(isTFE, tfeVDataOp, getLdStRegisterOperand<RC>.ret);
}
class getMUBUFInsDA<list<RegisterClass> vdataList, class getMUBUFInsDA<list<RegisterClass> vdataList,
list<RegisterClass> vaddrList=[], list<RegisterClass> vaddrList, bit isTFE> {
bit isLds = 0> {
RegisterClass vdataClass = !if(!empty(vdataList), ?, !head(vdataList)); RegisterClass vdataClass = !if(!empty(vdataList), ?, !head(vdataList));
RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList)); RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList));
RegisterOperand vdata_op = getLdStRegisterOperand<vdataClass>.ret; RegisterOperand vdata_op = getLdStVDataRegisterOperand<vdataClass, isTFE>.ret;
dag InsNoData = !if(!empty(vaddrList), dag InsNoData = !if(!empty(vaddrList),
(ins SReg_128:$srsrc, SCSrc_b32:$soffset, (ins SReg_128:$srsrc, SCSrc_b32:$soffset,
offset:$offset, CPol_0:$cpol), offset:$offset, CPol_0:$cpol),
(ins vaddrClass:$vaddr, SReg_128:$srsrc, SCSrc_b32:$soffset, (ins vaddrClass:$vaddr, SReg_128:$srsrc, SCSrc_b32:$soffset,
offset:$offset, CPol_0:$cpol) offset:$offset, CPol_0:$cpol)
); );
dag InsData = !if(!empty(vaddrList), dag InsData = !if(!empty(vaddrList),
(ins vdata_op:$vdata, SReg_128:$srsrc, (ins vdata_op:$vdata, SReg_128:$srsrc,
SCSrc_b32:$soffset, offset:$offset, CPol_0:$cpol), SCSrc_b32:$soffset, offset:$offset, CPol_0:$cpol),
(ins vdata_op:$vdata, vaddrClass:$vaddr, SReg_128:$srsrc, (ins vdata_op:$vdata, vaddrClass:$vaddr, SReg_128:$srsrc,
SCSrc_b32:$soffset, offset:$offset, CPol_0:$cpol) SCSrc_b32:$soffset, offset:$offset, CPol_0:$cpol)
); );
dag ret = !con( dag ret = !con(!if(!empty(vdataList), InsNoData, InsData), (ins SWZ_0:$swz));
!if(!empty(vdataList), InsNoData, InsData),
!if(isLds, (ins SWZ_0:$swz), (ins TFE_0:$tfe, SWZ_0:$swz))
);
} }
class getMUBUFElements<ValueType vt> { class getMUBUFElements<ValueType vt> {
int ret = int ret =
!if(!eq(vt, f16), 1, !if(!eq(vt, f16), 1,
!if(!eq(vt, v2f16), 2, !if(!eq(vt, v2f16), 2,
!if(!eq(vt, v3f16), 3, !if(!eq(vt, v3f16), 3,
!if(!eq(vt, v4f16), 4, !if(!eq(vt, v4f16), 4,
!if(!eq(vt.Size, 32), 1, !if(!eq(vt.Size, 32), 1,
!if(!eq(vt.Size, 64), 2, !if(!eq(vt.Size, 64), 2,
!if(!eq(vt.Size, 96), 3, !if(!eq(vt.Size, 96), 3,
!if(!eq(vt.Size, 128), 4, 0) !if(!eq(vt.Size, 128), 4, 0)
) )
) )
) )
) )
) )
) )
); );
} }
class getMUBUFIns<int addrKind, list<RegisterClass> vdataList=[], bit isLds = 0> { class getMUBUFIns<int addrKind, list<RegisterClass> vdataList, bit isTFE> {
dag ret = dag ret =
!if(!eq(addrKind, BUFAddrKind.Offset), getMUBUFInsDA<vdataList, [], isLds>.ret, !if(!eq(addrKind, BUFAddrKind.Offset), getMUBUFInsDA<vdataList, [], isTFE>.ret,
!if(!eq(addrKind, BUFAddrKind.OffEn), getMUBUFInsDA<vdataList, [VGPR_32], isLds>.ret, !if(!eq(addrKind, BUFAddrKind.OffEn), getMUBUFInsDA<vdataList, [VGPR_32], isTFE>.ret,
!if(!eq(addrKind, BUFAddrKind.IdxEn), getMUBUFInsDA<vdataList, [VGPR_32], isLds>.ret, !if(!eq(addrKind, BUFAddrKind.IdxEn), getMUBUFInsDA<vdataList, [VGPR_32], isTFE>.ret,
!if(!eq(addrKind, BUFAddrKind.BothEn), getMUBUFInsDA<vdataList, [VReg_64], isLds>.ret, !if(!eq(addrKind, BUFAddrKind.BothEn), getMUBUFInsDA<vdataList, [VReg_64], isTFE>.ret,
!if(!eq(addrKind, BUFAddrKind.Addr64), getMUBUFInsDA<vdataList, [VReg_64], isLds>.ret, !if(!eq(addrKind, BUFAddrKind.Addr64), getMUBUFInsDA<vdataList, [VReg_64], isTFE>.ret,
(ins)))))); (ins))))));
} }
class getMUBUFAsmOps<int addrKind> { class getMUBUFAsmOps<int addrKind> {
string Pfx = string Pfx =
!if(!eq(addrKind, BUFAddrKind.Offset), "off, $srsrc, $soffset", !if(!eq(addrKind, BUFAddrKind.Offset), "off, $srsrc, $soffset",
!if(!eq(addrKind, BUFAddrKind.OffEn), "$vaddr, $srsrc, $soffset offen", !if(!eq(addrKind, BUFAddrKind.OffEn), "$vaddr, $srsrc, $soffset offen",
!if(!eq(addrKind, BUFAddrKind.IdxEn), "$vaddr, $srsrc, $soffset idxen", !if(!eq(addrKind, BUFAddrKind.IdxEn), "$vaddr, $srsrc, $soffset idxen",
Show All 16 Lines
} }
class MUBUF_Load_Pseudo <string opName, class MUBUF_Load_Pseudo <string opName,
int addrKind, int addrKind,
ValueType vdata_vt, ValueType vdata_vt,
bit HasTiedDest = 0, bit HasTiedDest = 0,
bit isLds = 0, bit isLds = 0,
bit isLdsOpc = 0, bit isLdsOpc = 0,
bit isTFE = 0,
list<dag> pattern=[], list<dag> pattern=[],
// Workaround bug bz30254 // Workaround bug bz30254
int addrKindCopy = addrKind, int addrKindCopy = addrKind,
RegisterClass vdata_rc = getVregSrcForVT<vdata_vt>.ret, RegisterClass vdata_rc = getVregSrcForVT<vdata_vt>.ret,
RegisterOperand vdata_op = getLdStRegisterOperand<vdata_rc>.ret> RegisterOperand vdata_op = getLdStVDataRegisterOperand<vdata_rc, isTFE>.ret>
: MUBUF_Pseudo<opName, : MUBUF_Pseudo<opName,
!if(!or(isLds, isLdsOpc), (outs), (outs vdata_op:$vdata)), !if(!or(isLds, isLdsOpc), (outs), (outs vdata_op:$vdata)),
!con(getMUBUFIns<addrKindCopy, [], isLds>.ret, !con(getMUBUFIns<addrKindCopy, [], isTFE>.ret,
!if(HasTiedDest, (ins vdata_op:$vdata_in), (ins))), !if(HasTiedDest, (ins vdata_op:$vdata_in), (ins))),
!if(!or(isLds, isLdsOpc), " ", " $vdata, ") # getMUBUFAsmOps<addrKindCopy>.ret # "$cpol" # !if(!or(isLds, isLdsOpc), " ", " $vdata, ") # getMUBUFAsmOps<addrKindCopy>.ret # "$cpol" #
!if(isLds, " lds", "$tfe") # "$swz", !if(isLds, " lds", "") # !if(isTFE, " tfe", "") # "$swz",
pattern>, pattern>,
MUBUF_SetupAddr<addrKindCopy> { MUBUF_SetupAddr<addrKindCopy> {
let PseudoInstr = opName # !if(isLds, "_lds", "") # let PseudoInstr = opName # !if(isLds, "_lds", "") # !if(isTFE, "_tfe", "") #
"_" # getAddrName<addrKindCopy>.ret; "_" # getAddrName<addrKindCopy>.ret;
let AsmMatchConverter = !if(isLds, "cvtMubufLds", "cvtMubuf"); let AsmMatchConverter = "cvtMubuf";
let Constraints = !if(HasTiedDest, "$vdata = $vdata_in", ""); let Constraints = !if(HasTiedDest, "$vdata = $vdata_in", "");
let LGKM_CNT = isLds; let LGKM_CNT = isLds;
let has_vdata = !not(isLdsOpc); let has_vdata = !not(isLdsOpc);
let mayLoad = 1; let mayLoad = 1;
let mayStore = isLds; let mayStore = isLds;
let maybeAtomic = 1; let maybeAtomic = 1;
let Uses = !if(!or(isLds, isLdsOpc) , [EXEC, M0], [EXEC]); let Uses = !if(!or(isLds, isLdsOpc) , [EXEC, M0], [EXEC]);
let has_tfe = !not(isLds); let tfe = isTFE;
let lds = isLds; let lds = isLds;
let elements = getMUBUFElements<vdata_vt>.ret; let elements = getMUBUFElements<vdata_vt>.ret;
let VALU = isLds; let VALU = isLds;
} }
class MUBUF_Offset_Load_Pat <Instruction inst, ValueType load_vt = i32, SDPatternOperator ld = null_frag> : Pat < class MUBUF_Offset_Load_Pat <Instruction inst, ValueType load_vt = i32, SDPatternOperator ld = null_frag> : Pat <
(load_vt (ld (MUBUFOffset v4i32:$srsrc, i32:$soffset, i16:$offset))), (load_vt (ld (MUBUFOffset v4i32:$srsrc, i32:$soffset, i16:$offset))),
(load_vt (inst v4i32:$srsrc, i32:$soffset, i16:$offset)) (load_vt (inst v4i32:$srsrc, i32:$soffset, i16:$offset))
>; >;
class MUBUF_Addr64_Load_Pat <Instruction inst, class MUBUF_Addr64_Load_Pat <Instruction inst,
ValueType load_vt = i32, ValueType load_vt = i32,
SDPatternOperator ld = null_frag> : Pat < SDPatternOperator ld = null_frag> : Pat <
(load_vt (ld (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset, i16:$offset))), (load_vt (ld (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset, i16:$offset))),
(load_vt (inst i64:$vaddr, v4i32:$srsrc, i32:$soffset, i16:$offset)) (load_vt (inst i64:$vaddr, v4i32:$srsrc, i32:$soffset, i16:$offset))
>; >;
multiclass MUBUF_Pseudo_Load_Pats<string BaseInst, ValueType load_vt = i32, SDPatternOperator ld = null_frag> { multiclass MUBUF_Pseudo_Load_Pats<string BaseInst, ValueType load_vt = i32, SDPatternOperator ld = null_frag> {
def : MUBUF_Offset_Load_Pat<!cast<Instruction>(BaseInst#"_OFFSET"), load_vt, ld>; def : MUBUF_Offset_Load_Pat<!cast<Instruction>(BaseInst#"_OFFSET"), load_vt, ld>;
def : MUBUF_Addr64_Load_Pat<!cast<Instruction>(BaseInst#"_ADDR64"), load_vt, ld>; def : MUBUF_Addr64_Load_Pat<!cast<Instruction>(BaseInst#"_ADDR64"), load_vt, ld>;
} }
multiclass MUBUF_Pseudo_Loads_Helper<string opName, ValueType load_vt,
// FIXME: tfe can't be an operand because it requires a separate bit TiedDest, bit isLds, bit isTFE> {
// opcode because it needs an N+1 register class dest register.
multiclass MUBUF_Pseudo_Loads<string opName,
ValueType load_vt = i32,
bit TiedDest = 0,
bit isLds = 0> {
defvar legal_load_vt = !if(!eq(load_vt, v3f16), v4f16, load_vt); defvar legal_load_vt = !if(!eq(load_vt, v3f16), v4f16, load_vt);
def _OFFSET : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds>, def _OFFSET : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds, 0, isTFE>,
MUBUFAddr64Table<0, NAME # !if(isLds, "_LDS", "")>; MUBUFAddr64Table<0, NAME # !if(isLds, "_LDS", "")>;
def _ADDR64 : MUBUF_Load_Pseudo <opName, BUFAddrKind.Addr64, legal_load_vt, TiedDest, isLds>, def _ADDR64 : MUBUF_Load_Pseudo <opName, BUFAddrKind.Addr64, legal_load_vt, TiedDest, isLds, 0, isTFE>,
MUBUFAddr64Table<1, NAME # !if(isLds, "_LDS", "")>; MUBUFAddr64Table<1, NAME # !if(isLds, "_LDS", "")>;
def _OFFEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds>; def _OFFEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
def _IDXEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds>; def _IDXEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
def _BOTHEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds>; def _BOTHEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
let DisableWQM = 1 in { let DisableWQM = 1 in {
def _OFFSET_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds>; def _OFFSET_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds, 0, isTFE>;
def _OFFEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds>; def _OFFEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
def _IDXEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds>; def _IDXEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
def _BOTHEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds>; def _BOTHEN_exact : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds, 0, isTFE>;
} }
} }
multiclass MUBUF_Pseudo_Loads<string opName, ValueType load_vt = i32,
bit TiedDest = 0, bit isLds = 0> {
defm NAME : MUBUF_Pseudo_Loads_Helper<opName, load_vt, TiedDest, isLds, 0>;
if !not(isLds) then
defm _TFE : MUBUF_Pseudo_Loads_Helper<opName, load_vt, TiedDest, isLds, 1>;
}
multiclass MUBUF_Pseudo_Loads_Lds<string opName, ValueType load_vt = i32> { multiclass MUBUF_Pseudo_Loads_Lds<string opName, ValueType load_vt = i32> {
defm NAME : MUBUF_Pseudo_Loads<opName, load_vt>; defm NAME : MUBUF_Pseudo_Loads<opName, load_vt>;
defm _LDS : MUBUF_Pseudo_Loads<opName, load_vt, 0, 1>; defm _LDS : MUBUF_Pseudo_Loads<opName, load_vt, 0, 1>;
} }
multiclass MUBUF_Pseudo_Loads_LDSOpc<string opName, multiclass MUBUF_Pseudo_Loads_LDSOpc<string opName,
ValueType load_vt = i32, ValueType load_vt = i32,
bit TiedDest = 0, bit TiedDest = 0,
bit isLds = 0, bit isLds = 0,
bit isLdsOpc = 1> { bit isLdsOpc = 1> {
defvar legal_load_vt = !if(!eq(!cast<string>(load_vt), !cast<string>(v3f16)), v4f16, load_vt); defvar legal_load_vt = !if(!eq(!cast<string>(load_vt), !cast<string>(v3f16)), v4f16, load_vt);
def _OFFSET : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds, isLdsOpc>; def _OFFSET : MUBUF_Load_Pseudo <opName, BUFAddrKind.Offset, legal_load_vt, TiedDest, isLds, isLdsOpc>;
def _OFFEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds, isLdsOpc>; def _OFFEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, legal_load_vt, TiedDest, isLds, isLdsOpc>;
def _IDXEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds, isLdsOpc>; def _IDXEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, legal_load_vt, TiedDest, isLds, isLdsOpc>;
def _BOTHEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds, isLdsOpc>; def _BOTHEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.BothEn, legal_load_vt, TiedDest, isLds, isLdsOpc>;
} }
class MUBUF_Store_Pseudo <string opName, class MUBUF_Store_Pseudo <string opName,
int addrKind, int addrKind,
ValueType store_vt, ValueType store_vt,
bit isTFE = 0,
list<dag> pattern=[], list<dag> pattern=[],
// Workaround bug bz30254 // Workaround bug bz30254
int addrKindCopy = addrKind> int addrKindCopy = addrKind>
: MUBUF_Pseudo<opName, : MUBUF_Pseudo<opName,
(outs), (outs),
getMUBUFIns<addrKindCopy, [getVregSrcForVT<store_vt>.ret]>.ret, getMUBUFIns<addrKindCopy, [getVregSrcForVT<store_vt>.ret], isTFE>.ret,
" $vdata, " # getMUBUFAsmOps<addrKindCopy>.ret # "$cpol$tfe$swz", " $vdata, " # getMUBUFAsmOps<addrKindCopy>.ret # "$cpol" #
!if(isTFE, " tfe", "") # "$swz",
pattern>, pattern>,
MUBUF_SetupAddr<addrKindCopy> { MUBUF_SetupAddr<addrKindCopy> {
let PseudoInstr = opName # "_" # getAddrName<addrKindCopy>.ret; let PseudoInstr = opName # "_" # !if(isTFE, "_tfe", "") #
getAddrName<addrKindCopy>.ret;
let mayLoad = 0; let mayLoad = 0;
let mayStore = 1; let mayStore = 1;
let maybeAtomic = 1; let maybeAtomic = 1;
let elements = getMUBUFElements<store_vt>.ret; let elements = getMUBUFElements<store_vt>.ret;
let tfe = isTFE;
} }
multiclass MUBUF_Pseudo_Stores<string opName, multiclass MUBUF_Pseudo_Stores_Helper<string opName, ValueType store_vt,
ValueType store_vt = i32, SDPatternOperator st, bit isTFE> {
SDPatternOperator st = null_frag> {
defvar legal_store_vt = !if(!eq(store_vt, v3f16), v4f16, store_vt); defvar legal_store_vt = !if(!eq(store_vt, v3f16), v4f16, store_vt);
def _OFFSET : MUBUF_Store_Pseudo <opName, BUFAddrKind.Offset, legal_store_vt, def _OFFSET : MUBUF_Store_Pseudo <opName, BUFAddrKind.Offset, legal_store_vt, isTFE,
[(st legal_store_vt:$vdata, (MUBUFOffset v4i32:$srsrc, i32:$soffset, [(st legal_store_vt:$vdata, (MUBUFOffset v4i32:$srsrc, i32:$soffset,
i16:$offset))]>, i16:$offset))]>,
MUBUFAddr64Table<0, NAME>; MUBUFAddr64Table<0, NAME>;
def _ADDR64 : MUBUF_Store_Pseudo <opName, BUFAddrKind.Addr64, legal_store_vt, def _ADDR64 : MUBUF_Store_Pseudo <opName, BUFAddrKind.Addr64, legal_store_vt, isTFE,
[(st legal_store_vt:$vdata, (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset, [(st legal_store_vt:$vdata, (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset,
i16:$offset))]>, i16:$offset))]>,
MUBUFAddr64Table<1, NAME>; MUBUFAddr64Table<1, NAME>;
def _OFFEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, legal_store_vt>; def _OFFEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, legal_store_vt, isTFE>;
def _IDXEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, legal_store_vt>; def _IDXEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, legal_store_vt, isTFE>;
def _BOTHEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.BothEn, legal_store_vt>; def _BOTHEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.BothEn, legal_store_vt, isTFE>;
let DisableWQM = 1 in { let DisableWQM = 1 in {
def _OFFSET_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.Offset, legal_store_vt>; def _OFFSET_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.Offset, legal_store_vt, isTFE>;
def _OFFEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, legal_store_vt>; def _OFFEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, legal_store_vt, isTFE>;
def _IDXEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, legal_store_vt>; def _IDXEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, legal_store_vt, isTFE>;
def _BOTHEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.BothEn, legal_store_vt>; def _BOTHEN_exact : MUBUF_Store_Pseudo <opName, BUFAddrKind.BothEn, legal_store_vt, isTFE>;
}
} }
multiclass MUBUF_Pseudo_Stores<string opName, ValueType store_vt = i32,
SDPatternOperator st = null_frag> {
defm NAME : MUBUF_Pseudo_Stores_Helper<opName, store_vt, st, 0>;
defm _TFE : MUBUF_Pseudo_Stores_Helper<opName, store_vt, null_frag, 1>;
} }
class MUBUF_Pseudo_Store_Lds<string opName> class MUBUF_Pseudo_Store_Lds<string opName>
: MUBUF_Pseudo<opName, : MUBUF_Pseudo<opName,
(outs), (outs),
(ins SReg_128:$srsrc, SCSrc_b32:$soffset, offset:$offset, CPol:$cpol, SWZ:$swz), (ins SReg_128:$srsrc, SCSrc_b32:$soffset, offset:$offset, CPol:$cpol, SWZ:$swz),
" $srsrc, $soffset$offset lds$cpol$swz"> { " $srsrc, $soffset$offset lds$cpol$swz"> {
let LGKM_CNT = 1; let LGKM_CNT = 1;
let mayLoad = 1; let mayLoad = 1;
let mayStore = 1; let mayStore = 1;
let maybeAtomic = 1; let maybeAtomic = 1;
let has_vdata = 0; let has_vdata = 0;
let has_vaddr = 0; let has_vaddr = 0;
let has_tfe = 0;
let lds = 1; let lds = 1;
let VALU = 1; let VALU = 1;
let Uses = [EXEC, M0]; let Uses = [EXEC, M0];
let AsmMatchConverter = "cvtMubufLds"; let AsmMatchConverter = "cvtMubuf";
} }
class getMUBUFAtomicInsDA<RegisterClass vdataClass, bit vdata_in, class getMUBUFAtomicInsDA<RegisterClass vdataClass, bit vdata_in,
list<RegisterClass> vaddrList=[]> { list<RegisterClass> vaddrList=[]> {
RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList)); RegisterClass vaddrClass = !if(!empty(vaddrList), ?, !head(vaddrList));
RegisterOperand vdata_op = getLdStRegisterOperand<vdataClass>.ret; RegisterOperand vdata_op = getLdStRegisterOperand<vdataClass>.ret;
dag ret = !if(vdata_in, dag ret = !if(vdata_in,
!if(!empty(vaddrList), !if(!empty(vaddrList),
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Lines : MUBUF_Pseudo<opName, outs, ins, asmOps, pattern>,
MUBUF_SetupAddr<addrKindCopy> { MUBUF_SetupAddr<addrKindCopy> {
let mayStore = 1; let mayStore = 1;
let mayLoad = 1; let mayLoad = 1;
let hasPostISelHook = 1; let hasPostISelHook = 1;
let hasSideEffects = 1; let hasSideEffects = 1;
let DisableWQM = 1; let DisableWQM = 1;
let has_glc = 0; let has_glc = 0;
let has_dlc = 0; let has_dlc = 0;
let has_tfe = 0;
let has_sccb = 1; let has_sccb = 1;
let maybeAtomic = 1; let maybeAtomic = 1;
let AsmMatchConverter = "cvtMubufAtomic"; let AsmMatchConverter = "cvtMubufAtomic";
} }
class MUBUF_AtomicNoRet_Pseudo<string opName, int addrKind, class MUBUF_AtomicNoRet_Pseudo<string opName, int addrKind,
RegisterClass vdataClass, RegisterClass vdataClass,
list<dag> pattern=[], list<dag> pattern=[],
▲ Show 20 Lines • Show All 533 Lines • ▼ Show 20 Lines
multiclass MUBUF_LoadIntrinsicPat<SDPatternOperator name, ValueType vt, multiclass MUBUF_LoadIntrinsicPat<SDPatternOperator name, ValueType vt,
string opcode, ValueType memoryVt = vt> { string opcode, ValueType memoryVt = vt> {
defvar st = !if(!eq(memoryVt, vt), name, mubuf_intrinsic_load<name, memoryVt>); defvar st = !if(!eq(memoryVt, vt), name, mubuf_intrinsic_load<name, memoryVt>);
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset,
timm:$auxiliary, 0)), timm:$auxiliary, 0)),
(!cast<MUBUF_Pseudo>(opcode # _OFFSET) SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MUBUF_Pseudo>(opcode # _OFFSET) SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset,
timm:$auxiliary, 0)), timm:$auxiliary, 0)),
(!cast<MUBUF_Pseudo>(opcode # _OFFEN) VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MUBUF_Pseudo>(opcode # _OFFEN) VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset,
timm:$auxiliary, timm)), timm:$auxiliary, timm)),
(!cast<MUBUF_Pseudo>(opcode # _IDXEN) VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MUBUF_Pseudo>(opcode # _IDXEN) VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset,
timm:$auxiliary, timm)), timm:$auxiliary, timm)),
(!cast<MUBUF_Pseudo>(opcode # _BOTHEN) (!cast<MUBUF_Pseudo>(opcode # _BOTHEN)
(REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1), (REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1),
SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
} }
defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, f32, "BUFFER_LOAD_FORMAT_X">; defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, f32, "BUFFER_LOAD_FORMAT_X">;
defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, i32, "BUFFER_LOAD_FORMAT_X">; defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, i32, "BUFFER_LOAD_FORMAT_X">;
defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v2f32, "BUFFER_LOAD_FORMAT_XY">; defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v2f32, "BUFFER_LOAD_FORMAT_XY">;
defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v2i32, "BUFFER_LOAD_FORMAT_XY">; defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v2i32, "BUFFER_LOAD_FORMAT_XY">;
defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v3f32, "BUFFER_LOAD_FORMAT_XYZ">; defm : MUBUF_LoadIntrinsicPat<SIbuffer_load_format, v3f32, "BUFFER_LOAD_FORMAT_XYZ">;
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
multiclass MUBUF_StoreIntrinsicPat<SDPatternOperator name, ValueType vt, multiclass MUBUF_StoreIntrinsicPat<SDPatternOperator name, ValueType vt,
string opcode, ValueType memoryVt = vt> { string opcode, ValueType memoryVt = vt> {
defvar st = !if(!eq(memoryVt, vt), name, mubuf_intrinsic_store<name, memoryVt>); defvar st = !if(!eq(memoryVt, vt), name, mubuf_intrinsic_store<name, memoryVt>);
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset,
timm:$auxiliary, 0), timm:$auxiliary, 0),
(!cast<MUBUF_Pseudo>(opcode # _OFFSET_exact) getVregSrcForVT<vt>.ret:$vdata, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MUBUF_Pseudo>(opcode # _OFFSET_exact) getVregSrcForVT<vt>.ret:$vdata, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset,
timm:$auxiliary, 0), timm:$auxiliary, 0),
(!cast<MUBUF_Pseudo>(opcode # _OFFEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (!cast<MUBUF_Pseudo>(opcode # _OFFEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset,
(as_i16timm $offset), (extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (as_i16timm $offset), (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset,
timm:$auxiliary, timm), timm:$auxiliary, timm),
(!cast<MUBUF_Pseudo>(opcode # _IDXEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (!cast<MUBUF_Pseudo>(opcode # _IDXEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset,
(as_i16timm $offset), (extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (as_i16timm $offset), (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset,
timm:$auxiliary, timm), timm:$auxiliary, timm),
(!cast<MUBUF_Pseudo>(opcode # _BOTHEN_exact) (!cast<MUBUF_Pseudo>(opcode # _BOTHEN_exact)
getVregSrcForVT<vt>.ret:$vdata, getVregSrcForVT<vt>.ret:$vdata,
(REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1), (REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1),
SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (extract_cpol $auxiliary), SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (extract_cpol $auxiliary),
0, (extract_swz $auxiliary)) (extract_swz $auxiliary))
>; >;
} }
defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, f32, "BUFFER_STORE_FORMAT_X">; defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, f32, "BUFFER_STORE_FORMAT_X">;
defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, i32, "BUFFER_STORE_FORMAT_X">; defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, i32, "BUFFER_STORE_FORMAT_X">;
defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v2f32, "BUFFER_STORE_FORMAT_XY">; defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v2f32, "BUFFER_STORE_FORMAT_XY">;
defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v2i32, "BUFFER_STORE_FORMAT_XY">; defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v2i32, "BUFFER_STORE_FORMAT_XY">;
defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v3f32, "BUFFER_STORE_FORMAT_XYZ">; defm : MUBUF_StoreIntrinsicPat<SIbuffer_store_format, v3f32, "BUFFER_STORE_FORMAT_XYZ">;
▲ Show 20 Lines • Show All 393 Lines • ▼ Show 20 Lines
} // End OtherPredicates = [Has16BitInsts] } // End OtherPredicates = [Has16BitInsts]
multiclass MUBUFScratchLoadPat <MUBUF_Pseudo InstrOffen, multiclass MUBUFScratchLoadPat <MUBUF_Pseudo InstrOffen,
MUBUF_Pseudo InstrOffset, MUBUF_Pseudo InstrOffset,
ValueType vt, PatFrag ld> { ValueType vt, PatFrag ld> {
def : GCNPat < def : GCNPat <
(vt (ld (MUBUFScratchOffen v4i32:$srsrc, i32:$vaddr, (vt (ld (MUBUFScratchOffen v4i32:$srsrc, i32:$vaddr,
i32:$soffset, u16imm:$offset))), i32:$soffset, u16imm:$offset))),
(InstrOffen $vaddr, $srsrc, $soffset, $offset, 0, 0, 0) (InstrOffen $vaddr, $srsrc, $soffset, $offset, 0, 0)
>; >;
def : GCNPat < def : GCNPat <
(vt (ld (MUBUFScratchOffset v4i32:$srsrc, i32:$soffset, u16imm:$offset))), (vt (ld (MUBUFScratchOffset v4i32:$srsrc, i32:$soffset, u16imm:$offset))),
(InstrOffset $srsrc, $soffset, $offset, 0, 0, 0) (InstrOffset $srsrc, $soffset, $offset, 0, 0)
>; >;
} }
// XXX - Is it possible to have a complex pattern in a PatFrag? // XXX - Is it possible to have a complex pattern in a PatFrag?
multiclass MUBUFScratchLoadPat_D16 <MUBUF_Pseudo InstrOffen, multiclass MUBUFScratchLoadPat_D16 <MUBUF_Pseudo InstrOffen,
MUBUF_Pseudo InstrOffset, MUBUF_Pseudo InstrOffset,
ValueType vt, PatFrag ld_frag> { ValueType vt, PatFrag ld_frag> {
def : GCNPat < def : GCNPat <
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines
multiclass MUBUFScratchStorePat <MUBUF_Pseudo InstrOffen, multiclass MUBUFScratchStorePat <MUBUF_Pseudo InstrOffen,
MUBUF_Pseudo InstrOffset, MUBUF_Pseudo InstrOffset,
ValueType vt, PatFrag st, ValueType vt, PatFrag st,
RegisterClass rc = VGPR_32> { RegisterClass rc = VGPR_32> {
def : GCNPat < def : GCNPat <
(st vt:$value, (MUBUFScratchOffen v4i32:$srsrc, i32:$vaddr, (st vt:$value, (MUBUFScratchOffen v4i32:$srsrc, i32:$vaddr,
i32:$soffset, u16imm:$offset)), i32:$soffset, u16imm:$offset)),
(InstrOffen rc:$value, $vaddr, $srsrc, $soffset, $offset, 0, 0, 0) (InstrOffen rc:$value, $vaddr, $srsrc, $soffset, $offset, 0, 0)
>; >;
def : GCNPat < def : GCNPat <
(st vt:$value, (MUBUFScratchOffset v4i32:$srsrc, i32:$soffset, (st vt:$value, (MUBUFScratchOffset v4i32:$srsrc, i32:$soffset,
u16imm:$offset)), u16imm:$offset)),
(InstrOffset rc:$value, $srsrc, $soffset, $offset, 0, 0, 0) (InstrOffset rc:$value, $srsrc, $soffset, $offset, 0, 0)
>; >;
} }
let OtherPredicates = [DisableFlatScratch] in { let OtherPredicates = [DisableFlatScratch] in {
defm : MUBUFScratchStorePat <BUFFER_STORE_BYTE_OFFEN, BUFFER_STORE_BYTE_OFFSET, i32, truncstorei8_private>; defm : MUBUFScratchStorePat <BUFFER_STORE_BYTE_OFFEN, BUFFER_STORE_BYTE_OFFSET, i32, truncstorei8_private>;
defm : MUBUFScratchStorePat <BUFFER_STORE_SHORT_OFFEN, BUFFER_STORE_SHORT_OFFSET, i32, truncstorei16_private>; defm : MUBUFScratchStorePat <BUFFER_STORE_SHORT_OFFEN, BUFFER_STORE_SHORT_OFFSET, i32, truncstorei16_private>;
defm : MUBUFScratchStorePat <BUFFER_STORE_BYTE_OFFEN, BUFFER_STORE_BYTE_OFFSET, i16, truncstorei8_private>; defm : MUBUFScratchStorePat <BUFFER_STORE_BYTE_OFFEN, BUFFER_STORE_BYTE_OFFSET, i16, truncstorei8_private>;
defm : MUBUFScratchStorePat <BUFFER_STORE_SHORT_OFFEN, BUFFER_STORE_SHORT_OFFSET, i16, store_private>; defm : MUBUFScratchStorePat <BUFFER_STORE_SHORT_OFFEN, BUFFER_STORE_SHORT_OFFSET, i16, store_private>;
Show All 29 Linesmulticlass MTBUF_LoadIntrinsicPat<SDPatternOperator name, ValueType vt,
string opcode, ValueType memoryVt = vt> { string opcode, ValueType memoryVt = vt> {
defvar st = !if(!eq(memoryVt, vt), name, mtbuf_intrinsic_load<name, memoryVt>); defvar st = !if(!eq(memoryVt, vt), name, mtbuf_intrinsic_load<name, memoryVt>);
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, 0)), timm:$format, timm:$auxiliary, 0)),
(!cast<MTBUF_Pseudo>(opcode # _OFFSET) SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MTBUF_Pseudo>(opcode # _OFFSET) SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(as_i8timm $format), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, timm)), timm:$format, timm:$auxiliary, timm)),
(!cast<MTBUF_Pseudo>(opcode # _IDXEN) VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MTBUF_Pseudo>(opcode # _IDXEN) VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(as_i8timm $format), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, 0)), timm:$format, timm:$auxiliary, 0)),
(!cast<MTBUF_Pseudo>(opcode # _OFFEN) VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (!cast<MTBUF_Pseudo>(opcode # _OFFEN) VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(as_i8timm $format), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(vt (st v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset, (vt (st v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, timm)), timm:$format, timm:$auxiliary, timm)),
(!cast<MTBUF_Pseudo>(opcode # _BOTHEN) (!cast<MTBUF_Pseudo>(opcode # _BOTHEN)
(REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1), (REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1),
SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset),
(as_i8timm $format), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
} }
defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, i32, "TBUFFER_LOAD_FORMAT_X">; defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, i32, "TBUFFER_LOAD_FORMAT_X">;
defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v2i32, "TBUFFER_LOAD_FORMAT_XY">; defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v2i32, "TBUFFER_LOAD_FORMAT_XY">;
defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v3i32, "TBUFFER_LOAD_FORMAT_XYZ">; defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v3i32, "TBUFFER_LOAD_FORMAT_XYZ">;
defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v4i32, "TBUFFER_LOAD_FORMAT_XYZW">; defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, v4i32, "TBUFFER_LOAD_FORMAT_XYZW">;
defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, f32, "TBUFFER_LOAD_FORMAT_X">; defm : MTBUF_LoadIntrinsicPat<SItbuffer_load, f32, "TBUFFER_LOAD_FORMAT_X">;
Show All 21 Linesmulticlass MTBUF_StoreIntrinsicPat<SDPatternOperator name, ValueType vt,
string opcode, ValueType memoryVt = vt> { string opcode, ValueType memoryVt = vt> {
defvar st = !if(!eq(memoryVt, vt), name, mtbuf_intrinsic_store<name, memoryVt>); defvar st = !if(!eq(memoryVt, vt), name, mtbuf_intrinsic_store<name, memoryVt>);
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, 0, 0, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, 0), timm:$format, timm:$auxiliary, 0),
(!cast<MTBUF_Pseudo>(opcode # _OFFSET_exact) getVregSrcForVT<vt>.ret:$vdata, SReg_128:$rsrc, SCSrc_b32:$soffset, (!cast<MTBUF_Pseudo>(opcode # _OFFSET_exact) getVregSrcForVT<vt>.ret:$vdata, SReg_128:$rsrc, SCSrc_b32:$soffset,
(as_i16timm $offset), (as_i8timm $format), (as_i16timm $offset), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, i32:$vindex, 0, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, timm), timm:$format, timm:$auxiliary, timm),
(!cast<MTBUF_Pseudo>(opcode # _IDXEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset, (!cast<MTBUF_Pseudo>(opcode # _IDXEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$vindex, SReg_128:$rsrc, SCSrc_b32:$soffset,
(as_i16timm $offset), (as_i8timm $format), (as_i16timm $offset), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset, (st vt:$vdata, v4i32:$rsrc, 0, i32:$voffset, i32:$soffset, timm:$offset,
timm:$format, timm:$auxiliary, 0), timm:$format, timm:$auxiliary, 0),
(!cast<MTBUF_Pseudo>(opcode # _OFFEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset, (!cast<MTBUF_Pseudo>(opcode # _OFFEN_exact) getVregSrcForVT<vt>.ret:$vdata, VGPR_32:$voffset, SReg_128:$rsrc, SCSrc_b32:$soffset,
(as_i16timm $offset), (as_i8timm $format), (as_i16timm $offset), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
def : GCNPat< def : GCNPat<
(st vt:$vdata, v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset, (st vt:$vdata, v4i32:$rsrc, i32:$vindex, i32:$voffset, i32:$soffset,
timm:$offset, timm:$format, timm:$auxiliary, timm), timm:$offset, timm:$format, timm:$auxiliary, timm),
(!cast<MTBUF_Pseudo>(opcode # _BOTHEN_exact) (!cast<MTBUF_Pseudo>(opcode # _BOTHEN_exact)
getVregSrcForVT<vt>.ret:$vdata, getVregSrcForVT<vt>.ret:$vdata,
(REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1), (REG_SEQUENCE VReg_64, VGPR_32:$vindex, sub0, VGPR_32:$voffset, sub1),
SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (as_i8timm $format), SReg_128:$rsrc, SCSrc_b32:$soffset, (as_i16timm $offset), (as_i8timm $format),
(extract_cpol $auxiliary), 0, (extract_swz $auxiliary)) (extract_cpol $auxiliary), (extract_swz $auxiliary))
>; >;
} }
defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, i32, "TBUFFER_STORE_FORMAT_X">; defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, i32, "TBUFFER_STORE_FORMAT_X">;
defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v2i32, "TBUFFER_STORE_FORMAT_XY">; defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v2i32, "TBUFFER_STORE_FORMAT_XY">;
defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v3i32, "TBUFFER_STORE_FORMAT_XYZ">; defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v3i32, "TBUFFER_STORE_FORMAT_XYZ">;
defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v4i32, "TBUFFER_STORE_FORMAT_XYZW">; defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, v4i32, "TBUFFER_STORE_FORMAT_XYZW">;
defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, f32, "TBUFFER_STORE_FORMAT_X">; defm : MTBUF_StoreIntrinsicPat<SItbuffer_store, f32, "TBUFFER_STORE_FORMAT_X">;
Show All 38 Lines
class MUBUF_Real_gfx11<bits<8> op, MUBUF_Pseudo ps, class MUBUF_Real_gfx11<bits<8> op, MUBUF_Pseudo ps,
string real_name = ps.Mnemonic> : string real_name = ps.Mnemonic> :
Base_MUBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, SIEncodingFamily.GFX11, real_name> { Base_MUBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, SIEncodingFamily.GFX11, real_name> {
let Inst{12} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{12} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{13} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value); let Inst{13} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value);
let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value); let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value);
let Inst{25-18} = op; let Inst{25-18} = op;
let Inst{53} = !if(ps.has_tfe, tfe, ?); let Inst{53} = ps.tfe;
let Inst{54} = ps.offen; let Inst{54} = ps.offen;
let Inst{55} = ps.idxen; let Inst{55} = ps.idxen;
} }
class Base_MUBUF_Real_Atomic_gfx11<bits<8> op, MUBUF_Pseudo ps, class Base_MUBUF_Real_Atomic_gfx11<bits<8> op, MUBUF_Pseudo ps,
string real_name> : string real_name> :
MUBUF_Real_gfx11<op, ps, real_name> { MUBUF_Real_gfx11<op, ps, real_name> {
let Inst{13} = cpol{CPolBit.DLC}; let Inst{13} = cpol{CPolBit.DLC};
} }
class Base_MUBUF_Real_gfx6_gfx7_gfx10<bits<7> op, MUBUF_Pseudo ps, int ef> : class Base_MUBUF_Real_gfx6_gfx7_gfx10<bits<7> op, MUBUF_Pseudo ps, int ef> :
Base_MUBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, ef> { Base_MUBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, ef> {
let Inst{12} = ps.offen; let Inst{12} = ps.offen;
let Inst{13} = ps.idxen; let Inst{13} = ps.idxen;
let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value); let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value);
let Inst{16} = ps.lds; let Inst{16} = ps.lds;
let Inst{24-18} = op; let Inst{24-18} = op;
let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
} }
class MUBUF_Real_gfx10<bits<8> op, MUBUF_Pseudo ps> : class MUBUF_Real_gfx10<bits<8> op, MUBUF_Pseudo ps> :
Base_MUBUF_Real_gfx6_gfx7_gfx10<op{6-0}, ps, SIEncodingFamily.GFX10> { Base_MUBUF_Real_gfx6_gfx7_gfx10<op{6-0}, ps, SIEncodingFamily.GFX10> {
let Inst{15} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value); let Inst{15} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value);
let Inst{25} = op{7}; let Inst{25} = op{7};
} }
Show All 14 Lines
// gfx11 instruction that accept both old and new assembler name. // gfx11 instruction that accept both old and new assembler name.
class Pre_gfx11_MUBUF_Name <string mnemonic, string real_name> : class Pre_gfx11_MUBUF_Name <string mnemonic, string real_name> :
MnemonicAlias<mnemonic, real_name>, Requires<[isGFX11Plus]>; MnemonicAlias<mnemonic, real_name>, Requires<[isGFX11Plus]>;
class MUBUF_Real_gfx11_impl<bits<8> op, string ps_name, string real_name> : class MUBUF_Real_gfx11_impl<bits<8> op, string ps_name, string real_name> :
MUBUF_Real_gfx11<op, !cast<MUBUF_Pseudo>(ps_name), real_name>; MUBUF_Real_gfx11<op, !cast<MUBUF_Pseudo>(ps_name), real_name>;
let AssemblerPredicate = isGFX11Only, DecoderNamespace = "GFX11" in let AssemblerPredicate = isGFX11Only, DecoderNamespace = "GFX11" in
multiclass MUBUF_Real_AllAddr_gfx11_Renamed_Impl<bits<8> op, string real_name> { multiclass MUBUF_Real_AllAddr_gfx11_Renamed_Impl2<bits<8> op, string real_name> {
def _BOTHEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_BOTHEN", real_name>; def _BOTHEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_BOTHEN", real_name>;
def _IDXEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_IDXEN", real_name>; def _IDXEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_IDXEN", real_name>;
def _OFFEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_OFFEN", real_name>; def _OFFEN_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_OFFEN", real_name>;
def _OFFSET_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_OFFSET", real_name>; def _OFFSET_gfx11 : MUBUF_Real_gfx11_impl<op, NAME # "_OFFSET", real_name>;
} }
multiclass MUBUF_Real_AllAddr_gfx11_Renamed_Impl<bits<8> op, string real_name,
bit hasTFE = 1> {
defm NAME : MUBUF_Real_AllAddr_gfx11_Renamed_Impl2<op, real_name>;
if hasTFE then
defm _TFE : MUBUF_Real_AllAddr_gfx11_Renamed_Impl2<op, real_name>;
}
// Non-renamed, non-atomic gfx11 mubuf instructions. // Non-renamed, non-atomic gfx11 mubuf instructions.
multiclass MUBUF_Real_AllAddr_gfx11<bits<8> op> : multiclass MUBUF_Real_AllAddr_gfx11<bits<8> op, bit hasTFE = 1> :
MUBUF_Real_AllAddr_gfx11_Renamed_Impl<op, get_MUBUF_ps<NAME>.Mnemonic>; MUBUF_Real_AllAddr_gfx11_Renamed_Impl<op, get_MUBUF_ps<NAME>.Mnemonic, hasTFE>;
multiclass MUBUF_Real_AllAddr_gfx11_Renamed<bits<8> op, string real_name> : multiclass MUBUF_Real_AllAddr_gfx11_Renamed<bits<8> op, string real_name> :
MUBUF_Real_AllAddr_gfx11_Renamed_Impl<op, real_name> { MUBUF_Real_AllAddr_gfx11_Renamed_Impl<op, real_name> {
def : Pre_gfx11_MUBUF_Name<get_MUBUF_ps<NAME>.Mnemonic, real_name>; def : Pre_gfx11_MUBUF_Name<get_MUBUF_ps<NAME>.Mnemonic, real_name>;
} }
class MUBUF_Real_Atomic_gfx11_impl<bits<8> op, string ps_name, class MUBUF_Real_Atomic_gfx11_impl<bits<8> op, string ps_name,
string real_name> : string real_name> :
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines
defm BUFFER_LOAD_FORMAT_X : MUBUF_Real_AllAddr_gfx11<0x000>; defm BUFFER_LOAD_FORMAT_X : MUBUF_Real_AllAddr_gfx11<0x000>;
defm BUFFER_LOAD_FORMAT_XY : MUBUF_Real_AllAddr_gfx11<0x001>; defm BUFFER_LOAD_FORMAT_XY : MUBUF_Real_AllAddr_gfx11<0x001>;
defm BUFFER_LOAD_FORMAT_XYZ : MUBUF_Real_AllAddr_gfx11<0x002>; defm BUFFER_LOAD_FORMAT_XYZ : MUBUF_Real_AllAddr_gfx11<0x002>;
defm BUFFER_LOAD_FORMAT_XYZW : MUBUF_Real_AllAddr_gfx11<0x003>; defm BUFFER_LOAD_FORMAT_XYZW : MUBUF_Real_AllAddr_gfx11<0x003>;
defm BUFFER_LOAD_SBYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x011, "buffer_load_i8">; defm BUFFER_LOAD_SBYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x011, "buffer_load_i8">;
defm BUFFER_LOAD_SSHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x013, "buffer_load_i16">; defm BUFFER_LOAD_SSHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x013, "buffer_load_i16">;
defm BUFFER_LOAD_UBYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x010, "buffer_load_u8">; defm BUFFER_LOAD_UBYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x010, "buffer_load_u8">;
defm BUFFER_LOAD_USHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x012, "buffer_load_u16">; defm BUFFER_LOAD_USHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x012, "buffer_load_u16">;
defm BUFFER_LOAD_LDS_B32 : MUBUF_Real_AllAddr_gfx11<0x031>; defm BUFFER_LOAD_LDS_B32 : MUBUF_Real_AllAddr_gfx11<0x031, 0>;
defm BUFFER_LOAD_LDS_FORMAT_X : MUBUF_Real_AllAddr_gfx11<0x032>; defm BUFFER_LOAD_LDS_FORMAT_X : MUBUF_Real_AllAddr_gfx11<0x032, 0>;
defm BUFFER_LOAD_LDS_I8 : MUBUF_Real_AllAddr_gfx11<0x02e>; defm BUFFER_LOAD_LDS_I8 : MUBUF_Real_AllAddr_gfx11<0x02e, 0>;
defm BUFFER_LOAD_LDS_I16 : MUBUF_Real_AllAddr_gfx11<0x030>; defm BUFFER_LOAD_LDS_I16 : MUBUF_Real_AllAddr_gfx11<0x030, 0>;
defm BUFFER_LOAD_LDS_U8 : MUBUF_Real_AllAddr_gfx11<0x02d>; defm BUFFER_LOAD_LDS_U8 : MUBUF_Real_AllAddr_gfx11<0x02d, 0>;
defm BUFFER_LOAD_LDS_U16 : MUBUF_Real_AllAddr_gfx11<0x02f>; defm BUFFER_LOAD_LDS_U16 : MUBUF_Real_AllAddr_gfx11<0x02f, 0>;
defm BUFFER_STORE_BYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x018, "buffer_store_b8">; defm BUFFER_STORE_BYTE : MUBUF_Real_AllAddr_gfx11_Renamed<0x018, "buffer_store_b8">;
defm BUFFER_STORE_SHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x019, "buffer_store_b16">; defm BUFFER_STORE_SHORT : MUBUF_Real_AllAddr_gfx11_Renamed<0x019, "buffer_store_b16">;
defm BUFFER_STORE_DWORD : MUBUF_Real_AllAddr_gfx11_Renamed<0x01A, "buffer_store_b32">; defm BUFFER_STORE_DWORD : MUBUF_Real_AllAddr_gfx11_Renamed<0x01A, "buffer_store_b32">;
defm BUFFER_STORE_DWORDX2 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01B, "buffer_store_b64">; defm BUFFER_STORE_DWORDX2 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01B, "buffer_store_b64">;
defm BUFFER_STORE_DWORDX3 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01C, "buffer_store_b96">; defm BUFFER_STORE_DWORDX3 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01C, "buffer_store_b96">;
defm BUFFER_STORE_DWORDX4 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01D, "buffer_store_b128">; defm BUFFER_STORE_DWORDX4 : MUBUF_Real_AllAddr_gfx11_Renamed<0x01D, "buffer_store_b128">;
defm BUFFER_STORE_FORMAT_D16_X : MUBUF_Real_AllAddr_gfx11_Renamed<0x00C, "buffer_store_d16_format_x">; defm BUFFER_STORE_FORMAT_D16_X : MUBUF_Real_AllAddr_gfx11_Renamed<0x00C, "buffer_store_d16_format_x">;
defm BUFFER_STORE_FORMAT_D16_XY : MUBUF_Real_AllAddr_gfx11_Renamed<0x00D, "buffer_store_d16_format_xy">; defm BUFFER_STORE_FORMAT_D16_XY : MUBUF_Real_AllAddr_gfx11_Renamed<0x00D, "buffer_store_d16_format_xy">;
Show All 39 Lines
defm BUFFER_ATOMIC_XOR : MUBUF_Real_Atomic_gfx11_Renamed<0x03E, "buffer_atomic_xor_b32">; defm BUFFER_ATOMIC_XOR : MUBUF_Real_Atomic_gfx11_Renamed<0x03E, "buffer_atomic_xor_b32">;
defm BUFFER_ATOMIC_XOR_X2 : MUBUF_Real_Atomic_gfx11_Renamed<0x04B, "buffer_atomic_xor_b64">; defm BUFFER_ATOMIC_XOR_X2 : MUBUF_Real_Atomic_gfx11_Renamed<0x04B, "buffer_atomic_xor_b64">;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// MUBUF - GFX10. // MUBUF - GFX10.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
let AssemblerPredicate = isGFX10Only, DecoderNamespace = "GFX10" in { let AssemblerPredicate = isGFX10Only, DecoderNamespace = "GFX10" in {
multiclass MUBUF_Real_AllAddr_gfx10<bits<8> op> { multiclass MUBUF_Real_AllAddr_Helper_gfx10<bits<8> op> {
def _BOTHEN_gfx10 : def _BOTHEN_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
def _IDXEN_gfx10 : def _IDXEN_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _OFFEN_gfx10 : def _OFFEN_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _OFFSET_gfx10 : def _OFFSET_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
} }
multiclass MUBUF_Real_AllAddr_Lds_gfx10<bits<8> op> { multiclass MUBUF_Real_AllAddr_gfx10<bits<8> op> {
defm NAME : MUBUF_Real_AllAddr_Helper_gfx10<op>;
defm _TFE : MUBUF_Real_AllAddr_Helper_gfx10<op>;
}
multiclass MUBUF_Real_AllAddr_Lds_gfx10<bits<8> op, bit isTFE = 0> {
dpUnsubmitted
Not Done
ReplyInline Actions

Is there a reason to use both isTFE and noTFE? The expressions like !not(noTFE) are difficult to read.

dp: Is there a reason to use both `isTFE` and `noTFE`? The expressions like `!not(noTFE)` are…
kosarevAuthorUnsubmitted
Done
ReplyInline Actions

noTFE means we don't want the TFE version, for which isTFE is true, so not the same thing. !not(noTFE) indeed catches the eye, but I have no better options and it looks rather minor. Would appreciate any suggestions, though.

kosarev: `noTFE` means we don't want the TFE version, for which `isTFE` is true, so not the same thing.
dpUnsubmitted
Not Done
ReplyInline Actions

Maybe hasTFE or enableTFE (with default=1) would be more readable?

dp: Maybe `hasTFE` or `enableTFE` (with default=1) would be more readable?
def _OFFSET_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
def _OFFEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
if !not(isTFE) then {
def _LDS_OFFSET_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>; def _LDS_OFFSET_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>;
def _LDS_OFFEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>; def _LDS_OFFEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>;
def _LDS_IDXEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>; def _LDS_IDXEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>;
def _LDS_BOTHEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>; def _LDS_BOTHEN_gfx10 : MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>;
} }
}
multiclass MUBUF_Real_Atomics_RTN_gfx10<bits<8> op> { multiclass MUBUF_Real_Atomics_RTN_gfx10<bits<8> op> {
def _BOTHEN_RTN_gfx10 : def _BOTHEN_RTN_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN_RTN")>, MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN_RTN")>,
AtomicNoRet<NAME # "_BOTHEN_gfx10", 1>; AtomicNoRet<NAME # "_BOTHEN_gfx10", 1>;
def _IDXEN_RTN_gfx10 : def _IDXEN_RTN_gfx10 :
MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN_RTN")>, MUBUF_Real_gfx10<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN_RTN")>,
AtomicNoRet<NAME # "_IDXEN_gfx10", 1>; AtomicNoRet<NAME # "_IDXEN_gfx10", 1>;
def _OFFEN_RTN_gfx10 : def _OFFEN_RTN_gfx10 :
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines
let AssemblerPredicate = isGFX7Only, DecoderNamespace = "GFX7" in { let AssemblerPredicate = isGFX7Only, DecoderNamespace = "GFX7" in {
multiclass MUBUF_Real_gfx7<bits<8> op> { multiclass MUBUF_Real_gfx7<bits<8> op> {
def _gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME)>; def _gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME)>;
} }
} // End AssemblerPredicate = isGFX7Only, DecoderNamespace = "GFX7" } // End AssemblerPredicate = isGFX7Only, DecoderNamespace = "GFX7"
let AssemblerPredicate = isGFX6GFX7, DecoderNamespace = "GFX6GFX7" in { let AssemblerPredicate = isGFX6GFX7, DecoderNamespace = "GFX6GFX7" in {
multiclass MUBUF_Real_AllAddr_gfx6_gfx7<bits<8> op> { multiclass MUBUF_Real_AllAddr_Helper_gfx6_gfx7<bits<8> op> {
def _ADDR64_gfx6_gfx7 : def _ADDR64_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>; MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>;
def _BOTHEN_gfx6_gfx7 : def _BOTHEN_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
def _IDXEN_gfx6_gfx7 : def _IDXEN_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _OFFEN_gfx6_gfx7 : def _OFFEN_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _OFFSET_gfx6_gfx7 : def _OFFSET_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
} }
multiclass MUBUF_Real_AllAddr_Lds_gfx6_gfx7<bits<8> op> { multiclass MUBUF_Real_AllAddr_gfx6_gfx7<bits<8> op> {
defm NAME : MUBUF_Real_AllAddr_Helper_gfx6_gfx7<op>;
defm _TFE : MUBUF_Real_AllAddr_Helper_gfx6_gfx7<op>;
}
multiclass MUBUF_Real_AllAddr_Lds_gfx6_gfx7<bits<8> op, bit isTFE = 0> {
def _OFFSET_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
def _ADDR64_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>; def _ADDR64_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>;
def _OFFEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
if !not(isTFE) then {
def _LDS_OFFSET_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>; def _LDS_OFFSET_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>;
def _LDS_ADDR64_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_ADDR64")>; def _LDS_ADDR64_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_ADDR64")>;
def _LDS_OFFEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>; def _LDS_OFFEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>;
def _LDS_IDXEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>; def _LDS_IDXEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>;
def _LDS_BOTHEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>; def _LDS_BOTHEN_gfx6_gfx7 : MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>;
} }
}
multiclass MUBUF_Real_Atomics_gfx6_gfx7<bits<8> op> { multiclass MUBUF_Real_Atomics_gfx6_gfx7<bits<8> op> {
def _ADDR64_gfx6_gfx7 : def _ADDR64_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>, MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>,
AtomicNoRet<NAME # "_ADDR64_gfx6_gfx7", 0>; AtomicNoRet<NAME # "_ADDR64_gfx6_gfx7", 0>;
def _BOTHEN_gfx6_gfx7 : def _BOTHEN_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>, MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>,
AtomicNoRet<NAME # "_BOTHEN_gfx6_gfx7", 0>; AtomicNoRet<NAME # "_BOTHEN_gfx6_gfx7", 0>;
def _IDXEN_gfx6_gfx7 : def _IDXEN_gfx6_gfx7 :
Show All 20 Lines def _OFFEN_RTN_gfx6_gfx7 :
AtomicNoRet<NAME # "_OFFEN_gfx6_gfx7", 1>; AtomicNoRet<NAME # "_OFFEN_gfx6_gfx7", 1>;
def _OFFSET_RTN_gfx6_gfx7 : def _OFFSET_RTN_gfx6_gfx7 :
MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET_RTN")>, MUBUF_Real_gfx6_gfx7<op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET_RTN")>,
AtomicNoRet<NAME # "_OFFSET_gfx6_gfx7", 1>; AtomicNoRet<NAME # "_OFFSET_gfx6_gfx7", 1>;
} }
} // End AssemblerPredicate = isGFX6GFX7, DecoderNamespace = "GFX6GFX7" } // End AssemblerPredicate = isGFX6GFX7, DecoderNamespace = "GFX6GFX7"
multiclass MUBUF_Real_AllAddr_gfx6_gfx7_gfx10<bits<8> op> : multiclass MUBUF_Real_AllAddr_gfx6_gfx7_gfx10<bits<8> op> :
MUBUF_Real_AllAddr_gfx6_gfx7<op>, MUBUF_Real_AllAddr_gfx10<op>; MUBUF_Real_AllAddr_gfx6_gfx7<op>, MUBUF_Real_AllAddr_gfx10<op>;
dpUnsubmitted
Not Done
ReplyInline Actions

noTFE is always 0 and may be removed.

dp: `noTFE` is always 0 and may be removed.
multiclass MUBUF_Real_AllAddr_Lds_gfx6_gfx7_gfx10<bits<8> op> : multiclass MUBUF_Real_AllAddr_Lds_Helper_gfx6_gfx7_gfx10<bits<8> op, bit isTFE = 0> :
MUBUF_Real_AllAddr_Lds_gfx6_gfx7<op>, MUBUF_Real_AllAddr_Lds_gfx10<op>; MUBUF_Real_AllAddr_Lds_gfx6_gfx7<op, isTFE>,
MUBUF_Real_AllAddr_Lds_gfx10<op, isTFE>;
multiclass MUBUF_Real_AllAddr_Lds_gfx6_gfx7_gfx10<bits<8> op> {
defm NAME : MUBUF_Real_AllAddr_Lds_Helper_gfx6_gfx7_gfx10<op>;
defm _TFE : MUBUF_Real_AllAddr_Lds_Helper_gfx6_gfx7_gfx10<op, 1>;
}
multiclass MUBUF_Real_Atomics_gfx6_gfx7_gfx10<bits<8> op> : multiclass MUBUF_Real_Atomics_gfx6_gfx7_gfx10<bits<8> op> :
MUBUF_Real_Atomics_gfx6_gfx7<op>, MUBUF_Real_Atomics_gfx10<op>; MUBUF_Real_Atomics_gfx6_gfx7<op>, MUBUF_Real_Atomics_gfx10<op>;
// FIXME-GFX6: Following instructions are available only on GFX6. // FIXME-GFX6: Following instructions are available only on GFX6.
//defm BUFFER_ATOMIC_RSUB : MUBUF_Real_Atomics_gfx6 <0x034>; //defm BUFFER_ATOMIC_RSUB : MUBUF_Real_Atomics_gfx6 <0x034>;
//defm BUFFER_ATOMIC_RSUB_X2 : MUBUF_Real_Atomics_gfx6 <0x054>; //defm BUFFER_ATOMIC_RSUB_X2 : MUBUF_Real_Atomics_gfx6 <0x054>;
▲ Show 20 Lines • Show All 78 Lines • ▼ Show 20 Lines
class Base_MTBUF_Real_gfx11<bits<4> op, MTBUF_Pseudo ps, class Base_MTBUF_Real_gfx11<bits<4> op, MTBUF_Pseudo ps,
string real_name = ps.Mnemonic> : string real_name = ps.Mnemonic> :
Base_MTBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, SIEncodingFamily.GFX11, real_name> { Base_MTBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, SIEncodingFamily.GFX11, real_name> {
let Inst{12} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{12} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{13} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value); let Inst{13} = !if(ps.has_dlc, cpol{CPolBit.DLC}, ps.dlc_value);
let Inst{18-15} = op; let Inst{18-15} = op;
let Inst{25-19} = format; let Inst{25-19} = format;
let Inst{53} = !if(ps.has_tfe, tfe, ?); let Inst{53} = ps.tfe;
let Inst{54} = ps.offen; let Inst{54} = ps.offen;
let Inst{55} = ps.idxen; let Inst{55} = ps.idxen;
} }
class Base_MTBUF_Real_gfx6_gfx7_gfx10<bits<3> op, MTBUF_Pseudo ps, int ef> : class Base_MTBUF_Real_gfx6_gfx7_gfx10<bits<3> op, MTBUF_Pseudo ps, int ef> :
Base_MTBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, ef> { Base_MTBUF_Real_gfx6_gfx7_gfx10_gfx11<ps, ef> {
let Inst{12} = ps.offen; let Inst{12} = ps.offen;
let Inst{13} = ps.idxen; let Inst{13} = ps.idxen;
let Inst{18-16} = op; let Inst{18-16} = op;
let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// MTBUF - GFX11. // MTBUF - GFX11.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
let AssemblerPredicate = isGFX11Only, DecoderNamespace = "GFX11" in let AssemblerPredicate = isGFX11Only, DecoderNamespace = "GFX11" in
multiclass MTBUF_Real_AllAddr_gfx11_Renamed_Impl<bits<4> op, string real_name> { multiclass MTBUF_Real_AllAddr_gfx11_Renamed_Impl<bits<4> op, string real_name> {
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 Linesclass MUBUF_Real_Base_vi <bits<7> op, MUBUF_Pseudo ps, int Enc,
let Inst{63-56} = !if(ps.has_soffset, soffset, ?); let Inst{63-56} = !if(ps.has_soffset, soffset, ?);
} }
class MUBUF_Real_vi <bits<7> op, MUBUF_Pseudo ps, bit has_sccb = ps.has_sccb> : class MUBUF_Real_vi <bits<7> op, MUBUF_Pseudo ps, bit has_sccb = ps.has_sccb> :
MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.VI, has_sccb> { MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.VI, has_sccb> {
let AssemblerPredicate = isGFX8GFX9NotGFX90A; let AssemblerPredicate = isGFX8GFX9NotGFX90A;
let DecoderNamespace = "GFX8"; let DecoderNamespace = "GFX8";
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
} }
class MUBUF_Real_gfx90a <bits<7> op, MUBUF_Pseudo ps, class MUBUF_Real_gfx90a <bits<7> op, MUBUF_Pseudo ps,
bit has_sccb = ps.has_sccb> : bit has_sccb = ps.has_sccb> :
MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.GFX90A, has_sccb> { MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.GFX90A, has_sccb> {
let AssemblerPredicate = isGFX90APlus; let AssemblerPredicate = isGFX90APlus;
let DecoderNamespace = "GFX90A"; let DecoderNamespace = "GFX90A";
let AsmString = ps.Mnemonic # !subst("$sccb", !if(has_sccb, "$sccb",""), let AsmString = ps.Mnemonic # !subst("$sccb", !if(has_sccb, "$sccb",""),
!subst("$tfe", "", ps.AsmOperands)); ps.AsmOperands);
let Inst{55} = acc; let Inst{55} = acc;
} }
class MUBUF_Real_gfx940 <bits<7> op, MUBUF_Pseudo ps> : class MUBUF_Real_gfx940 <bits<7> op, MUBUF_Pseudo ps> :
MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.GFX940> { MUBUF_Real_Base_vi<op, ps, SIEncodingFamily.GFX940> {
let AssemblerPredicate = isGFX940Plus; let AssemblerPredicate = isGFX940Plus;
let DecoderNamespace = "GFX9"; let DecoderNamespace = "GFX9";
let AsmString = ps.Mnemonic # !subst("$tfe", "", ps.AsmOperands); let AsmString = ps.Mnemonic # ps.AsmOperands;
let Inst{55} = acc; let Inst{55} = acc;
} }
multiclass MUBUF_Real_vi_gfx90a<bits<7> op, MUBUF_Pseudo ps> { multiclass MUBUF_Real_vi_gfx90a<bits<7> op, MUBUF_Pseudo ps, bit isTFE = 0> {
def _vi : MUBUF_Real_vi<op, ps>; def _vi : MUBUF_Real_vi<op, ps>;
if !not(isTFE) then {
foreach _ = BoolToList<!not(ps.FPAtomic)>.ret in foreach _ = BoolToList<!not(ps.FPAtomic)>.ret in
def _gfx90a : MUBUF_Real_gfx90a<op, ps>; def _gfx90a : MUBUF_Real_gfx90a<op, ps>;
}
foreach _ = BoolToList<ps.FPAtomic>.ret in { foreach _ = BoolToList<ps.FPAtomic>.ret in {
def _gfx90a : MUBUF_Real_gfx90a<op, ps, 0> { def _gfx90a : MUBUF_Real_gfx90a<op, ps, 0> {
let SubtargetPredicate = isGFX90AOnly; let SubtargetPredicate = isGFX90AOnly;
let AssemblerPredicate = isGFX90AOnly; let AssemblerPredicate = isGFX90AOnly;
} }
def _gfx940 : MUBUF_Real_gfx940<op, ps>; def _gfx940 : MUBUF_Real_gfx940<op, ps>;
} }
} }
multiclass MUBUF_Real_AllAddr_vi<bits<7> op> { multiclass MUBUF_Real_AllAddr_Helper_vi<bits<7> op, bit isTFE = 0> {
defm _OFFSET : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; defm _OFFSET : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET"), isTFE>;
defm _OFFEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; defm _OFFEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN"), isTFE>;
defm _IDXEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; defm _IDXEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN"), isTFE>;
defm _BOTHEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; defm _BOTHEN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN"), isTFE>;
} }
multiclass MUBUF_Real_AllAddr_Lds_vi<bits<7> op> { multiclass MUBUF_Real_AllAddr_vi<bits<7> op, bit hasTFE = 1> {
defm NAME : MUBUF_Real_AllAddr_Helper_vi<op>;
if hasTFE then
defm _TFE : MUBUF_Real_AllAddr_Helper_vi<op, 1>;
}
multiclass MUBUF_Real_AllAddr_Lds_Helper_vi<bits<7> op, bit isTFE = 0> {
def _OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
def _OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
if !not(isTFE) then {
def _LDS_OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>; def _LDS_OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>;
def _LDS_OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>; def _LDS_OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>;
def _LDS_IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>; def _LDS_IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>;
def _LDS_BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>; def _LDS_BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>;
def _OFFSET_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
def _OFFEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
def _LDS_OFFSET_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>; def _LDS_OFFSET_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>;
def _LDS_OFFEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>; def _LDS_OFFEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>;
def _LDS_IDXEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>; def _LDS_IDXEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>;
def _LDS_BOTHEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>; def _LDS_BOTHEN_gfx90a : MUBUF_Real_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>;
} }
}
multiclass MUBUF_Real_AllAddr_Lds_vi<bits<7> op> {
defm NAME : MUBUF_Real_AllAddr_Lds_Helper_vi<op>;
defm _TFE : MUBUF_Real_AllAddr_Lds_Helper_vi<op, 1>;
}
class MUBUF_Real_gfx80 <bits<7> op, MUBUF_Pseudo ps> : class MUBUF_Real_gfx80 <bits<7> op, MUBUF_Pseudo ps> :
MUBUF_Real<ps>, MUBUF_Real<ps>,
Enc64, Enc64,
SIMCInstr<ps.PseudoInstr, SIEncodingFamily.GFX80> { SIMCInstr<ps.PseudoInstr, SIEncodingFamily.GFX80> {
let AssemblerPredicate=HasUnpackedD16VMem; let AssemblerPredicate=HasUnpackedD16VMem;
let DecoderNamespace="GFX80_UNPACKED"; let DecoderNamespace="GFX80_UNPACKED";
let Inst{11-0} = !if(ps.has_offset, offset, ?); let Inst{11-0} = !if(ps.has_offset, offset, ?);
let Inst{12} = ps.offen; let Inst{12} = ps.offen;
let Inst{13} = ps.idxen; let Inst{13} = ps.idxen;
let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value); let Inst{14} = !if(ps.has_glc, cpol{CPolBit.GLC}, ps.glc_value);
let Inst{16} = ps.lds; let Inst{16} = ps.lds;
let Inst{17} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{17} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{24-18} = op; let Inst{24-18} = op;
let Inst{31-26} = 0x38; //encoding let Inst{31-26} = 0x38; //encoding
let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?); let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?);
let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?); let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?);
let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?); let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?);
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
let Inst{63-56} = !if(ps.has_soffset, soffset, ?); let Inst{63-56} = !if(ps.has_soffset, soffset, ?);
} }
multiclass MUBUF_Real_AllAddr_gfx80<bits<7> op> { multiclass MUBUF_Real_AllAddr_Helper_gfx80<bits<7> op> {
def _OFFSET_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>;
def _OFFEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_gfx80 : MUBUF_Real_gfx80 <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>;
} }
multiclass MUBUF_Real_AllAddr_gfx80<bits<7> op> {
dpUnsubmitted
Not Done
ReplyInline Actions

It looks like noTFE is always 0 and may be removed.

dp: It looks like `noTFE` is always 0 and may be removed.
defm NAME : MUBUF_Real_AllAddr_Helper_gfx80<op>;
defm _TFE : MUBUF_Real_AllAddr_Helper_gfx80<op>;
}
multiclass MUBUF_Real_Atomic_vi<bits<7> op> : multiclass MUBUF_Real_Atomic_vi<bits<7> op> :
MUBUF_Real_AllAddr_vi<op> { MUBUF_Real_AllAddr_vi<op, 0> {
defm _OFFSET_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET_RTN")>; defm _OFFSET_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET_RTN")>;
defm _OFFEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN_RTN")>; defm _OFFEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN_RTN")>;
defm _IDXEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN_RTN")>; defm _IDXEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN_RTN")>;
defm _BOTHEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN_RTN")>; defm _BOTHEN_RTN : MUBUF_Real_vi_gfx90a <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN_RTN")>;
} }
defm BUFFER_LOAD_FORMAT_X : MUBUF_Real_AllAddr_Lds_vi <0x00>; defm BUFFER_LOAD_FORMAT_X : MUBUF_Real_AllAddr_Lds_vi <0x00>;
defm BUFFER_LOAD_FORMAT_XY : MUBUF_Real_AllAddr_vi <0x01>; defm BUFFER_LOAD_FORMAT_XY : MUBUF_Real_AllAddr_vi <0x01>;
▲ Show 20 Lines • Show All 123 Lines • ▼ Show 20 Linesclass MTBUF_Real_Base_vi <bits<4> op, MTBUF_Pseudo ps, int Enc> :
let Inst{22-19} = dfmt; let Inst{22-19} = dfmt;
let Inst{25-23} = nfmt; let Inst{25-23} = nfmt;
let Inst{31-26} = 0x3a; //encoding let Inst{31-26} = 0x3a; //encoding
let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?); let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?);
let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?); let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?);
let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?); let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?);
let Inst{53} = !if(ps.has_sccb, cpol{CPolBit.SCC}, ps.sccb_value); let Inst{53} = !if(ps.has_sccb, cpol{CPolBit.SCC}, ps.sccb_value);
let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
let Inst{63-56} = !if(ps.has_soffset, soffset, ?); let Inst{63-56} = !if(ps.has_soffset, soffset, ?);
} }
class MTBUF_Real_vi <bits<4> op, MTBUF_Pseudo ps> : class MTBUF_Real_vi <bits<4> op, MTBUF_Pseudo ps> :
MTBUF_Real_Base_vi <op, ps, SIEncodingFamily.VI> { MTBUF_Real_Base_vi <op, ps, SIEncodingFamily.VI> {
let AssemblerPredicate = isGFX8GFX9NotGFX90A; let AssemblerPredicate = isGFX8GFX9NotGFX90A;
let DecoderNamespace = "GFX8"; let DecoderNamespace = "GFX8";
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
} }
class MTBUF_Real_gfx90a <bits<4> op, MTBUF_Pseudo ps> : class MTBUF_Real_gfx90a <bits<4> op, MTBUF_Pseudo ps> :
MTBUF_Real_Base_vi <op, ps, SIEncodingFamily.GFX90A> { MTBUF_Real_Base_vi <op, ps, SIEncodingFamily.GFX90A> {
let AssemblerPredicate = isGFX90APlus; let AssemblerPredicate = isGFX90APlus;
let DecoderNamespace = "GFX90A"; let DecoderNamespace = "GFX90A";
let AsmString = ps.Mnemonic # !subst("$tfe", "", ps.AsmOperands); let AsmString = ps.Mnemonic # ps.AsmOperands;
let Inst{55} = acc; let Inst{55} = acc;
} }
multiclass MTBUF_Real_vi_gfx90a<bits<4> op, MTBUF_Pseudo ps> { multiclass MTBUF_Real_vi_gfx90a<bits<4> op, MTBUF_Pseudo ps> {
def _vi : MTBUF_Real_vi<op, ps>; def _vi : MTBUF_Real_vi<op, ps>;
def _gfx90a : MTBUF_Real_gfx90a<op, ps>; def _gfx90a : MTBUF_Real_gfx90a<op, ps>;
} }
Show All 19 Linesclass MTBUF_Real_gfx80 <bits<4> op, MTBUF_Pseudo ps> :
let Inst{18-15} = op; let Inst{18-15} = op;
let Inst{22-19} = dfmt; let Inst{22-19} = dfmt;
let Inst{25-23} = nfmt; let Inst{25-23} = nfmt;
let Inst{31-26} = 0x3a; //encoding let Inst{31-26} = 0x3a; //encoding
let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?); let Inst{39-32} = !if(ps.has_vaddr, vaddr, ?);
let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?); let Inst{47-40} = !if(ps.has_vdata, vdata{7-0}, ?);
let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?); let Inst{52-48} = !if(ps.has_srsrc, srsrc{6-2}, ?);
let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?); let Inst{54} = !if(ps.has_slc, cpol{CPolBit.SLC}, ?);
let Inst{55} = !if(ps.has_tfe, tfe, ?); let Inst{55} = ps.tfe;
let Inst{63-56} = !if(ps.has_soffset, soffset, ?); let Inst{63-56} = !if(ps.has_soffset, soffset, ?);
} }
multiclass MTBUF_Real_AllAddr_gfx80<bits<4> op> { multiclass MTBUF_Real_AllAddr_gfx80<bits<4> op> {
def _OFFSET_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_OFFSET")>; def _OFFSET_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_OFFSET")>;
def _OFFEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_OFFEN")>; def _OFFEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_OFFEN")>;
def _IDXEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_IDXEN")>; def _IDXEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_IDXEN")>;
def _BOTHEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_BOTHEN")>; def _BOTHEN_gfx80 : MTBUF_Real_gfx80 <op, !cast<MTBUF_Pseudo>(NAME#"_BOTHEN")>;
▲ Show 20 Lines • Show All 71 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/Disassembler/AMDGPUDisassembler.cpp

llvm/lib/Target/AMDGPU/SIInstrInfo.td

llvm/lib/Target/AMDGPU/SILoadStoreOptimizer.cpp

llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-load-atomic-global.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-load-global.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-load-global.s96.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-load-private.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-store-global.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-store-global.s96.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-store-private.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/lds-zero-initializer.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f32.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.i8.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.s.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f32.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir

llvm/test/CodeGen/AMDGPU/accvgpr-spill-scc-clobber.mir

llvm/test/CodeGen/AMDGPU/av_spill_cross_bb_usage.mir

llvm/test/CodeGen/AMDGPU/branch-relax-no-terminators.mir

llvm/test/CodeGen/AMDGPU/break-vmem-soft-clauses.mir

llvm/test/CodeGen/AMDGPU/buffer-intrinsics-mmo-offsets.ll

llvm/test/CodeGen/AMDGPU/clamp-omod-special-case.mir

llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir

llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir

llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir

llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir

llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir

llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir

llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir

llvm/test/CodeGen/AMDGPU/couldnt-join-subrange-3.mir

llvm/test/CodeGen/AMDGPU/csr-sgpr-spill-live-ins.mir

llvm/test/CodeGen/AMDGPU/dagcombine-lshr-and-cmp.ll

llvm/test/CodeGen/AMDGPU/divergence-driven-trunc-to-i1.ll

llvm/test/CodeGen/AMDGPU/extract_subvector_vec4_vec3.ll

llvm/test/CodeGen/AMDGPU/fold-fi-mubuf.mir

llvm/test/CodeGen/AMDGPU/fold-imm-copy.mir

llvm/test/CodeGen/AMDGPU/fold-imm-f16-f32.mir

llvm/test/CodeGen/AMDGPU/fold-immediate-output-mods.mir

llvm/test/CodeGen/AMDGPU/fold-multiple.mir

llvm/test/CodeGen/AMDGPU/frame-index.mir

llvm/test/CodeGen/AMDGPU/greedy-liverange-priority.mir

llvm/test/CodeGen/AMDGPU/hard-clauses.mir

llvm/test/CodeGen/AMDGPU/hazard-buffer-store-v-interp.mir

llvm/test/CodeGen/AMDGPU/hazard-hidden-bundle.mir

llvm/test/CodeGen/AMDGPU/hazard-in-bundle.mir

llvm/test/CodeGen/AMDGPU/indirect-addressing-term.ll

llvm/test/CodeGen/AMDGPU/insert-skips-flat-vmem-ds.mir

llvm/test/CodeGen/AMDGPU/insert-waitcnts-exp.mir

llvm/test/CodeGen/AMDGPU/inserted-wait-states.mir

llvm/test/CodeGen/AMDGPU/invert-br-undef-vcc.mir

llvm/test/CodeGen/AMDGPU/lds-branch-vmem-hazard.mir

llvm/test/CodeGen/AMDGPU/lds-dma-waitcnt.mir

llvm/test/CodeGen/AMDGPU/lds-zero-initializer.ll

llvm/test/CodeGen/AMDGPU/load-store-opt-dlc.mir

llvm/test/CodeGen/AMDGPU/load-store-opt-scc.mir

llvm/test/CodeGen/AMDGPU/mai-hazards-gfx90a.mir

llvm/test/CodeGen/AMDGPU/mai-hazards-gfx940.mir

llvm/test/CodeGen/AMDGPU/mai-hazards.mir

llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir

llvm/test/CodeGen/AMDGPU/memory-legalizer-multiple-mem-operands-atomics.mir

llvm/test/CodeGen/AMDGPU/memory-legalizer-multiple-mem-operands-nontemporal-1.mir

llvm/test/CodeGen/AMDGPU/memory-legalizer-multiple-mem-operands-nontemporal-2.mir

llvm/test/CodeGen/AMDGPU/memory_clause.mir

llvm/test/CodeGen/AMDGPU/merge-image-load-gfx10.mir

llvm/test/CodeGen/AMDGPU/merge-image-load-gfx11.mir

llvm/test/CodeGen/AMDGPU/merge-image-load.mir

llvm/test/CodeGen/AMDGPU/merge-image-sample-gfx10.mir

llvm/test/CodeGen/AMDGPU/merge-image-sample-gfx11.mir

llvm/test/CodeGen/AMDGPU/merge-image-sample.mir

llvm/test/CodeGen/AMDGPU/merge-load-store.mir

llvm/test/CodeGen/AMDGPU/merge-tbuffer.mir

llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.mir

llvm/test/CodeGen/AMDGPU/nsa-vmem-hazard.mir

llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir

llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir

llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll

llvm/test/CodeGen/AMDGPU/pei-build-av-spill.mir

llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr.mir

llvm/test/CodeGen/AMDGPU/pei-build-spill.mir

llvm/test/CodeGen/AMDGPU/pei-reg-scavenger-position.mir

llvm/test/CodeGen/AMDGPU/pei-scavenge-sgpr-carry-out.mir

llvm/test/CodeGen/AMDGPU/pei-scavenge-sgpr-gfx9.mir

llvm/test/CodeGen/AMDGPU/pei-scavenge-sgpr.mir

llvm/test/CodeGen/AMDGPU/pei-scavenge-vgpr-spill.mir

llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir

llvm/test/CodeGen/AMDGPU/postra-bundle-memops.mir

llvm/test/CodeGen/AMDGPU/power-sched-no-instr-sunit.mir

llvm/test/CodeGen/AMDGPU/regalloc-introduces-copy-sgpr-to-agpr.mir

llvm/test/CodeGen/AMDGPU/regcoal-subrange-join-seg.mir

llvm/test/CodeGen/AMDGPU/regcoalesce-dbg.mir

llvm/test/CodeGen/AMDGPU/release-vgprs.mir

llvm/test/CodeGen/AMDGPU/rename-independent-subregs-mac-operands.mir

llvm/test/CodeGen/AMDGPU/same-slot-agpr-sgpr.mir

llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir

llvm/test/CodeGen/AMDGPU/sched-crash-dbg-value.mir

llvm/test/CodeGen/AMDGPU/sched-handleMoveUp-subreg-def-across-subreg-def.mir

llvm/test/CodeGen/AMDGPU/schedule-barrier.mir

llvm/test/CodeGen/AMDGPU/schedule-ilp.mir

llvm/test/CodeGen/AMDGPU/sgpr-spill-to-vmem-scc-clobber.mir

llvm/test/CodeGen/AMDGPU/sgpr-spill-vmem-large-frame.mir

llvm/test/CodeGen/AMDGPU/sgpr-spill.mir

llvm/test/CodeGen/AMDGPU/shrink-vop3-carry-out.mir

llvm/test/CodeGen/AMDGPU/si-fold-copy-kills.mir

llvm/test/CodeGen/AMDGPU/spill-agpr-partially-undef.mir

llvm/test/CodeGen/AMDGPU/spill-agpr.mir

llvm/test/CodeGen/AMDGPU/spill-reg-tuple-super-reg-use.mir

llvm/test/CodeGen/AMDGPU/spill-special-sgpr.mir

llvm/test/CodeGen/AMDGPU/splitkit-copy-live-lanes.mir

llvm/test/CodeGen/AMDGPU/splitkit-getsubrangeformask.ll

llvm/test/CodeGen/AMDGPU/unallocatable-bundle-regression.mir

llvm/test/CodeGen/AMDGPU/vccz-corrupt-bug-workaround.mir

llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir

llvm/test/CodeGen/AMDGPU/vgpr-spill-scc-clobber.mir

llvm/test/CodeGen/AMDGPU/vgpr-spill.mir

llvm/test/CodeGen/AMDGPU/vmem-to-salu-hazard.mir

llvm/test/CodeGen/AMDGPU/vmem-vcc-hazard.mir

llvm/test/CodeGen/AMDGPU/waitcnt-bvh.mir

llvm/test/CodeGen/AMDGPU/waitcnt-loop-irreducible.mir

llvm/test/CodeGen/AMDGPU/waitcnt-no-redundant.mir

llvm/test/CodeGen/AMDGPU/waitcnt-overflow.mir

llvm/test/CodeGen/AMDGPU/waitcnt-vmcnt-loop.mir

llvm/test/CodeGen/AMDGPU/waitcnt-vmem-waw.mir

llvm/test/CodeGen/AMDGPU/waitcnt.mir

llvm/test/CodeGen/AMDGPU/wqm.mir

llvm/test/CodeGen/MIR/AMDGPU/custom-pseudo-source-values.ll

llvm/test/CodeGen/MIR/AMDGPU/mir-canon-multi.mir

llvm/test/CodeGen/MIR/AMDGPU/parse-order-reserved-regs.mir

llvm/test/CodeGen/MIR/AMDGPU/target-index-operands.mir

llvm/test/MC/AMDGPU/gfx10_err_pos.s

llvm/test/MC/AMDGPU/gfx11_asm_mubuf.s

llvm/test/MC/AMDGPU/gfx11_asm_mubuf_alias.s

llvm/test/MC/AMDGPU/gfx90a_err.s

llvm/test/MC/AMDGPU/mubuf-gfx10.s

llvm/test/MC/AMDGPU/mubuf-gfx9.s

llvm/test/MC/AMDGPU/mubuf.s