This is an archive of the discontinued LLVM Phabricator instance.

Differential D159140

[GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND
ClosedPublic

Authored by tobias-stadler on Aug 29 2023, 2:18 PM.

Details

Reviewers
aemerson
arsenm
tschuett
dzhidzhoev
Commits
rG373c343a77a7: Reland: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND
rG3686a0b611c6: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND
Summary

The legalizer currently generates lots of G_AND artifacts.
For example between boolean uses and defs there is always a G_AND with a mask of 1, but when the target uses ZeroOrOneBooleanContents, this is unnecessary.
Currently these artifacts have to be removed using post-legalize combines.
Omitting these artifacts at their source in the artifact combiner has a few advantages:

  • We know that the emitted G_AND is very likely to be useless, so our KnownBits call is likely worth it.
  • The G_AND and G_CONSTANT can interrupt e.g. G_UADDE/... sequences generated during legalization of wide adds which makes it harder to detect these sequences in the instruction selector (e.g. useful to prevent unnecessary reloading of AArch64 NZCV register).
  • This cleans up a lot of legalizer output and even improves compilation-times.

AArch64 CTMark geomean: O0 -5.6% size..text; O0 and O3 ~-0.9% compilation-time (instruction count).

Since this introduces KnownBits into code-paths used by O0, I reduced the default recursion depth.
This doesn't seem to make a difference in CTMark, but should prevent excessive recursive calls in the worst case.

Diff Detail

Event Timeline

tobias-stadler created this revision.Aug 29 2023, 2:18 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 29 2023, 2:18 PM
Herald added subscribers: luke, sunshaoce, frasercrmck and 26 others. · View Herald Transcript
tobias-stadler requested review of this revision.Aug 29 2023, 2:18 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 29 2023, 2:18 PM
Herald added subscribers: llvm-commits, wangpc, MaskRay, wdng. · View Herald Transcript
Harbormaster completed remote builds in B255627: Diff 554493.Aug 29 2023, 6:44 PM
tobias-stadler edited the summary of this revision. (Show Details)Aug 29 2023, 9:23 PM
aemerson added a comment.Aug 29 2023, 10:58 PM

We only emit the mask G_CONSTANT when necessary. Even when the G_AND is combined away later, the constant sometimes ends up being reused by other instructions instead of becoming dead.

I'm a bit confused by how this could happen. Does this happen with optimizations?

The wording suggests that a later transform needs a G_AND, and probably the CSEMIRBuilder returns a reference to dead one? Why would other instructions need a dead instruction?

arsenm added inline comments.Aug 30 2023, 6:24 AM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

How much of the benefit is there from only handling the constant case?

tobias-stadler added a comment.Aug 30 2023, 6:24 AM
In D159140#4627276, @aemerson wrote:

We only emit the mask G_CONSTANT when necessary. Even when the G_AND is combined away later, the constant sometimes ends up being reused by other instructions instead of becoming dead.

I'm a bit confused by how this could happen. Does this happen with optimizations?

The wording suggests that a later transform needs a G_AND, and probably the CSEMIRBuilder returns a reference to dead one? Why would other instructions need a dead instruction?

It happens at O3. You can take a look at https://godbolt.org/z/G97Kv6bza (this is taken from one of the Atomics tests). Maybe "reused" is the wrong wording. %47 is the mask constant emitted for the G_AND, but is then also used in various other places right after the legalizer. I assume this is intended CSE behavior, but I'd prefer this not to happen between G_UADDE sequences, so that these patterns can easily be detected by the selector again. Getting rid of the useless G_ANDs everywhere seemed like the cleanest solution and happens to also be a win in every CTMark metric.
Some more context: When I implemented wide adds in the selector, I opted to prevent unnecessary NZCV reloads by looking at the previous instruction to determine if NZCV will already be set correctly. This seems like the a good solution because it doesn't require custom legalization or changes to the way we handle flags (I think X86 does copies to and from the flags register and let's RegAlloc do the heavy lifting). The example also hits another issue though: RegBankAlloc decides to move one of the vRegs to fpr and inserts a copy, which also inhibits the optimization. I will have to look into if I missed some more edge cases like this. I think in the worst case we can always do some more cleanup in PostSelectOptimize, since we already do liveness analysis there. Though I'd prefer to make legalized G_UADDE sequences reliably detectable by the selector, so that we can still get good codegen without going the X86 route or resorting to additional optimizations.

tobias-stadler added inline comments.Aug 30 2023, 6:44 AM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

I'm not quite sure what you mean by "constant case". Do you mean after a single computeKnownBitsImpl calll? I can try that by reducing MaxDepth even more and will report. I believe most of the benefit comes from the boolean case. I'm not aware of a method to determine if we can ignore the AND other than how KnownBits does it using BooleanContents for applicable instructions, in which case I wouldn't want to duplicate this functionality here. Maybe we can add a MaxDepth override for cases like this?

tobias-stadler added inline comments.Aug 30 2023, 7:28 AM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

MaxDepth = 1 compared to MaxDepth = 6 doesn't seem to measurably affect compile-time, but geomean size..text goes from -5.6% to -5.5%. I don't think there's much to gain by adding extra complexity here just too avoid KnownBits.

aemerson added a comment.Aug 30 2023, 7:53 AM
In D159140#4628211, @tobias-stadler wrote:
In D159140#4627276, @aemerson wrote:

We only emit the mask G_CONSTANT when necessary. Even when the G_AND is combined away later, the constant sometimes ends up being reused by other instructions instead of becoming dead.

I'm a bit confused by how this could happen. Does this happen with optimizations?

The wording suggests that a later transform needs a G_AND, and probably the CSEMIRBuilder returns a reference to dead one? Why would other instructions need a dead instruction?

It happens at O3. You can take a look at https://godbolt.org/z/G97Kv6bza (this is taken from one of the Atomics tests). Maybe "reused" is the wrong wording. %47 is the mask constant emitted for the G_AND, but is then also used in various other places right after the legalizer. I assume this is intended CSE behavior, but I'd prefer this not to happen between G_UADDE sequences, so that these patterns can easily be detected by the selector again. Getting rid of the useless G_ANDs everywhere seemed like the cleanest solution and happens to also be a win in every CTMark metric.
Some more context: When I implemented wide adds in the selector, I opted to prevent unnecessary NZCV reloads by looking at the previous instruction to determine if NZCV will already be set correctly. This seems like the a good solution because it doesn't require custom legalization or changes to the way we handle flags (I think X86 does copies to and from the flags register and let's RegAlloc do the heavy lifting). The example also hits another issue though: RegBankAlloc decides to move one of the vRegs to fpr and inserts a copy, which also inhibits the optimization. I will have to look into if I missed some more edge cases like this. I think in the worst case we can always do some more cleanup in PostSelectOptimize, since we already do liveness analysis there. Though I'd prefer to make legalized G_UADDE sequences reliably detectable by the selector, so that we can still get good codegen without going the X86 route or resorting to additional optimizations.

I see what you mean now. Is there any code size impact with optimizations (-Os or -O3)? I expect there to be none but just want to check.

arsenm added inline comments.Aug 30 2023, 7:55 AM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

I mean by the input is a literal G_CONSTANT or copy of G_CONSTANT

tobias-stadler added a comment.Aug 30 2023, 8:41 AM
In D159140#4628523, @aemerson wrote:

I see what you mean now. Is there any code size impact with optimizations (-Os or -O3)? I expect there to be none but just want to check.

"size" is completely unchanged on -Os and -O3.
Very minor improvements on -Os and -O3 "size..text" on ~half of the benchmarks, no difference on the others. Biggest difference on lencod -O3 682928.00->682860.00.

tobias-stadler added inline comments.Aug 30 2023, 10:10 AM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

I tried it out. Only checking when lookThroughCopyInstrs(AndSrc) is a G_CONSTANT brings no benefit at all. A simple example of G_ZEXT(G_TRUNC(G_CONSTANT)) is already combined away by the other combines in the artifact combiner. The only difference between me directly checking for G_CONSTANT in this combine is that we end up with one less stray COPY.

tobias-stadler added inline comments.Aug 30 2023, 4:51 PM
llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h
143

Sorry, I didn't phrase this clearly. I first ran the "constant-case-only" version against O0 CTMark, which didn't show any difference in code size and compile-time compared to the current main branch. I then ran a simple hand-coded G_ZEXT(G_TRUNC(G_CONSTANT)) example against both versions to make sure the additional check did the right thing, which confirmed that there shouldn't be much of a difference in the first place. I couldn't think of and haven't seen examples in the tests where a G_CONSTANT is truncated and then extended again in such a way that this combine would emit better code than the other combines (apart from removing some COPY instructions, which also happens in some of the test differences).

tobias-stadler added a comment.Sep 12 2023, 4:20 PM

Gentle Ping.

tobias-stadler updated this revision to Diff 557343.Sep 25 2023, 6:18 PM
tobias-stadler edited the summary of this revision. (Show Details)

Rebase.

Harbormaster completed remote builds in B257597: Diff 557343.Sep 25 2023, 7:17 PM
tobias-stadler added a comment.Sep 27 2023, 2:57 PM

@arsenm @aemerson Hi, do you have any further comments on this or are you ok with this change?

aemerson accepted this revision.Sep 27 2023, 3:30 PM
In D159140#4651531, @tobias-stadler wrote:

@arsenm @aemerson Hi, do you have any further comments on this or are you ok with this change?

I think the benefits outweigh the costs in this case. I would just put a comment in the code explaining exactly why this combine is there, since someone may try to remove it in future if they don't notice any impact on final codegen.

This revision is now accepted and ready to land.Sep 27 2023, 3:30 PM
tobias-stadler updated this revision to Diff 557461.Sep 28 2023, 12:46 PM

Add suggested comment + Rebase.

tobias-stadler updated this revision to Diff 557469.Sep 28 2023, 2:19 PM

Fix broken test.

Harbormaster completed remote builds in B257674: Diff 557469.Sep 28 2023, 5:06 PM
Closed by commit rG3686a0b611c6: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND (authored by tobias-stadler). · Explain WhySep 28 2023, 5:15 PM
This revision was automatically updated to reflect the committed changes.
tobias-stadler added a commit: rG3686a0b611c6: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND.
tobias-stadler added a reverting change: rG305fbc1b3203: Revert "[GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND".Sep 28 2023, 6:44 PM
GitHub <noreply@github.com> mentioned this in rGb1a6b2cc409e: [AArch64][GlobalISel] Fix miscompile on carry-in selection (#68840).Oct 19 2023, 10:51 AM
tobias-stadler reopened this revision.Nov 1 2023, 12:00 PM

This was reverted due to an issue fixed by https://github.com/llvm/llvm-project/pull/68840 and should now be ready to reland.

This revision is now accepted and ready to land.Nov 1 2023, 12:00 PM
tobias-stadler updated this revision to Diff 557964.Nov 1 2023, 12:03 PM

Rebase, fix tests, and rerun CI before relanding.

Harbormaster completed remote builds in B257989: Diff 557964.Nov 1 2023, 1:48 PM
Closed by commit rG373c343a77a7: Reland: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND (authored by tobias-stadler). · Explain WhyNov 1 2023, 4:18 PM
This revision was automatically updated to reflect the committed changes.
tobias-stadler added a commit: rG373c343a77a7: Reland: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND.

Large Diff

This large diff affects 198 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
include/
llvm/
CodeGen/
GlobalISel/
6 lines
24 lines
lib/
CodeGen/
GlobalISel/
10 lines
2 lines
test/
CodeGen/
AArch64/
Atomics/
1180 lines
780 lines
1180 lines
1180 lines
1180 lines
780 lines
1180 lines
GlobalISel/
52 lines
2 lines
2 lines
4 lines
4 lines
4 lines
31 lines
33 lines
17 lines
8 lines
25 lines
51 lines
12 lines
4 lines
4 lines
2 lines
28 lines
24 lines
16 lines
2 lines
32 lines
8 lines
2 lines
14 lines
4 lines
6 lines
73 lines
2 lines
2 lines
15 lines
11 lines
37 lines
58 lines
73 lines
16 lines
15 lines
11 lines
37 lines
39 lines
20 lines
17 lines
13 lines
20 lines
17 lines
13 lines
2 lines
3 lines
30 lines
AMDGPU/
GlobalISel/
2 lines
15 lines
34 lines
798 lines
799 lines
63 lines
26 lines
46 lines
14 lines
372 lines
48 lines
7 lines
21 lines
15 lines
19 lines
33 lines
4 lines
40 lines
34 lines
40 lines
34 lines
50 lines
50 lines
34 lines
33 lines
206 lines
347 lines
36 lines
78 lines
27 lines
91 lines
14 lines
7 lines
34 lines
296 lines
330 lines
811 lines
952 lines
620 lines
454 lines
658 lines
42 lines
31 lines
4 lines
73 lines
15 lines
38 lines
16 lines
72 lines
2 lines
376 lines
21 lines
123 lines
9 lines
12 lines
147 lines
15 lines
38 lines
830 lines
66 lines
8 lines
99 lines
86 lines
115 lines
70 lines
70 lines
73 lines
65 lines
57 lines
123 lines
145 lines
99 lines
86 lines
42 lines
114 lines
ARM/
GlobalISel/
13 lines
Mips/
GlobalISel/
legalizer/
222 lines
451 lines
42 lines
36 lines
170 lines
672 lines
290 lines
141 lines
342 lines
159 lines
198 lines
llvm-ir/
5 lines
74 lines
7 lines
34 lines
9 lines
48 lines
4 lines
8 lines
1 line
56 lines
4 lines
RISCV/
GlobalISel/
legalizer/
rv32/
4 lines
24 lines
56 lines
136 lines
11 lines
64 lines
132 lines
6 lines
96 lines
32 lines
32 lines
8 lines
24 lines
32 lines
32 lines
rv64/
2 lines
26 lines
4 lines
142 lines
11 lines
12 lines
64 lines
144 lines
6 lines
12 lines
102 lines
28 lines
28 lines
8 lines
26 lines
34 lines
34 lines
X86/
GlobalISel/
90 lines
132 lines
390 lines
6 lines
6 lines
41 lines
3 lines
120 lines
unittests/
CodeGen/
GlobalISel/
2 lines

Diff 557967

llvm/include/llvm/CodeGen/GlobalISel/GISelKnownBits.h

llvm/include/llvm/CodeGen/GlobalISel/LegalizationArtifactCombiner.h

Show All 29 Lines
#define DEBUG_TYPE "legalizer" #define DEBUG_TYPE "legalizer"
namespace llvm { namespace llvm {
class LegalizationArtifactCombiner { class LegalizationArtifactCombiner {
MachineIRBuilder &Builder; MachineIRBuilder &Builder;
MachineRegisterInfo &MRI; MachineRegisterInfo &MRI;
const LegalizerInfo &LI; const LegalizerInfo &LI;
GISelKnownBits *KB;
static bool isArtifactCast(unsigned Opc) { static bool isArtifactCast(unsigned Opc) {
switch (Opc) { switch (Opc) {
case TargetOpcode::G_TRUNC: case TargetOpcode::G_TRUNC:
case TargetOpcode::G_SEXT: case TargetOpcode::G_SEXT:
case TargetOpcode::G_ZEXT: case TargetOpcode::G_ZEXT:
case TargetOpcode::G_ANYEXT: case TargetOpcode::G_ANYEXT:
return true; return true;
default: default:
return false; return false;
} }
} }
public: public:
LegalizationArtifactCombiner(MachineIRBuilder &B, MachineRegisterInfo &MRI, LegalizationArtifactCombiner(MachineIRBuilder &B, MachineRegisterInfo &MRI,
const LegalizerInfo &LI) const LegalizerInfo &LI,
: Builder(B), MRI(MRI), LI(LI) {} GISelKnownBits *KB = nullptr)
: Builder(B), MRI(MRI), LI(LI), KB(KB) {}
bool tryCombineAnyExt(MachineInstr &MI, bool tryCombineAnyExt(MachineInstr &MI,
SmallVectorImpl<MachineInstr *> &DeadInsts, SmallVectorImpl<MachineInstr *> &DeadInsts,
SmallVectorImpl<Register> &UpdatedDefs, SmallVectorImpl<Register> &UpdatedDefs,
GISelObserverWrapper &Observer) { GISelObserverWrapper &Observer) {
using namespace llvm::MIPatternMatch; using namespace llvm::MIPatternMatch;
assert(MI.getOpcode() == TargetOpcode::G_ANYEXT); assert(MI.getOpcode() == TargetOpcode::G_ANYEXT);
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Lines if (mi_match(SrcReg, MRI, m_GTrunc(m_Reg(TruncSrc))) ||
mi_match(SrcReg, MRI, m_GSExt(m_Reg(SextSrc)))) { mi_match(SrcReg, MRI, m_GSExt(m_Reg(SextSrc)))) {
LLT DstTy = MRI.getType(DstReg); LLT DstTy = MRI.getType(DstReg);
if (isInstUnsupported({TargetOpcode::G_AND, {DstTy}}) || if (isInstUnsupported({TargetOpcode::G_AND, {DstTy}}) ||
isConstantUnsupported(DstTy)) isConstantUnsupported(DstTy))
return false; return false;
LLVM_DEBUG(dbgs() << ".. Combine MI: " << MI;); LLVM_DEBUG(dbgs() << ".. Combine MI: " << MI;);
LLT SrcTy = MRI.getType(SrcReg); LLT SrcTy = MRI.getType(SrcReg);
APInt MaskVal = APInt::getAllOnes(SrcTy.getScalarSizeInBits()); APInt MaskVal = APInt::getAllOnes(SrcTy.getScalarSizeInBits());
auto Mask = Builder.buildConstant(
DstTy, MaskVal.zext(DstTy.getScalarSizeInBits()));
if (SextSrc && (DstTy != MRI.getType(SextSrc))) if (SextSrc && (DstTy != MRI.getType(SextSrc)))
SextSrc = Builder.buildSExtOrTrunc(DstTy, SextSrc).getReg(0); SextSrc = Builder.buildSExtOrTrunc(DstTy, SextSrc).getReg(0);
if (TruncSrc && (DstTy != MRI.getType(TruncSrc))) if (TruncSrc && (DstTy != MRI.getType(TruncSrc)))
TruncSrc = Builder.buildAnyExtOrTrunc(DstTy, TruncSrc).getReg(0); TruncSrc = Builder.buildAnyExtOrTrunc(DstTy, TruncSrc).getReg(0);
Builder.buildAnd(DstReg, SextSrc ? SextSrc : TruncSrc, Mask); APInt ExtMaskVal = MaskVal.zext(DstTy.getScalarSizeInBits());
Register AndSrc = SextSrc ? SextSrc : TruncSrc;
// Elide G_AND and mask constant if possible.
// The G_AND would also be removed by the post-legalize redundant_and
arsenmUnsubmitted
Not Done
ReplyInline Actions

How much of the benefit is there from only handling the constant case?

arsenm: How much of the benefit is there from only handling the constant case?
tobias-stadlerAuthorUnsubmitted
Done
ReplyInline Actions

I'm not quite sure what you mean by "constant case". Do you mean after a single computeKnownBitsImpl calll? I can try that by reducing MaxDepth even more and will report. I believe most of the benefit comes from the boolean case. I'm not aware of a method to determine if we can ignore the AND other than how KnownBits does it using BooleanContents for applicable instructions, in which case I wouldn't want to duplicate this functionality here. Maybe we can add a MaxDepth override for cases like this?

tobias-stadler: I'm not quite sure what you mean by "constant case". Do you mean after a single…
arsenmUnsubmitted
Not Done
ReplyInline Actions

I mean by the input is a literal G_CONSTANT or copy of G_CONSTANT

arsenm: I mean by the input is a literal G_CONSTANT or copy of G_CONSTANT
tobias-stadlerAuthorUnsubmitted
Done
ReplyInline Actions

I tried it out. Only checking when lookThroughCopyInstrs(AndSrc) is a G_CONSTANT brings no benefit at all. A simple example of G_ZEXT(G_TRUNC(G_CONSTANT)) is already combined away by the other combines in the artifact combiner. The only difference between me directly checking for G_CONSTANT in this combine is that we end up with one less stray COPY.

tobias-stadler: I tried it out. Only checking when lookThroughCopyInstrs(AndSrc) is a G_CONSTANT brings no…
tobias-stadlerAuthorUnsubmitted
Done
ReplyInline Actions

Sorry, I didn't phrase this clearly. I first ran the "constant-case-only" version against O0 CTMark, which didn't show any difference in code size and compile-time compared to the current main branch. I then ran a simple hand-coded G_ZEXT(G_TRUNC(G_CONSTANT)) example against both versions to make sure the additional check did the right thing, which confirmed that there shouldn't be much of a difference in the first place. I couldn't think of and haven't seen examples in the tests where a G_CONSTANT is truncated and then extended again in such a way that this combine would emit better code than the other combines (apart from removing some COPY instructions, which also happens in some of the test differences).

tobias-stadler: Sorry, I didn't phrase this clearly. I first ran the "constant-case-only" version against O0…
tobias-stadlerAuthorUnsubmitted
Done
ReplyInline Actions

MaxDepth = 1 compared to MaxDepth = 6 doesn't seem to measurably affect compile-time, but geomean size..text goes from -5.6% to -5.5%. I don't think there's much to gain by adding extra complexity here just too avoid KnownBits.

tobias-stadler: MaxDepth = 1 compared to MaxDepth = 6 doesn't seem to measurably affect compile-time, but…
// combine, but in this very common case, eliding early and regardless of
// OptLevel results in significant compile-time and O0 code-size
// improvements. Inserting unnecessary instructions between boolean defs
// and uses hinders a lot of folding during ISel.
if (KB && (KB->getKnownZeroes(AndSrc) | ExtMaskVal).isAllOnes()) {
replaceRegOrBuildCopy(DstReg, AndSrc, MRI, Builder, UpdatedDefs,
Observer);
} else {
auto Mask = Builder.buildConstant(DstTy, ExtMaskVal);
Builder.buildAnd(DstReg, AndSrc, Mask);
}
markInstAndDefDead(MI, *MRI.getVRegDef(SrcReg), DeadInsts); markInstAndDefDead(MI, *MRI.getVRegDef(SrcReg), DeadInsts);
return true; return true;
} }
// zext(zext x) -> (zext x) // zext(zext x) -> (zext x)
Register ZextSrc; Register ZextSrc;
if (mi_match(SrcReg, MRI, m_GZExt(m_Reg(ZextSrc)))) { if (mi_match(SrcReg, MRI, m_GZExt(m_Reg(ZextSrc)))) {
LLVM_DEBUG(dbgs() << ".. Combine MI: " << MI); LLVM_DEBUG(dbgs() << ".. Combine MI: " << MI);
▲ Show 20 Lines • Show All 1,365 Lines • Show Last 20 Lines

llvm/lib/CodeGen/GlobalISel/GISelKnownBits.cpp

llvm/lib/CodeGen/GlobalISel/Legalizer.cpp

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2_lse128.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll

llvm/test/CodeGen/AArch64/GlobalISel/arm64-atomic.ll

llvm/test/CodeGen/AArch64/GlobalISel/combine-ext-debugloc.mir

llvm/test/CodeGen/AArch64/GlobalISel/combine-sext-debugloc.mir

llvm/test/CodeGen/AArch64/GlobalISel/huge-switch.ll

llvm/test/CodeGen/AArch64/GlobalISel/invoke-region.ll

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-hoisted-constants.ll

llvm/test/CodeGen/AArch64/GlobalISel/legalize-add.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-cmp.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ctlz.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ctpop.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-cttz.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-div.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ext-cse.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ext-csedebug-output.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ext.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-extracts.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-freeze.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-fshl.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-fshr.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-inserts.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-intrinsic-min-max.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-itofp.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-load-store.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-merge-values.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-mul.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-phi-insertpt-decrement.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-phi.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ptrtoint.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-rem.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-sadde.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-saddo.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-saddsat.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-select.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-shift.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-simple.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ssube.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ssubo.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-ssubsat.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-sub.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-uadd-sat.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-uadde.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-uaddo.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-usub-sat.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-usube.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalize-usubo.mir

llvm/test/CodeGen/AArch64/GlobalISel/legalizer-combiner-zext-trunc-crash.mir

llvm/test/CodeGen/AArch64/GlobalISel/retry-artifact-combine.mir

llvm/test/CodeGen/AArch64/zext.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/artifact-combiner-cse-leaves-dead-cast.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/artifact-combiner-unmerge-values.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/artifact-combiner-zext.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/fshl.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/fshr.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/inst-select-load-private.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-add.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-and.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-anyext.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ashr.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-bitcast.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-bitreverse.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-bswap.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ctlz-zero-undef.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ctlz.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ctpop.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-cttz.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fabs.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fadd.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fcopysign.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fma.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fmaxnum.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fminnum.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fmul.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fneg.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fshl.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fshr.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fsub.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-icmp.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-implicit-def.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-insert.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-intrinsic-round.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.store.2d.d16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.s.buffer.load.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-load-constant.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-load-flat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-load-global.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-load-local.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-load-private.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-lshr.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-merge-values.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-or.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-phi.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ptrmask.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-rotl-rotr.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-saddo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-saddsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-select.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-sext-inreg.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-sext.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-shl.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-shuffle-vector.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-shuffle-vector.s16.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-smulh.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-smulo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-sshlsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ssubo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ssubsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-store-global.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-store.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-trunc.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-uaddo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-uaddsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-udiv.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-umax.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-umin.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-umulh.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-umulo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-unmerge-values.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-urem.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-ushlsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-usubo.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-usubsat.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-xor.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-zext.mir

llvm/test/CodeGen/ARM/GlobalISel/arm-legalize-fp.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/add.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/bitwise.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/constants.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/ctlz.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/cttz.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/fptosi_and_fptoui.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/icmp.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/jump_table_and_brjt.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/mul.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/select.mir

llvm/test/CodeGen/Mips/GlobalISel/legalizer/sub.mir

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/add.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/bitwise.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/ctlz.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/cttz.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/fptosi_and_fptoui.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/icmp.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/jump_table_and_brjt.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/mul.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/select.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sitofp_and_uitofp.ll

llvm/test/CodeGen/Mips/GlobalISel/llvm-ir/sub.ll

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-abs.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-add.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-addo-subo.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-ashr.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-bswap.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-icmp.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-lshr.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-mul-ext.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-shl.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-smax.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-smin.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-store.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-sub.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-umax.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv32/legalize-umin.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-abs.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-add.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-addo-subo.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-ashr.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-bswap.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-div.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-icmp.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-lshr.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-mul-ext.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-rem.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-shl.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-smax.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-smin.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-store.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-sub.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-umax.mir

llvm/test/CodeGen/RISCV/GlobalISel/legalizer/rv64/legalize-umin.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-cmp.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-ext-x86-64.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-ext.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-trailing-zeros-undef.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-trailing-zeros.mir

llvm/test/CodeGen/X86/GlobalISel/legalize-trunc.mir

llvm/test/CodeGen/X86/GlobalISel/lshr-scalar.ll

llvm/test/CodeGen/X86/GlobalISel/x86_64-legalize-zext.mir

llvm/unittests/CodeGen/GlobalISel/LegalizerTest.cpp