This is an archive of the discontinued LLVM Phabricator instance.

Differential D84252

[amdgpu] Run SROA after loop unrolling.
ClosedPublic

Authored by hliao on Jul 21 2020, 10:55 AM.

Details

Reviewers
rampitec
arsenm
Commits
rG1f4e7463b5e3: [amdgpu] Run SROA after loop unrolling.
Summary
  • There are promotable allocas after loop unrolling.

Diff Detail

Event Timeline

hliao created this revision.Jul 21 2020, 10:55 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 21 2020, 10:55 AM
Herald added subscribers: llvm-commits, nikic, kerbowa and 9 others. · View Herald Transcript
arsenm added a comment.Jul 21 2020, 10:57 AM

I would expect the generic code to add this

arsenm added a comment.Jul 21 2020, 10:58 AM

Also could use a test that shows this helping

Harbormaster completed remote builds in B65117: Diff 279579.Jul 21 2020, 11:36 AM
hliao added a comment.Jul 21 2020, 2:19 PM
In D84252#2164763, @arsenm wrote:

I would expect the generic code to add this

Yeah, I also feel that way. But, let do it as a target-specific one first. I will study it with cpu2k3 or cpu2k6 benchmarks to learn whether how they are benefited.

rampitec added a comment.Jul 28 2020, 3:00 PM

A test could be nice.

hliao updated this revision to Diff 289246.Sep 1 2020, 12:12 PM

Add a simple enough test case.

Herald added a subscriber: zzheng. · View Herald TranscriptSep 1 2020, 12:12 PM
hliao updated this revision to Diff 289248.Sep 1 2020, 12:18 PM

clean up.

rampitec added inline comments.Sep 1 2020, 12:28 PM
llvm/test/CodeGen/AMDGPU/extra-sroa-after-unroll.ll
10

You need to check something with a non-negative pattern.

hliao updated this revision to Diff 289255.Sep 1 2020, 12:40 PM

revise the test case

hliao marked an inline comment as done.Sep 1 2020, 12:40 PM
This comment was removed by hliao.
rampitec accepted this revision.Sep 1 2020, 12:54 PM

LGTM with a nit: please run opt -instnamer on the test before submission.

This revision is now accepted and ready to land.Sep 1 2020, 12:54 PM
Harbormaster completed remote builds in B70279: Diff 289246.Sep 1 2020, 12:57 PM
Harbormaster completed remote builds in B70281: Diff 289248.Sep 1 2020, 1:05 PM
This revision was landed with ongoing or failed builds.Sep 1 2020, 1:10 PM
Closed by commit rG1f4e7463b5e3: [amdgpu] Run SROA after loop unrolling. (authored by hliao). · Explain Why
This revision was automatically updated to reflect the committed changes.
hliao added a commit: rG1f4e7463b5e3: [amdgpu] Run SROA after loop unrolling..
Harbormaster completed remote builds in B70284: Diff 289255.Sep 1 2020, 1:14 PM

Revision Contents

PathSize
llvm/
lib/
Target/
AMDGPU/
8 lines
test/
CodeGen/
AMDGPU/
90 lines
6 lines

Diff 289261

llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

Show First 20 Lines • Show All 476 Lines • ▼ Show 20 Lines [EnableOpt](const PassManagerBuilder &, legacy::PassManagerBase &PM) {
// and before other cleanup optimizations. // and before other cleanup optimizations.
PM.add(createAMDGPULowerKernelAttributesPass()); PM.add(createAMDGPULowerKernelAttributesPass());
// Promote alloca to vector before SROA and loop unroll. If we manage // Promote alloca to vector before SROA and loop unroll. If we manage
// to eliminate allocas before unroll we may choose to unroll less. // to eliminate allocas before unroll we may choose to unroll less.
if (EnableOpt) if (EnableOpt)
PM.add(createAMDGPUPromoteAllocaToVector()); PM.add(createAMDGPUPromoteAllocaToVector());
}); });
Builder.addExtension(
PassManagerBuilder::EP_LoopOptimizerEnd,
[](const PassManagerBuilder &, legacy::PassManagerBase &PM) {
// Add SROA after loop unrolling as more promotable patterns are
// exposed after small loops are fully unrolled.
PM.add(createSROAPass());
});
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// R600 Target Machine (R600 -> Cayman) // R600 Target Machine (R600 -> Cayman)
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
R600TargetMachine::R600TargetMachine(const Target &T, const Triple &TT, R600TargetMachine::R600TargetMachine(const Target &T, const Triple &TT,
StringRef CPU, StringRef FS, StringRef CPU, StringRef FS,
▲ Show 20 Lines • Show All 726 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/extra-sroa-after-unroll.ll

  • This file was added.
; RUN: opt -instnamer -O1 -mtriple=amdgcn-- -S -o - %s | FileCheck -check-prefixes=ALL,O1 %s
; RUN: opt -instnamer -O2 -mtriple=amdgcn-- -S -o - %s | FileCheck -check-prefixes=ALL,O2 %s
; RUN: opt -instnamer -O3 -mtriple=amdgcn-- -S -o - %s | FileCheck -check-prefixes=ALL,O3 %s
target datalayout = "A5"
; ALL-LABEL: t0
; O1-NOT: alloca
; O2-NOT: alloca
; O3-NOT: alloca
; ALL-COUNT-27: = load
rampitecUnsubmitted
Done
ReplyInline Actions

You need to check something with a non-negative pattern.

rampitec: You need to check something with a non-negative pattern.
; ALL-COUNT-26: = add
define protected amdgpu_kernel void @t0(i32 addrspace(1)* %p.coerce) #0 {
entry:
%p = alloca i32*, align 8, addrspace(5)
%p.ascast = addrspacecast i32* addrspace(5)* %p to i32**
%p.addr = alloca i32*, align 8, addrspace(5)
%p.addr.ascast = addrspacecast i32* addrspace(5)* %p.addr to i32**
%t = alloca [27 x i32], align 16, addrspace(5)
%t.ascast = addrspacecast [27 x i32] addrspace(5)* %t to [27 x i32]*
%sum = alloca i32, align 4, addrspace(5)
%sum.ascast = addrspacecast i32 addrspace(5)* %sum to i32*
%i = alloca i32, align 4, addrspace(5)
%i.ascast = addrspacecast i32 addrspace(5)* %i to i32*
%cleanup.dest.slot = alloca i32, align 4, addrspace(5)
%0 = addrspacecast i32 addrspace(1)* %p.coerce to i32*
store i32* %0, i32** %p.ascast, align 8
%p1 = load i32*, i32** %p.ascast, align 8
store i32* %p1, i32** %p.addr.ascast, align 8
%1 = bitcast [27 x i32] addrspace(5)* %t to i8 addrspace(5)*
call void @llvm.lifetime.start.p5i8(i64 48, i8 addrspace(5)* %1)
%arraydecay = getelementptr inbounds [27 x i32], [27 x i32]* %t.ascast, i64 0, i64 0
%2 = load i32*, i32** %p.addr.ascast, align 8
call void @copy(i32* %arraydecay, i32* %2, i32 27)
%3 = bitcast i32 addrspace(5)* %sum to i8 addrspace(5)*
call void @llvm.lifetime.start.p5i8(i64 4, i8 addrspace(5)* %3)
store i32 0, i32* %sum.ascast, align 4
%4 = bitcast i32 addrspace(5)* %i to i8 addrspace(5)*
call void @llvm.lifetime.start.p5i8(i64 4, i8 addrspace(5)* %4)
store i32 0, i32* %i.ascast, align 4
br label %for.cond
for.cond: ; preds = %for.inc, %entry
%5 = load i32, i32* %i.ascast, align 4
%cmp = icmp slt i32 %5, 27
br i1 %cmp, label %for.body, label %for.cond.cleanup
for.cond.cleanup: ; preds = %for.cond
%6 = bitcast i32 addrspace(5)* %i to i8 addrspace(5)*
call void @llvm.lifetime.end.p5i8(i64 4, i8 addrspace(5)* %6)
br label %for.end
for.body: ; preds = %for.cond
%7 = load i32, i32* %i.ascast, align 4
%idxprom = sext i32 %7 to i64
%arrayidx = getelementptr inbounds [27 x i32], [27 x i32]* %t.ascast, i64 0, i64 %idxprom
%8 = load i32, i32* %arrayidx, align 4
%9 = load i32, i32* %sum.ascast, align 4
%add = add nsw i32 %9, %8
store i32 %add, i32* %sum.ascast, align 4
br label %for.inc
for.inc: ; preds = %for.body
%10 = load i32, i32* %i.ascast, align 4
%inc = add nsw i32 %10, 1
store i32 %inc, i32* %i.ascast, align 4
br label %for.cond
for.end: ; preds = %for.cond.cleanup
%11 = load i32, i32* %sum.ascast, align 4
%12 = load i32*, i32** %p.addr.ascast, align 8
store i32 %11, i32* %12, align 4
%13 = bitcast i32 addrspace(5)* %sum to i8 addrspace(5)*
call void @llvm.lifetime.end.p5i8(i64 4, i8 addrspace(5)* %13)
%14 = bitcast [27 x i32] addrspace(5)* %t to i8 addrspace(5)*
call void @llvm.lifetime.end.p5i8(i64 48, i8 addrspace(5)* %14)
ret void
}
define internal void @copy(i32* %d, i32* %s, i32 %N) {
entry:
%d8 = bitcast i32* %d to i8*
%s8 = bitcast i32* %s to i8*
%N8 = mul i32 %N, 4
tail call void @llvm.memcpy.p0i8.p0i8.i32(i8* %d8, i8* %s8, i32 %N8, i1 false)
ret void
}
declare void @llvm.lifetime.start.p5i8(i64 immarg, i8 addrspace(5)* nocapture)
declare void @llvm.lifetime.end.p5i8(i64 immarg, i8 addrspace(5)* nocapture)
declare void @llvm.memcpy.p0i8.p0i8.i32(i8* nocapture, i8* nocapture, i32, i1)

llvm/test/CodeGen/AMDGPU/opt-pipeline.ll

Show First 20 Lines • Show All 157 Lines • ▼ Show 20 Lines
; GCN-O1-NEXT: LCSSA Verifier ; GCN-O1-NEXT: LCSSA Verifier
; GCN-O1-NEXT: Loop-Closed SSA Form Pass ; GCN-O1-NEXT: Loop-Closed SSA Form Pass
; GCN-O1-NEXT: Scalar Evolution Analysis ; GCN-O1-NEXT: Scalar Evolution Analysis
; GCN-O1-NEXT: Loop Pass Manager ; GCN-O1-NEXT: Loop Pass Manager
; GCN-O1-NEXT: Induction Variable Simplification ; GCN-O1-NEXT: Induction Variable Simplification
; GCN-O1-NEXT: Recognize loop idioms ; GCN-O1-NEXT: Recognize loop idioms
; GCN-O1-NEXT: Delete dead loops ; GCN-O1-NEXT: Delete dead loops
; GCN-O1-NEXT: Unroll loops ; GCN-O1-NEXT: Unroll loops
; GCN-O1-NEXT: SROA
; GCN-O1-NEXT: Phi Values Analysis ; GCN-O1-NEXT: Phi Values Analysis
; GCN-O1-NEXT: Function Alias Analysis Results
; GCN-O1-NEXT: Memory Dependence Analysis ; GCN-O1-NEXT: Memory Dependence Analysis
; GCN-O1-NEXT: MemCpy Optimization ; GCN-O1-NEXT: MemCpy Optimization
; GCN-O1-NEXT: Sparse Conditional Constant Propagation ; GCN-O1-NEXT: Sparse Conditional Constant Propagation
; GCN-O1-NEXT: Demanded bits analysis ; GCN-O1-NEXT: Demanded bits analysis
; GCN-O1-NEXT: Bit-Tracking Dead Code Elimination ; GCN-O1-NEXT: Bit-Tracking Dead Code Elimination
; GCN-O1-NEXT: Function Alias Analysis Results ; GCN-O1-NEXT: Function Alias Analysis Results
; GCN-O1-NEXT: Lazy Branch Probability Analysis ; GCN-O1-NEXT: Lazy Branch Probability Analysis
; GCN-O1-NEXT: Lazy Block Frequency Analysis ; GCN-O1-NEXT: Lazy Block Frequency Analysis
▲ Show 20 Lines • Show All 301 Lines • ▼ Show 20 Lines
; GCN-O2-NEXT: LCSSA Verifier ; GCN-O2-NEXT: LCSSA Verifier
; GCN-O2-NEXT: Loop-Closed SSA Form Pass ; GCN-O2-NEXT: Loop-Closed SSA Form Pass
; GCN-O2-NEXT: Scalar Evolution Analysis ; GCN-O2-NEXT: Scalar Evolution Analysis
; GCN-O2-NEXT: Loop Pass Manager ; GCN-O2-NEXT: Loop Pass Manager
; GCN-O2-NEXT: Induction Variable Simplification ; GCN-O2-NEXT: Induction Variable Simplification
; GCN-O2-NEXT: Recognize loop idioms ; GCN-O2-NEXT: Recognize loop idioms
; GCN-O2-NEXT: Delete dead loops ; GCN-O2-NEXT: Delete dead loops
; GCN-O2-NEXT: Unroll loops ; GCN-O2-NEXT: Unroll loops
; GCN-O2-NEXT: SROA
; GCN-O2-NEXT: Function Alias Analysis Results
; GCN-O2-NEXT: MergedLoadStoreMotion ; GCN-O2-NEXT: MergedLoadStoreMotion
; GCN-O2-NEXT: Phi Values Analysis ; GCN-O2-NEXT: Phi Values Analysis
; GCN-O2-NEXT: Function Alias Analysis Results ; GCN-O2-NEXT: Function Alias Analysis Results
; GCN-O2-NEXT: Memory Dependence Analysis ; GCN-O2-NEXT: Memory Dependence Analysis
; GCN-O2-NEXT: Lazy Branch Probability Analysis ; GCN-O2-NEXT: Lazy Branch Probability Analysis
; GCN-O2-NEXT: Lazy Block Frequency Analysis ; GCN-O2-NEXT: Lazy Block Frequency Analysis
; GCN-O2-NEXT: Optimization Remark Emitter ; GCN-O2-NEXT: Optimization Remark Emitter
; GCN-O2-NEXT: Global Value Numbering ; GCN-O2-NEXT: Global Value Numbering
▲ Show 20 Lines • Show All 342 Lines • ▼ Show 20 Lines
; GCN-O3-NEXT: LCSSA Verifier ; GCN-O3-NEXT: LCSSA Verifier
; GCN-O3-NEXT: Loop-Closed SSA Form Pass ; GCN-O3-NEXT: Loop-Closed SSA Form Pass
; GCN-O3-NEXT: Scalar Evolution Analysis ; GCN-O3-NEXT: Scalar Evolution Analysis
; GCN-O3-NEXT: Loop Pass Manager ; GCN-O3-NEXT: Loop Pass Manager
; GCN-O3-NEXT: Induction Variable Simplification ; GCN-O3-NEXT: Induction Variable Simplification
; GCN-O3-NEXT: Recognize loop idioms ; GCN-O3-NEXT: Recognize loop idioms
; GCN-O3-NEXT: Delete dead loops ; GCN-O3-NEXT: Delete dead loops
; GCN-O3-NEXT: Unroll loops ; GCN-O3-NEXT: Unroll loops
; GCN-O3-NEXT: SROA
; GCN-O3-NEXT: Function Alias Analysis Results
; GCN-O3-NEXT: MergedLoadStoreMotion ; GCN-O3-NEXT: MergedLoadStoreMotion
; GCN-O3-NEXT: Phi Values Analysis ; GCN-O3-NEXT: Phi Values Analysis
; GCN-O3-NEXT: Function Alias Analysis Results ; GCN-O3-NEXT: Function Alias Analysis Results
; GCN-O3-NEXT: Memory Dependence Analysis ; GCN-O3-NEXT: Memory Dependence Analysis
; GCN-O3-NEXT: Lazy Branch Probability Analysis ; GCN-O3-NEXT: Lazy Branch Probability Analysis
; GCN-O3-NEXT: Lazy Block Frequency Analysis ; GCN-O3-NEXT: Lazy Block Frequency Analysis
; GCN-O3-NEXT: Optimization Remark Emitter ; GCN-O3-NEXT: Optimization Remark Emitter
; GCN-O3-NEXT: Global Value Numbering ; GCN-O3-NEXT: Global Value Numbering
▲ Show 20 Lines • Show All 197 Lines • Show Last 20 Lines