This is an archive of the discontinued LLVM Phabricator instance.

Differential D148199

[AMDGPU] Add a new command line argument amdgpu-atomic-optimizer-use-dpp
AbandonedPublic

Authored by pravinjagtap on Apr 13 2023, 12:18 AM.

Details

Reviewers
foad
ruiling
b-sumner
arsenm
nhaehnle
Summary

The current implementation of atomic optimization uses DPP with WWM for
scan computations. This new command line argument has exactly the same
behavior as old amdgpu-atomic-optimizations.

The new approach proposed in D147408, required to have two different
code paths for DPP and iterative scan. The implementation will be
selected based on this command line argument.

This will allow the the graphics pipeline to set the new argument
instead of the old one.

Diff Detail

Event Timeline

pravinjagtap created this revision.Apr 13 2023, 12:18 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 13 2023, 12:18 AM
Herald added subscribers: kosarev, StephenFan, kerbowa and 6 others. · View Herald Transcript
pravinjagtap requested review of this revision.Apr 13 2023, 12:18 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 13 2023, 12:18 AM
Herald added subscribers: llvm-commits, wdng. · View Herald Transcript
Harbormaster completed remote builds in B225262: Diff 513083.Apr 13 2023, 1:10 AM
foad added inline comments.Apr 13 2023, 1:56 AM
llvm/test/CodeGen/AMDGPU/atomic_load_add.ll
1 ↗(On Diff #513083)

I guess these tests really want to disable the atomic optimizer pass, to avoid widespread changes to the generated code. If so, they should continue using -amdgpu-atomic-optimizations and we should continue to support that option with its current meaning.

pravinjagtap added inline comments.Apr 13 2023, 2:26 AM
llvm/test/CodeGen/AMDGPU/atomic_load_add.ll
1 ↗(On Diff #513083)

If I understand correctly, the tests which uses -amdgpu-atomic-optimizations=*true* can be updated with a new option? whereas the tests which uses -amdgpu-atomic-optimizations=*false* should continue to support old option?

I thought we will be deprecating -amdgpu-atomic-optimizations option completely and Atomic Optimizer Pass will depend on only a new option -amdgpu-atomic-optimizer-use-dpp.

foad added inline comments.Apr 13 2023, 2:38 AM
llvm/test/CodeGen/AMDGPU/atomic_load_add.ll
1 ↗(On Diff #513083)

If I understand correctly, the tests which uses -amdgpu-atomic-optimizations=*true* can be updated with a new option? whereas the tests which uses -amdgpu-atomic-optimizations=*false* should continue to support old option?

Yes that makes sense.

I thought we will be deprecating -amdgpu-atomic-optimizations option completely

That was my first suggestion, but if we have lots of tests that use it then maybe we should keep it just for the tests? I don't feel too strongly about it.

pravinjagtap updated this revision to Diff 513158.Apr 13 2023, 3:47 AM

Rebased and addressed review comment about test flag

pravinjagtap edited the summary of this revision. (Show Details)Apr 13 2023, 3:48 AM
Harbormaster completed remote builds in B225300: Diff 513158.Apr 13 2023, 4:45 AM
arsenm added inline comments.Apr 14 2023, 2:53 PM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The behavior change could be a pass parameter.

foad added inline comments.Apr 16 2023, 11:22 PM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

Two flags to enable the same pass is bad

That's intentional and temporary. It's to avoid a flag-day change for downstream users like LLPC when they switch over to the new option.

pravinjagtap added inline comments.Apr 16 2023, 11:54 PM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The behavior change could be a pass parameter.

1121–1122

The behavior change could be a pass parameter.

If I understand correctly you are suggesting to have string here instead of boolean true or false value as parameter? Something like:

-amdgpu-atomic-optimizer-scan=dpp or -amdgpu-atomic-optimizer-scan=iterative and based on the parameter value of this new command line we will be taking different code paths

pravinjagtap marked 2 inline comments as not done.Apr 17 2023, 12:15 AM
pravinjagtap added inline comments.Apr 19 2023, 2:10 AM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

and this doesn't change the pass behavior

Keeping the pass behavior same was also intentional with new command line argument. This will buy some time for users to adapt to a new command line argument.

arsenm added inline comments.Apr 19 2023, 6:34 AM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

No, I mean move away from command line arguments entirely in favor of the new pass managers pass manager options (e.g. https://github.com/llvm/llvm-project/blob/139f678c7849d90eff715da99d80bbb8b75a79a5/llvm/lib/Transforms/Scalar/SROA.cpp#L5159)

Adding a second flag which simply controls adding the pass to the pipeline as is done here is pointless

foad added inline comments.Apr 19 2023, 12:20 PM
llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
1121–1122

No, I mean move away from command line arguments entirely in favor of the new pass managers pass manager options

What would a frontend like LLPC have to do to set such an option? Currently it calls LLVMTargetMachine::addPassesToEmitFile which calls TargetPassConfig::addISelPasses which calls into GCNPassConfig to create a bunch of passes including AMDGPUAtomicOptimizerPass.

arsenm requested changes to this revision.Jun 27 2023, 3:15 PM

I think this is obsolete now

This revision now requires changes to proceed.Jun 27 2023, 3:15 PM
pravinjagtap abandoned this revision.Jun 28 2023, 1:42 AM

Revision Contents

Diff 513158

llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

Show First 20 Lines • Show All 270 Lines • ▼ Show 20 Lines
// Enable atomic optimization // Enable atomic optimization
static cl::opt<bool> EnableAtomicOptimizations( static cl::opt<bool> EnableAtomicOptimizations(
"amdgpu-atomic-optimizations", "amdgpu-atomic-optimizations",
cl::desc("Enable atomic optimizations"), cl::desc("Enable atomic optimizations"),
cl::init(false), cl::init(false),
cl::Hidden); cl::Hidden);
// Enable atomic optimization with DPP
static cl::opt<bool> EnableAtomicOptimizationsUsingDPP(
"amdgpu-atomic-optimizer-use-dpp",
cl::desc("Enable atomic optimizations using DPP"), cl::init(false),
cl::Hidden);
// Enable Mode register optimization // Enable Mode register optimization
static cl::opt<bool> EnableSIModeRegisterPass( static cl::opt<bool> EnableSIModeRegisterPass(
"amdgpu-mode-register", "amdgpu-mode-register",
cl::desc("Enable mode register pass"), cl::desc("Enable mode register pass"),
cl::init(true), cl::init(true),
cl::Hidden); cl::Hidden);
// Enable GFX11+ s_delay_alu insertion // Enable GFX11+ s_delay_alu insertion
▲ Show 20 Lines • Show All 820 Lines • ▼ Show 20 Lines
} }
bool GCNPassConfig::addPreISel() { bool GCNPassConfig::addPreISel() {
AMDGPUPassConfig::addPreISel(); AMDGPUPassConfig::addPreISel();
if (TM->getOptLevel() > CodeGenOpt::None) if (TM->getOptLevel() > CodeGenOpt::None)
addPass(createAMDGPULateCodeGenPreparePass()); addPass(createAMDGPULateCodeGenPreparePass());
if (isPassEnabled(EnableAtomicOptimizations, CodeGenOpt::Less)) { if (isPassEnabled(EnableAtomicOptimizations, CodeGenOpt::Less) ||
isPassEnabled(EnableAtomicOptimizationsUsingDPP, CodeGenOpt::Less)) {
arsenmUnsubmitted
Not Done
ReplyInline Actions

Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The behavior change could be a pass parameter.

arsenm: Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The…
foadUnsubmitted
Not Done
ReplyInline Actions

Two flags to enable the same pass is bad

That's intentional and temporary. It's to avoid a flag-day change for downstream users like LLPC when they switch over to the new option.

foad: > Two flags to enable the same pass is bad That's intentional and temporary. It's to avoid a…
pravinjagtapAuthorUnsubmitted
Not Done
ReplyInline Actions

Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The behavior change could be a pass parameter.

pravinjagtap: > Two flags to enable the same pass is bad, and this doesn't change the pass behavior. The…
pravinjagtapAuthorUnsubmitted
Not Done
ReplyInline Actions

The behavior change could be a pass parameter.

If I understand correctly you are suggesting to have string here instead of boolean true or false value as parameter? Something like:

-amdgpu-atomic-optimizer-scan=dpp or -amdgpu-atomic-optimizer-scan=iterative and based on the parameter value of this new command line we will be taking different code paths

pravinjagtap: > The behavior change could be a pass parameter. If I understand correctly you are suggesting…
pravinjagtapAuthorUnsubmitted
Done
ReplyInline Actions

and this doesn't change the pass behavior

Keeping the pass behavior same was also intentional with new command line argument. This will buy some time for users to adapt to a new command line argument.

pravinjagtap: > and this doesn't change the pass behavior Keeping the pass behavior same was also intentional…
arsenmUnsubmitted
Not Done
ReplyInline Actions

No, I mean move away from command line arguments entirely in favor of the new pass managers pass manager options (e.g. https://github.com/llvm/llvm-project/blob/139f678c7849d90eff715da99d80bbb8b75a79a5/llvm/lib/Transforms/Scalar/SROA.cpp#L5159)

Adding a second flag which simply controls adding the pass to the pipeline as is done here is pointless

arsenm: No, I mean move away from command line arguments entirely in favor of the new pass managers…
foadUnsubmitted
Not Done
ReplyInline Actions

No, I mean move away from command line arguments entirely in favor of the new pass managers pass manager options

What would a frontend like LLPC have to do to set such an option? Currently it calls LLVMTargetMachine::addPassesToEmitFile which calls TargetPassConfig::addISelPasses which calls into GCNPassConfig to create a bunch of passes including AMDGPUAtomicOptimizerPass.

foad: > No, I mean move away from command line arguments entirely in favor of the new pass managers…
addPass(createAMDGPUAtomicOptimizerPass()); addPass(createAMDGPUAtomicOptimizerPass());
} }
if (TM->getOptLevel() > CodeGenOpt::None) if (TM->getOptLevel() > CodeGenOpt::None)
addPass(createSinkingPass()); addPass(createSinkingPass());
// Merge divergent exit nodes. StructurizeCFG won't recognize the multi-exit // Merge divergent exit nodes. StructurizeCFG won't recognize the multi-exit
// regions formed by them. // regions formed by them.
▲ Show 20 Lines • Show All 483 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_buffer.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -march=amdgcn -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s ; RUN: llc -march=amdgcn -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s
; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s ; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s
declare i32 @llvm.amdgcn.workitem.id.x() declare i32 @llvm.amdgcn.workitem.id.x()
declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32 immarg) declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32 immarg)
declare i32 @llvm.amdgcn.struct.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32, i32 immarg) declare i32 @llvm.amdgcn.struct.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32, i32 immarg)
declare i32 @llvm.amdgcn.raw.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32 immarg) declare i32 @llvm.amdgcn.raw.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32 immarg)
; Show what the atomic optimization pass will do for raw buffers. ; Show what the atomic optimization pass will do for raw buffers.
▲ Show 20 Lines • Show All 2,000 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_global_pointer.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -march=amdgcn -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GFX7LESS %s ; RUN: llc -march=amdgcn -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GFX7LESS %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX8 %s ; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX8 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX9 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX9 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1064 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1064 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1032 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1032 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1164 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1164 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1132 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1132 %s
declare i32 @llvm.amdgcn.workitem.id.x() declare i32 @llvm.amdgcn.workitem.id.x()
; Show what the atomic optimization pass will do for global pointers. ; Show what the atomic optimization pass will do for global pointers.
define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, ptr addrspace(1) %inout) { define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, ptr addrspace(1) %inout) {
; GFX7LESS-LABEL: add_i32_constant: ; GFX7LESS-LABEL: add_i32_constant:
; GFX7LESS: ; %bb.0: ; %entry ; GFX7LESS: ; %bb.0: ; %entry
▲ Show 20 Lines • Show All 3,153 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_local_pointer.ll

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --force-update ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --force-update
; RUN: llc -march=amdgcn -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX7LESS %s ; RUN: llc -march=amdgcn -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX7LESS %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s ; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1064 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1064 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1032 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX1032 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1164 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1164 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1132 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX1132 %s
declare i32 @llvm.amdgcn.workitem.id.x() declare i32 @llvm.amdgcn.workitem.id.x()
@local_var32 = addrspace(3) global i32 undef, align 4 @local_var32 = addrspace(3) global i32 undef, align 4
@local_var64 = addrspace(3) global i64 undef, align 8 @local_var64 = addrspace(3) global i64 undef, align 8
; Show what the atomic optimization pass will do for local pointers. ; Show what the atomic optimization pass will do for local pointers.
▲ Show 20 Lines • Show All 6,743 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_pixelshader.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=amdgcn-- -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX7 %s ; RUN: llc -mtriple=amdgcn-- -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX7 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX8 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=tonga -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX8 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX9 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX89,GFX9 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1064 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1064 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1032 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1032 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1164 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1164 %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizations=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1132 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -mattr=-flat-for-global -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs -simplifycfg-require-and-preserve-domtree=1 < %s | FileCheck -enable-var-scope -check-prefixes=GFX1132 %s
declare i1 @llvm.amdgcn.wqm.vote(i1) declare i1 @llvm.amdgcn.wqm.vote(i1)
declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32 immarg) declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32 immarg)
declare void @llvm.amdgcn.raw.buffer.store.f32(float, <4 x i32>, i32, i32, i32 immarg) declare void @llvm.amdgcn.raw.buffer.store.f32(float, <4 x i32>, i32, i32, i32 immarg)
; Show what the atomic optimization pass will do for raw buffers. ; Show what the atomic optimization pass will do for raw buffers.
define amdgpu_ps void @add_i32_constant(<4 x i32> inreg %out, <4 x i32> inreg %inout) { define amdgpu_ps void @add_i32_constant(<4 x i32> inreg %out, <4 x i32> inreg %inout) {
▲ Show 20 Lines • Show All 620 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_raw_buffer.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -march=amdgcn -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s ; RUN: llc -march=amdgcn -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s
; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s ; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s
declare i32 @llvm.amdgcn.workitem.id.x() declare i32 @llvm.amdgcn.workitem.id.x()
declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32) declare i32 @llvm.amdgcn.raw.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32)
declare i32 @llvm.amdgcn.raw.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32) declare i32 @llvm.amdgcn.raw.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32)
; Show what the atomic optimization pass will do for raw buffers. ; Show what the atomic optimization pass will do for raw buffers.
define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, <4 x i32> %inout) { define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, <4 x i32> %inout) {
▲ Show 20 Lines • Show All 1,651 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/atomic_optimizations_struct_buffer.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -march=amdgcn -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s ; RUN: llc -march=amdgcn -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX6 %s
; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s ; RUN: llc -march=amdgcn -mcpu=tonga -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX8 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX10W32 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-wavefrontsize32,+wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W64 %s
; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizations=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s ; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+wavefrontsize32,-wavefrontsize64 -amdgpu-atomic-optimizer-use-dpp=true -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX11,GFX11W32 %s
declare i32 @llvm.amdgcn.workitem.id.x() declare i32 @llvm.amdgcn.workitem.id.x()
declare i32 @llvm.amdgcn.struct.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32, i32) declare i32 @llvm.amdgcn.struct.buffer.atomic.add(i32, <4 x i32>, i32, i32, i32, i32)
declare i32 @llvm.amdgcn.struct.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32, i32) declare i32 @llvm.amdgcn.struct.buffer.atomic.sub(i32, <4 x i32>, i32, i32, i32, i32)
; Show what the atomic optimization pass will do for struct buffers. ; Show what the atomic optimization pass will do for struct buffers.
define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, <4 x i32> %inout) { define amdgpu_kernel void @add_i32_constant(ptr addrspace(1) %out, <4 x i32> %inout) {
▲ Show 20 Lines • Show All 1,883 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/divergence-at-use.ll

; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizations=true < %s | FileCheck %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -amdgpu-atomic-optimizer-use-dpp=true < %s | FileCheck %s
@local = addrspace(3) global i32 undef @local = addrspace(3) global i32 undef
define amdgpu_kernel void @reducible() { define amdgpu_kernel void @reducible() {
; CHECK-LABEL: reducible: ; CHECK-LABEL: reducible:
; CHECK-NOT: dpp ; CHECK-NOT: dpp
entry: entry:
%x = call i32 @llvm.amdgcn.workitem.id.x() %x = call i32 @llvm.amdgcn.workitem.id.x()
Show All 35 Lines