This is an archive of the discontinued LLVM Phabricator instance.

Differential D89596

[AMDGPU] Update Memory Model in AMDGPUUsage.rst
ClosedPublic

Authored by scott.linder on Oct 16 2020, 2:26 PM.

Details

Reviewers
t-tye
rampitec
kzhuravl
Commits
rGfb37943cc8be: [AMDGPU] Update Memory Model in AMDGPUUsage.rst
Summary

Mostly NFC, but some changes are "bug fixes" rather than just e.g.
formatting changes or typo corrections.

  • Fix typo "competing" -> "completing".
  • Document why waintcnt is added to stores and not loads for sequentially consistent ordering.
  • Lowercase some mentions of buffer_gl{0,1}_inv.
  • Make mentions of *cnt(0) consistently include the (0) count.
  • Remove some mentions of instructions for incorrect address spaces. For example, remove mention of flat_load from load atomic acquire workgroup global.
  • Re-flow some text to get all the target columns to fit in a 32-character wide column. Makes a future NFC patch to make these columns both 32-character wide more straightforward.

Modified cherry-pick of patch by Tony Tye

Diff Detail

Event Timeline

scott.linder created this revision.Oct 16 2020, 2:26 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 16 2020, 2:26 PM
Herald added subscribers: llvm-commits, kerbowa, t-tye and 6 others. · View Herald Transcript
scott.linder requested review of this revision.Oct 16 2020, 2:26 PM
Herald added a subscriber: wdng. · View Herald TranscriptOct 16 2020, 2:26 PM
scott.linder added reviewers: t-tye, rampitec, kzhuravl.Oct 16 2020, 2:27 PM
t-tye accepted this revision.Oct 16 2020, 2:33 PM

LGTM

This revision is now accepted and ready to land.Oct 16 2020, 2:33 PM
Harbormaster completed remote builds in B75380: Diff 298756.Oct 16 2020, 3:46 PM
nhaehnle mentioned this in D89595: [AMDGPU] Update AMDGPUUsage.rst.Oct 19 2020, 10:01 AM
scott.linder updated this revision to Diff 300047.Oct 22 2020, 11:03 AM
scott.linder retitled this revision from [AMDGPU] Update AMDGPUUsage.rst to [AMDGPU] Update Memory Model in AMDGPUUsage.rst.
scott.linder edited the summary of this revision. (Show Details)

Upstream more fixes

Herald added a subscriber: jfb. · View Herald TranscriptOct 22 2020, 11:03 AM
scott.linder added a child revision: D89977: [NFC][AMDGPU] Resize Memory Model columns in AMDGPUUsage.rst.Oct 22 2020, 11:07 AM
Harbormaster completed remote builds in B76066: Diff 300047.Oct 22 2020, 12:00 PM
scott.linder updated this revision to Diff 300075.Oct 22 2020, 12:36 PM

Fix typo

Harbormaster completed remote builds in B76075: Diff 300075.Oct 22 2020, 1:10 PM
t-tye added inline comments.Oct 27 2020, 1:39 PM
llvm/docs/AMDGPUUsage.rst
5081

This row is just global. There is a separate row for generic. So:

2. buffer/global_store     2. buffer/global_store
scott.linder updated this revision to Diff 301385.Oct 28 2020, 12:25 PM
scott.linder edited the summary of this revision. (Show Details)

Address feedback.

Remove more instances of instructions which don't apply to a row's address space.

scott.linder marked an inline comment as done.Oct 28 2020, 12:26 PM
Harbormaster completed remote builds in B76789: Diff 301385.Oct 28 2020, 2:45 PM
t-tye accepted this revision.Oct 29 2020, 2:32 PM

LGTM

This revision was landed with ongoing or failed builds.Oct 29 2020, 4:10 PM
Closed by commit rGfb37943cc8be: [AMDGPU] Update Memory Model in AMDGPUUsage.rst (authored by scott.linder). · Explain Why
This revision was automatically updated to reflect the committed changes.
scott.linder added a commit: rGfb37943cc8be: [AMDGPU] Update Memory Model in AMDGPUUsage.rst.

Revision Contents

PathSize
llvm/
docs/
251 lines

Diff 301781

llvm/docs/AMDGPUUsage.rst

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 4,258 Lines • ▼ Show 20 Lines* The LDS memory has multiple request queues shared by the SIMDs of a
operations performed by the same wavefront. operations performed by the same wavefront.
* The vector memory operations are performed as wavefront wide operations. * The vector memory operations are performed as wavefront wide operations.
Completion of load/store/sample operations are reported to a wavefront in Completion of load/store/sample operations are reported to a wavefront in
execution order of other load/store/sample operations performed by that execution order of other load/store/sample operations performed by that
wavefront. wavefront.
* The vector memory operations access a vector L0 cache. There is a single L0 * The vector memory operations access a vector L0 cache. There is a single L0
cache per CU. Each SIMD of a CU accesses the same L0 cache. Therefore, no cache per CU. Each SIMD of a CU accesses the same L0 cache. Therefore, no
special action is required for coherence between the lanes of a single special action is required for coherence between the lanes of a single
wavefront. However, a ``BUFFER_GL0_INV`` is required for coherence between wavefront. However, a ``buffer_gl0_inv`` is required for coherence between
wavefronts executing in the same work-group as they may be executing on SIMDs wavefronts executing in the same work-group as they may be executing on SIMDs
of different CUs that access different L0s. A ``BUFFER_GL0_INV`` is also of different CUs that access different L0s. A ``buffer_gl0_inv`` is also
required for coherence between wavefronts executing in different work-groups required for coherence between wavefronts executing in different work-groups
as they may be executing on different WGPs. as they may be executing on different WGPs.
* The scalar memory operations access a scalar L0 cache shared by all wavefronts * The scalar memory operations access a scalar L0 cache shared by all wavefronts
on a WGP. The scalar and vector L0 caches are not coherent. However, scalar on a WGP. The scalar and vector L0 caches are not coherent. However, scalar
operations are used in a restricted way so do not impact the memory model. See operations are used in a restricted way so do not impact the memory model. See
:ref:`amdgpu-amdhsa-memory-spaces`. :ref:`amdgpu-amdhsa-memory-spaces`.
* The vector and scalar memory L0 caches use an L1 cache shared by all WGPs on * The vector and scalar memory L0 caches use an L1 cache shared by all WGPs on
the same SA. Therefore, no special action is required for coherence between the same SA. Therefore, no special action is required for coherence between
the wavefronts of a single work-group. However, a ``BUFFER_GL1_INV`` is the wavefronts of a single work-group. However, a ``buffer_gl1_inv`` is
required for coherence between wavefronts executing in different work-groups required for coherence between wavefronts executing in different work-groups
as they may be executing on different SAs that access different L1s. as they may be executing on different SAs that access different L1s.
* The L1 caches have independent quadrants to service disjoint ranges of virtual * The L1 caches have independent quadrants to service disjoint ranges of virtual
addresses. addresses.
* Each L0 cache has a separate request queue per L1 quadrant. Therefore, the * Each L0 cache has a separate request queue per L1 quadrant. Therefore, the
vector and scalar memory operations performed by different wavefronts, whether vector and scalar memory operations performed by different wavefronts, whether
executing in the same or different work-groups (which may be executing on executing in the same or different work-groups (which may be executing on
different CUs accessing different L0s), can be reordered relative to each different CUs accessing different L0s), can be reordered relative to each
▲ Show 20 Lines • Show All 145 Lines • ▼ Show 20 Lines atomicrmw unordered *any* *any* *Same as monotonic *Same as monotonic
atomic*. atomic*. atomic*. atomic*.
**Monotonic Atomic** **Monotonic Atomic**
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
load atomic monotonic - singlethread - global 1. buffer/global/flat_load 1. buffer/global/flat_load load atomic monotonic - singlethread - global 1. buffer/global/flat_load 1. buffer/global/flat_load
- wavefront - generic - wavefront - generic
load atomic monotonic - workgroup - global 1. buffer/global/flat_load 1. buffer/global/flat_load load atomic monotonic - workgroup - global 1. buffer/global/flat_load 1. buffer/global/flat_load
- generic glc=1 - generic glc=1
- If CU wavefront execution mode, omit glc=1. - If CU wavefront execution
mode, omit glc=1.
load atomic monotonic - singlethread - local 1. ds_load 1. ds_load load atomic monotonic - singlethread - local 1. ds_load 1. ds_load
- wavefront - wavefront
- workgroup - workgroup
load atomic monotonic - agent - global 1. buffer/global/flat_load 1. buffer/global/flat_load load atomic monotonic - agent - global 1. buffer/global/flat_load 1. buffer/global/flat_load
- system - generic glc=1 glc=1 dlc=1 - system - generic glc=1 glc=1 dlc=1
store atomic monotonic - singlethread - global 1. buffer/global/flat_store 1. buffer/global/flat_store store atomic monotonic - singlethread - global 1. buffer/global/flat_store 1. buffer/global/flat_store
- wavefront - generic - wavefront - generic
Show All 11 Lines .. table:: AMDHSA Memory Model Code Sequences GFX6-GFX10
atomicrmw monotonic - singlethread - local 1. ds_atomic 1. ds_atomic atomicrmw monotonic - singlethread - local 1. ds_atomic 1. ds_atomic
- wavefront - wavefront
- workgroup - workgroup
**Acquire Atomic** **Acquire Atomic**
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
load atomic acquire - singlethread - global 1. buffer/global/ds/flat_load 1. buffer/global/ds/flat_load load atomic acquire - singlethread - global 1. buffer/global/ds/flat_load 1. buffer/global/ds/flat_load
- wavefront - local - wavefront - local
- generic - generic
load atomic acquire - workgroup - global 1. buffer/global/flat_load 1. buffer/global_load glc=1 load atomic acquire - workgroup - global 1. buffer/global_load 1. buffer/global_load glc=1
- If CU wavefront execution mode, omit glc=1. - If CU wavefront execution
mode, omit glc=1.
2. s_waitcnt vmcnt(0) 2. s_waitcnt vmcnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Must happen before - Must happen before
the following buffer_gl0_inv the following buffer_gl0_inv
and before any following and before any following
global/generic global/generic
load/load load/load
atomic/store/store atomic/store/store
atomic/atomicrmw. atomic/atomicrmw.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
load atomic acquire - workgroup - local 1. ds_load 1. ds_load load atomic acquire - workgroup - local 1. ds_load 1. ds_load
2. s_waitcnt lgkmcnt(0) 2. s_waitcnt lgkmcnt(0) 2. s_waitcnt lgkmcnt(0) 2. s_waitcnt lgkmcnt(0)
- If OpenCL, omit. - If OpenCL, omit. - If OpenCL, omit. - If OpenCL, omit.
- Must happen before - Must happen before - Must happen before - Must happen before
any following the following buffer_gl0_inv any following the following buffer_gl0_inv
global/generic and before any following global/generic and before any following
load/load global/generic load/load load/load global/generic load/load
atomic/store/store atomic/store/store atomic/store/store atomic/store/store
atomic/atomicrmw. atomic/atomicrmw. atomic/atomicrmw. atomic/atomicrmw.
- Ensures any - Ensures any - Ensures any - Ensures any
following global following global following global following global
data read is no data read is no data read is no data read is no
older than the load older than the load older than the load older than the load
atomic value being atomic value being atomic value being atomic value being
acquired. acquired. acquired. acquired.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- If OpenCL, omit. - If OpenCL, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
load atomic acquire - workgroup - generic 1. flat_load 1. flat_load glc=1 load atomic acquire - workgroup - generic 1. flat_load 1. flat_load glc=1
- If CU wavefront execution mode, omit glc=1. - If CU wavefront execution
mode, omit glc=1.
2. s_waitcnt lgkmcnt(0) 2. s_waitcnt lgkmcnt(0) & 2. s_waitcnt lgkmcnt(0) 2. s_waitcnt lgkmcnt(0) &
vmcnt(0) vmcnt(0)
- If CU wavefront execution mode, omit vmcnt. - If CU wavefront execution
mode, omit vmcnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
lgkmcnt(0). lgkmcnt(0).
- Must happen before - Must happen before - Must happen before - Must happen before
any following the following any following the following
global/generic buffer_gl0_inv and any global/generic buffer_gl0_inv and any
load/load following global/generic load/load following global/generic
atomic/store/store load/load atomic/store/store load/load
atomic/atomicrmw. atomic/store/store atomic/atomicrmw. atomic/store/store
atomic/atomicrmw. atomic/atomicrmw.
- Ensures any - Ensures any - Ensures any - Ensures any
following global following global following global following global
data read is no data read is no data read is no data read is no
older than the load older than the load older than the load older than the load
atomic value being atomic value being atomic value being atomic value being
acquired. acquired. acquired. acquired.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
load atomic acquire - agent - global 1. buffer/global/flat_load 1. buffer/global_load load atomic acquire - agent - global 1. buffer/global_load 1. buffer/global_load
- system glc=1 glc=1 dlc=1 - system glc=1 glc=1 dlc=1
2. s_waitcnt vmcnt(0) 2. s_waitcnt vmcnt(0) 2. s_waitcnt vmcnt(0) 2. s_waitcnt vmcnt(0)
- Must happen before - Must happen before - Must happen before - Must happen before
following following following following
buffer_wbinvl1_vol. buffer_gl*_inv. buffer_wbinvl1_vol. buffer_gl*_inv.
- Ensures the load - Ensures the load - Ensures the load - Ensures the load
has completed has completed has completed has completed
Show All 38 Lines load atomic acquire - agent - generic 1. flat_load glc=1 1. flat_load glc=1 dlc=1
- Ensures that - Ensures that - Ensures that - Ensures that
following loads following loads following loads following loads
will not see stale will not see stale will not see stale will not see stale
global data. global data. global data. global data.
atomicrmw acquire - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic atomicrmw acquire - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic
- wavefront - local - wavefront - local
- generic - generic
atomicrmw acquire - workgroup - global 1. buffer/global/flat_atomic 1. buffer/global_atomic atomicrmw acquire - workgroup - global 1. buffer/global_atomic 1. buffer/global_atomic
2. s_waitcnt vm/vscnt(0) 2. s_waitcnt vm/vscnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
- Use vmcnt if atomic with mode, omit.
return and vscnt if atomic - Use vmcnt(0) if atomic with
with no-return. return and vscnt(0) if
atomic with no-return.
- Must happen before - Must happen before
the following buffer_gl0_inv the following buffer_gl0_inv
and before any following and before any following
global/generic global/generic
load/load load/load
atomic/store/store atomic/store/store
atomic/atomicrmw. atomic/atomicrmw.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acquire - workgroup - local 1. ds_atomic 1. ds_atomic atomicrmw acquire - workgroup - local 1. ds_atomic 1. ds_atomic
2. waitcnt lgkmcnt(0) 2. waitcnt lgkmcnt(0) 2. waitcnt lgkmcnt(0) 2. waitcnt lgkmcnt(0)
Show All 18 Lines atomicrmw acquire - workgroup - local 1. ds_atomic 1. ds_atomic
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acquire - workgroup - generic 1. flat_atomic 1. flat_atomic atomicrmw acquire - workgroup - generic 1. flat_atomic 1. flat_atomic
2. waitcnt lgkmcnt(0) 2. waitcnt lgkmcnt(0) & 2. waitcnt lgkmcnt(0) 2. waitcnt lgkmcnt(0) &
vm/vscnt(0) vm/vscnt(0)
- If CU wavefront execution mode, omit vm/vscnt. - If CU wavefront execution
mode, omit vm/vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
waitcnt lgkmcnt(0)..
- Use vmcnt if atomic with
return and vscnt if atomic
with no-return.
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Use vmcnt(0) if atomic with
return and vscnt(0) if
atomic with no-return.
- Must happen before - Must happen before - Must happen before - Must happen before
any following the following any following the following
global/generic buffer_gl0_inv. global/generic buffer_gl0_inv.
load/load load/load
atomic/store/store atomic/store/store
atomic/atomicrmw. atomic/atomicrmw.
- Ensures any - Ensures any - Ensures any - Ensures any
following global following global following global following global
data read is no data read is no data read is no data read is no
older than the older than the older than the older than the
atomicrmw value atomicrmw value atomicrmw value atomicrmw value
being acquired. being acquired. being acquired. being acquired.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acquire - agent - global 1. buffer/global/flat_atomic 1. buffer/global_atomic atomicrmw acquire - agent - global 1. buffer/global_atomic 1. buffer/global_atomic
- system 2. s_waitcnt vmcnt(0) 2. s_waitcnt vm/vscnt(0) - system 2. s_waitcnt vmcnt(0) 2. s_waitcnt vm/vscnt(0)
- Use vmcnt if atomic with - Use vmcnt(0) if atomic with
return and vscnt if atomic return and vscnt(0) if
with no-return. atomic with no-return.
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen before - Must happen before - Must happen before - Must happen before
following following following following
buffer_wbinvl1_vol. buffer_gl*_inv. buffer_wbinvl1_vol. buffer_gl*_inv.
- Ensures the - Ensures the - Ensures the - Ensures the
atomicrmw has atomicrmw has atomicrmw has atomicrmw has
completed before completed before completed before completed before
invalidating the invalidating the invalidating the invalidating the
Show All 13 Lines atomicrmw acquire - agent - global 1. buffer/global_atomic 1. buffer/global_atomic
global data. global data. global data. global data.
atomicrmw acquire - agent - generic 1. flat_atomic 1. flat_atomic atomicrmw acquire - agent - generic 1. flat_atomic 1. flat_atomic
- system 2. s_waitcnt vmcnt(0) & 2. s_waitcnt vm/vscnt(0) & - system 2. s_waitcnt vmcnt(0) & 2. s_waitcnt vm/vscnt(0) &
lgkmcnt(0) lgkmcnt(0) lgkmcnt(0) lgkmcnt(0)
- If OpenCL, omit - If OpenCL, omit - If OpenCL, omit - If OpenCL, omit
lgkmcnt(0). lgkmcnt(0). lgkmcnt(0). lgkmcnt(0).
- Use vmcnt if atomic with - Use vmcnt(0) if atomic with
return and vscnt if atomic return and vscnt(0) if
with no-return. atomic with no-return.
- Must happen before - Must happen before - Must happen before - Must happen before
following following following following
buffer_wbinvl1_vol. buffer_gl*_inv. buffer_wbinvl1_vol. buffer_gl*_inv.
- Ensures the - Ensures the - Ensures the - Ensures the
atomicrmw has atomicrmw has atomicrmw has atomicrmw has
completed before completed before completed before completed before
invalidating the invalidating the invalidating the invalidating the
cache. caches. cache. caches.
Show All 11 Lines atomicrmw acquire - agent - generic 1. flat_atomic 1. flat_atomic
will not see stale will not see stale will not see stale will not see stale
global data. global data. global data. global data.
fence acquire - singlethread *none* *none* *none* fence acquire - singlethread *none* *none* *none*
- wavefront - wavefront
fence acquire - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & fence acquire - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL and - If OpenCL and - If OpenCL and - If OpenCL and
address space is address space is address space is address space is
not generic, omit. not generic, omit not generic, omit. not generic, omit
lgkmcnt(0). lgkmcnt(0).
- If OpenCL and - If OpenCL and
address space is address space is
local, omit local, omit
vmcnt(0) and vscnt(0). vmcnt(0) and vscnt(0).
▲ Show 20 Lines • Show All 94 Lines • ▼ Show 20 Lines fence acquire - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
locations read must locations read must
be no older than be no older than
the value read by the value read by
the the
fence-paired-atomic. fence-paired-atomic.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
fence acquire - agent *none* 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) & fence acquire - agent *none* 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
- system vmcnt(0) vmcnt(0) & vscnt(0) - system vmcnt(0) vmcnt(0) & vscnt(0)
▲ Show 20 Lines • Show All 139 Lines • ▼ Show 20 Lines .. table:: AMDHSA Memory Model Code Sequences GFX6-GFX10
**Release Atomic** **Release Atomic**
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
store atomic release - singlethread - global 1. buffer/global/ds/flat_store 1. buffer/global/ds/flat_store store atomic release - singlethread - global 1. buffer/global/ds/flat_store 1. buffer/global/ds/flat_store
- wavefront - local - wavefront - local
- generic - generic
store atomic release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & store atomic release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
lgkmcnt(0). lgkmcnt(0).
- Must happen after - Must happen after
any preceding any preceding
local/generic local/generic
load/store/load load/store/load
atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw.
Show All 32 Lines store atomic release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
store that is being store that is being store that is being store that is being
released. released. released. released.
2. buffer/global/flat_store 2. buffer/global_store 2. buffer/global_store 2. buffer/global_store
t-tyeUnsubmitted
Done
ReplyInline Actions

This row is just global. There is a separate row for generic. So:

2. buffer/global_store     2. buffer/global_store
t-tye: This row is just global. There is a separate row for generic. So: ``` 2. buffer/global_store…
store atomic release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0) store atomic release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- If OpenCL, omit. - If OpenCL, omit.
- Could be split into - Could be split into
separate s_waitcnt separate s_waitcnt
vmcnt(0) and s_waitcnt vmcnt(0) and s_waitcnt
vscnt(0) to allow vscnt(0) to allow
them to be them to be
independently moved independently moved
according to the according to the
Show All 20 Lines store atomic release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
performing the performing the
store that is being store that is being
released. released.
1. ds_store 2. ds_store 1. ds_store 2. ds_store
store atomic release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & store atomic release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
lgkmcnt(0). lgkmcnt(0).
- Must happen after - Must happen after
any preceding any preceding
local/generic local/generic
load/store/load load/store/load
atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw.
Show All 17 Lines store atomic release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
any preceding any preceding
global/generic global/generic
store/store store/store
atomic/ atomic/
atomicrmw-no-return-value. atomicrmw-no-return-value.
- s_waitcnt lgkmcnt(0) - s_waitcnt lgkmcnt(0)
must happen after must happen after
any preceding any preceding
local/generic load/store/load local/generic
atomic/store atomic/atomicrmw. load/store/load
atomic/store
atomic/atomicrmw.
- Must happen before - Must happen before - Must happen before - Must happen before
the following the following the following the following
store. store. store. store.
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Lines store atomic release - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to memory have to memory have to memory have to memory have
completed before completed before completed before completed before
performing the performing the performing the performing the
store that is being store that is being store that is being store that is being
released. released. released. released.
2. buffer/global/ds/flat_store 2. buffer/global/ds/flat_store 2. buffer/global/flat_store 2. buffer/global/flat_store
atomicrmw release - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic atomicrmw release - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic
- wavefront - local - wavefront - local
- generic - generic
atomicrmw release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & atomicrmw release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit.
- Must happen after - Must happen after
any preceding any preceding
local/generic local/generic
load/store/load load/store/load
atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw.
Show All 32 Lines atomicrmw release - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
atomicrmw that is atomicrmw that is atomicrmw that is atomicrmw that is
being released. being released. being released. being released.
2. buffer/global/flat_atomic 2. buffer/global_atomic 2. buffer/global_atomic 2. buffer/global_atomic
atomicrmw release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0) atomicrmw release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- If OpenCL, omit. - If OpenCL, omit.
- Could be split into - Could be split into
separate s_waitcnt separate s_waitcnt
vmcnt(0) and s_waitcnt vmcnt(0) and s_waitcnt
vscnt(0) to allow vscnt(0) to allow
them to be them to be
independently moved independently moved
according to the according to the
Show All 20 Lines atomicrmw release - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
performing the performing the
store that is being store that is being
released. released.
1. ds_atomic 2. ds_atomic 1. ds_atomic 2. ds_atomic
atomicrmw release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & atomicrmw release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen after - Must happen after
any preceding any preceding
local/generic local/generic
load/store/load load/store/load
atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw.
Show All 17 Lines atomicrmw release - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
any preceding any preceding
global/generic global/generic
store/store store/store
atomic/ atomic/
atomicrmw-no-return-value. atomicrmw-no-return-value.
- s_waitcnt lgkmcnt(0) - s_waitcnt lgkmcnt(0)
must happen after must happen after
any preceding any preceding
local/generic load/store/load local/generic
atomic/store atomic/atomicrmw. load/store/load
atomic/store
atomic/atomicrmw.
- Must happen before - Must happen before - Must happen before - Must happen before
the following the following the following the following
atomicrmw. atomicrmw. atomicrmw. atomicrmw.
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Lines atomicrmw release - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lkkmcnt(0) &
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to global and local to global and local to global and local to global and local
have completed have completed have completed have completed
before performing before performing before performing before performing
the atomicrmw that the atomicrmw that the atomicrmw that the atomicrmw that
is being released. is being released. is being released. is being released.
2. buffer/global/ds/flat_atomic 2. buffer/global/ds/flat_atomic 2. buffer/global/flat_atomic 2. buffer/global/flat_atomic
fence release - singlethread *none* *none* *none* fence release - singlethread *none* *none* *none*
- wavefront - wavefront
fence release - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & fence release - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL and - If OpenCL and - If OpenCL and - If OpenCL and
address space is address space is address space is address space is
not generic, omit. not generic, omit not generic, omit. not generic, omit
lgkmcnt(0). lgkmcnt(0).
- If OpenCL and - If OpenCL and
address space is address space is
local, omit local, omit
vmcnt(0) and vscnt(0). vmcnt(0) and vscnt(0).
▲ Show 20 Lines • Show All 141 Lines • ▼ Show 20 Lines
**Acquire-Release Atomic** **Acquire-Release Atomic**
---------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------------
atomicrmw acq_rel - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic atomicrmw acq_rel - singlethread - global 1. buffer/global/ds/flat_atomic 1. buffer/global/ds/flat_atomic
- wavefront - local - wavefront - local
- generic - generic
atomicrmw acq_rel - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & atomicrmw acq_rel - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
s_waitcnt lgkmcnt(0). s_waitcnt lgkmcnt(0).
- Must happen after - Must happen after - Must happen after - Must happen after
any preceding any preceding any preceding any preceding
local/generic local/generic local/generic local/generic
load/store/load load/store/load load/store/load load/store/load
atomic/store atomic/store atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw. atomic/atomicrmw. atomic/atomicrmw.
Show All 17 Lines atomicrmw acq_rel - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
any preceding any preceding
global/generic global/generic
store/store store/store
atomic/ atomic/
atomicrmw-no-return-value. atomicrmw-no-return-value.
- s_waitcnt lgkmcnt(0) - s_waitcnt lgkmcnt(0)
must happen after must happen after
any preceding any preceding
local/generic load/store/load local/generic
atomic/store atomic/atomicrmw. load/store/load
atomic/store
atomic/atomicrmw.
- Must happen before - Must happen before - Must happen before - Must happen before
the following the following the following the following
atomicrmw. atomicrmw. atomicrmw. atomicrmw.
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
atomicrmw that is atomicrmw that is atomicrmw that is atomicrmw that is
being released. being released. being released. being released.
2. buffer/global/flat_atomic 2. buffer/global_atomic 2. buffer/global_atomic 2. buffer/global_atomic
3. s_waitcnt vm/vscnt(0) 3. s_waitcnt vm/vscnt(0)
- If CU wavefront execution mode, omit vm/vscnt. - If CU wavefront execution
- Use vmcnt if atomic with mode, omit vm/vscnt(0).
return and vscnt if atomic - Use vmcnt(0) if atomic with
with no-return. return and vscnt(0) if
atomic with no-return.
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen before - Must happen before
the following the following
buffer_gl0_inv. buffer_gl0_inv.
- Ensures any - Ensures any
following global following global
data read is no data read is no
older than the older than the
atomicrmw value atomicrmw value
being acquired. being acquired.
4. buffer_gl0_inv 4. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acq_rel - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0) atomicrmw acq_rel - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- If OpenCL, omit. - If OpenCL, omit.
- Could be split into - Could be split into
separate s_waitcnt separate s_waitcnt
vmcnt(0) and s_waitcnt vmcnt(0) and s_waitcnt
vscnt(0) to allow vscnt(0) to allow
them to be them to be
independently moved independently moved
according to the according to the
Show All 35 Lines atomicrmw acq_rel - workgroup - local 1. waitcnt vmcnt(0) & vscnt(0)
following global following global following global following global
data read is no data read is no data read is no data read is no
older than the load older than the load older than the load older than the load
atomic value being atomic value being atomic value being atomic value being
acquired. acquired. acquired. acquired.
4. buffer_gl0_inv 4. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- If OpenCL omit. - If OpenCL omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acq_rel - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & atomicrmw acq_rel - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen after - Must happen after
any preceding any preceding
local/generic local/generic
load/store/load load/store/load
atomic/store atomic/store
atomic/atomicrmw. atomic/atomicrmw.
Show All 17 Lines atomicrmw acq_rel - workgroup - generic 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
any preceding any preceding
global/generic global/generic
store/store store/store
atomic/ atomic/
atomicrmw-no-return-value. atomicrmw-no-return-value.
- s_waitcnt lgkmcnt(0) - s_waitcnt lgkmcnt(0)
must happen after must happen after
any preceding any preceding
local/generic load/store/load local/generic
atomic/store atomic/atomicrmw. load/store/load
atomic/store
atomic/atomicrmw.
- Must happen before - Must happen before - Must happen before - Must happen before
the following the following the following the following
atomicrmw. atomicrmw. atomicrmw. atomicrmw.
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to local have have to local have have
completed before completed before completed before completed before
performing the performing the performing the performing the
atomicrmw that is atomicrmw that is atomicrmw that is atomicrmw that is
being released. being released. being released. being released.
2. flat_atomic 2. flat_atomic 2. flat_atomic 2. flat_atomic
3. s_waitcnt lgkmcnt(0) 3. s_waitcnt lgkmcnt(0) & 3. s_waitcnt lgkmcnt(0) 3. s_waitcnt lgkmcnt(0) &
vm/vscnt(0) vm/vscnt(0)
- If CU wavefront execution mode, omit vm/vscnt. - If CU wavefront execution
mode, omit vm/vscnt(0).
- If OpenCL, omit. - If OpenCL, omit - If OpenCL, omit. - If OpenCL, omit
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen before - Must happen before - Must happen before - Must happen before
any following the following any following the following
global/generic buffer_gl0_inv. global/generic buffer_gl0_inv.
load/load load/load
atomic/store/store atomic/store/store
atomic/atomicrmw. atomic/atomicrmw.
- Ensures any - Ensures any - Ensures any - Ensures any
following global following global following global following global
data read is no data read is no data read is no data read is no
older than the load older than the load older than the load older than the load
atomic value being atomic value being atomic value being atomic value being
acquired. acquired. acquired. acquired.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
atomicrmw acq_rel - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) & atomicrmw acq_rel - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
- system vmcnt(0) vmcnt(0) & vscnt(0) - system vmcnt(0) vmcnt(0) & vscnt(0)
Show All 34 Lines atomicrmw acq_rel - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
- Ensures that all - Ensures that all - Ensures that all - Ensures that all
memory operations memory operations memory operations memory operations
to global have to global have to global have to global have
completed before completed before completed before completed before
performing the performing the performing the performing the
atomicrmw that is atomicrmw that is atomicrmw that is atomicrmw that is
being released. being released. being released. being released.
2. buffer/global/flat_atomic 2. buffer/global_atomic 2. buffer/global_atomic 2. buffer/global_atomic
3. s_waitcnt vmcnt(0) 3. s_waitcnt vm/vscnt(0) 3. s_waitcnt vmcnt(0) 3. s_waitcnt vm/vscnt(0)
- Use vmcnt if atomic with - Use vmcnt(0) if atomic with
return and vscnt if atomic return and vscnt(0) if
with no-return. atomic with no-return.
waitcnt lgkmcnt(0). waitcnt lgkmcnt(0).
- Must happen before - Must happen before - Must happen before - Must happen before
following following following following
buffer_wbinvl1_vol. buffer_gl*_inv. buffer_wbinvl1_vol. buffer_gl*_inv.
- Ensures the - Ensures the - Ensures the - Ensures the
atomicrmw has atomicrmw has atomicrmw has atomicrmw has
completed before completed before completed before completed before
invalidating the invalidating the invalidating the invalidating the
▲ Show 20 Lines • Show All 58 Lines • ▼ Show 20 Lines atomicrmw acq_rel - agent - generic 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
being released. being released. being released. being released.
2. flat_atomic 2. flat_atomic 2. flat_atomic 2. flat_atomic
3. s_waitcnt vmcnt(0) & 3. s_waitcnt vm/vscnt(0) & 3. s_waitcnt vmcnt(0) & 3. s_waitcnt vm/vscnt(0) &
lgkmcnt(0) lgkmcnt(0) lgkmcnt(0) lgkmcnt(0)
- If OpenCL, omit - If OpenCL, omit - If OpenCL, omit - If OpenCL, omit
lgkmcnt(0). lgkmcnt(0). lgkmcnt(0). lgkmcnt(0).
- Use vmcnt if atomic with - Use vmcnt(0) if atomic with
return and vscnt if atomic return and vscnt(0) if
with no-return. atomic with no-return.
- Must happen before - Must happen before - Must happen before - Must happen before
following following following following
buffer_wbinvl1_vol. buffer_gl*_inv. buffer_wbinvl1_vol. buffer_gl*_inv.
- Ensures the - Ensures the - Ensures the - Ensures the
atomicrmw has atomicrmw has atomicrmw has atomicrmw has
completed before completed before completed before completed before
invalidating the invalidating the invalidating the invalidating the
cache. caches. cache. caches.
Show All 11 Lines atomicrmw acq_rel - agent - generic 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
will not see stale will not see stale will not see stale will not see stale
global data. global data. global data. global data.
fence acq_rel - singlethread *none* *none* *none* fence acq_rel - singlethread *none* *none* *none*
- wavefront - wavefront
fence acq_rel - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & fence acq_rel - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
vmcnt(0) & vscnt(0) vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- If OpenCL and - If OpenCL and - If OpenCL and - If OpenCL and
address space is address space is address space is address space is
not generic, omit. not generic, omit not generic, omit. not generic, omit
lgkmcnt(0). lgkmcnt(0).
- If OpenCL and - If OpenCL and
address space is address space is
local, omit local, omit
vmcnt(0) and vscnt(0). vmcnt(0) and vscnt(0).
▲ Show 20 Lines • Show All 102 Lines • ▼ Show 20 Lines fence acq_rel - workgroup *none* 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
locations read must locations read must
be no older than be no older than
the value read by the value read by
the the
acquire-fence-paired-atomic. acquire-fence-paired-atomic.
3. buffer_gl0_inv 3. buffer_gl0_inv
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Ensures that - Ensures that
following following
loads will not see loads will not see
stale data. stale data.
fence acq_rel - agent *none* 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) & fence acq_rel - agent *none* 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
- system vmcnt(0) vmcnt(0) & vscnt(0) - system vmcnt(0) vmcnt(0) & vscnt(0)
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Lines
load atomic seq_cst - singlethread - global *Same as corresponding *Same as corresponding load atomic seq_cst - singlethread - global *Same as corresponding *Same as corresponding
- wavefront - local load atomic acquire, load atomic acquire, - wavefront - local load atomic acquire, load atomic acquire,
- generic except must generated except must generated - generic except must generated except must generated
all instructions even all instructions even all instructions even all instructions even
for OpenCL.* for OpenCL.* for OpenCL.* for OpenCL.*
load atomic seq_cst - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) & load atomic seq_cst - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
- generic vmcnt(0) & vscnt(0) - generic vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit vmcnt and - If CU wavefront execution
vscnt. mode, omit vmcnt(0) and
vscnt(0).
- Could be split into - Could be split into
separate s_waitcnt separate s_waitcnt
vmcnt(0), s_waitcnt vmcnt(0), s_waitcnt
vscnt(0) and s_waitcnt vscnt(0) and s_waitcnt
lgkmcnt(0) to allow lgkmcnt(0) to allow
them to be them to be
independently moved independently moved
according to the according to the
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines load atomic seq_cst - workgroup - global 1. s_waitcnt lgkmcnt(0) 1. s_waitcnt lgkmcnt(0) &
release is release is release is release is
prevented by the prevented by the prevented by the prevented by the
waitcnt of waitcnt of waitcnt of waitcnt of
the release, but the release, but the release, but the release, but
there is nothing there is nothing there is nothing there is nothing
preventing a store preventing a store preventing a store preventing a store
release followed by release followed by release followed by release followed by
load acquire from load acquire from load acquire from load acquire from
competing out of competing out of completing out of completing out of
order.) order.) order. The waitcnt order. The waitcnt
could be placed after could be placed after
seq_store or before seq_store or before
the seq_load. We the seq_load. We
choose the load to choose the load to
make the waitcnt be make the waitcnt be
as late as possible as late as possible
so that the store so that the store
may have already may have already
completed.) completed.)
2. *Following 2. *Following 2. *Following 2. *Following
instructions same as instructions same as instructions same as instructions same as
corresponding load corresponding load corresponding load corresponding load
atomic acquire, atomic acquire, atomic acquire, atomic acquire,
except must generated except must generated except must generated except must generated
all instructions even all instructions even all instructions even all instructions even
for OpenCL.* for OpenCL.* for OpenCL.* for OpenCL.*
load atomic seq_cst - workgroup - local *Same as corresponding load atomic seq_cst - workgroup - local *Same as corresponding
load atomic acquire, load atomic acquire,
except must generated except must generated
all instructions even all instructions even
for OpenCL.* for OpenCL.*
1. s_waitcnt vmcnt(0) & vscnt(0) 1. s_waitcnt vmcnt(0) & vscnt(0)
- If CU wavefront execution mode, omit. - If CU wavefront execution
mode, omit.
- Could be split into - Could be split into
separate s_waitcnt separate s_waitcnt
vmcnt(0) and s_waitcnt vmcnt(0) and s_waitcnt
vscnt(0) to allow vscnt(0) to allow
them to be them to be
independently moved independently moved
according to the according to the
following rules. following rules.
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines load atomic seq_cst - workgroup - local *Same as corresponding
release is release is
prevented by the prevented by the
waitcnt of waitcnt of
the release, but the release, but
there is nothing there is nothing
preventing a store preventing a store
release followed by release followed by
load acquire from load acquire from
competing out of completing out of
order.) order. The waitcnt
could be placed after
seq_store or before
the seq_load. We
choose the load to
make the waitcnt be
as late as possible
so that the store
may have already
completed.)
2. *Following 2. *Following
instructions same as instructions same as
corresponding load corresponding load
atomic acquire, atomic acquire,
except must generated except must generated
all instructions even all instructions even
for OpenCL.* for OpenCL.*
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines load atomic seq_cst - agent - global 1. s_waitcnt lgkmcnt(0) & 1. s_waitcnt lgkmcnt(0) &
release is release is release is release is
prevented by the prevented by the prevented by the prevented by the
waitcnt of waitcnt of waitcnt of waitcnt of
the release, but the release, but the release, but the release, but
there is nothing there is nothing there is nothing there is nothing
preventing a store preventing a store preventing a store preventing a store
release followed by release followed by release followed by release followed by
load acquire from load acquire from load acquire from load acquire from
competing out of competing out of completing out of completing out of
order.) order.) order. The waitcnt order. The waitcnt
could be placed after could be placed after
seq_store or before seq_store or before
the seq_load. We the seq_load. We
choose the load to choose the load to
make the waitcnt be make the waitcnt be
as late as possible as late as possible
so that the store so that the store
may have already may have already
completed.) completed.)
2. *Following 2. *Following 2. *Following 2. *Following
instructions same as instructions same as instructions same as instructions same as
corresponding load corresponding load corresponding load corresponding load
atomic acquire, atomic acquire, atomic acquire, atomic acquire,
except must generated except must generated except must generated except must generated
all instructions even all instructions even all instructions even all instructions even
for OpenCL.* for OpenCL.* for OpenCL.* for OpenCL.*
▲ Show 20 Lines • Show All 1,507 Lines • Show Last 20 Lines