AMDGPU: Add support for cross address space synchronization scopes

[kzhuravl]

This still requires some more tests to be added (which I am working on), and docs updated. I just wanted to get this out here for some early code reviews and testing on hw.

[arsenm]

lib/Target/AMDGPU/AMDGPUMachineModuleInfo.h

72 ↗

(On Diff #191194)

Should expand name, "As" is not clear

94 ↗

(On Diff #191194)

Ditto for the rest

[kzhuravl]

Added docs and tests. Addressed review feedback.

[rampitec]

LGTM

llvm/trunk/docs/AMDGPUUsage.rst

 
 This is different to the OpenCL [OpenCL]_ memory model which does not have scope
 inclusion and requires the memory scopes to exactly match. However, this
 is conservatively correct for OpenCL.
 
   .. table:: AMDHSA LLVM Sync Scopes
      :name: amdgpu-amdhsa-llvm-sync-scopes-table
 
-     ================ ==========================================================
-     LLVM Sync Scope  Description
-     ================ ==========================================================
-     *none*           The default: ``system``.
+     ======================= ===================================================
+     LLVM Sync Scope         Description
+     ======================= ===================================================
+     *none*                  The default: ``system``.
 
-                      Synchronizes with, and participates in modification and
-                      seq_cst total orderings with, other operations (except
-                      image operations) for all address spaces (except private,
-                      or generic that accesses private) provided the other
-                      operation's sync scope is:
+                             Synchronizes with, and participates in modification
+                             and seq_cst total orderings with, other operations
+                             (except image operations) for all address spaces
+                             (except private, or generic that accesses private)
+                             provided the other operation's sync scope is:
 
                              - ``system``.
-                      - ``agent`` and executed by a thread on the same agent.
-                      - ``workgroup`` and executed by a thread in the same
-                        workgroup.
-                      - ``wavefront`` and executed by a thread in the same
-                        wavefront.
+                             - ``agent`` and executed by a thread on the same
+                               agent.
+                             - ``workgroup`` and executed by a thread in the
+                               same workgroup.
+                             - ``wavefront`` and executed by a thread in the
+                               same wavefront.
 
-     ``agent``        Synchronizes with, and participates in modification and
-                      seq_cst total orderings with, other operations (except
-                      image operations) for all address spaces (except private,
-                      or generic that accesses private) provided the other
-                      operation's sync scope is:
-
-                      - ``system`` or ``agent`` and executed by a thread on the
-                        same agent.
-                      - ``workgroup`` and executed by a thread in the same
-                        workgroup.
-                      - ``wavefront`` and executed by a thread in the same
-                        wavefront.
+     ``agent``               Synchronizes with, and participates in modification
+                             and seq_cst total orderings with, other operations
+                             (except image operations) for all address spaces
+                             (except private, or generic that accesses private)
+                             provided the other operation's sync scope is:
+
+                             - ``system`` or ``agent`` and executed by a thread
+                               on the same agent.
+                             - ``workgroup`` and executed by a thread in the
+                               same workgroup.
+                             - ``wavefront`` and executed by a thread in the
+                               same wavefront.
 
-     ``workgroup``    Synchronizes with, and participates in modification and
-                      seq_cst total orderings with, other operations (except
-                      image operations) for all address spaces (except private,
-                      or generic that accesses private) provided the other
-                      operation's sync scope is:
-
-                      - ``system``, ``agent`` or ``workgroup`` and executed by a
-                        thread in the same workgroup.
-                      - ``wavefront`` and executed by a thread in the same
-                        wavefront.
+     ``workgroup``           Synchronizes with, and participates in modification
+                             and seq_cst total orderings with, other operations
+                             (except image operations) for all address spaces
+                             (except private, or generic that accesses private)
+                             provided the other operation's sync scope is:
+
+                             - ``system``, ``agent`` or ``workgroup`` and
+                               executed by a thread in the same workgroup.
+                             - ``wavefront`` and executed by a thread in the
+                               same wavefront.
+
+     ``wavefront``           Synchronizes with, and participates in modification
+                             and seq_cst total orderings with, other operations
+                             (except image operations) for all address spaces
+                             (except private, or generic that accesses private)
+                             provided the other operation's sync scope is:
+
+                             - ``system``, ``agent``, ``workgroup`` or
+                               ``wavefront`` and executed by a thread in the
+                               same wavefront.
+
+     ``singlethread``        Only synchronizes with, and participates in
+                             modification and seq_cst total orderings with,
+                             other operations (except image operations) running
+                             in the same thread for all address spaces (for
+                             example, in signal handlers).
+
+     ``one-as``              Same as ``system`` but only synchronizes with other
+                             operations within the same address space.
+
+     ``agent-one-as``        Same as ``agent`` but only synchronizes with other
+                             operations within the same address space.
+
+     ``workgroup-one-as``    Same as ``workgroup`` but only synchronizes with
+                             other operations within the same address space.
+
+     ``wavefront-one-as``    Same as ``wavefront`` but only synchronizes with
+                             other operations within the same address space.
 
-     ``wavefront``    Synchronizes with, and participates in modification and
-                      seq_cst total orderings with, other operations (except
-                      image operations) for all address spaces (except private,
-                      or generic that accesses private) provided the other
-                      operation's sync scope is:
-
-                      - ``system``, ``agent``, ``workgroup`` or ``wavefront``
-                        and executed by a thread in the same wavefront.
-
-     ``singlethread`` Only synchronizes with, and participates in modification
-                      and seq_cst total orderings with, other operations (except
-                      image operations) running in the same thread for all
-                      address spaces (for example, in signal handlers).
-     ================ ==========================================================
+     ``singlethread-one-as`` Same as ``singlethread`` but only synchronizes with
+                             other operations within the same address space.
+     ======================= ===================================================
 
 AMDGPU Intrinsics
 -----------------
 
 The AMDGPU backend implements the following LLVM IR intrinsics.
 
 *This section is WIP.*
 

llvm/trunk/lib/Target/AMDGPU/AMDGPUMachineModuleInfo.h

 namespace llvm {
 
 class AMDGPUMachineModuleInfo final : public MachineModuleInfoELF {
 private:
 
   // All supported memory/synchronization scopes can be found here:
   //   http://llvm.org/docs/AMDGPUUsage.html#memory-scopes
 
-  /// Agent synchronization scope ID.
+  /// Agent synchronization scope ID (cross address space).
   SyncScope::ID AgentSSID;
-  /// Workgroup synchronization scope ID.
+  /// Workgroup synchronization scope ID (cross address space).
   SyncScope::ID WorkgroupSSID;
-  /// Wavefront synchronization scope ID.
+  /// Wavefront synchronization scope ID (cross address space).
   SyncScope::ID WavefrontSSID;
+  /// System synchronization scope ID (single address space).
+  SyncScope::ID SystemOneAddressSpaceSSID;
+  /// Agent synchronization scope ID (single address space).
+  SyncScope::ID AgentOneAddressSpaceSSID;
+  /// Workgroup synchronization scope ID (single address space).
+  SyncScope::ID WorkgroupOneAddressSpaceSSID;
+  /// Wavefront synchronization scope ID (single address space).
+  SyncScope::ID WavefrontOneAddressSpaceSSID;
+  /// Single thread synchronization scope ID (single address space).
+  SyncScope::ID SingleThreadOneAddressSpaceSSID;
 
   /// In AMDGPU target synchronization scopes are inclusive, meaning a
   /// larger synchronization scope is inclusive of a smaller synchronization
   /// scope.
   ///
   /// \returns \p SSID's inclusion ordering, or "None" if \p SSID is not
   /// supported by the AMDGPU target.
   Optional<uint8_t> getSyncScopeInclusionOrdering(SyncScope::ID SSID) const {
-    if (SSID == SyncScope::SingleThread)
+    if (SSID == SyncScope::SingleThread ||
+        SSID == getSingleThreadOneAddressSpaceSSID())
       return 0;
-    else if (SSID == getWavefrontSSID())
+    else if (SSID == getWavefrontSSID() ||
+             SSID == getWavefrontOneAddressSpaceSSID())
       return 1;
-    else if (SSID == getWorkgroupSSID())
+    else if (SSID == getWorkgroupSSID() ||
+             SSID == getWorkgroupOneAddressSpaceSSID())
       return 2;
-    else if (SSID == getAgentSSID())
+    else if (SSID == getAgentSSID() ||
+             SSID == getAgentOneAddressSpaceSSID())
       return 3;
-    else if (SSID == SyncScope::System)
+    else if (SSID == SyncScope::System ||
+             SSID == getSystemOneAddressSpaceSSID())
       return 4;
 
     return None;
   }
 
+  /// \returns True if \p SSID is restricted to single address space, false
+  /// otherwise
+  bool isOneAddressSpace(SyncScope::ID SSID) const {
+    return SSID == getSingleThreadOneAddressSpaceSSID() ||
+        SSID == getWavefrontOneAddressSpaceSSID() ||
+        SSID == getWorkgroupOneAddressSpaceSSID() ||
+        SSID == getAgentOneAddressSpaceSSID() ||
+        SSID == getSystemOneAddressSpaceSSID();
+  }
+
 public:
   AMDGPUMachineModuleInfo(const MachineModuleInfo &MMI);
 
-  /// \returns Agent synchronization scope ID.
+  /// \returns Agent synchronization scope ID (cross address space).
   SyncScope::ID getAgentSSID() const {
     return AgentSSID;
   }
-  /// \returns Workgroup synchronization scope ID.
+  /// \returns Workgroup synchronization scope ID (cross address space).
   SyncScope::ID getWorkgroupSSID() const {
     return WorkgroupSSID;
   }
-  /// \returns Wavefront synchronization scope ID.
+  /// \returns Wavefront synchronization scope ID (cross address space).
   SyncScope::ID getWavefrontSSID() const {
     return WavefrontSSID;
   }
+  /// \returns System synchronization scope ID (single address space).
+  SyncScope::ID getSystemOneAddressSpaceSSID() const {
+    return SystemOneAddressSpaceSSID;
+  }
+  /// \returns Agent synchronization scope ID (single address space).
+  SyncScope::ID getAgentOneAddressSpaceSSID() const {
+    return AgentOneAddressSpaceSSID;
+  }
+  /// \returns Workgroup synchronization scope ID (single address space).
+  SyncScope::ID getWorkgroupOneAddressSpaceSSID() const {
+    return WorkgroupOneAddressSpaceSSID;
+  }
+  /// \returns Wavefront synchronization scope ID (single address space).
+  SyncScope::ID getWavefrontOneAddressSpaceSSID() const {
+    return WavefrontOneAddressSpaceSSID;
+  }
+  /// \returns Single thread synchronization scope ID (single address space).
+  SyncScope::ID getSingleThreadOneAddressSpaceSSID() const {
+    return SingleThreadOneAddressSpaceSSID;
+  }
 
   /// In AMDGPU target synchronization scopes are inclusive, meaning a
   /// larger synchronization scope is inclusive of a smaller synchronization
   /// scope.
   ///
   /// \returns True if synchronization scope \p A is larger than or equal to
   /// synchronization scope \p B, false if synchronization scope \p A is smaller
   /// than synchronization scope \p B, or "None" if either synchronization scope
   /// \p A or \p B is not supported by the AMDGPU target.
   Optional<bool> isSyncScopeInclusion(SyncScope::ID A, SyncScope::ID B) const {
     const auto &AIO = getSyncScopeInclusionOrdering(A);
     const auto &BIO = getSyncScopeInclusionOrdering(B);
     if (!AIO || !BIO)
       return None;
 
-    return AIO.getValue() > BIO.getValue();
+    bool IsAOneAddressSpace = isOneAddressSpace(A);
+    bool IsBOneAddressSpace = isOneAddressSpace(B);
+
+    return AIO.getValue() >= BIO.getValue() &&
+        (IsAOneAddressSpace == IsBOneAddressSpace || !IsAOneAddressSpace);
   }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_LIB_TARGET_AMDGPU_AMDGPUMACHINEMODULEINFO_H

llvm/trunk/lib/Target/AMDGPU/AMDGPUMachineModuleInfo.cpp

 namespace llvm {
 
 AMDGPUMachineModuleInfo::AMDGPUMachineModuleInfo(const MachineModuleInfo &MMI)
     : MachineModuleInfoELF(MMI) {
   LLVMContext &CTX = MMI.getModule()->getContext();
   AgentSSID = CTX.getOrInsertSyncScopeID("agent");
   WorkgroupSSID = CTX.getOrInsertSyncScopeID("workgroup");
   WavefrontSSID = CTX.getOrInsertSyncScopeID("wavefront");
+  SystemOneAddressSpaceSSID =
+      CTX.getOrInsertSyncScopeID("one-as");
+  AgentOneAddressSpaceSSID =
+      CTX.getOrInsertSyncScopeID("agent-one-as");
+  WorkgroupOneAddressSpaceSSID =
+      CTX.getOrInsertSyncScopeID("workgroup-one-as");
+  WavefrontOneAddressSpaceSSID =
+      CTX.getOrInsertSyncScopeID("wavefront-one-as");
+  SingleThreadOneAddressSpaceSSID =
+      CTX.getOrInsertSyncScopeID("singlethread-one-as");
 }
 
 } // end namespace llvm

llvm/trunk/lib/Target/AMDGPU/SIMemoryLegalizer.cpp

void SIMemOpAccess::reportUnsupported(const MachineBasicBlock::iterator &MI,
   const Function &Func = MI->getParent()->getParent()->getFunction();
   DiagnosticInfoUnsupported Diag(Func, Msg, MI->getDebugLoc());
   Func.getContext().diagnose(Diag);
 }
 
 Optional<std::tuple<SIAtomicScope, SIAtomicAddrSpace, bool>>
 SIMemOpAccess::toSIAtomicScope(SyncScope::ID SSID,
                                SIAtomicAddrSpace InstrScope) const {
-  /// TODO: For now assume OpenCL memory model which treats each
-  /// address space as having a separate happens-before relation, and
-  /// so an instruction only has ordering with respect to the address
-  /// space it accesses, and if it accesses multiple address spaces it
-  /// does not require ordering of operations in different address
-  /// spaces.
   if (SSID == SyncScope::System)
     return std::make_tuple(SIAtomicScope::SYSTEM,
+                           SIAtomicAddrSpace::ATOMIC,
+                           true);
+  if (SSID == MMI->getAgentSSID())
+    return std::make_tuple(SIAtomicScope::AGENT,
+                           SIAtomicAddrSpace::ATOMIC,
+                           true);
+  if (SSID == MMI->getWorkgroupSSID())
+    return std::make_tuple(SIAtomicScope::WORKGROUP,
+                           SIAtomicAddrSpace::ATOMIC,
+                           true);
+  if (SSID == MMI->getWavefrontSSID())
+    return std::make_tuple(SIAtomicScope::WAVEFRONT,
+                           SIAtomicAddrSpace::ATOMIC,
+                           true);
+  if (SSID == SyncScope::SingleThread)
+    return std::make_tuple(SIAtomicScope::SINGLETHREAD,
+                           SIAtomicAddrSpace::ATOMIC,
+                           true);
+  if (SSID == MMI->getSystemOneAddressSpaceSSID())
+    return std::make_tuple(SIAtomicScope::SYSTEM,
                            SIAtomicAddrSpace::ATOMIC & InstrScope,
                            false);
-  if (SSID == MMI->getAgentSSID())
+  if (SSID == MMI->getAgentOneAddressSpaceSSID())
     return std::make_tuple(SIAtomicScope::AGENT,
                            SIAtomicAddrSpace::ATOMIC & InstrScope,
                            false);
-  if (SSID == MMI->getWorkgroupSSID())
+  if (SSID == MMI->getWorkgroupOneAddressSpaceSSID())
     return std::make_tuple(SIAtomicScope::WORKGROUP,
                            SIAtomicAddrSpace::ATOMIC & InstrScope,
                            false);
-  if (SSID == MMI->getWavefrontSSID())
+  if (SSID == MMI->getWavefrontOneAddressSpaceSSID())
     return std::make_tuple(SIAtomicScope::WAVEFRONT,
                            SIAtomicAddrSpace::ATOMIC & InstrScope,
                            false);
-  if (SSID == SyncScope::SingleThread)
+  if (SSID == MMI->getSingleThreadOneAddressSpaceSSID())
     return std::make_tuple(SIAtomicScope::SINGLETHREAD,
                            SIAtomicAddrSpace::ATOMIC & InstrScope,
                            false);
-  /// TODO: To support HSA Memory Model need to add additional memory
-  /// scopes that specify that do require cross address space
-  /// ordering.
   return None;
 }
 
 SIAtomicAddrSpace SIMemOpAccess::toSIAtomicAddrSpace(unsigned AS) const {
   if (AS == AMDGPUAS::FLAT_ADDRESS)
     return SIAtomicAddrSpace::FLAT;
   if (AS == AMDGPUAS::GLOBAL_ADDRESS)
     return SIAtomicAddrSpace::GLOBAL;
bool SIGfx6CacheControl::insertWait(MachineBasicBlock::iterator &MI,
   MachineBasicBlock &MBB = *MI->getParent();
   DebugLoc DL = MI->getDebugLoc();
 
   if (Pos == Position::AFTER)
     ++MI;
 
   bool VMCnt = false;
   bool LGKMCnt = false;
-  bool EXPCnt = false;
 
   if ((AddrSpace & SIAtomicAddrSpace::GLOBAL) != SIAtomicAddrSpace::NONE) {
     switch (Scope) {
     case SIAtomicScope::SYSTEM:
     case SIAtomicScope::AGENT:
-      VMCnt = true;
+      VMCnt |= true;
       break;
     case SIAtomicScope::WORKGROUP:
     case SIAtomicScope::WAVEFRONT:
     case SIAtomicScope::SINGLETHREAD:
       // The L1 cache keeps all memory operations in order for
       // wavefronts in the same work-group.
       break;
     default:
       llvm_unreachable("Unsupported synchronization scope");
     }
   }
 
   if ((AddrSpace & SIAtomicAddrSpace::LDS) != SIAtomicAddrSpace::NONE) {
     switch (Scope) {
     case SIAtomicScope::SYSTEM:
     case SIAtomicScope::AGENT:
     case SIAtomicScope::WORKGROUP:
       // If no cross address space ordering then an LDS waitcnt is not
       // needed as LDS operations for all waves are executed in a
       // total global ordering as observed by all waves. Required if
       // also synchronizing with global/GDS memory as LDS operations
       // could be reordered with respect to later global/GDS memory
       // operations of the same wave.
-      LGKMCnt = IsCrossAddrSpaceOrdering;
+      LGKMCnt |= IsCrossAddrSpaceOrdering;
       break;
     case SIAtomicScope::WAVEFRONT:
     case SIAtomicScope::SINGLETHREAD:
       // The LDS keeps all memory operations in order for
       // the same wavesfront.
       break;
     default:
       llvm_unreachable("Unsupported synchronization scope");
     }
   }
 
   if ((AddrSpace & SIAtomicAddrSpace::GDS) != SIAtomicAddrSpace::NONE) {
     switch (Scope) {
     case SIAtomicScope::SYSTEM:
     case SIAtomicScope::AGENT:
       // If no cross address space ordering then an GDS waitcnt is not
       // needed as GDS operations for all waves are executed in a
       // total global ordering as observed by all waves. Required if
       // also synchronizing with global/LDS memory as GDS operations
       // could be reordered with respect to later global/LDS memory
       // operations of the same wave.
-      EXPCnt = IsCrossAddrSpaceOrdering;
+      LGKMCnt |= IsCrossAddrSpaceOrdering;
       break;
     case SIAtomicScope::WORKGROUP:
     case SIAtomicScope::WAVEFRONT:
     case SIAtomicScope::SINGLETHREAD:
       // The GDS keeps all memory operations in order for
       // the same work-group.
       break;
     default:
       llvm_unreachable("Unsupported synchronization scope");
     }
   }
 
-  if (VMCnt || LGKMCnt || EXPCnt) {
+  if (VMCnt || LGKMCnt) {
     unsigned WaitCntImmediate =
       AMDGPU::encodeWaitcnt(IV,
                             VMCnt ? 0 : getVmcntBitMask(IV),
-                            EXPCnt ? 0 : getExpcntBitMask(IV),
+                            getExpcntBitMask(IV),
                             LGKMCnt ? 0 : getLgkmcntBitMask(IV));
     BuildMI(MBB, MI, DL, TII->get(AMDGPU::S_WAITCNT)).addImm(WaitCntImmediate);
     Changed = true;
   }
 
   if (Pos == Position::AFTER)
     --MI;
 

llvm/trunk/test/CodeGen/AMDGPU/atomicrmw-nand.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -check-prefix=GCN %s
 
 define i32 @atomic_nand_i32_lds(i32 addrspace(3)* %ptr) nounwind {
 ; GCN-LABEL: atomic_nand_i32_lds:
 ; GCN:       ; %bb.0:
 ; GCN-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GCN-NEXT:    ds_read_b32 v2, v0
 ; GCN-NEXT:    s_mov_b64 s[6:7], 0
 ; GCN-NEXT:  BB0_1: ; %atomicrmw.start
 ; GCN-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; GCN-NEXT:    s_waitcnt lgkmcnt(0)
 ; GCN-NEXT:    v_not_b32_e32 v1, v2
 ; GCN-NEXT:    v_or_b32_e32 v1, -5, v1
+; GCN-NEXT:    s_waitcnt vmcnt(0) lgkmcnt(0)
 ; GCN-NEXT:    ds_cmpst_rtn_b32 v1, v0, v2, v1
-; GCN-NEXT:    s_waitcnt lgkmcnt(0)
+; GCN-NEXT:    s_waitcnt vmcnt(0) lgkmcnt(0)
+; GCN-NEXT:    buffer_wbinvl1_vol
 ; GCN-NEXT:    v_cmp_eq_u32_e32 vcc, v1, v2
 ; GCN-NEXT:    v_mov_b32_e32 v2, v1
 ; GCN-NEXT:    s_or_b64 s[6:7], vcc, s[6:7]
 ; GCN-NEXT:    s_andn2_b64 exec, exec, s[6:7]
 ; GCN-NEXT:    s_cbranch_execnz BB0_1
 ; GCN-NEXT:  ; %bb.2: ; %atomicrmw.end
 ; GCN-NEXT:    s_or_b64 exec, exec, s[6:7]
 ; GCN-NEXT:    v_mov_b32_e32 v0, v1

llvm/trunk/test/CodeGen/AMDGPU/llvm.amdgcn.update.dpp.ll

 ; VI-NEXT: v_mov_b32_dpp {{v[0-9]+}}, [[REG]] quad_perm:[1,0,3,2] row_mask:0xf bank_mask:0xf
 @0 = internal unnamed_addr addrspace(3) global [448 x i32] undef, align 4
 define weak_odr amdgpu_kernel void @dpp_test1(i32* %arg) local_unnamed_addr {
 bb:
   %tmp = tail call i32 @llvm.amdgcn.workitem.id.x()
   %tmp1 = zext i32 %tmp to i64
   %tmp2 = getelementptr inbounds [448 x i32], [448 x i32] addrspace(3)* @0, i32 0, i32 %tmp
   %tmp3 = load i32, i32 addrspace(3)* %tmp2, align 4
-  fence syncscope("workgroup") release
+  fence syncscope("workgroup-one-as") release
   tail call void @llvm.amdgcn.s.barrier()
-  fence syncscope("workgroup") acquire
+  fence syncscope("workgroup-one-as") acquire
   %tmp4 = add nsw i32 %tmp3, %tmp3
   %tmp5 = tail call i32 @llvm.amdgcn.update.dpp.i32(i32 0, i32 %tmp4, i32 177, i32 15, i32 15, i1 zeroext false)
   %tmp6 = add nsw i32 %tmp5, %tmp4
   %tmp7 = getelementptr inbounds i32, i32* %arg, i64 %tmp1
   store i32 %tmp6, i32* %tmp7, align 4
   ret void
 }
 
 declare i32 @llvm.amdgcn.workitem.id.x()
 declare void @llvm.amdgcn.s.barrier()
 declare i32 @llvm.amdgcn.update.dpp.i32(i32, i32, i32, i32, i32, i1) #0
 
 attributes #0 = { nounwind readnone convergent }

llvm/trunk/test/CodeGen/AMDGPU/local-atomics-fp.ll

 }
 
 ; GCN-LABEL: {{^}}lds_ds_fadd:
 ; VI-DAG: s_mov_b32 m0
 ; GFX9-NOT: m0
 ; HAS-ATOMICS-DAG: v_mov_b32_e32 [[V0:v[0-9]+]], 0x42280000
 ; HAS-ATOMICS: ds_add_rtn_f32 [[V2:v[0-9]+]], [[V1:v[0-9]+]], [[V0]] offset:32
 ; HAS-ATOMICS: ds_add_f32 [[V3:v[0-9]+]], [[V0]] offset:64
-; HAS-ATOMICS: s_waitcnt lgkmcnt(1)
+; HAS-ATOMICS: s_waitcnt vmcnt(0) lgkmcnt(0)
 ; HAS-ATOMICS: ds_add_rtn_f32 {{v[0-9]+}}, {{v[0-9]+}}, [[V2]]
 define amdgpu_kernel void @lds_ds_fadd(float addrspace(1)* %out, float addrspace(3)* %ptrf, i32 %idx) {
   %idx.add = add nuw i32 %idx, 4
   %shl0 = shl i32 %idx.add, 3
   %shl1 = shl i32 %idx.add, 4
   %ptr0 = inttoptr i32 %shl0 to float addrspace(3)*
   %ptr1 = inttoptr i32 %shl1 to float addrspace(3)*
   %a1 = atomicrmw fadd float addrspace(3)* %ptr0, float 4.2e+1 seq_cst
   %a2 = atomicrmw fadd float addrspace(3)* %ptr1, float 4.2e+1 seq_cst
   %a3 = atomicrmw fadd float addrspace(3)* %ptrf, float %a1 seq_cst
   store float %a3, float addrspace(1)* %out
   ret void
 }
 
+; GCN-LABEL: {{^}}lds_ds_fadd_one_as:
+; VI-DAG: s_mov_b32 m0
+; GFX9-NOT: m0
+; HAS-ATOMICS-DAG: v_mov_b32_e32 [[V0:v[0-9]+]], 0x42280000
+; HAS-ATOMICS: ds_add_rtn_f32 [[V2:v[0-9]+]], [[V1:v[0-9]+]], [[V0]] offset:32
+; HAS-ATOMICS: ds_add_f32 [[V3:v[0-9]+]], [[V0]] offset:64
+; HAS-ATOMICS: s_waitcnt lgkmcnt(1)
+; HAS-ATOMICS: ds_add_rtn_f32 {{v[0-9]+}}, {{v[0-9]+}}, [[V2]]
+define amdgpu_kernel void @lds_ds_fadd_one_as(float addrspace(1)* %out, float addrspace(3)* %ptrf, i32 %idx) {
+  %idx.add = add nuw i32 %idx, 4
+  %shl0 = shl i32 %idx.add, 3
+  %shl1 = shl i32 %idx.add, 4
+  %ptr0 = inttoptr i32 %shl0 to float addrspace(3)*
+  %ptr1 = inttoptr i32 %shl1 to float addrspace(3)*
+  %a1 = atomicrmw fadd float addrspace(3)* %ptr0, float 4.2e+1 syncscope("one-as") seq_cst
+  %a2 = atomicrmw fadd float addrspace(3)* %ptr1, float 4.2e+1 syncscope("one-as") seq_cst
+  %a3 = atomicrmw fadd float addrspace(3)* %ptrf, float %a1 syncscope("one-as") seq_cst
+  store float %a3, float addrspace(1)* %out
+  ret void
+}
+
 ; GCN-LABEL: {{^}}lds_atomic_fadd_ret_f64:
 ; GCN: ds_read_b64
 ; GCN: v_add_f64
 ; GCN: ds_cmpst_rtn_b64
 ; GCN: s_cbranch_execnz
 define double @lds_atomic_fadd_ret_f64(double addrspace(3)* %ptr) nounwind {
   %result = atomicrmw fadd double addrspace(3)* %ptr, double 4.0 seq_cst
   ret double %result

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-amdpal.ll

 ; RUN: llc -mtriple=amdgcn--amdpal -mcpu=gfx600 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--amdpal -mcpu=gfx700 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--amdpal -mcpu=gfx800 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--amdpal -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 
 ; FUNC-LABEL: {{^}}system_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_acquire() {
 entry:
   fence acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_release() {
 entry:
   fence release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_acq_rel() {
 entry:
   fence acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_seq_cst() {
 entry:
   fence seq_cst
   ret void
 }
 
+; FUNC-LABEL: {{^}}system_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acquire() {
+entry:
+  fence syncscope("one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_release() {
+entry:
+  fence syncscope("one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acq_rel() {
+entry:
+  fence syncscope("one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_seq_cst() {
+entry:
+  fence syncscope("one-as") seq_cst
+  ret void
+}
+
 ; FUNC-LABEL: {{^}}singlethread_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @singlethread_acquire() {
 entry:
   fence syncscope("singlethread") acquire
   ret void

 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @singlethread_seq_cst() {
 entry:
   fence syncscope("singlethread") seq_cst
   ret void
 }
 
+; FUNC-LABEL: {{^}}singlethread_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acquire() {
+entry:
+  fence syncscope("singlethread-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_release() {
+entry:
+  fence syncscope("singlethread-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acq_rel() {
+entry:
+  fence syncscope("singlethread-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_seq_cst() {
+entry:
+  fence syncscope("singlethread-one-as") seq_cst
+  ret void
+}
+
 ; FUNC-LABEL: {{^}}agent_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acquire() {
 entry:
   fence syncscope("agent") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_release() {
 entry:
   fence syncscope("agent") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acq_rel() {
 entry:
   fence syncscope("agent") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_seq_cst() {
 entry:
   fence syncscope("agent") seq_cst
   ret void
 }
 
+; FUNC-LABEL: {{^}}agent_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acquire() {
+entry:
+  fence syncscope("agent-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_release() {
+entry:
+  fence syncscope("agent-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acq_rel() {
+entry:
+  fence syncscope("agent-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_seq_cst() {
+entry:
+  fence syncscope("agent-one-as") seq_cst
+  ret void
+}
+
 ; FUNC-LABEL: {{^}}workgroup_acquire:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acquire() {
 entry:
   fence syncscope("workgroup") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_release:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_release() {
 entry:
   fence syncscope("workgroup") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_acq_rel:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acq_rel() {
 entry:
   fence syncscope("workgroup") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_seq_cst:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_seq_cst() {
 entry:
   fence syncscope("workgroup") seq_cst
   ret void
 }
 
+; FUNC-LABEL: {{^}}workgroup_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acquire() {
+entry:
+  fence syncscope("workgroup-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_release() {
+entry:
+  fence syncscope("workgroup-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acq_rel() {
+entry:
+  fence syncscope("workgroup-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_seq_cst() {
+entry:
+  fence syncscope("workgroup-one-as") seq_cst
+  ret void
+}
+
 ; FUNC-LABEL: {{^}}wavefront_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @wavefront_acquire() {
 entry:
   fence syncscope("wavefront") acquire
   ret void

 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @wavefront_seq_cst() {
 entry:
   fence syncscope("wavefront") seq_cst
   ret void
 }
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acquire() {
+entry:
+  fence syncscope("wavefront-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_release() {
+entry:
+  fence syncscope("wavefront-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acq_rel() {
+entry:
+  fence syncscope("wavefront-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_seq_cst() {
+entry:
+  fence syncscope("wavefront-one-as") seq_cst
+  ret void
+}

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-atomic-cmpxchg.ll

 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8 %s
 
 ; GCN-LABEL: {{^}}system_monotonic_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @system_monotonic_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in monotonic monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acquire_monotonic:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acquire_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in acquire monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_release_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @system_release_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in release monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acq_rel_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acq_rel_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in acq_rel monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_seq_cst_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_seq_cst_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in seq_cst monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acquire_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acquire_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in acquire acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_release_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_release_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in release acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acq_rel_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acq_rel_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in acq_rel acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_seq_cst_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_seq_cst_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in seq_cst acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_seq_cst_seq_cst:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_seq_cst_seq_cst(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in seq_cst seq_cst
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_monotonic_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
-; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
-; GCN-NOT:   buffer_wbinvl1_vol
+; GCN-LABEL: {{^}}system_one_as_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_monotonic_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") monotonic monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acquire_monotonic:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acquire_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") acquire monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_release_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_release_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") release monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acq_rel_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acq_rel_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") acq_rel monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_seq_cst_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_seq_cst_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") seq_cst monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acquire_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acquire_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") acquire acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_release_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_release_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") release acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acq_rel_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acq_rel_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") acq_rel acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_seq_cst_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_seq_cst_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") seq_cst acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_seq_cst_seq_cst:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_seq_cst_seq_cst(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("one-as") seq_cst seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_monotonic_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") monotonic monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acquire_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acquire_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") acquire monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_release_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_release_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") release monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acq_rel_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acq_rel_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") acq_rel monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_seq_cst_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_seq_cst_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") seq_cst monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acquire_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acquire_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") acquire acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_release_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_release_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") release acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acq_rel_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acq_rel_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") acq_rel acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_seq_cst_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_seq_cst_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") seq_cst acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_seq_cst_seq_cst:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_seq_cst_seq_cst(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread") seq_cst seq_cst
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_monotonic_monotonic:
+; GCN-LABEL: {{^}}singlethread_one_as_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_monotonic_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") monotonic monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acquire_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acquire_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") acquire monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_release_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_release_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") release monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acq_rel_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acq_rel_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") acq_rel monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_seq_cst_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") seq_cst monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acquire_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acquire_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") acquire acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_release_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_release_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") release acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acq_rel_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acq_rel_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") acq_rel acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_seq_cst_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") seq_cst acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst_seq_cst:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_seq_cst_seq_cst(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("singlethread-one-as") seq_cst seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_monotonic_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") monotonic monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acquire_monotonic:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acquire_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") acquire monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_release_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_release_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") release monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acq_rel_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acq_rel_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") acq_rel monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_seq_cst_monotonic:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_seq_cst_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") seq_cst monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acquire_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acquire_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") acquire acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_release_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_release_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") release acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acq_rel_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acq_rel_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") acq_rel acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_seq_cst_acquire:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_seq_cst_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") seq_cst acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_seq_cst_seq_cst:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_seq_cst_seq_cst(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent") seq_cst seq_cst
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_monotonic_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
-; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-LABEL: {{^}}agent_one_as_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_monotonic_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") monotonic monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acquire_monotonic:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acquire_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") acquire monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_release_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_release_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") release monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acq_rel_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acq_rel_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") acq_rel monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_seq_cst_monotonic:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_seq_cst_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") seq_cst monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acquire_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acquire_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") acquire acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_release_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_release_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") release acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acq_rel_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acq_rel_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") acq_rel acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_seq_cst_acquire:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_seq_cst_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") seq_cst acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_seq_cst_seq_cst:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_seq_cst_seq_cst(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("agent-one-as") seq_cst seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_monotonic_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") monotonic monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acquire_monotonic:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acquire_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") acquire monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_release_monotonic:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_release_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") release monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acq_rel_monotonic:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acq_rel_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") acq_rel monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_seq_cst_monotonic:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_seq_cst_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") seq_cst monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acquire_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acquire_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") acquire acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_release_acquire:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_release_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") release acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acq_rel_acquire:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acq_rel_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") acq_rel acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_seq_cst_acquire:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_seq_cst_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") seq_cst acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_seq_cst_seq_cst:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_seq_cst_seq_cst(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup") seq_cst seq_cst
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_monotonic_monotonic:
+; GCN-LABEL: {{^}}workgroup_one_as_monotonic_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_monotonic_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") monotonic monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acquire_monotonic:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acquire_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") acquire monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_release_monotonic:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_release_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") release monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acq_rel_monotonic:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acq_rel_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") acq_rel monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst_monotonic:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_seq_cst_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") seq_cst monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acquire_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acquire_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") acquire acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_release_acquire:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_release_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") release acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acq_rel_acquire:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acq_rel_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") acq_rel acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst_acquire:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_seq_cst_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") seq_cst acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst_seq_cst:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_seq_cst_seq_cst(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("workgroup-one-as") seq_cst seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_monotonic_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") monotonic monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acquire_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acquire_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") acquire monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_release_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_release_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") release monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acq_rel_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acq_rel_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") acq_rel monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_seq_cst_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_seq_cst_monotonic(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") seq_cst monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acquire_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acquire_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") acquire acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_release_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_release_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") release acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acq_rel_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acq_rel_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") acq_rel acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_seq_cst_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_seq_cst_acquire(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") seq_cst acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_seq_cst_seq_cst:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_seq_cst_seq_cst(
     i32* %out, i32 %in, i32 %old) {
 entry:
   %gep = getelementptr i32, i32* %out, i32 4
   %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront") seq_cst seq_cst
   ret void
 }
+
+; GCN-LABEL: {{^}}wavefront_one_as_monotonic_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_monotonic_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") monotonic monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acquire_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acquire_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") acquire monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_release_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_release_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") release monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acq_rel_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acq_rel_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") acq_rel monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_seq_cst_monotonic(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") seq_cst monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acquire_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acquire_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") acquire acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_release_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_release_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") release acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acq_rel_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acq_rel_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") acq_rel acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_seq_cst_acquire(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") seq_cst acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_cmpswap v[{{[0-9]+\:[0-9]+}}], v[{{[0-9]+}}:{{[0-9]+}}]{{( offset:[0-9]+)*}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_seq_cst_seq_cst(
+    i32* %out, i32 %in, i32 %old) {
+entry:
+  %gep = getelementptr i32, i32* %out, i32 4
+  %val = cmpxchg volatile i32* %gep, i32 %old, i32 %in syncscope("wavefront-one-as") seq_cst seq_cst
+  ret void
+}

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-atomic-fence.ll

 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx600 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN,GFX6,GFX68 %s
 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN,GFX8,GFX68 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN,GFX8,GFX68 %s
 
-; FUNC-LABEL: {{^}}system_acquire:
+; FUNC-LABEL: {{^}}system_one_as_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
 ; GFX6:       s_waitcnt vmcnt(0){{$}}
 ; GFX6-NEXT:  buffer_wbinvl1{{$}}
 ; GFX8:       s_waitcnt vmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acquire() {
+entry:
+  fence syncscope("one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_release() {
+entry:
+  fence syncscope("one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GFX6:       buffer_wbinvl1{{$}}
+; GFX8:       buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acq_rel() {
+entry:
+  fence syncscope("one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GFX6:       buffer_wbinvl1{{$}}
+; GFX8:       buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_seq_cst() {
+entry:
+  fence syncscope("one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acquire() {
+entry:
+  fence syncscope("singlethread-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_release() {
+entry:
+  fence syncscope("singlethread-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acq_rel() {
+entry:
+  fence syncscope("singlethread-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_seq_cst() {
+entry:
+  fence syncscope("singlethread-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GFX6:       s_waitcnt vmcnt(0){{$}}
+; GFX6-NEXT:  buffer_wbinvl1{{$}}
+; GFX8:       s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acquire() {
+entry:
+  fence syncscope("agent-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_release() {
+entry:
+  fence syncscope("agent-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GFX6:       buffer_wbinvl1{{$}}
+; GFX8:       buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acq_rel() {
+entry:
+  fence syncscope("agent-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GFX6:       buffer_wbinvl1{{$}}
+; GFX8:       buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_seq_cst() {
+entry:
+  fence syncscope("agent-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_acquire:
+; GCN:        %bb.0
+; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acquire() {
+entry:
+  fence syncscope("workgroup-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_release:
+; GCN:        %bb.0
+; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_release() {
+entry:
+  fence syncscope("workgroup-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_acq_rel:
+; GCN:        %bb.0
+; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acq_rel() {
+entry:
+  fence syncscope("workgroup-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_seq_cst:
+; GCN:        %bb.0
+; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_seq_cst() {
+entry:
+  fence syncscope("workgroup-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acquire() {
+entry:
+  fence syncscope("wavefront-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_release() {
+entry:
+  fence syncscope("wavefront-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acq_rel() {
+entry:
+  fence syncscope("wavefront-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_seq_cst() {
+entry:
+  fence syncscope("wavefront-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GFX6:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX6-NEXT:  buffer_wbinvl1{{$}}
+; GFX8:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol{{$}}
+; GCN:        s_endpgm
 define amdgpu_kernel void @system_acquire() {
 entry:
   fence acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_release() {
 entry:
   fence release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX6:       buffer_wbinvl1{{$}}
 ; GFX8:       buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_acq_rel() {
 entry:
   fence acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX6:       buffer_wbinvl1{{$}}
 ; GFX8:       buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_seq_cst() {
 entry:
   fence seq_cst
   ret void
 }

 entry:
   fence syncscope("singlethread") seq_cst
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GFX6:       s_waitcnt vmcnt(0){{$}}
+; GFX6:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX6-NEXT:  buffer_wbinvl1{{$}}
-; GFX8:       s_waitcnt vmcnt(0){{$}}
+; GFX8:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acquire() {
 entry:
   fence syncscope("agent") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_release() {
 entry:
   fence syncscope("agent") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX6:       buffer_wbinvl1{{$}}
 ; GFX8:       buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acq_rel() {
 entry:
   fence syncscope("agent") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX6:       buffer_wbinvl1{{$}}
 ; GFX8:       buffer_wbinvl1_vol{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_seq_cst() {
 entry:
   fence syncscope("agent") seq_cst
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_acquire:
 ; GCN:        %bb.0
-; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GFX68-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acquire() {
 entry:
   fence syncscope("workgroup") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_release:
 ; GCN:        %bb.0
-; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GFX68-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_release() {
 entry:
   fence syncscope("workgroup") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_acq_rel:
 ; GCN:        %bb.0
-; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GFX68-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acq_rel() {
 entry:
   fence syncscope("workgroup") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_seq_cst:
 ; GCN:        %bb.0
-; GFX68-NOT:  s_waitcnt vmcnt(0){{$}}
+; GFX68-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_seq_cst() {
 entry:
   fence syncscope("workgroup") seq_cst
   ret void
 }
 

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir

bb.1.atomic:
 
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 15, 0 :: (non-temporal dereferenceable invariant load 4 from `i32 addrspace(4)* undef`)
     dead $vgpr0 = V_MOV_B32_e32 -1, implicit $exec
     dead $vgpr0 = V_MOV_B32_e32 61440, implicit $exec
     $sgpr4_sgpr5 = S_MOV_B64 0
     S_WAITCNT 127
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
     S_WAITCNT 3952
-    BUFFER_ATOMIC_SMAX_ADDR64 killed $vgpr0, killed $vgpr1_vgpr2, killed $sgpr4_sgpr5_sgpr6_sgpr7, 0, 400, 0, implicit $exec :: (volatile load seq_cst 4 from %ir.gep)
+    BUFFER_ATOMIC_SMAX_ADDR64 killed $vgpr0, killed $vgpr1_vgpr2, killed $sgpr4_sgpr5_sgpr6_sgpr7, 0, 400, 0, implicit $exec :: (volatile load syncscope("one-as") seq_cst 4 from %ir.gep)
 
   bb.2.exit:
     liveins: $sgpr2_sgpr3
 
     $exec = S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
     S_ENDPGM 0
 
 ...
 

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-atomic-rmw.ll

 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8 %s
 
-; GCN-LABEL: {{^}}system_monotonic:
+; GCN-LABEL: {{^}}system_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_monotonic(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("one-as") monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acquire(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("one-as") acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_release:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_release(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("one-as") release
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_acq_rel:
+; GCN:         s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:    flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:    s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_acq_rel(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("one-as") acq_rel
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_one_as_seq_cst:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @system_one_as_seq_cst(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("one-as") seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_monotonic(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread-one-as") monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acquire(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread-one-as") acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_release:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_release(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread-one-as") release
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_acq_rel:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_acq_rel(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread-one-as") acq_rel
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @singlethread_one_as_seq_cst(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread-one-as") seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_monotonic(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent-one-as") monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acquire(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent-one-as") acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_release:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_release(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent-one-as") release
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_acq_rel:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_acq_rel(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent-one-as") acq_rel
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_one_as_seq_cst:
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NEXT:  buffer_wbinvl1_vol
+define amdgpu_kernel void @agent_one_as_seq_cst(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent-one-as") seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_monotonic(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup-one-as") monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acquire(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup-one-as") acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_release:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_release(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup-one-as") release
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_acq_rel:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_acq_rel(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup-one-as") acq_rel
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst:
+; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @workgroup_one_as_seq_cst(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup-one-as") seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_monotonic(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront-one-as") monotonic
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acquire(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront-one-as") acquire
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_release:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_release(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront-one-as") release
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_acq_rel:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_acq_rel(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront-one-as") acq_rel
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
+define amdgpu_kernel void @wavefront_one_as_seq_cst(
+    i32* %out, i32 %in) {
+entry:
+  %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront-one-as") seq_cst
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @system_monotonic(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acquire(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_release:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @system_release(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in release
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_acq_rel:
-; GCN:         s_waitcnt vmcnt(0){{$}}
+; GCN:         s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:    flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @system_acq_rel(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in acq_rel
   ret void
 }
 
 ; GCN-LABEL: {{^}}system_seq_cst:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @system_seq_cst(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in seq_cst
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_monotonic(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread") monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acquire(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread") acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_release:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_release(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread") release
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_acq_rel:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_acq_rel(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread") acq_rel
   ret void
 }
 
 ; GCN-LABEL: {{^}}singlethread_seq_cst:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @singlethread_seq_cst(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("singlethread") seq_cst
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_monotonic(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent") monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acquire(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent") acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_release:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_release(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent") release
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_acq_rel:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_acq_rel(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent") acq_rel
   ret void
 }
 
 ; GCN-LABEL: {{^}}agent_seq_cst:
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NEXT:  buffer_wbinvl1_vol
 define amdgpu_kernel void @agent_seq_cst(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("agent") seq_cst
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_monotonic(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup") monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acquire:
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acquire(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup") acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_release:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_release(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup") release
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_acq_rel:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_acq_rel(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup") acq_rel
   ret void
 }
 
 ; GCN-LABEL: {{^}}workgroup_seq_cst:
-; GFX8-NOT:   s_waitcnt vmcnt(0){{$}}
+; GFX8-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GFX8-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @workgroup_seq_cst(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("workgroup") seq_cst
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_monotonic:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_monotonic(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront") monotonic
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acquire:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acquire(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront") acquire
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_release:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_release(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront") release
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_acq_rel:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_acq_rel(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront") acq_rel
   ret void
 }
 
 ; GCN-LABEL: {{^}}wavefront_seq_cst:
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:       flat_atomic_swap v[{{[0-9]+}}:{{[0-9]+}}], v{{[0-9]+}}{{$}}
-; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:   buffer_wbinvl1_vol
 define amdgpu_kernel void @wavefront_seq_cst(
     i32* %out, i32 %in) {
 entry:
   %val = atomicrmw volatile xchg i32* %out, i32 %in syncscope("wavefront") seq_cst
   ret void
 }

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-invalid-addrspace.mir

 # RUN: not llc -march=amdgcn -mcpu=gfx803 -run-pass si-memory-legalizer %s -o - 2>&1 | FileCheck -check-prefix=GCN %s
 
 ---
 
 # GCN: error: <unknown>:0:0: in function invalid_load void (): Unsupported atomic address space
 
 name:            invalid_load
 body:             |
   bb.0:
     liveins: $sgpr0_sgpr1
 
     $vgpr0 = V_MOV_B32_e32 $sgpr2, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr2_sgpr3
     $vgpr1 = V_MOV_B32_e32 killed $sgpr3, implicit $exec, implicit $sgpr2_sgpr3, implicit $exec
-    renamable $vgpr2 = FLAT_LOAD_DWORD killed renamable $vgpr0_vgpr1, 0, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load seq_cst 4 from `i32 addrspace(42)* undef`)
+    renamable $vgpr2 = FLAT_LOAD_DWORD killed renamable $vgpr0_vgpr1, 0, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load syncscope("one-as") seq_cst 4 from `i32 addrspace(42)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN: error: <unknown>:0:0: in function invalid_store void (): Unsupported atomic address space
 
 name:            invalid_store
 body:             |
   bb.0:
     liveins: $sgpr0_sgpr1
 
     $vgpr2 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
-    FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (volatile store syncscope("agent") seq_cst 4 into `i32 addrspace(42)* undef`)
+    FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (volatile store syncscope("agent-one-as") seq_cst 4 into `i32 addrspace(42)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN: error: <unknown>:0:0: in function invalid_cmpxchg void (): Unsupported atomic address space
 
 name:            invalid_cmpxchg
 body:             |
   bb.0:
     liveins: $sgpr0_sgpr1
 
     $vgpr3 = V_MOV_B32_e32 $sgpr1, implicit $exec, implicit-def $vgpr2_vgpr3, implicit $sgpr0_sgpr1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr4, implicit $exec, implicit $exec
     $vgpr1 = V_MOV_B32_e32 killed $sgpr5, implicit $exec, implicit $exec
     $vgpr2 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
-    FLAT_ATOMIC_CMPSWAP killed renamable $vgpr2_vgpr3, killed renamable $vgpr0_vgpr1, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load store syncscope("workgroup") seq_cst seq_cst 4 on `i32 addrspace(42)* undef`)
+    FLAT_ATOMIC_CMPSWAP killed renamable $vgpr2_vgpr3, killed renamable $vgpr0_vgpr1, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load store syncscope("workgroup-one-as") seq_cst seq_cst 4 on `i32 addrspace(42)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN: error: <unknown>:0:0: in function invalid_rmw void (): Unsupported atomic address space
 
 name:            invalid_rmw
 body:             |
   bb.0:
     liveins: $sgpr0_sgpr1
 
     $vgpr0 = V_MOV_B32_e32 $sgpr2, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr2_sgpr3
     $vgpr1 = V_MOV_B32_e32 killed $sgpr3, implicit $exec, implicit $sgpr2_sgpr3, implicit $exec
     $vgpr2 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    FLAT_ATOMIC_SWAP killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load store syncscope("wavefront") seq_cst 4 on `i32 addrspace(42)* undef`)
+    FLAT_ATOMIC_SWAP killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, implicit $exec, implicit $flat_scr :: (volatile load store syncscope("wavefront-one-as") seq_cst 4 on `i32 addrspace(42)* undef`)
     S_ENDPGM 0
 
 ...

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-load.ll

 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck --check-prefixes=GCN,GFX8,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck --check-prefixes=GCN,GFX8,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx900 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck --check-prefixes=GCN,GFX9,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck --check-prefixes=GCN,GFX9,GFX89 %s
 
 declare i32 @llvm.amdgcn.workitem.id.x()
 
-; GCN-LABEL: {{^}}system_unordered:
+; GCN-LABEL: {{^}}system_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @system_unordered(
+define amdgpu_kernel void @system_one_as_unordered(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in unordered, align 4
+  %val = load atomic i32, i32* %in syncscope("one-as") unordered, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}system_monotonic:
+; GCN-LABEL: {{^}}system_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @system_monotonic(
+define amdgpu_kernel void @system_one_as_monotonic(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in monotonic, align 4
+  %val = load atomic i32, i32* %in syncscope("one-as") monotonic, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}system_acquire:
+; GCN-LABEL: {{^}}system_one_as_acquire:
 ; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
 ; GCN:        flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NEXT: buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @system_acquire(
+define amdgpu_kernel void @system_one_as_acquire(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in acquire, align 4
+  %val = load atomic i32, i32* %in syncscope("one-as") acquire, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}system_seq_cst:
+; GCN-LABEL: {{^}}system_one_as_seq_cst:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NEXT: buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @system_seq_cst(
+define amdgpu_kernel void @system_one_as_seq_cst(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in seq_cst, align 4
+  %val = load atomic i32, i32* %in syncscope("one-as") seq_cst, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_unordered:
+; GCN-LABEL: {{^}}singlethread_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @singlethread_unordered(
+define amdgpu_kernel void @singlethread_one_as_unordered(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("singlethread") unordered, align 4
+  %val = load atomic i32, i32* %in syncscope("singlethread-one-as") unordered, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_monotonic:
+; GCN-LABEL: {{^}}singlethread_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @singlethread_monotonic(
+define amdgpu_kernel void @singlethread_one_as_monotonic(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("singlethread") monotonic, align 4
+  %val = load atomic i32, i32* %in syncscope("singlethread-one-as") monotonic, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_acquire:
+; GCN-LABEL: {{^}}singlethread_one_as_acquire:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @singlethread_acquire(
+define amdgpu_kernel void @singlethread_one_as_acquire(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("singlethread") acquire, align 4
+  %val = load atomic i32, i32* %in syncscope("singlethread-one-as") acquire, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_seq_cst:
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @singlethread_seq_cst(
+define amdgpu_kernel void @singlethread_one_as_seq_cst(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("singlethread") seq_cst, align 4
+  %val = load atomic i32, i32* %in syncscope("singlethread-one-as") seq_cst, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_unordered:
+; GCN-LABEL: {{^}}agent_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @agent_unordered(
+define amdgpu_kernel void @agent_one_as_unordered(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("agent") unordered, align 4
+  %val = load atomic i32, i32* %in syncscope("agent-one-as") unordered, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_monotonic:
+; GCN-LABEL: {{^}}agent_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @agent_monotonic(
+define amdgpu_kernel void @agent_one_as_monotonic(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("agent") monotonic, align 4
+  %val = load atomic i32, i32* %in syncscope("agent-one-as") monotonic, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_acquire:
+; GCN-LABEL: {{^}}agent_one_as_acquire:
 ; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
 ; GCN:        flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NEXT: buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @agent_acquire(
+define amdgpu_kernel void @agent_one_as_acquire(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("agent") acquire, align 4
+  %val = load atomic i32, i32* %in syncscope("agent-one-as") acquire, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_seq_cst:
+; GCN-LABEL: {{^}}agent_one_as_seq_cst:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
 ; GCN-NEXT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NEXT: buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @agent_seq_cst(
+define amdgpu_kernel void @agent_one_as_seq_cst(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("agent") seq_cst, align 4
+  %val = load atomic i32, i32* %in syncscope("agent-one-as") seq_cst, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_unordered:
+; GCN-LABEL: {{^}}workgroup_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @workgroup_unordered(
+define amdgpu_kernel void @workgroup_one_as_unordered(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("workgroup") unordered, align 4
+  %val = load atomic i32, i32* %in syncscope("workgroup-one-as") unordered, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_monotonic:
+; GCN-LABEL: {{^}}workgroup_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @workgroup_monotonic(
+define amdgpu_kernel void @workgroup_one_as_monotonic(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("workgroup") monotonic, align 4
+  %val = load atomic i32, i32* %in syncscope("workgroup-one-as") monotonic, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_acquire:
+; GCN-LABEL: {{^}}workgroup_one_as_acquire:
 ; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
 ; GFX89:      flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GFX89-NOT:  s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT:  buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @workgroup_acquire(
+define amdgpu_kernel void @workgroup_one_as_acquire(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("workgroup") acquire, align 4
+  %val = load atomic i32, i32* %in syncscope("workgroup-one-as") acquire, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_seq_cst:
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst:
 ; GFX89-NOT:  s_waitcnt vmcnt(0){{$}}
 ; GFX89:      flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GFX89-NOT:  s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT:  buffer_wbinvl1_vol
 ; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @workgroup_seq_cst(
+define amdgpu_kernel void @workgroup_one_as_seq_cst(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("workgroup") seq_cst, align 4
+  %val = load atomic i32, i32* %in syncscope("workgroup-one-as") seq_cst, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_unordered:
+; GCN-LABEL: {{^}}wavefront_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @wavefront_unordered(
+define amdgpu_kernel void @wavefront_one_as_unordered(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("wavefront") unordered, align 4
+  %val = load atomic i32, i32* %in syncscope("wavefront-one-as") unordered, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_monotonic:
+; GCN-LABEL: {{^}}wavefront_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @wavefront_monotonic(
+define amdgpu_kernel void @wavefront_one_as_monotonic(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("wavefront") monotonic, align 4
+  %val = load atomic i32, i32* %in syncscope("wavefront-one-as") monotonic, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_acquire:
+; GCN-LABEL: {{^}}wavefront_one_as_acquire:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @wavefront_acquire(
+define amdgpu_kernel void @wavefront_one_as_acquire(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("wavefront") acquire, align 4
+  %val = load atomic i32, i32* %in syncscope("wavefront-one-as") acquire, align 4
   store i32 %val, i32* %out
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_seq_cst:
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GFX89-NOT: buffer_wbinvl1_vol
 ; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
-define amdgpu_kernel void @wavefront_seq_cst(
+define amdgpu_kernel void @wavefront_one_as_seq_cst(
     i32* %in, i32* %out) {
 entry:
-  %val = load atomic i32, i32* %in syncscope("wavefront") seq_cst, align 4
+  %val = load atomic i32, i32* %in syncscope("wavefront-one-as") seq_cst, align 4
   store i32 %val, i32* %out
   ret void
 }
 
 ; GCN-LABEL: {{^}}nontemporal_private_0:
 ; GFX89: buffer_load_dword v{{[0-9]+}}, v{{[0-9]+}}, s[{{[0-9]+}}:{{[0-9]+}}], s{{[0-9]+}} offen glc slc{{$}}
 define amdgpu_kernel void @nontemporal_private_0(
     i32 addrspace(5)* %in, i32* %out) {

 entry:
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %val.gep = getelementptr inbounds i32, i32* %in, i32 %tid
   %val = load i32, i32* %val.gep, align 4, !nontemporal !0
   store i32 %val, i32* %out
   ret void
 }
 
+; GCN-LABEL: {{^}}system_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @system_unordered(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in unordered, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @system_monotonic(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in monotonic, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:        flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NEXT: buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @system_acquire(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in acquire, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_seq_cst:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NEXT: buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @system_seq_cst(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in seq_cst, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @singlethread_unordered(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("singlethread") unordered, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @singlethread_monotonic(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("singlethread") monotonic, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @singlethread_acquire(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("singlethread") acquire, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @singlethread_seq_cst(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("singlethread") seq_cst, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @agent_unordered(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("agent") unordered, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @agent_monotonic(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("agent") monotonic, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:        flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NEXT: buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @agent_acquire(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("agent") acquire, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_seq_cst:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}] glc{{$}}
+; GCN-NEXT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NEXT: buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @agent_seq_cst(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("agent") seq_cst, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @workgroup_unordered(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("workgroup") unordered, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89:     flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @workgroup_monotonic(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("workgroup") monotonic, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_acquire:
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89:      flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GFX89:      s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT:  buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @workgroup_acquire(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("workgroup") acquire, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_seq_cst:
+; GFX89-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89:      flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GFX89:      s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT:  buffer_wbinvl1_vol
+; GCN:        flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @workgroup_seq_cst(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("workgroup") seq_cst, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @wavefront_unordered(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("wavefront") unordered, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @wavefront_monotonic(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("wavefront") monotonic, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_acquire:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @wavefront_acquire(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("wavefront") acquire, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_load_dword [[RET:v[0-9]+]], v[{{[0-9]+}}:{{[0-9]+}}]{{$}}
+; GCN:       s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GFX89-NOT: buffer_wbinvl1_vol
+; GCN:       flat_store_dword v{{\[[0-9]+:[0-9]+\]}}, [[RET]]
+define amdgpu_kernel void @wavefront_seq_cst(
+    i32* %in, i32* %out) {
+entry:
+  %val = load atomic i32, i32* %in syncscope("wavefront") seq_cst, align 4
+  store i32 %val, i32* %out
+  ret void
+}
+
 !0 = !{i32 1}

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-local.mir

 
 name:            load_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") unordered 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") unordered 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") monotonic 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") monotonic 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") acquire 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") acquire 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") seq_cst 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") seq_cst 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") unordered 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") unordered 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") monotonic 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") monotonic 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") acquire 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") acquire 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") seq_cst 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") seq_cst 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") unordered 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") unordered 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") monotonic 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") monotonic 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") acquire 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") acquire 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") seq_cst 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") seq_cst 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent") unordered 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") unordered 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent") monotonic 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") monotonic 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent") acquire 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") acquire 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent") seq_cst 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") seq_cst 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load unordered 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") unordered 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load monotonic 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") monotonic 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load acquire 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") acquire 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load seq_cst 4 from `i32 addrspace(3)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 0, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") seq_cst 4 from `i32 addrspace(3)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") unordered 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") monotonic 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") release 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") seq_cst 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") unordered 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") monotonic 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") release 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") seq_cst 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") unordered 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") monotonic 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") release 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") seq_cst 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent") unordered 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent") monotonic 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent") release 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent") seq_cst 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_system_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store unordered 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_system_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store monotonic 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_system_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store release 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_system_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store seq_cst 4 into `i32 addrspace(3)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") unordered 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") unordered 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") monotonic 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") monotonic 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") acquire 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") acquire 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") release 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") release 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_acq_rel
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_acq_rel
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") acq_rel 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") acq_rel 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") seq_cst 4 into `i32 addrspace(3)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 0, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") seq_cst 4 into `i32 addrspace(3)* undef`)
     S_ENDPGM 0
 
 ...

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-mesa3d.ll

 ; RUN: llc -mtriple=amdgcn--mesa3d -mcpu=gfx600 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--mesa3d -mcpu=gfx700 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--mesa3d -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 ; RUN: llc -mtriple=amdgcn--mesa3d -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -check-prefixes=FUNC,GCN %s
 
-; FUNC-LABEL: {{^}}system_acquire:
+; FUNC-LABEL: {{^}}system_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acquire() {
+entry:
+  fence syncscope("one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_release() {
+entry:
+  fence syncscope("one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_acq_rel() {
+entry:
+  fence syncscope("one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @system_one_as_seq_cst() {
+entry:
+  fence syncscope("one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acquire() {
+entry:
+  fence syncscope("singlethread-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_release() {
+entry:
+  fence syncscope("singlethread-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_acq_rel() {
+entry:
+  fence syncscope("singlethread-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}singlethread_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @singlethread_one_as_seq_cst() {
+entry:
+  fence syncscope("singlethread-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN-NEXT:   buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acquire() {
+entry:
+  fence syncscope("agent-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_release() {
+entry:
+  fence syncscope("agent-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_acq_rel() {
+entry:
+  fence syncscope("agent-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}agent_one_as_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        buffer_wbinvl1{{$}}
+; GCN:        s_endpgm
+define amdgpu_kernel void @agent_one_as_seq_cst() {
+entry:
+  fence syncscope("agent-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acquire() {
+entry:
+  fence syncscope("workgroup-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_release() {
+entry:
+  fence syncscope("workgroup-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_acq_rel() {
+entry:
+  fence syncscope("workgroup-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}workgroup_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @workgroup_one_as_seq_cst() {
+entry:
+  fence syncscope("workgroup-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acquire() {
+entry:
+  fence syncscope("wavefront-one-as") acquire
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_release:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_release() {
+entry:
+  fence syncscope("wavefront-one-as") release
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_acq_rel:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_acq_rel() {
+entry:
+  fence syncscope("wavefront-one-as") acq_rel
+  ret void
+}
+
+; FUNC-LABEL: {{^}}wavefront_one_as_seq_cst:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_endpgm
+define amdgpu_kernel void @wavefront_one_as_seq_cst() {
+entry:
+  fence syncscope("wavefront-one-as") seq_cst
+  ret void
+}
+
+; FUNC-LABEL: {{^}}system_acquire:
+; GCN:        %bb.0
+; GCN-NOT:    ATOMIC_FENCE
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_acquire() {
 entry:
   fence acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_release() {
 entry:
   fence release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_acq_rel() {
 entry:
   fence acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}system_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @system_seq_cst() {
 entry:
   fence seq_cst
   ret void
 }
 

 entry:
   fence syncscope("singlethread") seq_cst
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_acquire:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NEXT:   buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acquire() {
 entry:
   fence syncscope("agent") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_release:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_release() {
 entry:
   fence syncscope("agent") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_acq_rel:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_acq_rel() {
 entry:
   fence syncscope("agent") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}agent_seq_cst:
 ; GCN:        %bb.0
 ; GCN-NOT:    ATOMIC_FENCE
-; GCN:        s_waitcnt vmcnt(0){{$}}
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN:        buffer_wbinvl1{{$}}
 ; GCN:        s_endpgm
 define amdgpu_kernel void @agent_seq_cst() {
 entry:
   fence syncscope("agent") seq_cst
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_acquire:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acquire() {
 entry:
   fence syncscope("workgroup") acquire
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_release:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_release() {
 entry:
   fence syncscope("workgroup") release
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_acq_rel:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_acq_rel() {
 entry:
   fence syncscope("workgroup") acq_rel
   ret void
 }
 
 ; FUNC-LABEL: {{^}}workgroup_seq_cst:
 ; GCN:        %bb.0
-; GCN-NOT:    s_waitcnt vmcnt(0){{$}}
+; GCN-NOT:    s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
 ; GCN-NOT:    ATOMIC_FENCE
 ; GCN:        s_endpgm
 define amdgpu_kernel void @workgroup_seq_cst() {
 entry:
   fence syncscope("workgroup") seq_cst
   ret void
 }
 

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-multiple-mem-operands-atomics.mir

bb.1:
     $vgpr0 = V_MOV_B32_e32 4, implicit $exec
 
   bb.3:
     liveins: $sgpr3, $sgpr4_sgpr5, $sgpr8_sgpr9_sgpr10_sgpr11, $vgpr0, $sgpr0
 
     S_WAITCNT 127
     $sgpr0 = S_LSHL_B32 killed $sgpr0, 2, implicit-def dead $scc
     $vgpr0 = V_ADD_I32_e32 killed $sgpr0, killed $vgpr0, implicit-def dead $vcc, implicit $exec
-    $vgpr0 = BUFFER_LOAD_DWORD_OFFEN killed $vgpr0, killed $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr3, 0, 0, 0, 0, implicit $exec :: (load syncscope("agent") unordered 4 from `i32 addrspace(1)* undef`), (load syncscope("workgroup") seq_cst 4 from `[8192 x i32] addrspace(5)* undef`)
+    $vgpr0 = BUFFER_LOAD_DWORD_OFFEN killed $vgpr0, killed $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr3, 0, 0, 0, 0, implicit $exec :: (load syncscope("agent-one-as") unordered 4 from `i32 addrspace(1)* undef`), (load syncscope("workgroup-one-as") seq_cst 4 from `[8192 x i32] addrspace(5)* undef`)
     $vgpr1 = V_MOV_B32_e32 $sgpr4, implicit $exec, implicit-def $vgpr1_vgpr2, implicit $sgpr4_sgpr5
     $vgpr2 = V_MOV_B32_e32 killed $sgpr5, implicit $exec, implicit $sgpr4_sgpr5, implicit $exec
     S_WAITCNT 3952
     FLAT_STORE_DWORD killed $vgpr1_vgpr2, killed $vgpr0, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32 addrspace(1)* undef`)
     S_ENDPGM 0
 
 ...

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-region.mir

 
 name:            load_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 1, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") unordered 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 1, 0, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") unordered 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") monotonic 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") monotonic 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") acquire 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") acquire 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread") seq_cst 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("singlethread-one-as") seq_cst 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") unordered 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") unordered 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") monotonic 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") monotonic 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") acquire 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") acquire 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_wavefront_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_wavefront_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront") seq_cst 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("wavefront-one-as") seq_cst 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") unordered 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") unordered 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") monotonic 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") monotonic 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") acquire 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") acquire 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_workgroup_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_workgroup_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup") seq_cst 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("workgroup-one-as") seq_cst 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent") unordered 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") unordered 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent") monotonic 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") monotonic 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent") acquire 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") acquire 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_agent_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_agent_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent") seq_cst 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("agent-one-as") seq_cst 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load unordered 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") unordered 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load monotonic 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") monotonic 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load acquire 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") acquire 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: load_system_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_READ_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       FLAT_STORE_DWORD
 
 name:            load_system_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM killed $sgpr0_sgpr1, 44, 0 :: (dereferenceable invariant load 8 from `i64 addrspace(4)* undef`, align 4, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr0 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
-    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load seq_cst 4 from `i32 addrspace(2)* undef`)
+    renamable $vgpr2 = DS_READ_B32 killed renamable $vgpr0, 0, 1, implicit $m0, implicit $exec :: (volatile load syncscope("one-as") seq_cst 4 from `i32 addrspace(2)* undef`)
     $vgpr0 = V_MOV_B32_e32 $sgpr0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $sgpr0_sgpr1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr1, implicit $exec, implicit $sgpr0_sgpr1, implicit $exec
     FLAT_STORE_DWORD killed renamable $vgpr0_vgpr1, killed renamable $vgpr2, 0, 0, 0, implicit $exec, implicit $flat_scr :: (store 4 into `i32* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") unordered 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") unordered 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") monotonic 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") monotonic 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") release 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") seq_cst 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") unordered 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") unordered 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") monotonic 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") monotonic 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") release 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_wavefront_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_wavefront_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront") seq_cst 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("wavefront-one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") unordered 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") unordered 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") monotonic 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") monotonic 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") release 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_workgroup_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_workgroup_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup") seq_cst 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("workgroup-one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent") unordered 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") unordered 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 ...
 ---
 
 # GCN-LABEL: name: store_agent_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent") monotonic 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") monotonic 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent") release 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_agent_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_agent_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent") seq_cst 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("agent-one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_unordered
 
 # GCN-LABEL: bb.0:

 name:            store_system_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store release 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: store_system_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRITE_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            store_system_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store seq_cst 4 into `i32 addrspace(2)* undef`)
+    DS_WRITE_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_unordered
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_unordered
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") unordered 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") unordered 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_monotonic
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_monotonic
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") monotonic 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") monotonic 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_acquire
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_acquire
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") acquire 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") acquire 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_release
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_release
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") release 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") release 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_acq_rel
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_acq_rel
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") acq_rel 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") acq_rel 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...
 ---
 
 # GCN-LABEL: name: atomicrmw_singlethread_seq_cst
 
 # GCN-LABEL: bb.0:
 # GCN-NOT:   S_WAITCNT
 # GCN:       DS_WRXCHG_RTN_B32
 # GCN-NOT:   S_WAITCNT
 # GCN:       S_ENDPGM 0
 
 name:            atomicrmw_singlethread_seq_cst
 body:             |
   bb.0:
     $sgpr2 = S_LOAD_DWORD_IMM $sgpr0_sgpr1, 36, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, addrspace 4)
     $sgpr0 = S_LOAD_DWORD_IMM killed $sgpr0_sgpr1, 40, 0 :: (dereferenceable invariant load 4 from `i32 addrspace(4)* undef`, align 8, addrspace 4)
     $m0 = S_MOV_B32 -1
     $vgpr1 = V_MOV_B32_e32 killed $sgpr2, implicit $exec, implicit $exec
     $vgpr0 = V_MOV_B32_e32 killed $sgpr0, implicit $exec, implicit $exec
-    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread") seq_cst 4 into `i32 addrspace(2)* undef`)
+    $vgpr2 = DS_WRXCHG_RTN_B32 killed renamable $vgpr0, killed renamable $vgpr1, 0, 1, implicit $m0, implicit $exec :: (volatile store syncscope("singlethread-one-as") seq_cst 4 into `i32 addrspace(2)* undef`)
     S_ENDPGM 0
 
 ...

llvm/trunk/test/CodeGen/AMDGPU/memory-legalizer-store.ll

 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck --check-prefixes=GCN,GFX8,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -verify-machineinstrs < %s | FileCheck --check-prefixes=GCN,GFX8,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd- -mcpu=gfx900 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck --check-prefixes=GCN,GFX9,GFX89 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck --check-prefixes=GCN,GFX9,GFX89 %s
 
 declare i32 @llvm.amdgcn.workitem.id.x()
 
-; GCN-LABEL: {{^}}system_unordered:
+; GCN-LABEL: {{^}}system_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @system_unordered(
+define amdgpu_kernel void @system_one_as_unordered(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out unordered, align 4
+  store atomic i32 %in, i32* %out syncscope("one-as") unordered, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}system_monotonic:
+; GCN-LABEL: {{^}}system_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @system_monotonic(
+define amdgpu_kernel void @system_one_as_monotonic(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out monotonic, align 4
+  store atomic i32 %in, i32* %out syncscope("one-as") monotonic, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}system_release:
+; GCN-LABEL: {{^}}system_one_as_release:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @system_release(
+define amdgpu_kernel void @system_one_as_release(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out release, align 4
+  store atomic i32 %in, i32* %out syncscope("one-as") release, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}system_seq_cst:
+; GCN-LABEL: {{^}}system_one_as_seq_cst:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @system_seq_cst(
+define amdgpu_kernel void @system_one_as_seq_cst(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out seq_cst, align 4
+  store atomic i32 %in, i32* %out syncscope("one-as") seq_cst, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_unordered:
+; GCN-LABEL: {{^}}singlethread_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @singlethread_unordered(
+define amdgpu_kernel void @singlethread_one_as_unordered(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("singlethread") unordered, align 4
+  store atomic i32 %in, i32* %out syncscope("singlethread-one-as") unordered, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_monotonic:
+; GCN-LABEL: {{^}}singlethread_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @singlethread_monotonic(
+define amdgpu_kernel void @singlethread_one_as_monotonic(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("singlethread") monotonic, align 4
+  store atomic i32 %in, i32* %out syncscope("singlethread-one-as") monotonic, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_release:
+; GCN-LABEL: {{^}}singlethread_one_as_release:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @singlethread_release(
+define amdgpu_kernel void @singlethread_one_as_release(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("singlethread") release, align 4
+  store atomic i32 %in, i32* %out syncscope("singlethread-one-as") release, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}singlethread_seq_cst:
+; GCN-LABEL: {{^}}singlethread_one_as_seq_cst:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @singlethread_seq_cst(
+define amdgpu_kernel void @singlethread_one_as_seq_cst(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("singlethread") seq_cst, align 4
+  store atomic i32 %in, i32* %out syncscope("singlethread-one-as") seq_cst, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_unordered:
+; GCN-LABEL: {{^}}agent_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @agent_unordered(
+define amdgpu_kernel void @agent_one_as_unordered(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("agent") unordered, align 4
+  store atomic i32 %in, i32* %out syncscope("agent-one-as") unordered, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_monotonic:
+; GCN-LABEL: {{^}}agent_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @agent_monotonic(
+define amdgpu_kernel void @agent_one_as_monotonic(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("agent") monotonic, align 4
+  store atomic i32 %in, i32* %out syncscope("agent-one-as") monotonic, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_release:
+; GCN-LABEL: {{^}}agent_one_as_release:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @agent_release(
+define amdgpu_kernel void @agent_one_as_release(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("agent") release, align 4
+  store atomic i32 %in, i32* %out syncscope("agent-one-as") release, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}agent_seq_cst:
+; GCN-LABEL: {{^}}agent_one_as_seq_cst:
 ; GCN:        s_waitcnt vmcnt(0){{$}}
 ; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @agent_seq_cst(
+define amdgpu_kernel void @agent_one_as_seq_cst(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("agent") seq_cst, align 4
+  store atomic i32 %in, i32* %out syncscope("agent-one-as") seq_cst, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_unordered:
+; GCN-LABEL: {{^}}workgroup_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @workgroup_unordered(
+define amdgpu_kernel void @workgroup_one_as_unordered(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("workgroup") unordered, align 4
+  store atomic i32 %in, i32* %out syncscope("workgroup-one-as") unordered, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_monotonic:
+; GCN-LABEL: {{^}}workgroup_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @workgroup_monotonic(
+define amdgpu_kernel void @workgroup_one_as_monotonic(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("workgroup") monotonic, align 4
+  store atomic i32 %in, i32* %out syncscope("workgroup-one-as") monotonic, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_release:
+; GCN-LABEL: {{^}}workgroup_one_as_release:
 ; GFX89-NOT:  s_waitcnt vmcnt(0){{$}}
 ; GCN:        flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @workgroup_release(
+define amdgpu_kernel void @workgroup_one_as_release(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("workgroup") release, align 4
+  store atomic i32 %in, i32* %out syncscope("workgroup-one-as") release, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}workgroup_seq_cst:
+; GCN-LABEL: {{^}}workgroup_one_as_seq_cst:
 ; GFX89-NOT:  s_waitcnt vmcnt(0){{$}}
 ; GCN:        flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @workgroup_seq_cst(
+define amdgpu_kernel void @workgroup_one_as_seq_cst(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("workgroup") seq_cst, align 4
+  store atomic i32 %in, i32* %out syncscope("workgroup-one-as") seq_cst, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_unordered:
+; GCN-LABEL: {{^}}wavefront_one_as_unordered:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @wavefront_unordered(
+define amdgpu_kernel void @wavefront_one_as_unordered(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("wavefront") unordered, align 4
+  store atomic i32 %in, i32* %out syncscope("wavefront-one-as") unordered, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_monotonic:
+; GCN-LABEL: {{^}}wavefront_one_as_monotonic:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @wavefront_monotonic(
+define amdgpu_kernel void @wavefront_one_as_monotonic(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("wavefront") monotonic, align 4
+  store atomic i32 %in, i32* %out syncscope("wavefront-one-as") monotonic, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_release:
+; GCN-LABEL: {{^}}wavefront_one_as_release:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @wavefront_release(
+define amdgpu_kernel void @wavefront_one_as_release(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("wavefront") release, align 4
+  store atomic i32 %in, i32* %out syncscope("wavefront-one-as") release, align 4
   ret void
 }
 
-; GCN-LABEL: {{^}}wavefront_seq_cst:
+; GCN-LABEL: {{^}}wavefront_one_as_seq_cst:
 ; GCN-NOT:   s_waitcnt vmcnt(0){{$}}
 ; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
-define amdgpu_kernel void @wavefront_seq_cst(
+define amdgpu_kernel void @wavefront_one_as_seq_cst(
     i32 %in, i32* %out) {
 entry:
-  store atomic i32 %in, i32* %out syncscope("wavefront") seq_cst, align 4
+  store atomic i32 %in, i32* %out syncscope("wavefront-one-as") seq_cst, align 4
   ret void
 }
 
 ; GCN-LABEL: {{^}}nontemporal_private_0:
 ; GFX89: buffer_store_dword v{{[0-9]+}}, v{{[0-9]+}}, s[{{[0-9]+}}:{{[0-9]+}}], s{{[0-9]+}} offen glc slc{{$}}
 define amdgpu_kernel void @nontemporal_private_0(
     i32* %in, i32 addrspace(5)* %out) {
 entry:

 entry:
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %val = load i32, i32* %in, align 4
   %out.gep = getelementptr inbounds i32, i32* %out, i32 %tid
   store i32 %val, i32* %out.gep, !nontemporal !0
   ret void
 }
 
+; GCN-LABEL: {{^}}system_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @system_unordered(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out unordered, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @system_monotonic(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out monotonic, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_release:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @system_release(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out release, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}system_seq_cst:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @system_seq_cst(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out seq_cst, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @singlethread_unordered(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("singlethread") unordered, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @singlethread_monotonic(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("singlethread") monotonic, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_release:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @singlethread_release(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("singlethread") release, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}singlethread_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @singlethread_seq_cst(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("singlethread") seq_cst, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @agent_unordered(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("agent") unordered, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @agent_monotonic(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("agent") monotonic, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_release:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @agent_release(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("agent") release, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}agent_seq_cst:
+; GCN:        s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN-NEXT:   flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @agent_seq_cst(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("agent") seq_cst, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @workgroup_unordered(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("workgroup") unordered, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @workgroup_monotonic(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("workgroup") monotonic, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_release:
+; GFX89-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:        flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @workgroup_release(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("workgroup") release, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}workgroup_seq_cst:
+; GFX89-NOT:  s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:        flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @workgroup_seq_cst(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("workgroup") seq_cst, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_unordered:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @wavefront_unordered(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("wavefront") unordered, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_monotonic:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @wavefront_monotonic(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("wavefront") monotonic, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_release:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @wavefront_release(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("wavefront") release, align 4
+  ret void
+}
+
+; GCN-LABEL: {{^}}wavefront_seq_cst:
+; GCN-NOT:   s_waitcnt vmcnt(0) lgkmcnt(0){{$}}
+; GCN:       flat_store_dword v[{{[0-9]+}}:{{[0-9]+}}], {{v[0-9]+}}{{$}}
+define amdgpu_kernel void @wavefront_seq_cst(
+    i32 %in, i32* %out) {
+entry:
+  store atomic i32 %in, i32* %out syncscope("wavefront") seq_cst, align 4
+  ret void
+}
+
 !0 = !{i32 1}