[DRAFT] Implementing new atomic orderings in LLVM and generate barriers for legacy __sync builtins. Support corresponding memory model in outline atomics as well.

[ilinpv]

Summary

The patch represents current effort to introduce new "sync_acq", "sync_rel", "sync_seq_cst" atomic orderings in LLVM. They used in sync builtins to generate additional barriers.
It also includes new outline atomics used for these orderings.
The work is in progress. I decided to share it with community early on to get feedback and opinions if it is the right way to implement it. At this moment it is implemented for AArch64 ( aarch64-sync-builtins.ll tests) and I give it a try for RISCV ( but it is not working yet, sync-builtins.ll tests show wrong codegen ). It can affect all targets.
It would be good to know if there are need on other targets to make __sync builtins stonger ( https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html ) like gcc does. I would appriciate any comments and knowledge sharing on this topic. Thank you in advance!

Motivation and requests for this work:
https://reviews.llvm.org/D91157
https://github.com/llvm/llvm-project/issues/29472

[jrtc27]

I can't obviously see a description of what the additional barriers implied by the sync variants is (which should be in an update to LangRef at the very least, if not also in the summary itself). Inferring it from the AArch64 assembly is also difficult, and the RISC-V lowering being identical to the non-sync forms confuses me further. I don't particularly want to trawl through the web of mailing list posts to try and find out which emails have the right information and which aren't relevant, either.

llvm/include/llvm-c/Core.h

360

These comments are not particularly insightful

llvm/include/llvm/IR/Instructions.h

660

You mean SyncSequentiallyConsistent?

670

Are these correct? Without a clear description of what SyncFoo add over and above Foo it's hard to know.

[jyknight]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65697 is the relevant discussion thread that added these to GCC. I personally never found it very convincing, even back in 2015 when they first made the change.

And, now, 7 years later, I'd be even more reluctant to add this to LLVM's core atomic memory model. By now, the __sync_* routines have been obsolete for over a decade. And they never really had a proper memory model description in the first place -- I don't see how they even make sense to use at all on a non-TSO (which is to say: non-X86) memory architecture, since they don't include proper atomic load/store operations.

[efriedma]

atomicrmw add [...] sync_seq_cst is supposed to be equivalent to fence seq_cst; atomicrmw add seq_cst; fence seq_cst, I think.

The extra fences aren't necessary for most uses of __sync_*, but the difference is theoretically visible for exotic synchronization primitives. At least the gcc developers thought the difference might be visible. They therefore decided to add the extra barriers to __sync_* on AArch64 (and maybe other targets?).

As far as I know, the issue is entirely theoretical; nobody has demonstrated any issue on actual hardware, even with a synthetic testcase.

If we do decide we need to do something here, I'd prefer to model this using explicit fences in the IR, as opposed to a new atomic ordering. Adding extra atomic orderings just makes everything harder to reason about.

[Wilco1]

The general requirement is that inline and outline atomics have identical behaviour, and that GCC and LLVM emit the same sequences. I agree sync is badly documented, so it's hard to figure whether an extra DMB barrier could actually make a difference, but it's best to be conservative with atomics. Also it was trivial to add (GCC just adds an extra flag for sync which then emits the extra barrier if the flag is set, so you don't need to introduce new atomic models).

However if sync primitives are hardly used in the real world then perhaps it is about time to deprecate them with annoying warnings, and completely remove support next year. Does that sound reasonable?

[asb]

In D129802#3653544, @efriedma wrote:

If we do decide we need to do something here, I'd prefer to model this using explicit fences in the IR, as opposed to a new atomic ordering. Adding extra atomic orderings just makes everything harder to reason about.

+1. I think the bar for adding new memory orderings should be incredibly high. Memory models are complicated enough without introducing esoteric orderings that aren't clearly specified.

[jyknight]

In D129802#3654943, @Wilco1 wrote:

The general requirement is that inline and outline atomics have identical behaviour, and that GCC and LLVM emit the same sequences. I agree sync is badly documented, so it's hard to figure whether an extra DMB barrier could actually make a difference, but it's best to be conservative with atomics. Also it was trivial to add (GCC just adds an extra flag for sync which then emits the extra barrier if the flag is set, so you don't need to introduce new atomic models).

But it _is_ a new atomic model. If we add this, then for every architecture LLVM supports, now or in the future, we'll need to figure out whether more barriers are needed in the "SYNC" mode vs C++11 standard memory orders. That's a non-trivial thing to ask.

However if sync primitives are hardly used in the real world then perhaps it is about time to deprecate them with annoying warnings, and completely remove support next year. Does that sound reasonable?

I don't know that they are "hardly used" in the real world -- there is certainly legacy code using them, although I haven't attempted to quantify the amount. Another interesting question would be: how much of that code is actually correct, at all. I'd be rather surprised if someone could identify a single piece of software which is actually correct when using the __sync_* operations on GCC, but is incorrect on Clang.

[ilinpv]

@sebpop could you ellaborate on __sync_* operations usage, are you getting issues with current Clang implementation? Do Clang need to keep supporting them and fix introducing new memory model? It seems we need compelling reasons to do that.

However if sync primitives are hardly used in the real world then perhaps it is about time to deprecate them with annoying warnings, and completely remove support next year. Does that sound reasonable?

I don't know that they are "hardly used" in the real world -- there is certainly legacy code using them, although I haven't attempted to quantify the amount. Another interesting question would be: how much of that code is actually correct, at all. I'd be rather surprised if someone could identify a single piece of software which is actually correct when using the __sync_* operations on GCC, but is incorrect on Clang.

clang/lib/CodeGen/CGBuiltin.cpp

 static Value *EmitNontemporalLoad(CodeGenFunction &CGF, const CallExpr *E) {
   Value *Address = CGF.EmitScalarExpr(E->getArg(0));
 
   LValue LV = CGF.MakeNaturalAlignAddrLValue(Address, E->getType());
   LV.setNontemporal(true);
   return CGF.EmitLoadOfScalar(LV, E->getExprLoc());
 }
 
-static RValue EmitBinaryAtomic(CodeGenFunction &CGF,
-                               llvm::AtomicRMWInst::BinOp Kind,
-                               const CallExpr *E) {
-  return RValue::get(MakeBinaryAtomicValue(CGF, Kind, E));
+static RValue EmitBinaryAtomic(
+    CodeGenFunction &CGF, llvm::AtomicRMWInst::BinOp Kind, const CallExpr *E,
+    AtomicOrdering Ordering = AtomicOrdering::SequentiallyConsistent) {
+  return RValue::get(MakeBinaryAtomicValue(CGF, Kind, E, Ordering));
 }
 
 /// Utility to insert an atomic instruction based Intrinsic::ID and
 /// the expression node, where the return value is the result of the
 /// operation.
 static RValue EmitBinaryAtomicPost(CodeGenFunction &CGF,
                                    llvm::AtomicRMWInst::BinOp Kind,
                                    const CallExpr *E,
static RValue EmitBinaryAtomicPost(CodeGenFunction &CGF,
 
   llvm::Value *Args[2];
   Args[1] = CGF.EmitScalarExpr(E->getArg(1));
   llvm::Type *ValueType = Args[1]->getType();
   Args[1] = EmitToInt(CGF, Args[1], T, IntType);
   Args[0] = CGF.Builder.CreateBitCast(DestPtr, IntPtrType);
 
   llvm::Value *Result = CGF.Builder.CreateAtomicRMW(
-      Kind, Args[0], Args[1], llvm::AtomicOrdering::SequentiallyConsistent);
+      Kind, Args[0], Args[1], llvm::AtomicOrdering::SyncSequentiallyConsistent);
   Result = CGF.Builder.CreateBinOp(Op, Result, Args[1]);
   if (Invert)
     Result =
         CGF.Builder.CreateBinOp(llvm::Instruction::Xor, Result,
                                 llvm::ConstantInt::getAllOnesValue(IntType));
   Result = EmitFromInt(CGF, Result, T, ValueType);
   return RValue::get(Result);
 }
 
 /// Utility to insert an atomic cmpxchg instruction.
 ///
 /// @param CGF The current codegen function.
 /// @param E   Builtin call expression to convert to cmpxchg.
 ///            arg0 - address to operate on
 ///            arg1 - value to compare with
 ///            arg2 - new value
 /// @param ReturnBool Specifies whether to return success flag of
 ///                   cmpxchg result or the old value.
 ///
 /// @returns result of cmpxchg, according to ReturnBool
 ///
 /// Note: In order to lower Microsoft's _InterlockedCompareExchange* intrinsics
 /// invoke the function EmitAtomicCmpXchgForMSIntrin.
-static Value *MakeAtomicCmpXchgValue(CodeGenFunction &CGF, const CallExpr *E,
-                                     bool ReturnBool) {
+static Value *
+MakeAtomicCmpXchgValue(CodeGenFunction &CGF, const CallExpr *E, bool ReturnBool,
+                       llvm::AtomicOrdering Ordering =
+                           llvm::AtomicOrdering::SequentiallyConsistent) {
   QualType T = ReturnBool ? E->getArg(1)->getType() : E->getType();
   llvm::Value *DestPtr = CGF.EmitScalarExpr(E->getArg(0));
   unsigned AddrSpace = DestPtr->getType()->getPointerAddressSpace();
 
   llvm::IntegerType *IntType = llvm::IntegerType::get(
       CGF.getLLVMContext(), CGF.getContext().getTypeSize(T));
   llvm::Type *IntPtrType = IntType->getPointerTo(AddrSpace);
 
   Value *Args[3];
   Args[0] = CGF.Builder.CreateBitCast(DestPtr, IntPtrType);
   Args[1] = CGF.EmitScalarExpr(E->getArg(1));
   llvm::Type *ValueType = Args[1]->getType();
   Args[1] = EmitToInt(CGF, Args[1], T, IntType);
   Args[2] = EmitToInt(CGF, CGF.EmitScalarExpr(E->getArg(2)), T, IntType);
 
-  Value *Pair = CGF.Builder.CreateAtomicCmpXchg(
-      Args[0], Args[1], Args[2], llvm::AtomicOrdering::SequentiallyConsistent,
-      llvm::AtomicOrdering::SequentiallyConsistent);
+  Value *Pair = CGF.Builder.CreateAtomicCmpXchg(Args[0], Args[1], Args[2],
+                                                Ordering, Ordering);
   if (ReturnBool)
     // Extract boolean success flag and zext it to int.
     return CGF.Builder.CreateZExt(CGF.Builder.CreateExtractValue(Pair, 1),
                                   CGF.ConvertType(E->getType()));
   else
     // Extract old value and emit it using the same type as compare value.
     return EmitFromInt(CGF, CGF.Builder.CreateExtractValue(Pair, 0), T,
                        ValueType);

   case Builtin::BI__sync_lock_release:
   case Builtin::BI__sync_swap:
     llvm_unreachable("Shouldn't make it through sema");
   case Builtin::BI__sync_fetch_and_add_1:
   case Builtin::BI__sync_fetch_and_add_2:
   case Builtin::BI__sync_fetch_and_add_4:
   case Builtin::BI__sync_fetch_and_add_8:
   case Builtin::BI__sync_fetch_and_add_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Add, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Add, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
   case Builtin::BI__sync_fetch_and_sub_1:
   case Builtin::BI__sync_fetch_and_sub_2:
   case Builtin::BI__sync_fetch_and_sub_4:
   case Builtin::BI__sync_fetch_and_sub_8:
   case Builtin::BI__sync_fetch_and_sub_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Sub, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Sub, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
   case Builtin::BI__sync_fetch_and_or_1:
   case Builtin::BI__sync_fetch_and_or_2:
   case Builtin::BI__sync_fetch_and_or_4:
   case Builtin::BI__sync_fetch_and_or_8:
   case Builtin::BI__sync_fetch_and_or_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Or, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Or, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
   case Builtin::BI__sync_fetch_and_and_1:
   case Builtin::BI__sync_fetch_and_and_2:
   case Builtin::BI__sync_fetch_and_and_4:
   case Builtin::BI__sync_fetch_and_and_8:
   case Builtin::BI__sync_fetch_and_and_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::And, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::And, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
   case Builtin::BI__sync_fetch_and_xor_1:
   case Builtin::BI__sync_fetch_and_xor_2:
   case Builtin::BI__sync_fetch_and_xor_4:
   case Builtin::BI__sync_fetch_and_xor_8:
   case Builtin::BI__sync_fetch_and_xor_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xor, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xor, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
   case Builtin::BI__sync_fetch_and_nand_1:
   case Builtin::BI__sync_fetch_and_nand_2:
   case Builtin::BI__sync_fetch_and_nand_4:
   case Builtin::BI__sync_fetch_and_nand_8:
   case Builtin::BI__sync_fetch_and_nand_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Nand, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Nand, E,
+                            llvm::AtomicOrdering::SyncSequentiallyConsistent);
 
   // Clang extensions: not overloaded yet.
   case Builtin::BI__sync_fetch_and_min:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Min, E);
   case Builtin::BI__sync_fetch_and_max:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Max, E);
   case Builtin::BI__sync_fetch_and_umin:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::UMin, E);
case Builtin::BI__sync_nand_and_fetch_16:
     return EmitBinaryAtomicPost(*this, llvm::AtomicRMWInst::Nand, E,
                                 llvm::Instruction::And, true);
 
   case Builtin::BI__sync_val_compare_and_swap_1:
   case Builtin::BI__sync_val_compare_and_swap_2:
   case Builtin::BI__sync_val_compare_and_swap_4:
   case Builtin::BI__sync_val_compare_and_swap_8:
   case Builtin::BI__sync_val_compare_and_swap_16:
-    return RValue::get(MakeAtomicCmpXchgValue(*this, E, false));
+    return RValue::get(MakeAtomicCmpXchgValue(
+        *this, E, false, llvm::AtomicOrdering::SyncSequentiallyConsistent));
 
   case Builtin::BI__sync_bool_compare_and_swap_1:
   case Builtin::BI__sync_bool_compare_and_swap_2:
   case Builtin::BI__sync_bool_compare_and_swap_4:
   case Builtin::BI__sync_bool_compare_and_swap_8:
   case Builtin::BI__sync_bool_compare_and_swap_16:
-    return RValue::get(MakeAtomicCmpXchgValue(*this, E, true));
+    return RValue::get(MakeAtomicCmpXchgValue(
+        *this, E, true, llvm::AtomicOrdering::SyncSequentiallyConsistent));
 
   case Builtin::BI__sync_swap_1:
   case Builtin::BI__sync_swap_2:
   case Builtin::BI__sync_swap_4:
   case Builtin::BI__sync_swap_8:
   case Builtin::BI__sync_swap_16:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xchg, E);
 
   case Builtin::BI__sync_lock_test_and_set_1:
   case Builtin::BI__sync_lock_test_and_set_2:
   case Builtin::BI__sync_lock_test_and_set_4:
   case Builtin::BI__sync_lock_test_and_set_8:
   case Builtin::BI__sync_lock_test_and_set_16:
-    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xchg, E);
+    return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xchg, E,
+                            llvm::AtomicOrdering::SyncAcquire);
 
   case Builtin::BI__sync_lock_release_1:
   case Builtin::BI__sync_lock_release_2:
   case Builtin::BI__sync_lock_release_4:
   case Builtin::BI__sync_lock_release_8:
   case Builtin::BI__sync_lock_release_16: {
     Value *Ptr = EmitScalarExpr(E->getArg(0));
     QualType ElTy = E->getArg(0)->getType()->getPointeeType();
     CharUnits StoreSize = getContext().getTypeSizeInChars(ElTy);
     llvm::Type *ITy = llvm::IntegerType::get(getLLVMContext(),
                                              StoreSize.getQuantity() * 8);
     Ptr = Builder.CreateBitCast(Ptr, ITy->getPointerTo());
     llvm::StoreInst *Store =
       Builder.CreateAlignedStore(llvm::Constant::getNullValue(ITy), Ptr,
                                  StoreSize);
-    Store->setAtomic(llvm::AtomicOrdering::Release);
+    Store->setAtomic(llvm::AtomicOrdering::SyncRelease);
     return RValue::get(nullptr);
   }
 
   case Builtin::BI__sync_synchronize: {
     // We assume this is supposed to correspond to a C++0x-style
     // sequentially-consistent fence (i.e. this is only usable for
     // synchronization, not device I/O or anything like that). This intrinsic
     // is really badly designed in the sense that in theory, there isn't

clang/lib/CodeGen/CGStmtOpenMP.cpp

static void emitOMPAtomicReadExpr(CodeGenFunction &CGF, llvm::AtomicOrdering AO,
   LValue VLValue = CGF.EmitLValue(V);
   RValue Res = emitSimpleAtomicLoad(CGF, AO, XLValue, Loc);
   // OpenMP, 2.17.7, atomic Construct
   // If the read or capture clause is specified and the acquire, acq_rel, or
   // seq_cst clause is specified then the strong flush on exit from the atomic
   // operation is also an acquire flush.
   switch (AO) {
   case llvm::AtomicOrdering::Acquire:
+  case llvm::AtomicOrdering::SyncAcquire:
   case llvm::AtomicOrdering::AcquireRelease:
   case llvm::AtomicOrdering::SequentiallyConsistent:
+  case llvm::AtomicOrdering::SyncSequentiallyConsistent:
     CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc,
                                          llvm::AtomicOrdering::Acquire);
     break;
   case llvm::AtomicOrdering::Monotonic:
   case llvm::AtomicOrdering::Release:
+  case llvm::AtomicOrdering::SyncRelease:
     break;
   case llvm::AtomicOrdering::NotAtomic:
   case llvm::AtomicOrdering::Unordered:
     llvm_unreachable("Unexpected ordering.");
   }
   CGF.emitOMPSimpleStore(VLValue, Res, X->getType().getNonReferenceType(), Loc);
   CGF.CGM.getOpenMPRuntime().checkAndEmitLastprivateConditional(CGF, V);
 }
 
 static void emitOMPAtomicWriteExpr(CodeGenFunction &CGF,
                                    llvm::AtomicOrdering AO, const Expr *X,
                                    const Expr *E, SourceLocation Loc) {
   // x = expr;
   assert(X->isLValue() && "X of 'omp atomic write' is not lvalue");
   emitSimpleAtomicStore(CGF, AO, CGF.EmitLValue(X), CGF.EmitAnyExpr(E));
   CGF.CGM.getOpenMPRuntime().checkAndEmitLastprivateConditional(CGF, X);
   // OpenMP, 2.17.7, atomic Construct
   // If the write, update, or capture clause is specified and the release,
   // acq_rel, or seq_cst clause is specified then the strong flush on entry to
   // the atomic operation is also a release flush.
   switch (AO) {
   case llvm::AtomicOrdering::Release:
+  case llvm::AtomicOrdering::SyncRelease:
   case llvm::AtomicOrdering::AcquireRelease:
   case llvm::AtomicOrdering::SequentiallyConsistent:
+  case llvm::AtomicOrdering::SyncSequentiallyConsistent:
     CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc,
                                          llvm::AtomicOrdering::Release);
     break;
   case llvm::AtomicOrdering::Acquire:
+  case llvm::AtomicOrdering::SyncAcquire:
   case llvm::AtomicOrdering::Monotonic:
     break;
   case llvm::AtomicOrdering::NotAtomic:
   case llvm::AtomicOrdering::Unordered:
     llvm_unreachable("Unexpected ordering.");
   }
 }
 
(void)CGF.EmitOMPAtomicSimpleUpdateExpr(
       XLValue, ExprRValue, BOUE->getOpcode(), IsXLHSInRHSPart, AO, Loc, Gen);
   CGF.CGM.getOpenMPRuntime().checkAndEmitLastprivateConditional(CGF, X);
   // OpenMP, 2.17.7, atomic Construct
   // If the write, update, or capture clause is specified and the release,
   // acq_rel, or seq_cst clause is specified then the strong flush on entry to
   // the atomic operation is also a release flush.
   switch (AO) {
   case llvm::AtomicOrdering::Release:
+  case llvm::AtomicOrdering::SyncRelease:
   case llvm::AtomicOrdering::AcquireRelease:
   case llvm::AtomicOrdering::SequentiallyConsistent:
+  case llvm::AtomicOrdering::SyncSequentiallyConsistent:
     CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc,
                                          llvm::AtomicOrdering::Release);
     break;
   case llvm::AtomicOrdering::Acquire:
+  case llvm::AtomicOrdering::SyncAcquire:
   case llvm::AtomicOrdering::Monotonic:
     break;
   case llvm::AtomicOrdering::NotAtomic:
   case llvm::AtomicOrdering::Unordered:
     llvm_unreachable("Unexpected ordering.");
   }
 }
 
if (CGF.CGM.getLangOpts().OpenMP < 51) {
     // If the write, update, or capture clause is specified and the release,
     // acq_rel, or seq_cst clause is specified then the strong flush on entry to
     // the atomic operation is also a release flush.
     // If the read or capture clause is specified and the acquire, acq_rel, or
     // seq_cst clause is specified then the strong flush on exit from the atomic
     // operation is also an acquire flush.
     switch (AO) {
     case llvm::AtomicOrdering::Release:
+    case llvm::AtomicOrdering::SyncRelease:
       CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc,
                                            llvm::AtomicOrdering::Release);
       break;
     case llvm::AtomicOrdering::Acquire:
+    case llvm::AtomicOrdering::SyncAcquire:
       CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc,
                                            llvm::AtomicOrdering::Acquire);
       break;
     case llvm::AtomicOrdering::AcquireRelease:
     case llvm::AtomicOrdering::SequentiallyConsistent:
+    case llvm::AtomicOrdering::SyncSequentiallyConsistent:
       CGF.CGM.getOpenMPRuntime().emitFlush(
           CGF, llvm::None, Loc, llvm::AtomicOrdering::AcquireRelease);
       break;
     case llvm::AtomicOrdering::Monotonic:
       break;
     case llvm::AtomicOrdering::NotAtomic:
     case llvm::AtomicOrdering::Unordered:
       llvm_unreachable("Unexpected ordering.");

clang/test/CodeGen/2010-01-13-MemBarrier.c

 // RUN: %clang_cc1 -no-opaque-pointers %s -emit-llvm -o - | FileCheck %s
 // rdar://7536390
 
 typedef unsigned __INT32_TYPE__ uint32_t;
 
 unsigned t(uint32_t *ptr, uint32_t val) {
   // CHECK:      @t
-  // CHECK: atomicrmw xchg i32* {{.*}} seq_cst, align 4
+  // CHECK: atomicrmw xchg i32* {{.*}} sync_acq, align 4
   return __sync_lock_test_and_set(ptr, val);
 }

clang/test/CodeGen/Atomics.c

 unsigned int ui;
 signed long long sll;
 unsigned long long ull;
 __int128 s128;
 unsigned  __int128 u128;
 
 void test_op_ignore (void) // CHECK-LABEL: define{{.*}} void @test_op_ignore
 {
-  (void) __sync_fetch_and_add (&sc, 1); // CHECK: atomicrmw add i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_add (&uc, 1); // CHECK: atomicrmw add i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_add (&ss, 1); // CHECK: atomicrmw add i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_add (&us, 1); // CHECK: atomicrmw add i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_add (&si, 1); // CHECK: atomicrmw add i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_add (&ui, 1); // CHECK: atomicrmw add i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_add (&sll, 1); // CHECK: atomicrmw add i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_add (&ull, 1); // CHECK: atomicrmw add i64* {{.*}} seq_cst, align 8
-
-  (void) __sync_fetch_and_sub (&sc, 1); // CHECK: atomicrmw sub i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_sub (&uc, 1); // CHECK: atomicrmw sub i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_sub (&ss, 1); // CHECK: atomicrmw sub i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_sub (&us, 1); // CHECK: atomicrmw sub i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_sub (&si, 1); // CHECK: atomicrmw sub i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_sub (&ui, 1); // CHECK: atomicrmw sub i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_sub (&sll, 1); // CHECK: atomicrmw sub i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_sub (&ull, 1); // CHECK: atomicrmw sub i64* {{.*}} seq_cst, align 8
-
-  (void) __sync_fetch_and_or (&sc, 1); // CHECK: atomicrmw or i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_or (&uc, 1); // CHECK: atomicrmw or i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_or (&ss, 1); // CHECK: atomicrmw or i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_or (&us, 1); // CHECK: atomicrmw or i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_or (&si, 1); // CHECK: atomicrmw or i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_or (&ui, 1); // CHECK: atomicrmw or i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_or (&sll, 1); // CHECK: atomicrmw or i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_or (&ull, 1); // CHECK: atomicrmw or i64* {{.*}} seq_cst, align 8
-
-  (void) __sync_fetch_and_xor (&sc, 1); // CHECK: atomicrmw xor i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_xor (&uc, 1); // CHECK: atomicrmw xor i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_xor (&ss, 1); // CHECK: atomicrmw xor i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_xor (&us, 1); // CHECK: atomicrmw xor i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_xor (&si, 1); // CHECK: atomicrmw xor i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_xor (&ui, 1); // CHECK: atomicrmw xor i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_xor (&sll, 1); // CHECK: atomicrmw xor i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_xor (&ull, 1); // CHECK: atomicrmw xor i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_xor (&u128, 1); // CHECK: atomicrmw xor i128* {{.*}} seq_cst, align 16
-  (void) __sync_fetch_and_xor (&s128, 1); // CHECK: atomicrmw xor i128* {{.*}} seq_cst, align 16
-
-  (void) __sync_fetch_and_nand (&sc, 1); // CHECK: atomicrmw nand i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_nand (&uc, 1); // CHECK: atomicrmw nand i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_nand (&ss, 1); // CHECK: atomicrmw nand i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_nand (&us, 1); // CHECK: atomicrmw nand i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_nand (&si, 1); // CHECK: atomicrmw nand i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_nand (&ui, 1); // CHECK: atomicrmw nand i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_nand (&sll, 1); // CHECK: atomicrmw nand i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_nand (&ull, 1); // CHECK: atomicrmw nand i64* {{.*}} seq_cst, align 8
-
-  (void) __sync_fetch_and_and (&sc, 1); // CHECK: atomicrmw and i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_and (&uc, 1); // CHECK: atomicrmw and i8* {{.*}} seq_cst, align 1
-  (void) __sync_fetch_and_and (&ss, 1); // CHECK: atomicrmw and i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_and (&us, 1); // CHECK: atomicrmw and i16* {{.*}} seq_cst, align 2
-  (void) __sync_fetch_and_and (&si, 1); // CHECK: atomicrmw and i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_and (&ui, 1); // CHECK: atomicrmw and i32* {{.*}} seq_cst, align 4
-  (void) __sync_fetch_and_and (&sll, 1); // CHECK: atomicrmw and i64* {{.*}} seq_cst, align 8
-  (void) __sync_fetch_and_and (&ull, 1); // CHECK: atomicrmw and i64* {{.*}} seq_cst, align 8
+  (void) __sync_fetch_and_add (&sc, 1); // CHECK: atomicrmw add i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_add (&uc, 1); // CHECK: atomicrmw add i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_add (&ss, 1); // CHECK: atomicrmw add i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_add (&us, 1); // CHECK: atomicrmw add i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_add (&si, 1); // CHECK: atomicrmw add i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_add (&ui, 1); // CHECK: atomicrmw add i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_add (&sll, 1); // CHECK: atomicrmw add i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_add (&ull, 1); // CHECK: atomicrmw add i64* {{.*}} sync_seq_cst, align 8
+
+  (void) __sync_fetch_and_sub (&sc, 1); // CHECK: atomicrmw sub i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_sub (&uc, 1); // CHECK: atomicrmw sub i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_sub (&ss, 1); // CHECK: atomicrmw sub i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_sub (&us, 1); // CHECK: atomicrmw sub i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_sub (&si, 1); // CHECK: atomicrmw sub i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_sub (&ui, 1); // CHECK: atomicrmw sub i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_sub (&sll, 1); // CHECK: atomicrmw sub i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_sub (&ull, 1); // CHECK: atomicrmw sub i64* {{.*}} sync_seq_cst, align 8
+
+  (void) __sync_fetch_and_or (&sc, 1); // CHECK: atomicrmw or i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_or (&uc, 1); // CHECK: atomicrmw or i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_or (&ss, 1); // CHECK: atomicrmw or i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_or (&us, 1); // CHECK: atomicrmw or i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_or (&si, 1); // CHECK: atomicrmw or i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_or (&ui, 1); // CHECK: atomicrmw or i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_or (&sll, 1); // CHECK: atomicrmw or i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_or (&ull, 1); // CHECK: atomicrmw or i64* {{.*}} sync_seq_cst, align 8
+
+  (void) __sync_fetch_and_xor (&sc, 1); // CHECK: atomicrmw xor i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_xor (&uc, 1); // CHECK: atomicrmw xor i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_xor (&ss, 1); // CHECK: atomicrmw xor i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_xor (&us, 1); // CHECK: atomicrmw xor i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_xor (&si, 1); // CHECK: atomicrmw xor i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_xor (&ui, 1); // CHECK: atomicrmw xor i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_xor (&sll, 1); // CHECK: atomicrmw xor i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_xor (&ull, 1); // CHECK: atomicrmw xor i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_xor (&u128, 1); // CHECK: atomicrmw xor i128* {{.*}} sync_seq_cst, align 16
+  (void) __sync_fetch_and_xor (&s128, 1); // CHECK: atomicrmw xor i128* {{.*}} sync_seq_cst, align 16
+
+  (void) __sync_fetch_and_nand (&sc, 1); // CHECK: atomicrmw nand i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_nand (&uc, 1); // CHECK: atomicrmw nand i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_nand (&ss, 1); // CHECK: atomicrmw nand i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_nand (&us, 1); // CHECK: atomicrmw nand i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_nand (&si, 1); // CHECK: atomicrmw nand i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_nand (&ui, 1); // CHECK: atomicrmw nand i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_nand (&sll, 1); // CHECK: atomicrmw nand i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_nand (&ull, 1); // CHECK: atomicrmw nand i64* {{.*}} sync_seq_cst, align 8
+
+  (void) __sync_fetch_and_and (&sc, 1); // CHECK: atomicrmw and i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_and (&uc, 1); // CHECK: atomicrmw and i8* {{.*}} sync_seq_cst, align 1
+  (void) __sync_fetch_and_and (&ss, 1); // CHECK: atomicrmw and i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_and (&us, 1); // CHECK: atomicrmw and i16* {{.*}} sync_seq_cst, align 2
+  (void) __sync_fetch_and_and (&si, 1); // CHECK: atomicrmw and i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_and (&ui, 1); // CHECK: atomicrmw and i32* {{.*}} sync_seq_cst, align 4
+  (void) __sync_fetch_and_and (&sll, 1); // CHECK: atomicrmw and i64* {{.*}} sync_seq_cst, align 8
+  (void) __sync_fetch_and_and (&ull, 1); // CHECK: atomicrmw and i64* {{.*}} sync_seq_cst, align 8
 
 }
 
 void test_fetch_and_op (void) // CHECK-LABEL: define{{.*}} void @test_fetch_and_op
 {
   sc = __sync_fetch_and_add (&sc, 11); // CHECK: atomicrmw add
   uc = __sync_fetch_and_add (&uc, 11); // CHECK: atomicrmw add
   ss = __sync_fetch_and_add (&ss, 11); // CHECK: atomicrmw add
void test_op_and_fetch (void)
   sll = __sync_and_and_fetch (&sll, uc); // CHECK: atomicrmw and
   ull = __sync_and_and_fetch (&ull, uc); // CHECK: atomicrmw and
 
 }
 
 void test_compare_and_swap (void)
 {
   sc = __sync_val_compare_and_swap (&sc, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} seq_cst, align 1
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} sync_seq_cst, align 1
   // CHECK: extractvalue { i8, i1 } [[PAIR]], 0
 
   uc = __sync_val_compare_and_swap (&uc, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} seq_cst, align 1
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} sync_seq_cst, align 1
   // CHECK: extractvalue { i8, i1 } [[PAIR]], 0
 
   ss = __sync_val_compare_and_swap (&ss, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} seq_cst, align 2
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} sync_seq_cst, align 2
   // CHECK: extractvalue { i16, i1 } [[PAIR]], 0
 
   us = __sync_val_compare_and_swap (&us, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} seq_cst, align 2
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} sync_seq_cst, align 2
   // CHECK: extractvalue { i16, i1 } [[PAIR]], 0
 
   si = __sync_val_compare_and_swap (&si, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 0
 
   ui = __sync_val_compare_and_swap (&ui, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 0
 
   sll = __sync_val_compare_and_swap (&sll, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} seq_cst, align 8
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} sync_seq_cst, align 8
   // CHECK: extractvalue { i64, i1 } [[PAIR]], 0
 
   ull = __sync_val_compare_and_swap (&ull, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} seq_cst, align 8
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} sync_seq_cst, align 8
   // CHECK: extractvalue { i64, i1 } [[PAIR]], 0
 
 
   ui = __sync_bool_compare_and_swap (&sc, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} seq_cst, align 1
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} sync_seq_cst, align 1
   // CHECK: extractvalue { i8, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&uc, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} seq_cst, align 1
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i8* {{.*}} sync_seq_cst, align 1
   // CHECK: extractvalue { i8, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&ss, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} seq_cst, align 2
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} sync_seq_cst, align 2
   // CHECK: extractvalue { i16, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&us, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} seq_cst, align 2
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i16* {{.*}} sync_seq_cst, align 2
   // CHECK: extractvalue { i16, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&si, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&ui, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i32* {{.*}} sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&sll, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} seq_cst, align 8
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} sync_seq_cst, align 8
   // CHECK: extractvalue { i64, i1 } [[PAIR]], 1
 
   ui = __sync_bool_compare_and_swap (&ull, uc, sc);
-  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} seq_cst, align 8
+  // CHECK: [[PAIR:%[a-z0-9._]+]] = cmpxchg i64* {{.*}} sync_seq_cst, align 8
   // CHECK: extractvalue { i64, i1 } [[PAIR]], 1
 }
 
 void test_lock (void)
 {
-  sc = __sync_lock_test_and_set (&sc, 1); // CHECK: atomicrmw xchg i8* {{.*}} seq_cst, align 1
-  uc = __sync_lock_test_and_set (&uc, 1); // CHECK: atomicrmw xchg i8* {{.*}} seq_cst, align 1
-  ss = __sync_lock_test_and_set (&ss, 1); // CHECK: atomicrmw xchg i16* {{.*}} seq_cst, align 2
-  us = __sync_lock_test_and_set (&us, 1); // CHECK: atomicrmw xchg i16* {{.*}} seq_cst, align 2
-  si = __sync_lock_test_and_set (&si, 1); // CHECK: atomicrmw xchg i32* {{.*}} seq_cst, align 4
-  ui = __sync_lock_test_and_set (&ui, 1); // CHECK: atomicrmw xchg i32* {{.*}} seq_cst, align 4
-  sll = __sync_lock_test_and_set (&sll, 1); // CHECK: atomicrmw xchg i64* {{.*}} seq_cst, align 8
-  ull = __sync_lock_test_and_set (&ull, 1); // CHECK: atomicrmw xchg i64* {{.*}} seq_cst, align 8
+  sc = __sync_lock_test_and_set (&sc, 1); // CHECK: atomicrmw xchg i8* {{.*}} sync_acq, align 1
+  uc = __sync_lock_test_and_set (&uc, 1); // CHECK: atomicrmw xchg i8* {{.*}} sync_acq, align 1
+  ss = __sync_lock_test_and_set (&ss, 1); // CHECK: atomicrmw xchg i16* {{.*}} sync_acq, align 2
+  us = __sync_lock_test_and_set (&us, 1); // CHECK: atomicrmw xchg i16* {{.*}} sync_acq, align 2
+  si = __sync_lock_test_and_set (&si, 1); // CHECK: atomicrmw xchg i32* {{.*}} sync_acq, align 4
+  ui = __sync_lock_test_and_set (&ui, 1); // CHECK: atomicrmw xchg i32* {{.*}} sync_acq, align 4
+  sll = __sync_lock_test_and_set (&sll, 1); // CHECK: atomicrmw xchg i64* {{.*}} sync_acq, align 8
+  ull = __sync_lock_test_and_set (&ull, 1); // CHECK: atomicrmw xchg i64* {{.*}} sync_acq, align 8
 
   __sync_synchronize (); // CHECK: fence seq_cst
 
-  __sync_lock_release (&sc); // CHECK: store atomic {{.*}} release, align 1
-  __sync_lock_release (&uc); // CHECK: store atomic {{.*}} release, align 1
-  __sync_lock_release (&ss); // CHECK: store atomic {{.*}} release, align 2
-  __sync_lock_release (&us); /// CHECK: store atomic {{.*}} release, align 2
-  __sync_lock_release (&si); // CHECK: store atomic {{.*}} release, align 4
-  __sync_lock_release (&ui); // CHECK: store atomic {{.*}} release, align 4
-  __sync_lock_release (&sll); // CHECK: store atomic {{.*}} release, align 8
-  __sync_lock_release (&ull); // CHECK: store atomic {{.*}} release, align 8
+  __sync_lock_release (&sc); // CHECK: store atomic {{.*}} sync_rel, align 1
+  __sync_lock_release (&uc); // CHECK: store atomic {{.*}} sync_rel, align 1
+  __sync_lock_release (&ss); // CHECK: store atomic {{.*}} sync_rel, align 2
+  __sync_lock_release (&us); /// CHECK: store atomic {{.*}} sync_rel, align 2
+  __sync_lock_release (&si); // CHECK: store atomic {{.*}} sync_rel, align 4
+  __sync_lock_release (&ui); // CHECK: store atomic {{.*}} sync_rel, align 4
+  __sync_lock_release (&sll); // CHECK: store atomic {{.*}} sync_rel, align 8
+  __sync_lock_release (&ull); // CHECK: store atomic {{.*}} sync_rel, align 8
 }
 
 void test_atomic(void) {
   ui = __atomic_fetch_min(&ui, 5, __ATOMIC_RELAXED); // CHECK: atomicrmw umin {{.*}} monotonic, align 4
   si = __atomic_fetch_min(&si, 5, __ATOMIC_SEQ_CST); // CHECK: atomicrmw min {{.*}} seq_cst, align 4
   ui = __atomic_fetch_max(&ui, 5, __ATOMIC_ACQUIRE); // CHECK: atomicrmw umax {{.*}} acquire, align 4
   si = __atomic_fetch_max(&si, 5, __ATOMIC_RELEASE); // CHECK: atomicrmw max {{.*}} release, align 4
 }

clang/test/CodeGen/X86/x86_64-atomic-128.c

 // RUN: %clang_cc1 -no-opaque-pointers -triple x86_64-linux-gnu -target-cpu core2 %s -S -emit-llvm -o - | FileCheck %s
 
 // All atomics up to 16 bytes should be emitted inline on x86_64. The
 // backend can reform __sync_whatever calls if necessary (e.g. the CPU
 // doesn't have cmpxchg16b).
 
 __int128 test_sync_call(__int128 *addr, __int128 val) {
   // CHECK-LABEL: @test_sync_call
-  // CHECK: atomicrmw add i128* {{.*}} seq_cst, align 16
+  // CHECK: atomicrmw add i128* {{.*}} sync_seq_cst, align 16
   return __sync_fetch_and_add(addr, val);
 }
 
 __int128 test_c11_call(_Atomic __int128 *addr, __int128 val) {
   // CHECK-LABEL: @test_c11_call
   // CHECK: atomicrmw sub i128* {{.*}} monotonic, align 16
   return __c11_atomic_fetch_sub(addr, val, 0);
 }

clang/test/CodeGen/atomic.c

 // RUN: %clang_cc1 -no-opaque-pointers %s -emit-llvm -o - -triple=i686-apple-darwin9 | FileCheck %s
 
 int atomic(void) {
   // non-sensical test for sync functions
   int old;
   int val = 1;
   char valc = 1;
   _Bool valb = 0;
   unsigned int uval = 1;
   int cmp = 0;
   int* ptrval;
 
   old = __sync_fetch_and_add(&val, 1);
-  // CHECK: atomicrmw add i32* %val, i32 1 seq_cst, align 4
+  // CHECK: atomicrmw add i32* %val, i32 1 sync_seq_cst, align 4
   
   old = __sync_fetch_and_sub(&valc, 2);
-  // CHECK: atomicrmw sub i8* %valc, i8 2 seq_cst, align 1
+  // CHECK: atomicrmw sub i8* %valc, i8 2 sync_seq_cst, align 1
   
   old = __sync_fetch_and_min(&val, 3);
   // CHECK: atomicrmw min i32* %val, i32 3 seq_cst, align 4
   
   old = __sync_fetch_and_max(&val, 4);
   // CHECK: atomicrmw max i32* %val, i32 4 seq_cst, align 4
   
   old = __sync_fetch_and_umin(&uval, 5u);
   // CHECK: atomicrmw umin i32* %uval, i32 5 seq_cst, align 4
   
   old = __sync_fetch_and_umax(&uval, 6u);
   // CHECK: atomicrmw umax i32* %uval, i32 6 seq_cst, align 4
   
   old = __sync_lock_test_and_set(&val, 7);
-  // CHECK: atomicrmw xchg i32* %val, i32 7 seq_cst, align 4
+  // CHECK: atomicrmw xchg i32* %val, i32 7 sync_acq, align 4
   
   old = __sync_swap(&val, 8);
   // CHECK: atomicrmw xchg i32* %val, i32 8 seq_cst, align 4
   
   old = __sync_val_compare_and_swap(&val, 4, 1976);
-  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* %val, i32 4, i32 1976 seq_cst seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* %val, i32 4, i32 1976 sync_seq_cst sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 0
 
   old = __sync_bool_compare_and_swap(&val, 4, 1976);
-  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* %val, i32 4, i32 1976 seq_cst seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* %val, i32 4, i32 1976 sync_seq_cst sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 1
 
   old = __sync_fetch_and_and(&val, 0x9);
-  // CHECK: atomicrmw and i32* %val, i32 9 seq_cst, align 4
+  // CHECK: atomicrmw and i32* %val, i32 9 sync_seq_cst, align 4
 
   old = __sync_fetch_and_or(&val, 0xa);
-  // CHECK: atomicrmw or i32* %val, i32 10 seq_cst, align 4
+  // CHECK: atomicrmw or i32* %val, i32 10 sync_seq_cst, align 4
 
   old = __sync_fetch_and_xor(&val, 0xb);
-  // CHECK: atomicrmw xor i32* %val, i32 11 seq_cst, align 4
+  // CHECK: atomicrmw xor i32* %val, i32 11 sync_seq_cst, align 4
  
   old = __sync_fetch_and_nand(&val, 0xc);
-  // CHECK: atomicrmw nand i32* %val, i32 12 seq_cst, align 4
+  // CHECK: atomicrmw nand i32* %val, i32 12 sync_seq_cst, align 4
  
   old = __sync_add_and_fetch(&val, 1);
-  // CHECK: atomicrmw add i32* %val, i32 1 seq_cst, align 4
+  // CHECK: atomicrmw add i32* %val, i32 1 sync_seq_cst, align 4
 
   old = __sync_sub_and_fetch(&val, 2);
-  // CHECK: atomicrmw sub i32* %val, i32 2 seq_cst, align 4
+  // CHECK: atomicrmw sub i32* %val, i32 2 sync_seq_cst, align 4
 
   old = __sync_and_and_fetch(&valc, 3);
-  // CHECK: atomicrmw and i8* %valc, i8 3 seq_cst, align 1
+  // CHECK: atomicrmw and i8* %valc, i8 3 sync_seq_cst, align 1
 
   old = __sync_or_and_fetch(&valc, 4);
-  // CHECK: atomicrmw or i8* %valc, i8 4 seq_cst, align 1
+  // CHECK: atomicrmw or i8* %valc, i8 4 sync_seq_cst, align 1
 
   old = __sync_xor_and_fetch(&valc, 5);
-  // CHECK: atomicrmw xor i8* %valc, i8 5 seq_cst, align 1
+  // CHECK: atomicrmw xor i8* %valc, i8 5 sync_seq_cst, align 1
  
   old = __sync_nand_and_fetch(&valc, 6);
-  // CHECK: atomicrmw nand i8* %valc, i8 6 seq_cst, align 1
+  // CHECK: atomicrmw nand i8* %valc, i8 6 sync_seq_cst, align 1
  
   __sync_val_compare_and_swap((void **)0, (void *)0, (void *)0);
-  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* null, i32 0, i32 0 seq_cst seq_cst, align 4
+  // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i32* null, i32 0, i32 0 sync_seq_cst sync_seq_cst, align 4
   // CHECK: extractvalue { i32, i1 } [[PAIR]], 0
 
   if ( __sync_val_compare_and_swap(&valb, 0, 1)) {
-    // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i8* %valb, i8 0, i8 1 seq_cst seq_cst, align 1
+    // CHECK: [[PAIR:%[a-z0-9_.]+]] = cmpxchg i8* %valb, i8 0, i8 1 sync_seq_cst sync_seq_cst, align 1
     // CHECK: [[VAL:%[a-z0-9_.]+]] = extractvalue { i8, i1 } [[PAIR]], 0
     // CHECK: trunc i8 [[VAL]] to i1
     old = 42;
   }
   
   __sync_bool_compare_and_swap((void **)0, (void *)0, (void *)0);
-  // CHECK: cmpxchg i32* null, i32 0, i32 0 seq_cst seq_cst, align 4
+  // CHECK: cmpxchg i32* null, i32 0, i32 0 sync_seq_cst sync_seq_cst, align 4
   
   __sync_lock_release(&val);
-  // CHECK: store atomic i32 0, {{.*}} release, align 4
+  // CHECK: store atomic i32 0, {{.*}} sync_rel, align 4
 
   __sync_lock_release(&ptrval);
-  // CHECK: store atomic i32 0, {{.*}} release, align 4
+  // CHECK: store atomic i32 0, {{.*}} sync_rel, align 4
 
   __sync_synchronize ();
   // CHECK: fence seq_cst
 
   return old;
 }
 
 // CHECK: @release_return
 void release_return(int *lock) {
   // Ensure this is actually returning void all the way through.
   return __sync_lock_release(lock);
-  // CHECK: store atomic {{.*}} release, align 4
+  // CHECK: store atomic {{.*}} sync_rel, align 4
 }
 
 
 // rdar://8461279 - Atomics with address spaces.
 // CHECK: @addrspace
 void addrspace(int  __attribute__((address_space(256))) * P) {
   __sync_bool_compare_and_swap(P, 0, 1);
-  // CHECK: cmpxchg i32 addrspace(256)*{{.*}}, i32 0, i32 1 seq_cst seq_cst, align 4
+  // CHECK: cmpxchg i32 addrspace(256)*{{.*}}, i32 0, i32 1 sync_seq_cst sync_seq_cst, align 4
 
   __sync_val_compare_and_swap(P, 0, 1);
-  // CHECK: cmpxchg i32 addrspace(256)*{{.*}}, i32 0, i32 1 seq_cst seq_cst, align 4
+  // CHECK: cmpxchg i32 addrspace(256)*{{.*}}, i32 0, i32 1 sync_seq_cst sync_seq_cst, align 4
 
   __sync_xor_and_fetch(P, 123);
-  // CHECK: atomicrmw xor i32 addrspace(256)*{{.*}}, i32 123 seq_cst, align 4
+  // CHECK: atomicrmw xor i32 addrspace(256)*{{.*}}, i32 123 sync_seq_cst, align 4
 }

clang/test/CodeGenCXX/atomic.cpp

 // RUN: %clang_cc1 -no-opaque-pointers %s -emit-llvm -o - -triple=i686-apple-darwin9 | FileCheck %s
 
 namespace PR11411 {
   template<typename _Tp> struct Ptr {
     void f();
   };
 
   // CHECK-LABEL: define linkonce_odr void @_ZN7PR114113PtrIiE1fEv
   // CHECK-NOT: ret
   template<typename _Tp> inline void Ptr<_Tp>::f() {
     int* _refcount;
-    // CHECK: atomicrmw add i32* {{.*}} seq_cst, align 4
+    // CHECK: atomicrmw add i32* {{.*}} sync_seq_cst, align 4
     __sync_fetch_and_add(_refcount, 1);
     // CHECK-NEXT: ret void
   }
   void f(Ptr<int> *a) { a->f(); }
 }
 
 namespace DelegatingParameter {
   // Check that we're delegating the complete ctor to the base
   // ctor, and that doesn't crash.
   // CHECK-LABEL: define void @_ZN19DelegatingParameter1SC1EU7_AtomicNS_1ZE
   // CHECK: call void @_ZN19DelegatingParameter1SC2EU7_AtomicNS_1ZE
   struct Z { int z[100]; };
   struct S { S(_Atomic Z); };
   S::S(_Atomic Z) {}
 }

compiler-rt/lib/builtins/CMakeLists.txt

 if(CMAKE_HOST_UNIX)
   set(COMPILER_RT_LINK_OR_COPY create_symlink)
 else()
   set(COMPILER_RT_LINK_OR_COPY copy)
 endif()
 
 foreach(pat cas swp ldadd ldclr ldeor ldset)
   foreach(size 1 2 4 8 16)
-    foreach(model 1 2 3 4)
+    foreach(model 1 2 3 4 5)
       if(pat STREQUAL "cas" OR NOT size STREQUAL "16")
         set(helper_asm "${OA_HELPERS_DIR}/outline_atomic_${pat}${size}_${model}.S")
         list(APPEND lse_builtins "${helper_asm}")
         list(APPEND arm64_lse_commands COMMAND ${CMAKE_COMMAND} -E ${COMPILER_RT_LINK_OR_COPY} "${CMAKE_CURRENT_SOURCE_DIR}/aarch64/lse.S" "${helper_asm}")
         set_source_files_properties("${helper_asm}"
           PROPERTIES
           COMPILE_DEFINITIONS "L_${pat};SIZE=${size};MODEL=${model}"
           INCLUDE_DIRECTORIES "${CMAKE_CURRENT_SOURCE_DIR}"

compiler-rt/lib/builtins/aarch64/lse.S

 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 #include "assembly.h"
 
 // Out-of-line LSE atomics helpers. Ported from libgcc library.
 // N = {1, 2, 4, 8}
 // M = {1, 2, 4, 8, 16}
-// ORDER = {'relax', 'acq', 'rel', 'acq_rel'}
+// ORDER = {'relax', 'acq', 'rel', 'acq_rel', 'sync'}
 // Routines implemented:
 //
 //  iM __aarch64_casM_ORDER(iM expected, iM desired, iM *ptr)
 //  iN __aarch64_swpN_ORDER(iN val, iN *ptr)
 //  iN __aarch64_ldaddN_ORDER(iN val, iN *ptr)
 //  iN __aarch64_ldclrN_ORDER(iN val, iN *ptr)
 //  iN __aarch64_ldeorN_ORDER(iN val, iN *ptr)
 //  iN __aarch64_ldsetN_ORDER(iN val, iN *ptr)

 
 #if !defined(__APPLE__)
 HIDDEN(__aarch64_have_lse_atomics)
 #else
 HIDDEN(___aarch64_have_lse_atomics)
 #endif
 
 // Generate mnemonics for
-// L_cas:                                 SIZE: 1,2,4,8,16 MODEL: 1,2,3,4
-// L_swp L_ldadd L_ldclr L_ldeor L_ldset: SIZE: 1,2,4,8    MODEL: 1,2,3,4
+// L_cas:                                 SIZE: 1,2,4,8,16 MODEL: 1,2,3,4,5
+// L_swp L_ldadd L_ldclr L_ldeor L_ldset: SIZE: 1,2,4,8    MODEL: 1,2,3,4,5
 
 #if SIZE == 1
 #define S b
 #define UXT uxtb
 #define B 0x00000000
 #elif SIZE == 2
 #define S h
 #define UXT uxth

 #endif // SIZE
 
 #if MODEL == 1
 #define SUFF _relax
 #define A
 #define L
 #define M 0x000000
 #define N 0x000000
+#define DMB
 #elif MODEL == 2
 #define SUFF _acq
 #define A a
 #define L
 #define M 0x400000
 #define N 0x800000
+#define DMB
 #elif MODEL == 3
 #define SUFF _rel
 #define A
 #define L l
 #define M 0x008000
 #define N 0x400000
+#define DMB
 #elif MODEL == 4
 #define SUFF _acq_rel
 #define A a
 #define L l
 #define M 0x408000
 #define N 0xc00000
+#define DMB
+#elif MODEL == 5
+#define SUFF _sync
+#ifdef L_swp
+#define A a
+#define L
+#define M 0x400000
+#define N 0x800000
+#else
+#define A a
+#define L l
+#define M 0x408000
+#define N 0xc00000
+#endif
+#define DMB dmb ish
 #else
 #error
 #endif // MODEL
 
 // Define register size.
 #define x(N) GLUE2(x, N)
 #define w(N) GLUE2(w, N)
 #if SIZE < 8
 #define s(N) w(N)
 #else
 #define s(N) x(N)
 #endif
 
 #define NAME(BASE) GLUE4(__aarch64_, BASE, SIZE, SUFF)
+#if MODEL == 5
+#define LDXR GLUE3(ld, xr, S)
+#else
 #define LDXR GLUE4(ld, A, xr, S)
+#endif
 #define STXR GLUE4(st, L, xr, S)
 
 // Define temporary registers.
 #define tmp0 16
 #define tmp1 17
 #define tmp2 15
 
 // Macro for branch to label if no LSE available
8:
         UXT    s(tmp0), s(0)
 0:
         LDXR   s(0), [x2]
         cmp    s(0), s(tmp0)
         bne    1f
         STXR   w(tmp1), s(1), [x2]
         cbnz   w(tmp1), 0b
 1:
+        DMB
         ret
 #else
+#if MODEL == 5
+#define LDXP GLUE2(ld, xp)
+#else
 #define LDXP GLUE3(ld, A, xp)
+#endif
 #define STXP GLUE3(st, L, xp)
 #ifdef HAS_ASM_LSE
 #define CASP GLUE3(casp, A, L)  x0, x1, x2, x3, [x4]
 #else
 #define CASP .inst 0x48207c82 + M
 #endif
 
         CASP   // x0, x1, x2, x3, [x4]
         ret
 8:
         mov    x(tmp0), x0
         mov    x(tmp1), x1
 0:
         LDXP   x0, x1, [x4]
         cmp    x0, x(tmp0)
         ccmp   x1, x(tmp1), #0, eq
         bne    1f
         STXP   w(tmp2), x2, x3, [x4]
         cbnz   w(tmp2), 0b
 1:
+        DMB
         ret
 #endif
 END_COMPILERRT_OUTLINE_FUNCTION(NAME(cas))
 #endif // L_cas
 
 #ifdef L_swp
 #ifdef HAS_ASM_LSE
 #define SWP GLUE4(swp, A, L, S)  s(0), s(0), [x1]
 #else
 #define SWP .inst 0x38208020 + B + N
 #endif
 DEFINE_COMPILERRT_OUTLINE_FUNCTION_UNMANGLED(NAME(swp))
         JUMP_IF_NOT_LSE 8f
         SWP    // s(0), s(0), [x1]
         ret
 8:
         mov    s(tmp0), s(0)
 0:
         LDXR   s(0), [x1]
         STXR   w(tmp1), s(tmp0), [x1]
         cbnz   w(tmp1), 0b
+        DBM
         ret
 END_COMPILERRT_OUTLINE_FUNCTION(NAME(swp))
 #endif // L_swp
 
 #if defined(L_ldadd) || defined(L_ldclr) ||                                    \
     defined(L_ldeor) || defined(L_ldset)
 
 #ifdef L_ldadd
DEFINE_COMPILERRT_OUTLINE_FUNCTION_UNMANGLED(NAME(LDNM))
         ret
 8:
         mov    s(tmp0), s(0)
 0:
         LDXR   s(0), [x1]
         OP     s(tmp1), s(0), s(tmp0)
         STXR   w(tmp2), s(tmp1), [x1]
         cbnz   w(tmp2), 0b
+        DBM
         ret
 END_COMPILERRT_OUTLINE_FUNCTION(NAME(LDNM))
 #endif // L_ldadd L_ldclr L_ldeor L_ldset
 
 NO_EXEC_STACK_DIRECTIVE
 
 // GNU property note for BTI and PAC
 GNU_PROPERTY_BTI_PAC
 
 #endif // __aarch64__

llvm/include/llvm-c/Core.h

 
 typedef enum {
   LLVMAtomicOrderingNotAtomic = 0, /**< A load or store which is not atomic */
   LLVMAtomicOrderingUnordered = 1, /**< Lowest level of atomicity, guarantees
                                      somewhat sane results, lock free. */
   LLVMAtomicOrderingMonotonic = 2, /**< guarantees that if you take all the
                                      operations affecting a specific address,
                                      a consistent ordering exists */
-  LLVMAtomicOrderingAcquire = 4, /**< Acquire provides a barrier of the sort
+  LLVMAtomicOrderingAcquire = 4,   /**< Acquire provides a barrier of the sort
                                      necessary to acquire a lock to access other
                                      memory with normal loads and stores. */
-  LLVMAtomicOrderingRelease = 5, /**< Release is similar to Acquire, but with
+  LLVMAtomicOrderingRelease = 5,   /**< Release is similar to Acquire, but with
                                      a barrier of the sort necessary to release
                                      a lock. */
   LLVMAtomicOrderingAcquireRelease = 6, /**< provides both an Acquire and a
                                           Release barrier (for fences and
                                           operations which both read and write
                                            memory). */
-  LLVMAtomicOrderingSequentiallyConsistent = 7 /**< provides Acquire semantics
+  LLVMAtomicOrderingSequentiallyConsistent = 7, /**< provides Acquire semantics
                                                   for loads and Release
                                                   semantics for stores.
                                                   Additionally, it guarantees
                                                   that a total ordering exists
                                                   between all
                                                   SequentiallyConsistent
                                                   operations. */
+  LLVMAtomicOrderingSyncAcquire = 8, /**< Acquire with additional barrier. */
+  LLVMAtomicOrderingSyncRelease = 9, /**< Release with additional barrier. */
+  LLVMAtomicOrderingSyncSequentiallyConsistent =
+      10 /**< SequentiallyConsistent
+            with additional barrier. */
 } LLVMAtomicOrdering;
 
 typedef enum {
     LLVMAtomicRMWBinOpXchg, /**< Set the new value and return the one old */
     LLVMAtomicRMWBinOpAdd, /**< Add a value and return the old one */
     LLVMAtomicRMWBinOpSub, /**< Subtract a value and return the old one */
     LLVMAtomicRMWBinOpAnd, /**< And a value and return the old one */
     LLVMAtomicRMWBinOpNand, /**< Not-And a value and return the old one */

llvm/include/llvm/AsmParser/LLToken.h

enum Kind {
   kw_volatile,
   kw_atomic,
   kw_unordered,
   kw_monotonic,
   kw_acquire,
   kw_release,
   kw_acq_rel,
   kw_seq_cst,
+  kw_sync_acq,
+  kw_sync_rel,
+  kw_sync_seq_cst,
   kw_syncscope,
   kw_nnan,
   kw_ninf,
   kw_nsz,
   kw_arcp,
   kw_contract,
   kw_reassoc,
   kw_afn,

llvm/include/llvm/Bitcode/LLVMBitCodes.h

 /// Encoded AtomicOrdering values.
 enum AtomicOrderingCodes {
   ORDERING_NOTATOMIC = 0,
   ORDERING_UNORDERED = 1,
   ORDERING_MONOTONIC = 2,
   ORDERING_ACQUIRE = 3,
   ORDERING_RELEASE = 4,
   ORDERING_ACQREL = 5,
-  ORDERING_SEQCST = 6
+  ORDERING_SEQCST = 6,
+  ORDERING_SYNC_ACQ = 7,
+  ORDERING_SYNC_REL = 8,
+  ORDERING_SYNC_SEQCST = 9
 };
 
 /// Markers and flags for call instruction.
 enum CallMarkersFlags {
   CALL_TAIL = 0,
   CALL_CCONV = 1,
   CALL_MUSTTAIL = 14,
   CALL_EXPLICIT_TYPE = 15,

llvm/include/llvm/IR/Instruction.h

class Instruction : public User,
   BasicBlock *Parent;
   DebugLoc DbgLoc;                         // 'dbg' Metadata cache.
 
   /// Relative order of this instruction in its parent basic block. Used for
   /// O(1) local dominance checks between instructions.
   mutable unsigned Order = 0;
 
 protected:
-  // The 15 first bits of `Value::SubclassData` are available for subclasses of
+  // The 16 first bits of `Value::SubclassData` are available for subclasses of
   // `Instruction` to use.
-  using OpaqueField = Bitfield::Element<uint16_t, 0, 15>;
+  using OpaqueField = Bitfield::Element<uint16_t, 0, 16>;
 
   // Template alias so that all Instruction storing alignment use the same
   // definiton.
   // Valid alignments are powers of two from 2^0 to 2^MaxAlignmentExponent =
   // 2^32. We store them as Log2(Alignment), so we need 6 bits to encode the 33
   // possible values.
   template <unsigned Offset>
   using AlignmentBitfieldElementT =
       typename Bitfield::Element<unsigned, Offset, 6,
                                  Value::MaxAlignmentExponent>;
 
   template <unsigned Offset>
   using BoolBitfieldElementT = typename Bitfield::Element<bool, Offset, 1>;
 
   template <unsigned Offset>
   using AtomicOrderingBitfieldElementT =
-      typename Bitfield::Element<AtomicOrdering, Offset, 3,
+      typename Bitfield::Element<AtomicOrdering, Offset, 4,
                                  AtomicOrdering::LAST>;
 
 private:
   // The last bit is used to store whether the instruction has metadata attached
   // or not.
-  using HasMetadataField = Bitfield::Element<bool, 15, 1>;
+  using HasMetadataField = Bitfield::Element<bool, 16, 1>;
 
 protected:
   ~Instruction(); // Use deleteValue() to delete a generic Instruction.
 
 public:
   Instruction(const Instruction &) = delete;
   Instruction &operator=(const Instruction &) = delete;
 

llvm/include/llvm/IR/Instructions.h

 /// original value as first element, and an i1 indicating success (true) or
 /// failure (false) as second element.
 ///
 class AtomicCmpXchgInst : public Instruction {
   void Init(Value *Ptr, Value *Cmp, Value *NewVal, Align Align,
             AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering,
             SyncScope::ID SSID);
 
-  template <unsigned Offset>
-  using AtomicOrderingBitfieldElement =
-      typename Bitfield::Element<AtomicOrdering, Offset, 3,
-                                 AtomicOrdering::LAST>;
+  //  template <unsigned Offset>
+  //  using AtomicOrderingBitfieldElement =
+  //      typename Bitfield::Element<AtomicOrdering, Offset, 4,
+  //                                 AtomicOrdering::LAST>;
 
 protected:
   // Note: Instruction needs to be a friend here to call cloneImpl.
   friend class Instruction;
 
   AtomicCmpXchgInst *cloneImpl() const;
 
 public:
unsigned getPointerAddressSpace() const {
     return getPointerOperand()->getType()->getPointerAddressSpace();
   }
 
   /// Returns the strongest permitted ordering on failure, given the
   /// desired ordering on success.
   ///
   /// If the comparison in a cmpxchg operation fails, there is no atomic store
   /// so release semantics cannot be provided. So this function drops explicit
-  /// Release requests from the AtomicOrdering. A SequentiallyConsistent
-  /// operation would remain SequentiallyConsistent.
+  /// Release requests from the AtomicOrdering. A SequentiallyConsistent and
+  /// StrongSequetiallyConsistent operations would remain
+  /// SequentiallyConsistent.
   static AtomicOrdering
   getStrongestFailureOrdering(AtomicOrdering SuccessOrdering) {
     switch (SuccessOrdering) {
     default:
       llvm_unreachable("invalid cmpxchg success ordering");
     case AtomicOrdering::Release:
+    case AtomicOrdering::SyncRelease:
     case AtomicOrdering::Monotonic:
       return AtomicOrdering::Monotonic;
     case AtomicOrdering::AcquireRelease:
     case AtomicOrdering::Acquire:
+    case AtomicOrdering::SyncAcquire:
       return AtomicOrdering::Acquire;
     case AtomicOrdering::SequentiallyConsistent:
+    case AtomicOrdering::SyncSequentiallyConsistent:
       return AtomicOrdering::SequentiallyConsistent;
     }
   }
 
   // Methods for support type inquiry through isa, cast, and dyn_cast:
   static bool classof(const Instruction *I) {
     return I->getOpcode() == Instruction::AtomicCmpXchg;
   }
enum BinOp : unsigned {
     FMin,
 
     FIRST_BINOP = Xchg,
     LAST_BINOP = FMin,
     BAD_BINOP
   };
 
 private:
-  template <unsigned Offset>
-  using AtomicOrderingBitfieldElement =
-      typename Bitfield::Element<AtomicOrdering, Offset, 3,
-                                 AtomicOrdering::LAST>;
+  //  template <unsigned Offset>
+  //  using AtomicOrderingBitfieldElement =
+  //      typename Bitfield::Element<AtomicOrdering, Offset, 4,
+  //                                 AtomicOrdering::LAST>;
 
   template <unsigned Offset>
   using BinOpBitfieldElement =
       typename Bitfield::Element<BinOp, Offset, 4, BinOp::LAST_BINOP>;
 
 public:
   AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, Align Alignment,
                 AtomicOrdering Ordering, SyncScope::ID SSID,

llvm/include/llvm/IR/RuntimeLibcalls.def

 HANDLE_LIBCALL(ATOMIC_FETCH_NAND_8, "__atomic_fetch_nand_8")
 HANDLE_LIBCALL(ATOMIC_FETCH_NAND_16, "__atomic_fetch_nand_16")
 
 // Out-of-line atomics libcalls
 #define HLCALLS(A, N)                                                          \
   HANDLE_LIBCALL(A##N##_RELAX, nullptr)                                        \
   HANDLE_LIBCALL(A##N##_ACQ, nullptr)                                          \
   HANDLE_LIBCALL(A##N##_REL, nullptr)                                          \
-  HANDLE_LIBCALL(A##N##_ACQ_REL, nullptr)
+  HANDLE_LIBCALL(A##N##_ACQ_REL, nullptr)                                      \
+  HANDLE_LIBCALL(A##N##_SYNC, nullptr)
 #define HLCALL5(A)                                                             \
   HLCALLS(A, 1) HLCALLS(A, 2) HLCALLS(A, 4) HLCALLS(A, 8) HLCALLS(A, 16)
 HLCALL5(OUTLINE_ATOMIC_CAS)
 HLCALL5(OUTLINE_ATOMIC_SWP)
 HLCALL5(OUTLINE_ATOMIC_LDADD)
 HLCALL5(OUTLINE_ATOMIC_LDSET)
 HLCALL5(OUTLINE_ATOMIC_LDCLR)
 HLCALL5(OUTLINE_ATOMIC_LDEOR)

llvm/include/llvm/Support/AtomicOrdering.h

enum class AtomicOrdering : unsigned {
   NotAtomic = 0,
   Unordered = 1,
   Monotonic = 2, // Equivalent to C++'s relaxed.
   // Consume = 3,  // Not specified yet.
   Acquire = 4,
   Release = 5,
   AcquireRelease = 6,
   SequentiallyConsistent = 7,
-  LAST = SequentiallyConsistent
+  // Special orderings for legacy __sync builtins specified by Intel Itanium
+  // Processor-specific Application Binary Interface section 7.4
+  // release-->sync_rel, acquire-->sync_acq, seq_cst-->sync_seq_cst
+  SyncAcquire = 8,
+  SyncRelease = 9,
+  SyncSequentiallyConsistent = 10,
+  LAST = SyncSequentiallyConsistent
 };
 
 bool operator<(AtomicOrdering, AtomicOrdering) = delete;
 bool operator>(AtomicOrdering, AtomicOrdering) = delete;
 bool operator<=(AtomicOrdering, AtomicOrdering) = delete;
 bool operator>=(AtomicOrdering, AtomicOrdering) = delete;
 
 // Validate an integral value which isn't known to fit within the enum's range
 // is a valid AtomicOrdering.
 template <typename Int> inline bool isValidAtomicOrdering(Int I) {
   return static_cast<Int>(AtomicOrdering::NotAtomic) <= I &&
-         I <= static_cast<Int>(AtomicOrdering::SequentiallyConsistent);
+         I <= static_cast<Int>(AtomicOrdering::SyncSequentiallyConsistent);
 }
 
 /// String used by LLVM IR to represent atomic ordering.
 inline const char *toIRString(AtomicOrdering ao) {
-  static const char *names[8] = {"not_atomic", "unordered", "monotonic",
-                                 "consume",    "acquire",   "release",
-                                 "acq_rel",    "seq_cst"};
+  static const char *names[11] = {"not_atomic", "unordered",   "monotonic",
+                                  "consume",    "acquire",     "release",
+                                  "acq_rel",    "seq_cst",     "sync_acq",
+                                  "sync_rel",   "sync_seq_cst"};
   return names[static_cast<size_t>(ao)];
 }
 
 /// Returns true if ao is stronger than other as defined by the AtomicOrdering
 /// lattice, which is based on C++'s definition.
 inline bool isStrongerThan(AtomicOrdering AO, AtomicOrdering Other) {
-  static const bool lookup[8][8] = {
-      //               NA     UN     RX     CO     AC     RE     AR     SC
-      /* NotAtomic */ {false, false, false, false, false, false, false, false},
-      /* Unordered */ { true, false, false, false, false, false, false, false},
-      /* relaxed   */ { true,  true, false, false, false, false, false, false},
-      /* consume   */ { true,  true,  true, false, false, false, false, false},
-      /* acquire   */ { true,  true,  true,  true, false, false, false, false},
-      /* release   */ { true,  true,  true, false, false, false, false, false},
-      /* acq_rel   */ { true,  true,  true,  true,  true,  true, false, false},
-      /* seq_cst   */ { true,  true,  true,  true,  true,  true,  true, false},
+  static const bool lookup[11][11] = {
+      //                  NA     UN     RX     CO     AC     RE     AR     SC
+      //                  SAC    SRE    SSC
+      /* NotAtomic    */ {false, false, false, false, false, false, false,
+                          false, false, false, false},
+      /* Unordered    */
+      {true, false, false, false, false, false, false, false, false, false,
+       false},
+      /* relaxed      */
+      {true, true, false, false, false, false, false, false, false, false,
+       false},
+      /* consume      */
+      {true, true, true, false, false, false, false, false, false, false,
+       false},
+      /* acquire      */
+      {true, true, true, true, false, false, false, false, false, false, false},
+      /* release      */
+      {true, true, true, false, false, false, false, false, false, false,
+       false},
+      /* acq_rel      */
+      {true, true, true, true, true, true, false, false, false, false, false},
+      /* seq_cst      */
+      {true, true, true, true, true, true, true, false, false, false, false},
+      /* sync_acq     */
+      {true, true, true, true, true, false, false, false, false, false, false},
+      /* sync_rel     */
+      {true, true, true, false, false, true, false, false, false, false, false},
+      /* sync_seq_cst */
+      {true, true, true, true, true, true, true, true, true, true, false},
   };
   return lookup[static_cast<size_t>(AO)][static_cast<size_t>(Other)];
 }
 
 inline bool isAtLeastOrStrongerThan(AtomicOrdering AO, AtomicOrdering Other) {
-  static const bool lookup[8][8] = {
-      //               NA     UN     RX     CO     AC     RE     AR     SC
-      /* NotAtomic */ { true, false, false, false, false, false, false, false},
-      /* Unordered */ { true,  true, false, false, false, false, false, false},
-      /* relaxed   */ { true,  true,  true, false, false, false, false, false},
-      /* consume   */ { true,  true,  true,  true, false, false, false, false},
-      /* acquire   */ { true,  true,  true,  true,  true, false, false, false},
-      /* release   */ { true,  true,  true, false, false,  true, false, false},
-      /* acq_rel   */ { true,  true,  true,  true,  true,  true,  true, false},
-      /* seq_cst   */ { true,  true,  true,  true,  true,  true,  true,  true},
+  static const bool lookup[11][11] = {
+      //                  NA     UN     RX     CO     AC     RE     AR     SC
+      //                  SAC    SRE    SSC
+      /* NotAtomic    */ {true, false, false, false, false, false, false, false,
+                          false, false, false},
+      /* Unordered    */
+      {true, true, false, false, false, false, false, false, false, false,
+       false},
+      /* relaxed      */
+      {true, true, true, false, false, false, false, false, false, false,
+       false},
+      /* consume      */
+      {true, true, true, true, false, false, false, false, false, false, false},
+      /* acquire      */
+      {true, true, true, true, true, false, false, false, false, false, false},
+      /* release      */
+      {true, true, true, false, false, true, false, false, false, false, false},
+      /* acq_rel      */
+      {true, true, true, true, true, true, true, false, false, false, false},
+      /* seq_cst      */
+      {true, true, true, true, true, true, true, true, false, false, false},
+      /* sync_acq     */
+      {true, true, true, true, true, false, false, false, true, false, false},
+      /* sync_rel     */
+      {true, true, true, false, false, true, false, false, false, true, false},
+      /* sync_seq_cst */
+      {true, true, true, true, true, true, true, true, true, true, true},
   };
   return lookup[static_cast<size_t>(AO)][static_cast<size_t>(Other)];
 }
 
 inline bool isStrongerThanUnordered(AtomicOrdering AO) {
   return isStrongerThan(AO, AtomicOrdering::Unordered);
 }
 
inline AtomicOrdering getMergedAtomicOrdering(AtomicOrdering AO,
                                               AtomicOrdering Other) {
   if ((AO == AtomicOrdering::Acquire && Other == AtomicOrdering::Release) ||
       (AO == AtomicOrdering::Release && Other == AtomicOrdering::Acquire))
     return AtomicOrdering::AcquireRelease;
   return isStrongerThan(AO, Other) ? AO : Other;
 }
 
 inline AtomicOrderingCABI toCABI(AtomicOrdering AO) {
-  static const AtomicOrderingCABI lookup[8] = {
-      /* NotAtomic */ AtomicOrderingCABI::relaxed,
-      /* Unordered */ AtomicOrderingCABI::relaxed,
-      /* relaxed   */ AtomicOrderingCABI::relaxed,
-      /* consume   */ AtomicOrderingCABI::consume,
-      /* acquire   */ AtomicOrderingCABI::acquire,
-      /* release   */ AtomicOrderingCABI::release,
-      /* acq_rel   */ AtomicOrderingCABI::acq_rel,
-      /* seq_cst   */ AtomicOrderingCABI::seq_cst,
-  };
+  static const AtomicOrderingCABI lookup[11] = {
+      /* NotAtomic    */ AtomicOrderingCABI::relaxed,
+      /* Unordered    */ AtomicOrderingCABI::relaxed,
+      /* relaxed      */ AtomicOrderingCABI::relaxed,
+      /* consume      */ AtomicOrderingCABI::consume,
+      /* acquire      */ AtomicOrderingCABI::acquire,
+      /* release      */ AtomicOrderingCABI::release,
+      /* acq_rel      */ AtomicOrderingCABI::acq_rel,
+      /* seq_cst      */ AtomicOrderingCABI::seq_cst,
+      /* sync_acq     */ AtomicOrderingCABI::acquire,
+      /* sync_rel     */ AtomicOrderingCABI::release,
+      /* sync_seq_cst */ AtomicOrderingCABI::seq_cst};
   return lookup[static_cast<size_t>(AO)];
 }
 
 } // end namespace llvm
 
 #endif // LLVM_SUPPORT_ATOMICORDERING_H

llvm/include/llvm/Target/TargetSelectionDAG.td

class PatFrags<dag ops, list<dag> frags, code pred = [{}],
   // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::Acquire
   bit IsAtomicOrderingAcquire = ?;
   // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::Release
   bit IsAtomicOrderingRelease = ?;
   // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::AcquireRelease
   bit IsAtomicOrderingAcquireRelease = ?;
   // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::SequentiallyConsistent
   bit IsAtomicOrderingSequentiallyConsistent = ?;
+  // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::SyncAcquire
+  bit IsAtomicOrderingSyncAcquire = ?;
+  // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::SyncRelease
+  bit IsAtomicOrderingSyncRelease = ?;
+  // cast<AtomicSDNode>(N)->getOrdering() == AtomicOrdering::SyncSequentiallyConsistent
+  bit IsAtomicOrderingSyncSequentiallyConsistent = ?;
 
   // isAcquireOrStronger(cast<AtomicSDNode>(N)->getOrdering())
   // !isAcquireOrStronger(cast<AtomicSDNode>(N)->getOrdering())
   bit IsAtomicOrderingAcquireOrStronger = ?;
 
   // isReleaseOrStronger(cast<AtomicSDNode>(N)->getOrdering())
   // !isReleaseOrStronger(cast<AtomicSDNode>(N)->getOrdering())
   bit IsAtomicOrderingReleaseOrStronger = ?;
def NAME#_acq_rel : PatFrag<(ops node:$ptr, node:$val),
     let IsAtomic = true;
     let IsAtomicOrderingAcquireRelease = true;
   }
   def NAME#_seq_cst : PatFrag<(ops node:$ptr, node:$val),
       (!cast<SDPatternOperator>(NAME) node:$ptr, node:$val)> {
     let IsAtomic = true;
     let IsAtomicOrderingSequentiallyConsistent = true;
   }
+  def NAME#_sync_acq : PatFrag<(ops node:$ptr, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$val)> {
+  //  let IsAtomic = true;
+    let IsAtomicOrderingSyncAcquire = true;
+  }
+  def NAME#_sync_rel : PatFrag<(ops node:$ptr, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$val)> {
+  //  let IsAtomic = true;
+    let IsAtomicOrderingSyncRelease = true;
+  }
+  def NAME#_sync_seq_cst : PatFrag<(ops node:$ptr, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$val)> {
+ //   let IsAtomic = true;
+    let IsAtomicOrderingSyncSequentiallyConsistent = true;
+  }
 }
 
 multiclass ternary_atomic_op_ord {
   def NAME#_monotonic : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
       (!cast<SDPatternOperator>(NAME) node:$ptr, node:$cmp, node:$val)> {
     let IsAtomic = true;
     let IsAtomicOrderingMonotonic = true;
   }
def NAME#_acq_rel : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
     let IsAtomic = true;
     let IsAtomicOrderingAcquireRelease = true;
   }
   def NAME#_seq_cst : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
       (!cast<SDPatternOperator>(NAME) node:$ptr, node:$cmp, node:$val)> {
     let IsAtomic = true;
     let IsAtomicOrderingSequentiallyConsistent = true;
   }
+  def NAME#_sync_acq : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$cmp, node:$val)> {
+  //  let IsAtomic = true;
+    let IsAtomicOrderingSyncAcquire = true;
+  }
+  def NAME#_sync_rel : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$cmp, node:$val)> {
+  //  let IsAtomic = true;
+    let IsAtomicOrderingSyncRelease = true;
+  }
+  def NAME#_sync_seq_cst : PatFrag<(ops node:$ptr, node:$cmp, node:$val),
+      (!cast<SDPatternOperator>(NAME) node:$ptr, node:$cmp, node:$val)> {
+ //   let IsAtomic = true;
+    let IsAtomicOrderingSyncSequentiallyConsistent = true;
+  }
 }
 
 multiclass binary_atomic_op<SDNode atomic_op, bit IsInt = 1> {
   def _8 : PatFrag<(ops node:$ptr, node:$val),
                    (atomic_op  node:$ptr, node:$val)> {
     let IsAtomic = true;
     let MemoryVT = !if(IsInt, i8, ?);
   }

llvm/lib/AsmParser/LLLexer.cpp

#define KEYWORD(STR)                                                           \
   KEYWORD(volatile);
   KEYWORD(atomic);
   KEYWORD(unordered);
   KEYWORD(monotonic);
   KEYWORD(acquire);
   KEYWORD(release);
   KEYWORD(acq_rel);
   KEYWORD(seq_cst);
+  KEYWORD(sync_acq);
+  KEYWORD(sync_rel);
+  KEYWORD(sync_seq_cst);
   KEYWORD(syncscope);
 
   KEYWORD(nnan);
   KEYWORD(ninf);
   KEYWORD(nsz);
   KEYWORD(arcp);
   KEYWORD(contract);
   KEYWORD(reassoc);

llvm/lib/AsmParser/LLParser.cpp

bool LLParser::parseOrdering(AtomicOrdering &Ordering) {
   // Not specified yet:
   // case lltok::kw_consume: Ordering = AtomicOrdering::Consume; break;
   case lltok::kw_acquire: Ordering = AtomicOrdering::Acquire; break;
   case lltok::kw_release: Ordering = AtomicOrdering::Release; break;
   case lltok::kw_acq_rel: Ordering = AtomicOrdering::AcquireRelease; break;
   case lltok::kw_seq_cst:
     Ordering = AtomicOrdering::SequentiallyConsistent;
     break;
+  case lltok::kw_sync_acq:
+    Ordering = AtomicOrdering::SyncAcquire;
+    break;
+  case lltok::kw_sync_rel:
+    Ordering = AtomicOrdering::SyncRelease;
+    break;
+  case lltok::kw_sync_seq_cst:
+    Ordering = AtomicOrdering::SyncSequentiallyConsistent;
+    break;
   }
   Lex.Lex();
   return false;
 }
 
 /// parseOptionalStackAlignment
 ///   ::= /* empty */
 ///   ::= 'alignstack' '(' 4 ')'

llvm/lib/Bitcode/Reader/BitcodeReader.cpp

 static AtomicOrdering getDecodedOrdering(unsigned Val) {
   switch (Val) {
   case bitc::ORDERING_NOTATOMIC: return AtomicOrdering::NotAtomic;
   case bitc::ORDERING_UNORDERED: return AtomicOrdering::Unordered;
   case bitc::ORDERING_MONOTONIC: return AtomicOrdering::Monotonic;
   case bitc::ORDERING_ACQUIRE: return AtomicOrdering::Acquire;
   case bitc::ORDERING_RELEASE: return AtomicOrdering::Release;
   case bitc::ORDERING_ACQREL: return AtomicOrdering::AcquireRelease;
+  case bitc::ORDERING_SYNC_ACQ:
+    return AtomicOrdering::SyncAcquire;
+  case bitc::ORDERING_SYNC_REL:
+    return AtomicOrdering::SyncRelease;
+  case bitc::ORDERING_SYNC_SEQCST:
+    return AtomicOrdering::SyncSequentiallyConsistent;
   default: // Map unknown orderings to sequentially-consistent.
   case bitc::ORDERING_SEQCST: return AtomicOrdering::SequentiallyConsistent;
   }
 }
 
 static Comdat::SelectionKind getDecodedComdatSelectionKind(unsigned Val) {
   switch (Val) {
   default: // Map unknown selection kinds to any.

llvm/lib/Bitcode/Writer/BitcodeWriter.cpp

static unsigned getEncodedOrdering(AtomicOrdering Ordering) {
   switch (Ordering) {
   case AtomicOrdering::NotAtomic: return bitc::ORDERING_NOTATOMIC;
   case AtomicOrdering::Unordered: return bitc::ORDERING_UNORDERED;
   case AtomicOrdering::Monotonic: return bitc::ORDERING_MONOTONIC;
   case AtomicOrdering::Acquire: return bitc::ORDERING_ACQUIRE;
   case AtomicOrdering::Release: return bitc::ORDERING_RELEASE;
   case AtomicOrdering::AcquireRelease: return bitc::ORDERING_ACQREL;
   case AtomicOrdering::SequentiallyConsistent: return bitc::ORDERING_SEQCST;
+  case AtomicOrdering::SyncAcquire:
+    return bitc::ORDERING_SYNC_ACQ;
+  case AtomicOrdering::SyncRelease:
+    return bitc::ORDERING_SYNC_REL;
+  case AtomicOrdering::SyncSequentiallyConsistent:
+    return bitc::ORDERING_SYNC_SEQCST;
   }
   llvm_unreachable("Invalid ordering");
 }
 
 static void writeStringRecord(BitstreamWriter &Stream, unsigned Code,
                               StringRef Str, unsigned AbbrevToUse) {
   SmallVector<unsigned, 64> Vals;
 

llvm/lib/CodeGen/AtomicExpandPass.cpp

OldResult = insertRMWCmpXchgLoop(Builder, PMV.WordType, PMV.AlignedAddr,
                                      PerformPartwordOp, createCmpXchgInstFun);
   } else {
     assert(ExpansionKind == TargetLoweringBase::AtomicExpansionKind::LLSC);
     OldResult = insertRMWLLSCLoop(Builder, PMV.WordType, PMV.AlignedAddr,
                                   PMV.AlignedAddrAlignment, MemOpOrder,
                                   PerformPartwordOp);
   }
 
+  bool HasAdditionalBarriers =
+      (MemOpOrder == AtomicOrdering::SyncAcquire ||
+       MemOpOrder == AtomicOrdering::SyncRelease ||
+       MemOpOrder == AtomicOrdering::SyncSequentiallyConsistent);
+  if (HasAdditionalBarriers)
+    TLI->emitTrailingFence(Builder, AI, MemOpOrder);
+
   Value *FinalOldResult = extractMaskedValue(Builder, OldResult, PMV);
   AI->replaceAllUsesWith(FinalOldResult);
   AI->eraseFromParent();
 }
 
 // Widen the bitwise atomicrmw (or/xor/and) to the minimum supported width.
 AtomicRMWInst *AtomicExpand::widenPartwordAtomicRMW(AtomicRMWInst *AI) {
   IRBuilder<> Builder(AI);

 
 void AtomicExpand::expandAtomicOpToLLSC(
     Instruction *I, Type *ResultType, Value *Addr, Align AddrAlign,
     AtomicOrdering MemOpOrder,
     function_ref<Value *(IRBuilder<> &, Value *)> PerformOp) {
   IRBuilder<> Builder(I);
   Value *Loaded = insertRMWLLSCLoop(Builder, ResultType, Addr, AddrAlign,
                                     MemOpOrder, PerformOp);
-
+  bool HasAdditionalBarriers =
+      (MemOpOrder == AtomicOrdering::SyncAcquire ||
+       MemOpOrder == AtomicOrdering::SyncRelease ||
+       MemOpOrder == AtomicOrdering::SyncSequentiallyConsistent);
+  if (HasAdditionalBarriers)
+    TLI->emitTrailingFence(Builder, I, MemOpOrder);
   I->replaceAllUsesWith(Loaded);
   I->eraseFromParent();
 }
 
 void AtomicExpand::expandAtomicRMWToMaskedIntrinsic(AtomicRMWInst *AI) {
   IRBuilder<> Builder(AI);
 
   PartwordMaskValues PMV =
Value *AtomicExpand::insertRMWLLSCLoop(
 
   Value *StoreSuccess =
       TLI->emitStoreConditional(Builder, NewVal, Addr, MemOpOrder);
   Value *TryAgain = Builder.CreateICmpNE(
       StoreSuccess, ConstantInt::get(IntegerType::get(Ctx, 32), 0), "tryagain");
   Builder.CreateCondBr(TryAgain, LoopBB, ExitBB);
 
   Builder.SetInsertPoint(ExitBB, ExitBB->begin());
+
   return Loaded;
 }
 
 /// Convert an atomic cmpxchg of a non-integral type to an integer cmpxchg of
 /// the equivalent bitwidth.  We used to not support pointer cmpxchg in the
 /// IR.  As a migration step, we convert back to what use to be the standard
 /// way to represent a pointer cmpxchg so that we can update backends one by
 /// one.
bool AtomicExpand::expandAtomicCmpXchg(AtomicCmpXchgInst *CI) {
   // since in other cases the extra blocks naturally collapse down to the
   // minimal loop. Unfortunately, this puts too much stress on later
   // optimisations so we avoid emitting the extra logic in those cases too.
   bool HasReleasedLoadBB = !CI->isWeak() && ShouldInsertFencesForAtomic &&
                            SuccessOrder != AtomicOrdering::Monotonic &&
                            SuccessOrder != AtomicOrdering::Acquire &&
                            !F->hasMinSize();
 
+  bool HasAdditionalBarriers =
+      (SuccessOrder == AtomicOrdering::SyncSequentiallyConsistent ||
+       SuccessOrder == AtomicOrdering::SyncAcquire ||
+       SuccessOrder == AtomicOrdering::SyncRelease);
+
   // There's no overhead for sinking the release barrier in a weak cmpxchg, so
   // do it even on minsize.
   bool UseUnconditionalReleaseBarrier = F->hasMinSize() && !CI->isWeak();
 
   // Given: cmpxchg some_op iN* %addr, iN %desired, iN %new success_ord fail_ord
   //
   // The full expansion we produce is:
   //     [...]
if (HasReleasedLoadBB) {
     // Update PHI node in TryStoreBB.
     LoadedTryStore->addIncoming(SecondLoad, ReleasedLoadBB);
   } else
     Builder.CreateUnreachable();
 
   // Make sure later instructions don't get reordered with a fence if
   // necessary.
   Builder.SetInsertPoint(SuccessBB);
-  if (ShouldInsertFencesForAtomic)
+  if (ShouldInsertFencesForAtomic || HasAdditionalBarriers)
     TLI->emitTrailingFence(Builder, CI, SuccessOrder);
   Builder.CreateBr(ExitBB);
 
   Builder.SetInsertPoint(NoStoreBB);
   PHINode *LoadedNoStore =
       Builder.CreatePHI(UnreleasedLoad->getType(), 2, "loaded.nostore");
   LoadedNoStore->addIncoming(UnreleasedLoad, StartBB);
   if (HasReleasedLoadBB)
     LoadedNoStore->addIncoming(SecondLoad, ReleasedLoadBB);
 
   // In the failing case, where we don't execute the store-conditional, the
   // target might want to balance out the load-linked with a dedicated
   // instruction (e.g., on ARM, clearing the exclusive monitor).
   TLI->emitAtomicCmpXchgNoStoreLLBalance(Builder);
   Builder.CreateBr(FailureBB);
 
   Builder.SetInsertPoint(FailureBB);
   PHINode *LoadedFailure =
       Builder.CreatePHI(UnreleasedLoad->getType(), 2, "loaded.failure");
   LoadedFailure->addIncoming(LoadedNoStore, NoStoreBB);
   if (CI->isWeak())
     LoadedFailure->addIncoming(LoadedTryStore, TryStoreBB);
-  if (ShouldInsertFencesForAtomic)
+  if (ShouldInsertFencesForAtomic || HasAdditionalBarriers)
     TLI->emitTrailingFence(Builder, CI, FailureOrder);
   Builder.CreateBr(ExitBB);
 
   // Finally, we have control-flow based knowledge of whether the cmpxchg
   // succeeded or not. We expose this to later passes by converting any
   // subsequent "icmp eq/ne %loaded, %oldval" into a use of an appropriate
   // PHI.
   Builder.SetInsertPoint(ExitBB, ExitBB->begin());

llvm/lib/CodeGen/MIRParser/MIParser.cpp

bool MIParser::parseOptionalAtomicOrdering(AtomicOrdering &Order) {
 
   Order = StringSwitch<AtomicOrdering>(Token.stringValue())
               .Case("unordered", AtomicOrdering::Unordered)
               .Case("monotonic", AtomicOrdering::Monotonic)
               .Case("acquire", AtomicOrdering::Acquire)
               .Case("release", AtomicOrdering::Release)
               .Case("acq_rel", AtomicOrdering::AcquireRelease)
               .Case("seq_cst", AtomicOrdering::SequentiallyConsistent)
+              .Case("sync_acq", AtomicOrdering::SyncAcquire)
+              .Case("sync_rel", AtomicOrdering::SyncRelease)
+              .Case("sync_seq_cst", AtomicOrdering::SyncSequentiallyConsistent)
               .Default(AtomicOrdering::NotAtomic);
 
   if (Order != AtomicOrdering::NotAtomic) {
     lex();
     return false;
   }
 
   return error("expected an atomic scope, ordering or a size specification");

llvm/lib/CodeGen/TargetLoweringBase.cpp

case AtomicOrdering::Acquire:
     break;
   case AtomicOrdering::Release:
     ModelN = 2;
     break;
   case AtomicOrdering::AcquireRelease:
   case AtomicOrdering::SequentiallyConsistent:
     ModelN = 3;
     break;
+  case AtomicOrdering::SyncAcquire:
+  case AtomicOrdering::SyncRelease:
+  case AtomicOrdering::SyncSequentiallyConsistent:
+    ModelN = 4;
+    break;
   default:
     return UNKNOWN_LIBCALL;
   }
 
 #define LCALLS(A, B)                                                           \
-  { A##B##_RELAX, A##B##_ACQ, A##B##_REL, A##B##_ACQ_REL }
+  { A##B##_RELAX, A##B##_ACQ, A##B##_REL, A##B##_ACQ_REL, A##B##_SYNC }
 #define LCALL5(A)                                                              \
   LCALLS(A, 1), LCALLS(A, 2), LCALLS(A, 4), LCALLS(A, 8), LCALLS(A, 16)
   switch (Opc) {
   case ISD::ATOMIC_CMP_SWAP: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_CAS)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_CAS)};
     return LC[ModeN][ModelN];
   }
   case ISD::ATOMIC_SWAP: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_SWP)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_SWP)};
     return LC[ModeN][ModelN];
   }
   case ISD::ATOMIC_LOAD_ADD: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_LDADD)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_LDADD)};
     return LC[ModeN][ModelN];
   }
   case ISD::ATOMIC_LOAD_OR: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_LDSET)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_LDSET)};
     return LC[ModeN][ModelN];
   }
   case ISD::ATOMIC_LOAD_CLR: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_LDCLR)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_LDCLR)};
     return LC[ModeN][ModelN];
   }
   case ISD::ATOMIC_LOAD_XOR: {
-    const Libcall LC[5][4] = {LCALL5(OUTLINE_ATOMIC_LDEOR)};
+    const Libcall LC[5][5] = {LCALL5(OUTLINE_ATOMIC_LDEOR)};
     return LC[ModeN][ModelN];
   }
   default:
     return UNKNOWN_LIBCALL;
   }
 #undef LCALLS
 #undef LCALL5
 }

llvm/lib/IR/Core.cpp

 }
 
 static AtomicOrdering mapFromLLVMOrdering(LLVMAtomicOrdering Ordering) {
   switch (Ordering) {
     case LLVMAtomicOrderingNotAtomic: return AtomicOrdering::NotAtomic;
     case LLVMAtomicOrderingUnordered: return AtomicOrdering::Unordered;
     case LLVMAtomicOrderingMonotonic: return AtomicOrdering::Monotonic;
     case LLVMAtomicOrderingAcquire: return AtomicOrdering::Acquire;
+    case LLVMAtomicOrderingSyncAcquire:
+      return AtomicOrdering::SyncAcquire;
     case LLVMAtomicOrderingRelease: return AtomicOrdering::Release;
+    case LLVMAtomicOrderingSyncRelease:
+      return AtomicOrdering::SyncRelease;
     case LLVMAtomicOrderingAcquireRelease:
       return AtomicOrdering::AcquireRelease;
     case LLVMAtomicOrderingSequentiallyConsistent:
       return AtomicOrdering::SequentiallyConsistent;
+    case LLVMAtomicOrderingSyncSequentiallyConsistent:
+      return AtomicOrdering::SyncSequentiallyConsistent;
   }
 
   llvm_unreachable("Invalid LLVMAtomicOrdering value!");
 }
 
 static LLVMAtomicOrdering mapToLLVMOrdering(AtomicOrdering Ordering) {
   switch (Ordering) {
     case AtomicOrdering::NotAtomic: return LLVMAtomicOrderingNotAtomic;
     case AtomicOrdering::Unordered: return LLVMAtomicOrderingUnordered;
     case AtomicOrdering::Monotonic: return LLVMAtomicOrderingMonotonic;
     case AtomicOrdering::Acquire: return LLVMAtomicOrderingAcquire;
     case AtomicOrdering::Release: return LLVMAtomicOrderingRelease;
-    case AtomicOrdering::AcquireRelease:
       return LLVMAtomicOrderingAcquireRelease;
     case AtomicOrdering::SequentiallyConsistent:
       return LLVMAtomicOrderingSequentiallyConsistent;
+    case AtomicOrdering::SyncAcquire:
+      return LLVMAtomicOrderingSyncAcquire;
+    case AtomicOrdering::SyncRelease:
+      return LLVMAtomicOrderingSyncRelease;
+    case AtomicOrdering::AcquireRelease:
+    case AtomicOrdering::SyncSequentiallyConsistent:
+      return LLVMAtomicOrderingSyncSequentiallyConsistent;
   }
 
   llvm_unreachable("Invalid AtomicOrdering value!");
 }
 
 static AtomicRMWInst::BinOp mapFromLLVMRMWBinOp(LLVMAtomicRMWBinOp BinOp) {
   switch (BinOp) {
     case LLVMAtomicRMWBinOpXchg: return AtomicRMWInst::Xchg;

llvm/lib/IR/Verifier.cpp

void Verifier::visitAtomicRMWInst(AtomicRMWInst &RMWI) {
   Check(AtomicRMWInst::FIRST_BINOP <= Op && Op <= AtomicRMWInst::LAST_BINOP,
         "Invalid binary operation!", &RMWI);
   visitInstruction(RMWI);
 }
 
 void Verifier::visitFenceInst(FenceInst &FI) {
   const AtomicOrdering Ordering = FI.getOrdering();
   Check(Ordering == AtomicOrdering::Acquire ||
+            Ordering == AtomicOrdering::SyncAcquire ||
             Ordering == AtomicOrdering::Release ||
+            Ordering == AtomicOrdering::SyncRelease ||
             Ordering == AtomicOrdering::AcquireRelease ||
-            Ordering == AtomicOrdering::SequentiallyConsistent,
-        "fence instructions may only have acquire, release, acq_rel, or "
-        "seq_cst ordering.",
+            Ordering == AtomicOrdering::SequentiallyConsistent ||
+            Ordering == AtomicOrdering::SyncSequentiallyConsistent,
+        "fence instructions may only have acquire, release, acq_rel, seq_cst, "
+        "sync_acq, sync_rel, or sync_seq_cst ordering.",
         &FI);
   visitInstruction(FI);
 }
 
 void Verifier::visitExtractValueInst(ExtractValueInst &EVI) {
   Check(ExtractValueInst::getIndexedType(EVI.getAggregateOperand()->getType(),
                                          EVI.getIndices()) == EVI.getType(),
         "Invalid ExtractValueInst operands!", &EVI);

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

if (Subtarget->outlineAtomics() && !Subtarget->hasLSE()) {
     setOperationAction(ISD::ATOMIC_LOAD_XOR, MVT::i8, LibCall);
     setOperationAction(ISD::ATOMIC_LOAD_XOR, MVT::i16, LibCall);
     setOperationAction(ISD::ATOMIC_LOAD_XOR, MVT::i32, LibCall);
     setOperationAction(ISD::ATOMIC_LOAD_XOR, MVT::i64, LibCall);
 #define LCALLNAMES(A, B, N)                                                    \
   setLibcallName(A##N##_RELAX, #B #N "_relax");                                \
   setLibcallName(A##N##_ACQ, #B #N "_acq");                                    \
   setLibcallName(A##N##_REL, #B #N "_rel");                                    \
-  setLibcallName(A##N##_ACQ_REL, #B #N "_acq_rel");
+  setLibcallName(A##N##_ACQ_REL, #B #N "_acq_rel");                            \
+  setLibcallName(A##N##_SYNC, #B #N "_sync");
 #define LCALLNAME4(A, B)                                                       \
   LCALLNAMES(A, B, 1)                                                          \
   LCALLNAMES(A, B, 2) LCALLNAMES(A, B, 4) LCALLNAMES(A, B, 8)
 #define LCALLNAME5(A, B)                                                       \
   LCALLNAMES(A, B, 1)                                                          \
   LCALLNAMES(A, B, 2)                                                          \
   LCALLNAMES(A, B, 4) LCALLNAMES(A, B, 8) LCALLNAMES(A, B, 16)
     LCALLNAME5(RTLIB::OUTLINE_ATOMIC_CAS, __aarch64_cas)
AArch64TargetLowering::shouldExpandAtomicCmpXchgInIR(
 
   return AtomicExpansionKind::LLSC;
 }
 
 Value *AArch64TargetLowering::emitLoadLinked(IRBuilderBase &Builder,
                                              Type *ValueTy, Value *Addr,
                                              AtomicOrdering Ord) const {
   Module *M = Builder.GetInsertBlock()->getParent()->getParent();
-  bool IsAcquire = isAcquireOrStronger(Ord);
+  // assert((Ord == AtomicOrdering::SyncAcquire ||
+  //        Ord == AtomicOrdering::SyncRelease) &&
+  //       "Sync acquire/release Memory Ordering not expected");
+  bool IsAcquire = isAcquireOrStronger(Ord) &&
+                   (Ord != llvm::AtomicOrdering::SyncSequentiallyConsistent);
 
   // Since i128 isn't legal and intrinsics don't get type-lowered, the ldrexd
   // intrinsic must return {i64, i64} and we have to recombine them into a
   // single i128 here.
   if (ValueTy->getPrimitiveSizeInBits() == 128) {
     Intrinsic::ID Int =
         IsAcquire ? Intrinsic::aarch64_ldaxp : Intrinsic::aarch64_ldxp;
     Function *Ldxr = Intrinsic::getDeclaration(M, Int);

llvm/lib/Target/RISCV/RISCVExpandAtomicPseudoInsts.cpp

 
 static unsigned getLRForRMW32(AtomicOrdering Ordering) {
   switch (Ordering) {
   default:
     llvm_unreachable("Unexpected AtomicOrdering");
   case AtomicOrdering::Monotonic:
     return RISCV::LR_W;
   case AtomicOrdering::Acquire:
+  case AtomicOrdering::SyncAcquire:
     return RISCV::LR_W_AQ;
   case AtomicOrdering::Release:
+  case AtomicOrdering::SyncRelease:
     return RISCV::LR_W;
   case AtomicOrdering::AcquireRelease:
     return RISCV::LR_W_AQ;
   case AtomicOrdering::SequentiallyConsistent:
+  case AtomicOrdering::SyncSequentiallyConsistent:
     return RISCV::LR_W_AQ_RL;
   }
 }
 
 static unsigned getSCForRMW32(AtomicOrdering Ordering) {
   switch (Ordering) {
   default:
     llvm_unreachable("Unexpected AtomicOrdering");
   case AtomicOrdering::Monotonic:
     return RISCV::SC_W;
   case AtomicOrdering::Acquire:
+  case AtomicOrdering::SyncAcquire:
     return RISCV::SC_W;
   case AtomicOrdering::Release:
+  case AtomicOrdering::SyncRelease:
     return RISCV::SC_W_RL;
   case AtomicOrdering::AcquireRelease:
     return RISCV::SC_W_RL;
   case AtomicOrdering::SequentiallyConsistent:
+  case AtomicOrdering::SyncSequentiallyConsistent:
     return RISCV::SC_W_AQ_RL;
   }
 }
 
 static unsigned getLRForRMW64(AtomicOrdering Ordering) {
   switch (Ordering) {
   default:
     llvm_unreachable("Unexpected AtomicOrdering");
   case AtomicOrdering::Monotonic:
     return RISCV::LR_D;
   case AtomicOrdering::Acquire:
+  case AtomicOrdering::SyncAcquire:
     return RISCV::LR_D_AQ;
   case AtomicOrdering::Release:
+  case AtomicOrdering::SyncRelease:
     return RISCV::LR_D;
   case AtomicOrdering::AcquireRelease:
     return RISCV::LR_D_AQ;
   case AtomicOrdering::SequentiallyConsistent:
+  case AtomicOrdering::SyncSequentiallyConsistent:
     return RISCV::LR_D_AQ_RL;
   }
 }
 
 static unsigned getSCForRMW64(AtomicOrdering Ordering) {
   switch (Ordering) {
   default:
     llvm_unreachable("Unexpected AtomicOrdering");
   case AtomicOrdering::Monotonic:
     return RISCV::SC_D;
   case AtomicOrdering::Acquire:
+  case AtomicOrdering::SyncAcquire:
     return RISCV::SC_D;
   case AtomicOrdering::Release:
+  case AtomicOrdering::SyncRelease:
     return RISCV::SC_D_RL;
   case AtomicOrdering::AcquireRelease:
     return RISCV::SC_D_RL;
   case AtomicOrdering::SequentiallyConsistent:
+  case AtomicOrdering::SyncSequentiallyConsistent:
     return RISCV::SC_D_AQ_RL;
   }
 }
 
 static unsigned getLRForRMW(AtomicOrdering Ordering, int Width) {
   if (Width == 32)
     return getLRForRMW32(Ordering);
   if (Width == 64)

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

if (Constraint.length() == 1) {
     }
   }
   TargetLowering::LowerAsmOperandForConstraint(Op, Constraint, Ops, DAG);
 }
 
 Instruction *RISCVTargetLowering::emitLeadingFence(IRBuilderBase &Builder,
                                                    Instruction *Inst,
                                                    AtomicOrdering Ord) const {
-  if (isa<LoadInst>(Inst) && Ord == AtomicOrdering::SequentiallyConsistent)
+  if (isa<LoadInst>(Inst) &&
+      (Ord == AtomicOrdering::SequentiallyConsistent ||
+       Ord == AtomicOrdering::SyncSequentiallyConsistent))
     return Builder.CreateFence(Ord);
   if (isa<StoreInst>(Inst) && isReleaseOrStronger(Ord))
     return Builder.CreateFence(AtomicOrdering::Release);
   return nullptr;
 }
 
 Instruction *RISCVTargetLowering::emitTrailingFence(IRBuilderBase &Builder,
                                                     Instruction *Inst,

llvm/lib/Target/RISCV/RISCVInstrInfo.td

 // fence acquire -> fence r, rw
 def : Pat<(atomic_fence (XLenVT 4), (timm)), (FENCE 0b10, 0b11)>;
 // fence release -> fence rw, w
 def : Pat<(atomic_fence (XLenVT 5), (timm)), (FENCE 0b11, 0b1)>;
 // fence acq_rel -> fence.tso
 def : Pat<(atomic_fence (XLenVT 6), (timm)), (FENCE_TSO)>;
 // fence seq_cst -> fence rw, rw
 def : Pat<(atomic_fence (XLenVT 7), (timm)), (FENCE 0b11, 0b11)>;
+// fence sync_acq -> fence r, rw
+def : Pat<(atomic_fence (XLenVT 8), (timm)), (FENCE 0b10, 0b11)>;
+// fence sync_rel -> fence rw, w
+def : Pat<(atomic_fence (XLenVT 9), (timm)), (FENCE 0b11, 0b1)>;
+// fence sync_seq_cst -> fence rw, rw
+def : Pat<(atomic_fence (XLenVT 10), (timm)), (FENCE 0b11, 0b11)>;
 
 // Lowering for atomic load and store is defined in RISCVInstrInfoA.td.
 // Although these are lowered to fence+load/store instructions defined in the
 // base RV32I/RV64I ISA, this lowering is only used when the A extension is
 // present. This is necessary as it isn't valid to mix __atomic_* libcalls
 // with inline atomic operations for the same object.
 
 /// Access to system registers

llvm/lib/Target/RISCV/RISCVInstrInfoA.td

multiclass AMOPat<string AtomicOp, string BaseInst> {
   def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_acquire"),
                   !cast<RVInst>(BaseInst#"_AQ")>;
   def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_release"),
                   !cast<RVInst>(BaseInst#"_RL")>;
   def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_acq_rel"),
                   !cast<RVInst>(BaseInst#"_AQ_RL")>;
   def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_seq_cst"),
                   !cast<RVInst>(BaseInst#"_AQ_RL")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_sync_acq"),
+                  !cast<RVInst>(BaseInst#"_AQ")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_sync_rel"),
+                  !cast<RVInst>(BaseInst#"_RL")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_sync_seq_cst"),
+                  !cast<RVInst>(BaseInst#"_AQ_RL")>;
 }
 
 defm : AMOPat<"atomic_swap_32", "AMOSWAP_W">;
 defm : AMOPat<"atomic_load_add_32", "AMOADD_W">;
 defm : AMOPat<"atomic_load_and_32", "AMOAND_W">;
 defm : AMOPat<"atomic_load_or_32", "AMOOR_W">;
 defm : AMOPat<"atomic_load_xor_32", "AMOXOR_W">;
 defm : AMOPat<"atomic_load_max_32", "AMOMAX_W">;
 defm : AMOPat<"atomic_load_min_32", "AMOMIN_W">;
 defm : AMOPat<"atomic_load_umax_32", "AMOMAXU_W">;
 defm : AMOPat<"atomic_load_umin_32", "AMOMINU_W">;
 
 def : Pat<(atomic_load_sub_32_monotonic GPR:$addr, GPR:$incr),
           (AMOADD_W GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(atomic_load_sub_32_acquire GPR:$addr, GPR:$incr),
           (AMOADD_W_AQ GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(atomic_load_sub_32_release GPR:$addr, GPR:$incr),
           (AMOADD_W_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(atomic_load_sub_32_acq_rel GPR:$addr, GPR:$incr),
           (AMOADD_W_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(atomic_load_sub_32_seq_cst GPR:$addr, GPR:$incr),
           (AMOADD_W_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_sync_acq GPR:$addr, GPR:$incr),
+          (AMOADD_W_AQ GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_sync_rel GPR:$addr, GPR:$incr),
+          (AMOADD_W_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_sync_seq_cst GPR:$addr, GPR:$incr),
+          (AMOADD_W_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 
 /// Pseudo AMOs
 
 class PseudoAMO : Pseudo<(outs GPR:$res, GPR:$scratch),
                          (ins GPR:$addr, GPR:$incr, ixlenimm:$ordering), []> {
   let Constraints = "@earlyclobber $res,@earlyclobber $scratch";
   let mayLoad = 1;
   let mayStore = 1;

 def : Pat<(atomic_load_nand_32_acquire GPR:$addr, GPR:$incr),
           (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 4)>;
 def : Pat<(atomic_load_nand_32_release GPR:$addr, GPR:$incr),
           (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 5)>;
 def : Pat<(atomic_load_nand_32_acq_rel GPR:$addr, GPR:$incr),
           (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 6)>;
 def : Pat<(atomic_load_nand_32_seq_cst GPR:$addr, GPR:$incr),
           (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 7)>;
+def : Pat<(atomic_load_nand_32_sync_acq GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 8)>;
+def : Pat<(atomic_load_nand_32_sync_rel GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 9)>;
+def : Pat<(atomic_load_nand_32_sync_seq_cst GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 10)>;
 
 class PseudoMaskedAMO
     : Pseudo<(outs GPR:$res, GPR:$scratch),
              (ins GPR:$addr, GPR:$incr, GPR:$mask, ixlenimm:$ordering), []> {
   let Constraints = "@earlyclobber $res,@earlyclobber $scratch";
   let mayLoad = 1;
   let mayStore = 1;
   let hasSideEffects = 0;
multiclass PseudoCmpXchgPat<string Op, Pseudo CmpXchgInst> {
   def : Pat<(!cast<PatFrag>(Op#"_acquire") GPR:$addr, GPR:$cmp, GPR:$new),
             (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 4)>;
   def : Pat<(!cast<PatFrag>(Op#"_release") GPR:$addr, GPR:$cmp, GPR:$new),
             (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 5)>;
   def : Pat<(!cast<PatFrag>(Op#"_acq_rel") GPR:$addr, GPR:$cmp, GPR:$new),
             (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 6)>;
   def : Pat<(!cast<PatFrag>(Op#"_seq_cst") GPR:$addr, GPR:$cmp, GPR:$new),
             (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 7)>;
+  def : Pat<(!cast<PatFrag>(Op#"_sync_acq") GPR:$addr, GPR:$cmp, GPR:$new),
+            (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 8)>;
+  def : Pat<(!cast<PatFrag>(Op#"_sync_rel") GPR:$addr, GPR:$cmp, GPR:$new),
+            (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 9)>;
+  def : Pat<(!cast<PatFrag>(Op#"_sync_seq_cst") GPR:$addr, GPR:$cmp, GPR:$new),
+            (CmpXchgInst GPR:$addr, GPR:$cmp, GPR:$new, 10)>;
 }
 
 def PseudoCmpXchg32 : PseudoCmpXchg;
 defm : PseudoCmpXchgPat<"atomic_cmp_swap_32", PseudoCmpXchg32>;
 
 def PseudoMaskedCmpXchg32
     : Pseudo<(outs GPR:$res, GPR:$scratch),
              (ins GPR:$addr, GPR:$cmpval, GPR:$newval, GPR:$mask,

 def : Pat<(i64 (atomic_load_sub_64_acquire GPR:$addr, GPR:$incr)),
           (AMOADD_D_AQ GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(i64 (atomic_load_sub_64_release GPR:$addr, GPR:$incr)),
           (AMOADD_D_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(i64 (atomic_load_sub_64_acq_rel GPR:$addr, GPR:$incr)),
           (AMOADD_D_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 def : Pat<(i64 (atomic_load_sub_64_seq_cst GPR:$addr, GPR:$incr)),
           (AMOADD_D_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(i64 (atomic_load_sub_64_sync_acq GPR:$addr, GPR:$incr)),
+          (AMOADD_D_AQ GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(i64 (atomic_load_sub_64_sync_rel GPR:$addr, GPR:$incr)),
+          (AMOADD_D_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(i64 (atomic_load_sub_64_sync_seq_cst GPR:$addr, GPR:$incr)),
+          (AMOADD_D_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
 
 /// 64-bit pseudo AMOs
 
 let Size = 20 in
 def PseudoAtomicLoadNand64 : PseudoAMO;
 // Ordering constants must be kept in sync with the AtomicOrdering enum in
 // AtomicOrdering.h.
 def : Pat<(i64 (atomic_load_nand_64_monotonic GPR:$addr, GPR:$incr)),
           (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 2)>;
 def : Pat<(i64 (atomic_load_nand_64_acquire GPR:$addr, GPR:$incr)),
           (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 4)>;
 def : Pat<(i64 (atomic_load_nand_64_release GPR:$addr, GPR:$incr)),
           (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 5)>;
 def : Pat<(i64 (atomic_load_nand_64_acq_rel GPR:$addr, GPR:$incr)),
           (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 6)>;
 def : Pat<(i64 (atomic_load_nand_64_seq_cst GPR:$addr, GPR:$incr)),
           (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 7)>;
+def : Pat<(i64 (atomic_load_nand_64_sync_acq GPR:$addr, GPR:$incr)),
+          (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 8)>;
+def : Pat<(i64 (atomic_load_nand_64_sync_rel GPR:$addr, GPR:$incr)),
+          (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 9)>;
+def : Pat<(i64 (atomic_load_nand_64_sync_seq_cst GPR:$addr, GPR:$incr)),
+          (PseudoAtomicLoadNand64 GPR:$addr, GPR:$incr, 10)>;
 
 def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_xchg_i64,
                          PseudoMaskedAtomicSwap32>;
 def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_add_i64,
                          PseudoMaskedAtomicLoadAdd32>;
 def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_sub_i64,
                          PseudoMaskedAtomicLoadSub32>;
 def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_nand_i64,

llvm/lib/Transforms/Instrumentation/DataFlowSanitizer.cpp

 static AtomicOrdering addAcquireOrdering(AtomicOrdering AO) {
   switch (AO) {
   case AtomicOrdering::NotAtomic:
     return AtomicOrdering::NotAtomic;
   case AtomicOrdering::Unordered:
   case AtomicOrdering::Monotonic:
   case AtomicOrdering::Acquire:
     return AtomicOrdering::Acquire;
+  case AtomicOrdering::SyncAcquire:
+    return AtomicOrdering::SyncAcquire;
   case AtomicOrdering::Release:
   case AtomicOrdering::AcquireRelease:
     return AtomicOrdering::AcquireRelease;
   case AtomicOrdering::SequentiallyConsistent:
     return AtomicOrdering::SequentiallyConsistent;
+  case AtomicOrdering::SyncRelease:
+  case AtomicOrdering::SyncSequentiallyConsistent:
+    return AtomicOrdering::SyncSequentiallyConsistent;
   }
   llvm_unreachable("Unknown ordering");
 }
 
 Value *StripPointerGEPsAndCasts(Value *V) {
   if (!V->getType()->isPointerTy())
     return V;
 

 static AtomicOrdering addReleaseOrdering(AtomicOrdering AO) {
   switch (AO) {
   case AtomicOrdering::NotAtomic:
     return AtomicOrdering::NotAtomic;
   case AtomicOrdering::Unordered:
   case AtomicOrdering::Monotonic:
   case AtomicOrdering::Release:
     return AtomicOrdering::Release;
+  case AtomicOrdering::SyncRelease:
+    return AtomicOrdering::SyncRelease;
   case AtomicOrdering::Acquire:
   case AtomicOrdering::AcquireRelease:
     return AtomicOrdering::AcquireRelease;
   case AtomicOrdering::SequentiallyConsistent:
     return AtomicOrdering::SequentiallyConsistent;
+  case AtomicOrdering::SyncAcquire:
+  case AtomicOrdering::SyncSequentiallyConsistent:
+    return AtomicOrdering::SyncSequentiallyConsistent;
   }
   llvm_unreachable("Unknown ordering");
 }
 
 void DFSanVisitor::visitStoreInst(StoreInst &SI) {
   auto &DL = SI.getModule()->getDataLayout();
   Value *Val = SI.getValueOperand();
   uint64_t Size = DL.getTypeStoreSize(Val->getType());

llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp

#endif
   AtomicOrdering addReleaseOrdering(AtomicOrdering a) {
     switch (a) {
       case AtomicOrdering::NotAtomic:
         return AtomicOrdering::NotAtomic;
       case AtomicOrdering::Unordered:
       case AtomicOrdering::Monotonic:
       case AtomicOrdering::Release:
         return AtomicOrdering::Release;
+      case AtomicOrdering::SyncRelease:
+        return AtomicOrdering::SyncRelease;
       case AtomicOrdering::Acquire:
       case AtomicOrdering::AcquireRelease:
         return AtomicOrdering::AcquireRelease;
       case AtomicOrdering::SequentiallyConsistent:
         return AtomicOrdering::SequentiallyConsistent;
+      case AtomicOrdering::SyncAcquire:
+      case AtomicOrdering::SyncSequentiallyConsistent:
+        return AtomicOrdering::SyncSequentiallyConsistent;
     }
     llvm_unreachable("Unknown ordering");
   }
 
   Value *makeAddReleaseOrderingTable(IRBuilder<> &IRB) {
     constexpr int NumOrderings = (int)AtomicOrderingCABI::seq_cst + 1;
     uint32_t OrderingTable[NumOrderings] = {};
 
#endif
   AtomicOrdering addAcquireOrdering(AtomicOrdering a) {
     switch (a) {
       case AtomicOrdering::NotAtomic:
         return AtomicOrdering::NotAtomic;
       case AtomicOrdering::Unordered:
       case AtomicOrdering::Monotonic:
       case AtomicOrdering::Acquire:
         return AtomicOrdering::Acquire;
+      case AtomicOrdering::SyncAcquire:
+        return AtomicOrdering::SyncAcquire;
       case AtomicOrdering::Release:
       case AtomicOrdering::AcquireRelease:
         return AtomicOrdering::AcquireRelease;
       case AtomicOrdering::SequentiallyConsistent:
         return AtomicOrdering::SequentiallyConsistent;
+      case AtomicOrdering::SyncRelease:
+      case AtomicOrdering::SyncSequentiallyConsistent:
+        return AtomicOrdering::SyncSequentiallyConsistent;
     }
     llvm_unreachable("Unknown ordering");
   }
 
   Value *makeAddAcquireOrderingTable(IRBuilder<> &IRB) {
     constexpr int NumOrderings = (int)AtomicOrderingCABI::seq_cst + 1;
     uint32_t OrderingTable[NumOrderings] = {};
 

llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp

switch (ord) {
     case AtomicOrdering::Unordered:              LLVM_FALLTHROUGH;
     case AtomicOrdering::Monotonic:              v = 0; break;
     // Not specified yet:
     // case AtomicOrdering::Consume:                v = 1; break;
     case AtomicOrdering::Acquire:                v = 2; break;
     case AtomicOrdering::Release:                v = 3; break;
     case AtomicOrdering::AcquireRelease:         v = 4; break;
     case AtomicOrdering::SequentiallyConsistent: v = 5; break;
+    case AtomicOrdering::SyncAcquire:
+      v = 6;
+      break;
+    case AtomicOrdering::SyncRelease:
+      v = 7;
+      break;
+    case AtomicOrdering::SyncSequentiallyConsistent:
+      v = 8;
+      break;
   }
   return IRB->getInt32(v);
 }
 
 // If a memset intrinsic gets inlined by the code gen, we will miss races on it.
 // So, we either need to ensure the intrinsic is not inlined, or instrument it.
 // We do not instrument memset/memmove/memcpy intrinsics (too complicated),
 // instead we simply replace them with regular function calls, which are then

llvm/test/CodeGen/AArch64/aarch64-sync-builtins.ll

 ; }
 ; void test_synchronize_64() {
 ;   __sync_synchronize();
 ; }
 
 define dso_local i32 @test_fetch_and_add_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_add_32:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB0_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
-; CHECK-NEXT:    add w9, w8, w1
-; CHECK-NEXT:    stlxr w10, w9, [x0]
+; CHECK-NEXT:    ldxr w0, [x8]
+; CHECK-NEXT:    add w9, w0, w1
+; CHECK-NEXT:    stlxr w10, w9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB0_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
-; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_add_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw add ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_fetch_and_add_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_add_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB1_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    add x9, x0, x1
 ; CHECK-NEXT:    stlxr w10, x9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB1_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_add_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw add ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 define dso_local i32 @test_fetch_and_sub_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_sub_32:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB2_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
-; CHECK-NEXT:    sub w9, w8, w1
-; CHECK-NEXT:    stlxr w10, w9, [x0]
+; CHECK-NEXT:    ldxr w0, [x8]
+; CHECK-NEXT:    sub w9, w0, w1
+; CHECK-NEXT:    stlxr w10, w9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB2_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
-; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_sub_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    neg w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw sub ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_fetch_and_sub_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_sub_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB3_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    sub x9, x0, x1
 ; CHECK-NEXT:    stlxr w10, x9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB3_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_sub_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    neg x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw sub ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 define dso_local i32 @test_fetch_and_or_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_or_32:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB4_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
-; CHECK-NEXT:    orr w9, w8, w1
-; CHECK-NEXT:    stlxr w10, w9, [x0]
+; CHECK-NEXT:    ldxr w0, [x8]
+; CHECK-NEXT:    orr w9, w0, w1
+; CHECK-NEXT:    stlxr w10, w9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB4_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
-; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_or_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw or ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_fetch_and_or_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_or_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB5_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    orr x9, x0, x1
 ; CHECK-NEXT:    stlxr w10, x9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB5_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_or_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw or ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 define dso_local i32 @test_fetch_and_and_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_and_32:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB6_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
-; CHECK-NEXT:    and w9, w8, w1
-; CHECK-NEXT:    stlxr w10, w9, [x0]
+; CHECK-NEXT:    ldxr w0, [x8]
+; CHECK-NEXT:    and w9, w0, w1
+; CHECK-NEXT:    stlxr w10, w9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB6_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
-; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_and_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mvn w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw and ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_fetch_and_and_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_and_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB7_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    and x9, x0, x1
 ; CHECK-NEXT:    stlxr w10, x9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB7_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_and_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mvn x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw and ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 define dso_local i32 @test_fetch_and_xor_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_xor_32:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB8_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
-; CHECK-NEXT:    eor w9, w8, w1
-; CHECK-NEXT:    stlxr w10, w9, [x0]
+; CHECK-NEXT:    ldxr w0, [x8]
+; CHECK-NEXT:    eor w9, w0, w1
+; CHECK-NEXT:    stlxr w10, w9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB8_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
-; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_xor_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xor ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_fetch_and_xor_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_xor_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB9_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    eor x9, x0, x1
 ; CHECK-NEXT:    stlxr w10, x9, [x8]
 ; CHECK-NEXT:    cbnz w10, .LBB9_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_fetch_and_xor_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xor ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 define dso_local i32 @test_add_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_add_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB10_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    add w8, w8, w1
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB10_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_add_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_sync
 ; OUTLINE-ATOMICS-NEXT:    add w0, w0, w19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw add ptr %p, i32 %v sync_seq_cst, align 4
   %1 = add i32 %0, %v
   ret i32 %1
 }
 
 define dso_local i64 @test_add_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_add_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB11_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    add x8, x8, x1
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB11_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_add_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_sync
 ; OUTLINE-ATOMICS-NEXT:    add x0, x0, x19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw add ptr %p, i64 %v sync_seq_cst, align 8
   %1 = add i64 %0, %v
   ret i64 %1
 }
 
 define dso_local i32 @test_sub_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sub_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB12_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    sub w8, w8, w1
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB12_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_sub_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    neg w0, w19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd4_sync
 ; OUTLINE-ATOMICS-NEXT:    sub w0, w0, w19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw sub ptr %p, i32 %v sync_seq_cst, align 4
   %1 = sub i32 %0, %v
   ret i32 %1
 }
 
 define dso_local i64 @test_sub_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sub_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB13_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    sub x8, x8, x1
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB13_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_sub_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    neg x0, x19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldadd8_sync
 ; OUTLINE-ATOMICS-NEXT:    sub x0, x0, x19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw sub ptr %p, i64 %v sync_seq_cst, align 8
   %1 = sub i64 %0, %v
   ret i64 %1
 }
 
 define dso_local i32 @test_or_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_or_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB14_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    orr w8, w8, w1
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB14_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_or_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset4_sync
 ; OUTLINE-ATOMICS-NEXT:    orr w0, w0, w19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw or ptr %p, i32 %v sync_seq_cst, align 4
   %1 = or i32 %0, %v
   ret i32 %1
 }
 
 define dso_local i64 @test_or_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_or_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB15_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    orr x8, x8, x1
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB15_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_or_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldset8_sync
 ; OUTLINE-ATOMICS-NEXT:    orr x0, x0, x19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw or ptr %p, i64 %v sync_seq_cst, align 8
   %1 = or i64 %0, %v
   ret i64 %1
 }
 
 define dso_local i32 @test_and_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_and_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB16_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    and w8, w8, w1
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB16_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_and_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mvn w0, w19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr4_sync
 ; OUTLINE-ATOMICS-NEXT:    and w0, w0, w19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw and ptr %p, i32 %v sync_seq_cst, align 4
   %1 = and i32 %0, %v
   ret i32 %1
 }
 
 define dso_local i64 @test_and_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_and_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB17_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    and x8, x8, x1
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB17_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_and_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mvn x0, x19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldclr8_sync
 ; OUTLINE-ATOMICS-NEXT:    and x0, x0, x19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw and ptr %p, i64 %v sync_seq_cst, align 8
   %1 = and i64 %0, %v
   ret i64 %1
 }
 
 define dso_local i32 @test_xor_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_xor_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB18_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    eor w8, w8, w1
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB18_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_xor_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor4_sync
 ; OUTLINE-ATOMICS-NEXT:    eor w0, w0, w19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xor ptr %p, i32 %v sync_seq_cst, align 4
   %1 = xor i32 %0, %v
   ret i32 %1
 }
 
 define dso_local i64 @test_xor_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_xor_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB19_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    eor x8, x8, x1
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB19_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_xor_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x19
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_ldeor8_sync
 ; OUTLINE-ATOMICS-NEXT:    eor x0, x0, x19
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xor ptr %p, i64 %v sync_seq_cst, align 8
   %1 = xor i64 %0, %v
   ret i64 %1
 }
 
 define dso_local i32 @test_nand_and_fetch_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_nand_and_fetch_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB20_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    and w8, w8, w1
 ; CHECK-NEXT:    mvn w8, w8
 ; CHECK-NEXT:    stlxr w9, w8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB20_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov w0, w8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_nand_and_fetch_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:  .LBB20_1: // %atomicrmw.start
 ; OUTLINE-ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
-; OUTLINE-ATOMICS-NEXT:    ldaxr w8, [x0]
+; OUTLINE-ATOMICS-NEXT:    ldxr w8, [x0]
 ; OUTLINE-ATOMICS-NEXT:    and w8, w8, w1
 ; OUTLINE-ATOMICS-NEXT:    mvn w8, w8
 ; OUTLINE-ATOMICS-NEXT:    stlxr w9, w8, [x0]
 ; OUTLINE-ATOMICS-NEXT:    cbnz w9, .LBB20_1
 ; OUTLINE-ATOMICS-NEXT:  // %bb.2: // %atomicrmw.end
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w8
+; OUTLINE-ATOMICS-NEXT:    dmb ish
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw nand ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw nand ptr %p, i32 %v sync_seq_cst, align 4
   %1 = and i32 %0, %v
   %2 = xor i32 %1, -1
   ret i32 %2
 }
 
 define dso_local i64 @test_nand_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_nand_and_fetch_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB21_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    and x8, x8, x1
 ; CHECK-NEXT:    mvn x8, x8
 ; CHECK-NEXT:    stlxr w9, x8, [x0]
 ; CHECK-NEXT:    cbnz w9, .LBB21_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    mov x0, x8
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_nand_and_fetch_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:  .LBB21_1: // %atomicrmw.start
 ; OUTLINE-ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
-; OUTLINE-ATOMICS-NEXT:    ldaxr x8, [x0]
+; OUTLINE-ATOMICS-NEXT:    ldxr x8, [x0]
 ; OUTLINE-ATOMICS-NEXT:    and x8, x8, x1
 ; OUTLINE-ATOMICS-NEXT:    mvn x8, x8
 ; OUTLINE-ATOMICS-NEXT:    stlxr w9, x8, [x0]
 ; OUTLINE-ATOMICS-NEXT:    cbnz w9, .LBB21_1
 ; OUTLINE-ATOMICS-NEXT:  // %bb.2: // %atomicrmw.end
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x8
+; OUTLINE-ATOMICS-NEXT:    dmb ish
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw nand ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw nand ptr %p, i64 %v sync_seq_cst, align 8
   %1 = and i64 %0, %v
   %2 = xor i64 %1, -1
   ret i64 %2
 }
 
 define dso_local i1 @test_bool_cas_32(ptr nocapture noundef %p, i32 noundef %old, i32 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_bool_cas_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB22_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w8, [x0]
+; CHECK-NEXT:    ldxr w8, [x0]
 ; CHECK-NEXT:    cmp w8, w1
 ; CHECK-NEXT:    b.ne .LBB22_4
 ; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
 ; CHECK-NEXT:    // in Loop: Header=BB22_1 Depth=1
 ; CHECK-NEXT:    stlxr w8, w2, [x0]
 ; CHECK-NEXT:    cbnz w8, .LBB22_1
 ; CHECK-NEXT:  // %bb.3:
 ; CHECK-NEXT:    mov w0, #1
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB22_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    clrex
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_bool_cas_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w19, w1
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov w1, w2
 ; OUTLINE-ATOMICS-NEXT:    mov x2, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas4_sync
 ; OUTLINE-ATOMICS-NEXT:    cmp w0, w19
 ; OUTLINE-ATOMICS-NEXT:    cset w0, eq
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i32 %old, i32 %new seq_cst seq_cst, align 4
+  %0 = cmpxchg ptr %p, i32 %old, i32 %new sync_seq_cst sync_seq_cst, align 4
   %1 = extractvalue { i32, i1 } %0, 1
   ret i1 %1
 }
 
 define dso_local i1 @test_bool_cas_64(ptr nocapture noundef %p, i64 noundef %old, i64 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_bool_cas_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .LBB23_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x8, [x0]
+; CHECK-NEXT:    ldxr x8, [x0]
 ; CHECK-NEXT:    cmp x8, x1
 ; CHECK-NEXT:    b.ne .LBB23_4
 ; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
 ; CHECK-NEXT:    // in Loop: Header=BB23_1 Depth=1
 ; CHECK-NEXT:    stlxr w8, x2, [x0]
 ; CHECK-NEXT:    cbnz w8, .LBB23_1
 ; CHECK-NEXT:  // %bb.3:
 ; CHECK-NEXT:    mov w0, #1
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB23_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    clrex
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_bool_cas_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x2
 ; OUTLINE-ATOMICS-NEXT:    mov x2, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas8_sync
 ; OUTLINE-ATOMICS-NEXT:    cmp x0, x19
 ; OUTLINE-ATOMICS-NEXT:    cset w0, eq
 ; OUTLINE-ATOMICS-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i64 %old, i64 %new seq_cst seq_cst, align 8
+  %0 = cmpxchg ptr %p, i64 %old, i64 %new sync_seq_cst sync_seq_cst, align 8
   %1 = extractvalue { i64, i1 } %0, 1
   ret i1 %1
 }
 
 define dso_local i32 @test_sync_val_cas_32(ptr nocapture noundef %p, i32 noundef %old, i32 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sync_val_cas_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB24_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr w0, [x8]
+; CHECK-NEXT:    ldxr w0, [x8]
 ; CHECK-NEXT:    cmp w0, w1
 ; CHECK-NEXT:    b.ne .LBB24_4
 ; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
 ; CHECK-NEXT:    // in Loop: Header=BB24_1 Depth=1
 ; CHECK-NEXT:    stlxr w9, w2, [x8]
 ; CHECK-NEXT:    cbnz w9, .LBB24_1
 ; CHECK-NEXT:  // %bb.3: // %cmpxchg.end
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB24_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    clrex
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_sync_val_cas_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov w1, w2
 ; OUTLINE-ATOMICS-NEXT:    mov x2, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i32 %old, i32 %new seq_cst seq_cst, align 4
+  %0 = cmpxchg ptr %p, i32 %old, i32 %new sync_seq_cst sync_seq_cst, align 4
   %1 = extractvalue { i32, i1 } %0, 0
   ret i32 %1
 }
 
 define dso_local i64 @test_sync_val_cas_64(ptr nocapture noundef %p, i64 noundef %old, i64 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sync_val_cas_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB25_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldaxr x0, [x8]
+; CHECK-NEXT:    ldxr x0, [x8]
 ; CHECK-NEXT:    cmp x0, x1
 ; CHECK-NEXT:    b.ne .LBB25_4
 ; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
 ; CHECK-NEXT:    // in Loop: Header=BB25_1 Depth=1
 ; CHECK-NEXT:    stlxr w9, x2, [x8]
 ; CHECK-NEXT:    cbnz w9, .LBB25_1
 ; CHECK-NEXT:  // %bb.3: // %cmpxchg.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB25_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    clrex
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_sync_val_cas_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x2
 ; OUTLINE-ATOMICS-NEXT:    mov x2, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_cas8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i64 %old, i64 %new seq_cst seq_cst, align 8
+  %0 = cmpxchg ptr %p, i64 %old, i64 %new sync_seq_cst sync_seq_cst, align 8
   %1 = extractvalue { i64, i1 } %0, 0
   ret i64 %1
 }
 
 define dso_local i32 @test_lock_test_and_set_32(ptr nocapture noundef %p, i32 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_lock_test_and_set_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB26_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxr w0, [x8]
-; CHECK-NEXT:    stlxr w9, w1, [x8]
+; CHECK-NEXT:    stxr w9, w1, [x8]
 ; CHECK-NEXT:    cbnz w9, .LBB26_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_lock_test_and_set_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov w0, w1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_swp4_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_swp4_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xchg ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xchg ptr %p, i32 %v sync_acq, align 4
   ret i32 %0
 }
 
 define dso_local i64 @test_lock_test_and_set_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_lock_test_and_set_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:  .LBB27_1: // %atomicrmw.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxr x0, [x8]
-; CHECK-NEXT:    stlxr w9, x1, [x8]
+; CHECK-NEXT:    stxr w9, x1, [x8]
 ; CHECK-NEXT:    cbnz w9, .LBB27_1
 ; CHECK-NEXT:  // %bb.2: // %atomicrmw.end
+; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_lock_test_and_set_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -16
 ; OUTLINE-ATOMICS-NEXT:    mov x8, x0
 ; OUTLINE-ATOMICS-NEXT:    mov x0, x1
 ; OUTLINE-ATOMICS-NEXT:    mov x1, x8
-; OUTLINE-ATOMICS-NEXT:    bl __aarch64_swp8_acq_rel
+; OUTLINE-ATOMICS-NEXT:    bl __aarch64_swp8_sync
 ; OUTLINE-ATOMICS-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  %0 = atomicrmw xchg ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xchg ptr %p, i64 %v sync_acq, align 8
   ret i64 %0
 }
 
 define dso_local void @test_lock_release_32(ptr nocapture noundef writeonly %p) local_unnamed_addr #1 {
 ; CHECK-LABEL: test_lock_release_32:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    stlr wzr, [x0]
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_lock_release_32:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stlr wzr, [x0]
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  store atomic i32 0, ptr %p release, align 4
+  store atomic i32 0, ptr %p sync_rel, align 4
   ret void
 }
 
 define dso_local void @test_lock_release_64(ptr nocapture noundef writeonly %p) local_unnamed_addr #1 {
 ; CHECK-LABEL: test_lock_release_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    stlr xzr, [x0]
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_lock_release_64:
 ; OUTLINE-ATOMICS:       // %bb.0: // %entry
 ; OUTLINE-ATOMICS-NEXT:    stlr xzr, [x0]
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
-  store atomic i64 0, ptr %p release, align 8
+  store atomic i64 0, ptr %p sync_rel, align 8
   ret void
 }
 
 define dso_local void @test_synchronize_64() local_unnamed_addr #0 {
 ; CHECK-LABEL: test_synchronize_64:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ret

llvm/test/CodeGen/RISCV/sync-builtins.ll

 ; }
 ; void test_synchronize_64() {
 ;   __sync_synchronize();
 ; }
 
 define dso_local signext i32 @test_fetch_and_add_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_add_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw add ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_fetch_and_add_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_add_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoadd.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw add ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 
 define dso_local signext i32 @test_fetch_and_sub_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_sub_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    neg a1, a1
-; CHECK-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw sub ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_fetch_and_sub_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_sub_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    neg a1, a1
-; CHECK-NEXT:    amoadd.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw sub ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 
 define dso_local signext i32 @test_fetch_and_or_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_or_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoor.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw or ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_fetch_and_or_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_or_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoor.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoor.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw or ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 
 define dso_local signext i32 @test_fetch_and_and_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_and_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoand.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw and ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_fetch_and_and_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_and_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoand.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoand.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw and ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 
 define dso_local signext i32 @test_fetch_and_xor_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_xor_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoxor.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xor ptr %p, i32 %v sync_seq_cst, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_fetch_and_xor_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_fetch_and_xor_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoxor.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoxor.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xor ptr %p, i64 %v sync_seq_cst, align 8
   ret i64 %0
 }
 
 
 define dso_local signext i32 @test_add_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_add_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    addw a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw add ptr %p, i32 %v sync_seq_cst, align 4
   %1 = add i32 %0, %v
   ret i32 %1
 }
 
 
 define dso_local i64 @test_add_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_add_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoadd.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoadd.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    add a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw add ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw add ptr %p, i64 %v sync_seq_cst, align 8
   %1 = add i64 %0, %v
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_sub_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sub_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    neg a2, a1
-; CHECK-NEXT:    amoadd.w.aqrl a0, a2, (a0)
+; CHECK-NEXT:    amoadd.w.aq a0, a2, (a0)
 ; CHECK-NEXT:    subw a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw sub ptr %p, i32 %v sync_seq_cst, align 4
   %1 = sub i32 %0, %v
   ret i32 %1
 }
 
 
 define dso_local i64 @test_sub_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sub_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    neg a2, a1
-; CHECK-NEXT:    amoadd.d.aqrl a0, a2, (a0)
+; CHECK-NEXT:    amoadd.d.aq a0, a2, (a0)
 ; CHECK-NEXT:    sub a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw sub ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw sub ptr %p, i64 %v sync_seq_cst, align 8
   %1 = sub i64 %0, %v
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_or_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_or_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoor.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    or a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw or ptr %p, i32 %v sync_seq_cst, align 4
   %1 = or i32 %0, %v
   ret i32 %1
 }
 
 
 define dso_local i64 @test_or_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_or_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoor.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoor.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    or a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw or ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw or ptr %p, i64 %v sync_seq_cst, align 8
   %1 = or i64 %0, %v
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_and_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_and_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoand.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    and a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw and ptr %p, i32 %v sync_seq_cst, align 4
   %1 = and i32 %0, %v
   ret i32 %1
 }
 
 
 define dso_local i64 @test_and_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_and_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoand.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoand.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    and a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw and ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw and ptr %p, i64 %v sync_seq_cst, align 8
   %1 = and i64 %0, %v
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_xor_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_xor_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoxor.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    xor a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xor ptr %p, i32 %v sync_seq_cst, align 4
   %1 = xor i32 %0, %v
   ret i32 %1
 }
 
 
 define dso_local i64 @test_xor_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_xor_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoxor.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoxor.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    xor a0, a0, a1
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xor ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xor ptr %p, i64 %v sync_seq_cst, align 8
   %1 = xor i64 %0, %v
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_nand_and_fetch_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_nand_and_fetch_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB20_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.w.aqrl a2, (a0)
+; CHECK-NEXT:    lr.w.aq a2, (a0)
 ; CHECK-NEXT:    and a3, a2, a1
 ; CHECK-NEXT:    not a3, a3
-; CHECK-NEXT:    sc.w.aqrl a3, a3, (a0)
+; CHECK-NEXT:    sc.w a3, a3, (a0)
 ; CHECK-NEXT:    bnez a3, .LBB20_1
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    and a0, a2, a1
 ; CHECK-NEXT:    not a0, a0
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw nand ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw nand ptr %p, i32 %v sync_seq_cst, align 4
   %1 = and i32 %0, %v
   %2 = xor i32 %1, -1
   ret i32 %2
 }
 
 
 define dso_local i64 @test_nand_and_fetch_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_nand_and_fetch_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB21_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.d.aqrl a2, (a0)
+; CHECK-NEXT:    lr.d.aq a2, (a0)
 ; CHECK-NEXT:    and a3, a2, a1
 ; CHECK-NEXT:    not a3, a3
-; CHECK-NEXT:    sc.d.aqrl a3, a3, (a0)
+; CHECK-NEXT:    sc.d a3, a3, (a0)
 ; CHECK-NEXT:    bnez a3, .LBB21_1
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    and a0, a2, a1
 ; CHECK-NEXT:    not a0, a0
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw nand ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw nand ptr %p, i64 %v sync_seq_cst, align 8
   %1 = and i64 %0, %v
   %2 = xor i64 %1, -1
   ret i64 %2
 }
 
 
 define dso_local zeroext i1 @test_bool_cas_32(ptr nocapture noundef %p, i32 noundef signext %old, i32 noundef signext %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_bool_cas_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB22_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.w.aqrl a3, (a0)
+; CHECK-NEXT:    lr.w.aq a3, (a0)
 ; CHECK-NEXT:    bne a3, a1, .LBB22_3
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    # in Loop: Header=BB22_1 Depth=1
-; CHECK-NEXT:    sc.w.aqrl a4, a2, (a0)
+; CHECK-NEXT:    sc.w a4, a2, (a0)
 ; CHECK-NEXT:    bnez a4, .LBB22_1
 ; CHECK-NEXT:  .LBB22_3: # %entry
 ; CHECK-NEXT:    xor a0, a3, a1
 ; CHECK-NEXT:    seqz a0, a0
 ; CHECK-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i32 %old, i32 %new seq_cst seq_cst, align 4
+  %0 = cmpxchg ptr %p, i32 %old, i32 %new sync_seq_cst sync_seq_cst, align 4
   %1 = extractvalue { i32, i1 } %0, 1
   ret i1 %1
 }
 
 
 define dso_local zeroext i1 @test_bool_cas_64(ptr nocapture noundef %p, i64 noundef %old, i64 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_bool_cas_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB23_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.d.aqrl a3, (a0)
+; CHECK-NEXT:    lr.d.aq a3, (a0)
 ; CHECK-NEXT:    bne a3, a1, .LBB23_3
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    # in Loop: Header=BB23_1 Depth=1
-; CHECK-NEXT:    sc.d.aqrl a4, a2, (a0)
+; CHECK-NEXT:    sc.d a4, a2, (a0)
 ; CHECK-NEXT:    bnez a4, .LBB23_1
 ; CHECK-NEXT:  .LBB23_3: # %entry
 ; CHECK-NEXT:    xor a0, a3, a1
 ; CHECK-NEXT:    seqz a0, a0
 ; CHECK-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i64 %old, i64 %new seq_cst seq_cst, align 8
+  %0 = cmpxchg ptr %p, i64 %old, i64 %new sync_seq_cst sync_seq_cst, align 8
   %1 = extractvalue { i64, i1 } %0, 1
   ret i1 %1
 }
 
 
 define dso_local signext i32 @test_sync_val_cas_32(ptr nocapture noundef %p, i32 noundef signext %old, i32 noundef signext %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sync_val_cas_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB24_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.w.aqrl a3, (a0)
+; CHECK-NEXT:    lr.w.aq a3, (a0)
 ; CHECK-NEXT:    bne a3, a1, .LBB24_3
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    # in Loop: Header=BB24_1 Depth=1
-; CHECK-NEXT:    sc.w.aqrl a4, a2, (a0)
+; CHECK-NEXT:    sc.w a4, a2, (a0)
 ; CHECK-NEXT:    bnez a4, .LBB24_1
 ; CHECK-NEXT:  .LBB24_3: # %entry
 ; CHECK-NEXT:    mv a0, a3
 ; CHECK-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i32 %old, i32 %new seq_cst seq_cst, align 4
+  %0 = cmpxchg ptr %p, i32 %old, i32 %new sync_seq_cst sync_seq_cst, align 4
   %1 = extractvalue { i32, i1 } %0, 0
   ret i32 %1
 }
 
 
 define dso_local i64 @test_sync_val_cas_64(ptr nocapture noundef %p, i64 noundef %old, i64 noundef %new) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_sync_val_cas_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:  .LBB25_1: # %entry
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    lr.d.aqrl a3, (a0)
+; CHECK-NEXT:    lr.d.aq a3, (a0)
 ; CHECK-NEXT:    bne a3, a1, .LBB25_3
 ; CHECK-NEXT:  # %bb.2: # %entry
 ; CHECK-NEXT:    # in Loop: Header=BB25_1 Depth=1
-; CHECK-NEXT:    sc.d.aqrl a4, a2, (a0)
+; CHECK-NEXT:    sc.d a4, a2, (a0)
 ; CHECK-NEXT:    bnez a4, .LBB25_1
 ; CHECK-NEXT:  .LBB25_3: # %entry
 ; CHECK-NEXT:    mv a0, a3
 ; CHECK-NEXT:    ret
 entry:
-  %0 = cmpxchg ptr %p, i64 %old, i64 %new seq_cst seq_cst, align 8
+  %0 = cmpxchg ptr %p, i64 %old, i64 %new sync_seq_cst sync_seq_cst, align 8
   %1 = extractvalue { i64, i1 } %0, 0
   ret i64 %1
 }
 
 
 define dso_local signext i32 @test_lock_test_and_set_32(ptr nocapture noundef %p, i32 noundef signext %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_lock_test_and_set_32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoswap.w.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoswap.w.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xchg ptr %p, i32 %v seq_cst, align 4
+  %0 = atomicrmw xchg ptr %p, i32 %v sync_acq, align 4
   ret i32 %0
 }
 
 
 define dso_local i64 @test_lock_test_and_set_64(ptr nocapture noundef %p, i64 noundef %v) local_unnamed_addr #0 {
 ; CHECK-LABEL: test_lock_test_and_set_64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    amoswap.d.aqrl a0, a1, (a0)
+; CHECK-NEXT:    amoswap.d.aq a0, a1, (a0)
 ; CHECK-NEXT:    ret
 entry:
-  %0 = atomicrmw xchg ptr %p, i64 %v seq_cst, align 8
+  %0 = atomicrmw xchg ptr %p, i64 %v sync_acq, align 8
   ret i64 %0
 }
 
 define dso_local void @test_lock_release_32(ptr nocapture noundef writeonly %p) local_unnamed_addr #1 {
 ; CHECK-LABEL: test_lock_release_32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    fence rw, w
 ; CHECK-NEXT:    sw zero, 0(a0)
 ; CHECK-NEXT:    ret
 entry:
-  store atomic i32 0, ptr %p release, align 4
+  store atomic i32 0, ptr %p sync_rel, align 4
   ret void
 }
 
 define dso_local void @test_lock_release_64(ptr nocapture noundef writeonly %p) local_unnamed_addr #1 {
 ; CHECK-LABEL: test_lock_release_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    fence rw, w
 ; CHECK-NEXT:    sd zero, 0(a0)
 ; CHECK-NEXT:    ret
 entry:
-  store atomic i64 0, ptr %p release, align 8
+  store atomic i64 0, ptr %p sync_rel, align 8
   ret void
 }
 
 
 define dso_local void @test_synchronize_64() local_unnamed_addr #0 {
 ; CHECK-LABEL: test_synchronize_64:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    fence rw, rw
 ; CHECK-NEXT:    ret
 entry:
   fence seq_cst
   ret void
 }