[CGP] Remove dead PHI nodes before elimination of mostly empty blocks

[skatkov]

Summary

Before elimination of mostly empty block it makes sense to remove dead PHI nodes.
It open more opportunity for elimination plus eliminates dead code itself.

It appeared that change results in failing many unit tests and some of them I've updated
and for another one I disable this optimization.
The pattern I observed in the tests is that there is a infinite loop without side effects.
As a result after elimination of dead phi node all other related instruction are also removed and
tests stops to check what it is expected.
I guess it is due to test is obtained by bugpoint.

I've added people related to these tests to check whether they are ok with update of the tests.

[efriedma]

AArch64 changes look fine; it looks like atomic lowering is generating weird code that gets simplified here. (It was mostly getting simplified anyway, but after register allocation, so the register couldn't be eliminated.)

[skatkov]

In D158503#4607337, @efriedma wrote:

AArch64 changes look fine; it looks like atomic lowering is generating weird code that gets simplified here. (It was mostly getting simplified anyway, but after register allocation, so the register couldn't be eliminated.)

Thank you, Eli for checking AARch64 changes!

[skatkov]

Hi Eli, as no other feedbacks for a while, may I ask you to review the patch (it looks like it is a simple dead code elimination) and approve if there is no objections?

I'll wait say for Tue to get any possible another feedbacks and land it if no any objections.

I guess if someone will want to update tests it can be done as follow-up.
What do you think?

[efriedma]

The change seems to be an improvement overall.

I don't see any reason anyone would object to the test changes that just disable the transform; that should preserve the original intent of the tests,

A few of the x86 test changes seem a bit tricky to review, particularly llvm/test/CodeGen/X86/tail-opts.ll

llvm/lib/CodeGen/CodeGenPrepare.cpp

887

Maybe this could use a slightly better comment, like "delete phi nodes that could block deleting other empty blocks).

[skatkov]

Thank you, Eli.

I've updated comment and revert back changes for tail-opts.ll

Please take a look.

[efriedma]

Please run some quick benchmarking to make sure this doesn't significantly slow down compile-time. (I don't think it should, but RecursivelyDeleteDeadPHINode is a recursive algorithm, so it isn't entirely obvious.)

Otherwise LGTM

[skatkov]

Thank you, Eli.

Let me do this in the following way, as I did not run compile time benchmarking for LLVM before, I'll land this patch and will monitor http://llvm-compile-time-tracker.com/index.php, if there will be some noticeable regression from my patch I will revert the patch and will think about what to do later.

I've added @nikic to reviewers as he monitored compile time as I know.

[skatkov]

I do not see any visible regressions: https://llvm-compile-time-tracker.com/compare.php?from=692344d87357ded619d216b265a9375f4326d8fb&to=a701b7e368b70688bb4b84dafcaa43fa7c9a3649&stat=instructions:u

llvm/lib/CodeGen/CodeGenPrepare.cpp

 static cl::opt<unsigned>
     HugeFuncThresholdInCGPP("cgpp-huge-func", cl::init(10000), cl::Hidden,
                             cl::desc("Least BB number of huge function."));
 
 static cl::opt<unsigned>
     MaxAddressUsersToScan("cgp-max-address-users-to-scan", cl::init(100),
                           cl::Hidden,
                           cl::desc("Max number of address users to look at"));
+
+static cl::opt<bool>
+    DisableDeletePHIs("disable-cgp-delete-phis", cl::Hidden, cl::init(false),
+                      cl::desc("Disable elimination of dead PHI nodes."));
+
 namespace {
 
 enum ExtType {
   ZeroExtension, // Zero extension has been seen.
   SignExtension, // Sign extension has been seen.
   BothExtension  // This extension type is used if we saw sext after
                  // ZeroExtension had been set, or if we saw zext after
                  // SignExtension had been set. It makes the type
if (BasicBlock *Preheader = L->getLoopPreheader())
       Preheaders.insert(Preheader);
   }
 
   bool MadeChange = false;
   // Copy blocks into a temporary array to avoid iterator invalidation issues
   // as we remove them.
   // Note that this intentionally skips the entry block.
   SmallVector<WeakTrackingVH, 16> Blocks;
-  for (auto &Block : llvm::drop_begin(F))
+  for (auto &Block : llvm::drop_begin(F)) {
+    // Delete phi nodes that could block deleting other empty blocks.
+    if (!DisableDeletePHIs)
+      MadeChange |= DeleteDeadPHIs(&Block, TLInfo);
     Blocks.push_back(&Block);
+  }
 
   for (auto &Block : Blocks) {
     BasicBlock *BB = cast_or_null<BasicBlock>(Block);
     if (!BB)
       continue;
     BasicBlock *DestBB = findDestBlockOfMergeableEmptyBlock(BB);
     if (!DestBB ||
         !isMergingEmptyBlockProfitable(BB, DestBB, Preheaders.count(BB)))

llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse2.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x0, x1, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x0, x1, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc3.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x0, x1, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8a.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x0, x1, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse2.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x1, x0, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x1, x0, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc3.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x1, x0, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8a.ll

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stxrh w9, w1, [x2]
+; -O1:    stxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
 ; -O1:    ldxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxrh w0, [x2]

 ; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrh w0, [x2]
 ; -O0:    cmp w0, w9, uxth
 ; -O0:    stlxrh w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrh w8, [x2]
 ; -O1:    cmp w8, w0, uxth
-; -O1:    stlxrh w9, w1, [x2]
+; -O1:    stlxrh wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
     %r = extractvalue { i16, i1 } %pair, 0
     ret i16 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stxr w9, w1, [x2]
+; -O1:    stxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
 ; -O1:    ldxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr w0, [x2]

 ; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr w0, [x2]
 ; -O0:    cmp w0, w9
 ; -O0:    stlxr w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr w8, [x2]
 ; -O1:    cmp w8, w0
-; -O1:    stlxr w9, w1, [x2]
+; -O1:    stlxr wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
     %r = extractvalue { i32, i1 } %pair, 0
     ret i32 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stxr w9, x1, [x2]
+; -O1:    stxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
 ; -O1:    ldxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
 ; -O0:    ldaxr x0, [x2]

 ; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxr x0, [x2]
 ; -O0:    cmp x0, x9
 ; -O0:    stlxr w8, x1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxr x8, [x2]
 ; -O1:    cmp x8, x0
-; -O1:    stlxr w9, x1, [x2]
+; -O1:    stlxr wzr, x1, [x2]
     %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
     %r = extractvalue { i64, i1 } %pair, 0
     ret i64 %r
 }
 
 define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
 ; -O0:    ldxp x1, x0, [x4]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stxrb w9, w1, [x2]
+; -O1:    stxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
 ; -O1:    ldxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
 ; -O0:    ldaxrb w0, [x2]

 ; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O0:    ldaxrb w0, [x2]
 ; -O0:    cmp w0, w9, uxtb
 ; -O0:    stlxrb w8, w1, [x2]
 ;
 ; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
 ; -O1:    ldaxrb w8, [x2]
 ; -O1:    cmp w8, w0, uxtb
-; -O1:    stlxrb w9, w1, [x2]
+; -O1:    stlxrb wzr, w1, [x2]
     %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
     %r = extractvalue { i8, i1 } %pair, 0
     ret i8 %r
 }
 
 define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
 ; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
 ; CHECK:    bl __atomic_compare_exchange

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-switch-bittest.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-; RUN: llc -mtriple aarch64 -aarch64-enable-atomic-cfg-tidy=0 -stop-after=irtranslator -global-isel -verify-machineinstrs %s -o - 2>&1 | FileCheck %s
+; RUN: llc -mtriple aarch64 -aarch64-enable-atomic-cfg-tidy=0 -stop-after=irtranslator -global-isel -verify-machineinstrs -disable-cgp-delete-phis %s -o - 2>&1 | FileCheck %s
 
 define i32 @test_bittest(i16 %p) {
   ; CHECK-LABEL: name: test_bittest
   ; CHECK: bb.1 (%ir-block.0):
   ; CHECK-NEXT:   successors: %bb.4(0x1b6db6db), %bb.5(0x64924925)
   ; CHECK-NEXT:   liveins: $w0
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0

llvm/test/CodeGen/AArch64/machine_cse.ll

 ; RUN: llc < %s -mtriple=aarch64-linux-gnuabi -O2 -tail-dup-placement=0 | FileCheck %s
 ; -tail-dup-placement causes tail duplication during layout. This breaks the
 ; assumptions of the test case as written (specifically, it creates an
 ; additional cmp instruction, creating a false positive), so we pass
 ; -tail-dup-placement=0 to restore the original behavior
 
 ; marked as external to prevent possible optimizations
 @a = external global i32
 @b = external global i32
 @c = external global i32
 @d = external global i32
 @e = external global i32
 
-define void @combine-sign-comparisons-by-cse(ptr %arg) {
+define i32 @combine-sign-comparisons-by-cse(ptr %arg) {
 ; CHECK: cmp
 ; CHECK: b.ge
 ; CHECK-NOT: cmp
 ; CHECK: b.le
 
 entry:
   %a = load i32, ptr @a, align 4
   %b = load i32, ptr @b, align 4
land.lhs.true3:
   br i1 %cmp4, label %return, label %if.end
 
 if.end:
   br label %return
 
 return:
   %retval.0 = phi i32 [ 0, %if.end ], [ 1, %land.lhs.true3 ], [ 1, %land.lhs.true ]
   store i32 %a, ptr %arg
-  ret void
+  ret i32 %retval.0
 }
 
 define void @combine_vector_zeros(ptr %p, ptr %q) {
 ; CHECK-LABEL: combine_vector_zeros:
 ; CHECK: movi v[[REG:[0-9]+]].2d, #0
 ; CHECK-NOT: movi
 ; CHECK: str d[[REG]], [x0]
 ; CHECK: str q[[REG]], [x1]

llvm/test/CodeGen/Hexagon/autohvx/lower-insert-elt.ll

-; RUN: llc -march=hexagon < %s | FileCheck %s
+; RUN: llc -march=hexagon -disable-cgp-delete-phis < %s | FileCheck %s
 ; REQUIRES: asserts
 
 ; Check that this testcase compiles successfully.
 ; CHECK: vextract
 
 target datalayout = "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048"
 target triple = "hexagon-unknown--elf"
 

llvm/test/CodeGen/Hexagon/postinc-offset.ll

-; RUN: llc -enable-aa-sched-mi -march=hexagon -mcpu=hexagonv5 -rdf-opt=0 \
+; RUN: llc -enable-aa-sched-mi -march=hexagon -mcpu=hexagonv5 -rdf-opt=0 -disable-cgp-delete-phis \
 ; RUN:      < %s | FileCheck %s
 
 ; CHECK: {
 ; CHECK: = memd([[REG0:(r[0-9]+)]]++#8)
 ; CHECK-NOT: memw([[REG0]]+#0) =
 ; CHECK: }
 
 

llvm/test/CodeGen/Hexagon/swp-carried-1.ll

-; RUN: llc -march=hexagon -rdf-opt=0 -disable-hexagon-misched -hexagon-initial-cfg-cleanup=0 -lsr-setupcost-depth-limit=1 < %s -pipeliner-experimental-cg=true | FileCheck %s
+; RUN: llc -march=hexagon -rdf-opt=0 -disable-hexagon-misched -hexagon-initial-cfg-cleanup=0 -lsr-setupcost-depth-limit=1 -disable-cgp-delete-phis < %s -pipeliner-experimental-cg=true | FileCheck %s
 
 ; Test that we generate the correct code when a loop carried value
 ; is scheduled one stage earlier than it's use. The code in
 ; isLoopCarried was returning false in this case, and the generated
 ; code was missing an copy.
 
 ; CHECK: loop0(.LBB0_[[LOOP:.]],
 ; CHECK: .LBB0_[[LOOP]]:

llvm/test/CodeGen/Hexagon/swp-epilog-phi12.ll

-; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 -pipeliner-experimental-cg=true < %s | FileCheck %s
+; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 -pipeliner-experimental-cg=true -disable-cgp-delete-phis < %s | FileCheck %s
 
 ; Test epilogue generation when reading loop-carried dependency from a previous
 ; stage. The first epilogue should read value from iteration N-1 of the kernel.
 
 ; CHECK: loop0
 ; CHECK: r{{[0-9]+}} = add([[REG0:r([0-9]+)]],#8)
 ; CHECK: [[REG0]] = [[REG1:r([0-9]+)]]
 ; CHECK: endloop0

llvm/test/CodeGen/Hexagon/swp-epilog-phi6.ll

-; RUN: llc -march=hexagon -O2 -debug-only=pipeliner -hexagon-initial-cfg-cleanup=0 < %s -o - 2>&1 > /dev/null | FileCheck %s
+; RUN: llc -march=hexagon -O2 -debug-only=pipeliner -hexagon-initial-cfg-cleanup=0 -disable-cgp-delete-phis < %s -o - 2>&1 > /dev/null | FileCheck %s
 ; REQUIRES: asserts
 
 ; Test that the phi in the first epilog block is getter the correct
 ; value from the kernel block. In this bug, the phi was using the value
 ; defined in the loop instead of the Phi valued defined in the kernel.
 ; We need to use the kernel's phi value (if the Phi in the kernel is the
 ; last definition).
 

llvm/test/CodeGen/Hexagon/swp-epilog-phi9.ll

-; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 < %s | FileCheck %s
+; RUN: llc -march=hexagon -hexagon-initial-cfg-cleanup=0 -disable-cgp-delete-phis < %s | FileCheck %s
 
 ; Test that we generate the correct Phi name in the last couple of epilog
 ; blocks, when there are 3 epilog blocks. The Phi was scheduled in stage
 ; 2, so the computation for the number of Phis needs to be adjusted when
 ; the incoming prolog block is from prolog 0 or prolog 1.
 ; Note: the pipeliner no longer generates a 3 stage pipeline for this test.
 
 ; CHECK: loop0

llvm/test/CodeGen/Hexagon/swp-lots-deps.ll

-; RUN: llc -march=hexagon -enable-pipeliner -stats -o /dev/null < %s 2>&1 -pipeliner-experimental-cg=true | FileCheck %s --check-prefix=STATS
+; RUN: llc -march=hexagon -enable-pipeliner -stats -o /dev/null < %s 2>&1 -pipeliner-experimental-cg=true -disable-cgp-delete-phis | FileCheck %s --check-prefix=STATS
 ; REQUIRES: asserts
 
 ; STATS: 1 pipeliner        - Number of loops software pipelined
 
 ; Function Attrs: nounwind
 define i64 @f0(i32 %a0, ptr %a1) #0 {
 b0:
   %v0 = icmp slt i32 %a0, 123469

llvm/test/CodeGen/Hexagon/swp-prolog-phi.ll

-; RUN: llc -march=hexagon -rdf-opt=0 < %s -pipeliner-experimental-cg=true | FileCheck %s
+; RUN: llc -march=hexagon -rdf-opt=0 < %s -pipeliner-experimental-cg=true -disable-cgp-delete-phis | FileCheck %s
 
 ; Test that we generate the correct name for a value in a prolog block. The
 ; pipeliner was using an incorrect value for an instruction in the 2nd prolog
 ; block for a value defined by a Phi. The result was that an instruction in
 ; the 1st and 2nd prolog blocks contain the same operands.
 
 ; CHECK: vcmp.gt([[VREG:(v[0-9]+)]].uh,v{{[0-9]+}}.uh)
 ; CHECK-NOT: vcmp.gt([[VREG]].uh,v{{[0-9]+}}.uh)

llvm/test/CodeGen/Hexagon/swp-tfri.ll

-; RUN: llc -march=hexagon -enable-pipeliner -hexagon-initial-cfg-cleanup=0 -stats -o /dev/null < %s 2>&1 -pipeliner-experimental-cg=true | FileCheck %s --check-prefix=STATS
+; RUN: llc -march=hexagon -enable-pipeliner -hexagon-initial-cfg-cleanup=0 -stats -o /dev/null < %s 2>&1 -pipeliner-experimental-cg=true -disable-cgp-delete-phis | FileCheck %s --check-prefix=STATS
 ; REQUIRES: asserts
 
 ; Check that we handle the case when a value is first defined in the loop.
 
 ; STATS: 1 pipeliner        - Number of loops software pipelined
 
 ; Function Attrs: nounwind
 define fastcc void @f0() #0 {

llvm/test/CodeGen/Hexagon/v6-unaligned-spill.ll

-; RUN: llc -march=hexagon < %s | FileCheck %s
+; RUN: llc -march=hexagon -disable-cgp-delete-phis < %s | FileCheck %s
 
 ; Test that we no longer generate an unaligned vector store for a spill when
 ; a function has an alloca.
 
 ; CHECK: vmem({{.*}}) =
 
 %s.0 = type { [5 x [4 x i8]], i32, i32, i32, i32 }
 

llvm/test/CodeGen/PowerPC/sink-side-effect.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=powerpc64le -mcpu=pwr9 -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -mtriple=powerpc64le -mcpu=pwr9 -verify-machineinstrs -disable-cgp-delete-phis < %s | FileCheck %s
 
 define double @zot(ptr %arg, ptr %arg1, ptr %arg2) {
 ; CHECK-LABEL: zot:
 ; CHECK:       # %bb.0: # %bb
 ; CHECK-NEXT:    bc 12, 20, .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %bb3
 ; CHECK-NEXT:    lhz 5, 0(5)
 ; CHECK-NEXT:    rlwinm. 5, 5, 28, 30, 31

llvm/test/CodeGen/PowerPC/sms-iterator.ll

 ; REQUIRES: asserts
 ; RUN: llc < %s -mtriple=powerpc64le-unknown-linux-gnu -verify-machineinstrs\
-; RUN:       -mcpu=pwr9 --ppc-enable-pipeliner -debug-only=pipeliner 2>&1 \
+; RUN:       -mcpu=pwr9 --ppc-enable-pipeliner -debug-only=pipeliner -disable-cgp-delete-phis 2>&1 \
 ; RUN:       >/dev/null | FileCheck %s
 
 %0 = type { i32, [16 x double] }
 
 ; CHECK: MII = 3 MAX_II = 13 (rec=3, res=2)
 
 define dso_local fastcc double @_ZN3povL9polysolveEiPdS0_() unnamed_addr #0 {
   br label %1

llvm/test/CodeGen/Thumb2/2011-06-07-TwoAddrEarlyClobber.ll

-; RUN: llc -mtriple=thumbv7-apple-darwin10 -arm-atomic-cfg-tidy=0 < %s | FileCheck %s
+; RUN: llc -mtriple=thumbv7-apple-darwin10 -arm-atomic-cfg-tidy=0 -disable-cgp-delete-phis < %s | FileCheck %s
 
 %struct.op = type { ptr, ptr, ptr, i32, i16, i16, i8, i8 }
 
 ; CHECK: Perl_ck_sort
 ; CHECK: ldr
 ; CHECK: mov [[REGISTER:(r[0-9]+)|(lr)]]
 ; CHECK: str {{(r[0-9])|(lr)}}, [[[REGISTER]]], #24
 

llvm/test/CodeGen/X86/2007-01-13-StackPtrIndex.ll

 ; CHECK-NEXT:    jmp .LBB0_3
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_9: # %b1606
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    je .LBB0_10
 ; CHECK-NEXT:  .LBB0_3: # %a29b173
 ; CHECK-NEXT:    # =>This Loop Header: Depth=1
-; CHECK-NEXT:    # Child Loop BB0_37 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_36 Depth 2
 ; CHECK-NEXT:    # Child Loop BB0_19 Depth 2
 ; CHECK-NEXT:    # Child Loop BB0_20 Depth 2
-; CHECK-NEXT:    # Child Loop BB0_21 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_23 Depth 2
-; CHECK-NEXT:    # Child Loop BB0_24 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_26 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_37 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_22 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_23 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_25 Depth 2
 ; CHECK-NEXT:    # Child Loop BB0_38 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_29 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_30 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_28 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_29 Depth 2
 ; CHECK-NEXT:    # Child Loop BB0_39 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_33 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_34 Depth 2
-; CHECK-NEXT:    # Child Loop BB0_36 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_32 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_33 Depth 2
+; CHECK-NEXT:    # Child Loop BB0_35 Depth 2
 ; CHECK-NEXT:    testl %eax, %eax
 ; CHECK-NEXT:    js .LBB0_4
 ; CHECK-NEXT:  # %bb.17: # %b179
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    testq %rdx, %rdx
 ; CHECK-NEXT:    js .LBB0_18
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_37: # %a30b
+; CHECK-NEXT:  .LBB0_36: # %a30b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    je .LBB0_37
+; CHECK-NEXT:    je .LBB0_36
 ; CHECK-NEXT:  .LBB0_18: # %b188
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    testb %r8b, %r8b
 ; CHECK-NEXT:    jne .LBB0_4
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_19: # %a30b294
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    je .LBB0_19
 ; CHECK-NEXT:  .LBB0_4: # %a33b
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    movl %esi, %r9d
 ; CHECK-NEXT:    orl %eax, %r9d
 ; CHECK-NEXT:    jns .LBB0_20
 ; CHECK-NEXT:  .LBB0_5: # %a50b
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    movl %eax, %r10d
 ; CHECK-NEXT:    orl %esi, %r10d
-; CHECK-NEXT:    jns .LBB0_26
+; CHECK-NEXT:    jns .LBB0_25
 ; CHECK-NEXT:  .LBB0_6: # %a57b
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    shrl $31, %r9d
 ; CHECK-NEXT:    testb %r9b, %r9b
-; CHECK-NEXT:    je .LBB0_30
+; CHECK-NEXT:    je .LBB0_29
 ; CHECK-NEXT:  .LBB0_7: # %a66b
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    shrl $31, %r10d
 ; CHECK-NEXT:    testb %r10b, %r10b
 ; CHECK-NEXT:    jne .LBB0_8
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_34: # %a74b
+; CHECK-NEXT:  .LBB0_33: # %a74b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    jne .LBB0_34
-; CHECK-NEXT:  # %bb.35: # %b1582
-; CHECK-NEXT:    # in Loop: Header=BB0_34 Depth=2
+; CHECK-NEXT:    jne .LBB0_33
+; CHECK-NEXT:  # %bb.34: # %b1582
+; CHECK-NEXT:    # in Loop: Header=BB0_33 Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    jne .LBB0_34
+; CHECK-NEXT:    jne .LBB0_33
 ; CHECK-NEXT:  .LBB0_8: # %a93b
 ; CHECK-NEXT:    # in Loop: Header=BB0_3 Depth=1
 ; CHECK-NEXT:    testl %eax, %eax
 ; CHECK-NEXT:    js .LBB0_9
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_36: # %a97b
+; CHECK-NEXT:  .LBB0_35: # %a97b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
 ; CHECK-NEXT:    addss %xmm0, %xmm1
 ; CHECK-NEXT:    addss %xmm0, %xmm1
 ; CHECK-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
 ; CHECK-NEXT:    addss %xmm0, %xmm2
 ; CHECK-NEXT:    addss %xmm1, %xmm2
 ; CHECK-NEXT:    movss %xmm2, i6000(%rip)
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    jne .LBB0_36
+; CHECK-NEXT:    jne .LBB0_35
 ; CHECK-NEXT:    jmp .LBB0_9
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_22: # %b463
+; CHECK-NEXT:  .LBB0_21: # %b377
 ; CHECK-NEXT:    # in Loop: Header=BB0_20 Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    je .LBB0_23
+; CHECK-NEXT:    je .LBB0_22
 ; CHECK-NEXT:  .LBB0_20: # %b341
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Loop Header: Depth=2
-; CHECK-NEXT:    # Child Loop BB0_21 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_37 Depth 3
 ; CHECK-NEXT:    testq %rsi, %rsi
-; CHECK-NEXT:    js .LBB0_22
+; CHECK-NEXT:    js .LBB0_21
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_21: # %a35b
+; CHECK-NEXT:  .LBB0_37: # %a35b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # Parent Loop BB0_20 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    je .LBB0_21
-; CHECK-NEXT:    jmp .LBB0_22
+; CHECK-NEXT:    je .LBB0_37
+; CHECK-NEXT:    jmp .LBB0_21
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_28: # %b1016
-; CHECK-NEXT:    # in Loop: Header=BB0_26 Depth=2
+; CHECK-NEXT:  .LBB0_27: # %b1016
+; CHECK-NEXT:    # in Loop: Header=BB0_25 Depth=2
 ; CHECK-NEXT:    testq %rsi, %rsi
 ; CHECK-NEXT:    jle .LBB0_6
-; CHECK-NEXT:  .LBB0_26: # %b858
+; CHECK-NEXT:  .LBB0_25: # %b858
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Loop Header: Depth=2
 ; CHECK-NEXT:    # Child Loop BB0_38 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_29 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_28 Depth 3
 ; CHECK-NEXT:    testq %rdx, %rdx
-; CHECK-NEXT:    js .LBB0_27
+; CHECK-NEXT:    js .LBB0_26
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_38: # %a53b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
-; CHECK-NEXT:    # Parent Loop BB0_26 Depth=2
+; CHECK-NEXT:    # Parent Loop BB0_25 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    je .LBB0_38
-; CHECK-NEXT:  .LBB0_27: # %b879
-; CHECK-NEXT:    # in Loop: Header=BB0_26 Depth=2
+; CHECK-NEXT:  .LBB0_26: # %b879
+; CHECK-NEXT:    # in Loop: Header=BB0_25 Depth=2
 ; CHECK-NEXT:    testb %r8b, %r8b
-; CHECK-NEXT:    jne .LBB0_28
+; CHECK-NEXT:    jne .LBB0_27
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_29: # %a53b1019
+; CHECK-NEXT:  .LBB0_28: # %a53b1019
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
-; CHECK-NEXT:    # Parent Loop BB0_26 Depth=2
+; CHECK-NEXT:    # Parent Loop BB0_25 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testq %rdx, %rdx
-; CHECK-NEXT:    jle .LBB0_29
-; CHECK-NEXT:    jmp .LBB0_28
+; CHECK-NEXT:    jle .LBB0_28
+; CHECK-NEXT:    jmp .LBB0_27
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_32: # %b1263
-; CHECK-NEXT:    # in Loop: Header=BB0_30 Depth=2
+; CHECK-NEXT:  .LBB0_31: # %b1263
+; CHECK-NEXT:    # in Loop: Header=BB0_29 Depth=2
 ; CHECK-NEXT:    testq %rdx, %rdx
 ; CHECK-NEXT:    jle .LBB0_7
-; CHECK-NEXT:  .LBB0_30: # %b1117
+; CHECK-NEXT:  .LBB0_29: # %b1117
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Loop Header: Depth=2
 ; CHECK-NEXT:    # Child Loop BB0_39 Depth 3
-; CHECK-NEXT:    # Child Loop BB0_33 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_32 Depth 3
 ; CHECK-NEXT:    testq %rsi, %rsi
-; CHECK-NEXT:    js .LBB0_31
+; CHECK-NEXT:    js .LBB0_30
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_39: # %a63b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
-; CHECK-NEXT:    # Parent Loop BB0_30 Depth=2
+; CHECK-NEXT:    # Parent Loop BB0_29 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testq %rsi, %rsi
 ; CHECK-NEXT:    jle .LBB0_39
-; CHECK-NEXT:  .LBB0_31: # %b1139
-; CHECK-NEXT:    # in Loop: Header=BB0_30 Depth=2
+; CHECK-NEXT:  .LBB0_30: # %b1139
+; CHECK-NEXT:    # in Loop: Header=BB0_29 Depth=2
 ; CHECK-NEXT:    testq %rsi, %rsi
-; CHECK-NEXT:    jle .LBB0_32
+; CHECK-NEXT:    jle .LBB0_31
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_33: # %a63b1266
+; CHECK-NEXT:  .LBB0_32: # %a63b1266
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
-; CHECK-NEXT:    # Parent Loop BB0_30 Depth=2
+; CHECK-NEXT:    # Parent Loop BB0_29 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testq %rsi, %rsi
-; CHECK-NEXT:    jle .LBB0_33
-; CHECK-NEXT:    jmp .LBB0_32
+; CHECK-NEXT:    jle .LBB0_32
+; CHECK-NEXT:    jmp .LBB0_31
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_25: # %b712
-; CHECK-NEXT:    # in Loop: Header=BB0_23 Depth=2
+; CHECK-NEXT:  .LBB0_24: # %b712
+; CHECK-NEXT:    # in Loop: Header=BB0_22 Depth=2
 ; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    je .LBB0_5
-; CHECK-NEXT:  .LBB0_23: # %b535
+; CHECK-NEXT:  .LBB0_22: # %b535
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
 ; CHECK-NEXT:    # => This Loop Header: Depth=2
-; CHECK-NEXT:    # Child Loop BB0_24 Depth 3
+; CHECK-NEXT:    # Child Loop BB0_23 Depth 3
 ; CHECK-NEXT:    testq %rdx, %rdx
-; CHECK-NEXT:    js .LBB0_25
+; CHECK-NEXT:    js .LBB0_24
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB0_24: # %a45b
+; CHECK-NEXT:  .LBB0_23: # %a45b
 ; CHECK-NEXT:    # Parent Loop BB0_3 Depth=1
-; CHECK-NEXT:    # Parent Loop BB0_23 Depth=2
+; CHECK-NEXT:    # Parent Loop BB0_22 Depth=2
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    testb %dil, %dil
-; CHECK-NEXT:    je .LBB0_24
-; CHECK-NEXT:    jmp .LBB0_25
+; CHECK-NEXT:    je .LBB0_23
+; CHECK-NEXT:    jmp .LBB0_24
 ; CHECK-NEXT:  .LBB0_10: # %a109b
 ; CHECK-NEXT:    movq %rbp, %rsp
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    retq
 b:
   %r = load i32, ptr %a0
   %r2 = load i32, ptr %a1

llvm/test/CodeGen/X86/2008-04-28-CoalescerBug.ll

 ; CHECK-NEXT:    subl %ecx, %edx
 ; CHECK-NEXT:    incl %edx
 ; CHECK-NEXT:    shrl $16, %edx
 ; CHECK-NEXT:    subl %edx, %ecx
 ; CHECK-NEXT:    testw %cx, %cx
 ; CHECK-NEXT:    je LBB0_4
 ; CHECK-NEXT:  ## %bb.8: ## %bb13236
 ; CHECK-NEXT:    ## in Loop: Header=BB0_4 Depth=1
-; CHECK-NEXT:    cmpw $-1, %cx
 ; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:    jne LBB0_4
 ; CHECK-NEXT:  ## %bb.9: ## %bb13572
 ; CHECK-NEXT:    ## in Loop: Header=BB0_4 Depth=1
 ; CHECK-NEXT:    movzwl %cx, %ecx
 ; CHECK-NEXT:    movl %ecx, %edx
 ; CHECK-NEXT:    shll $16, %edx
 ; CHECK-NEXT:    subl %ecx, %edx

llvm/test/CodeGen/X86/avx512-i1test.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mattr=+avx512f | FileCheck %s
-; RUN: llc < %s -mattr=+avx512f -mattr=+avx512vl -mattr=+avx512bw -mattr=+avx512dq | FileCheck %s
+; RUN: llc < %s -mattr=+avx512f -disable-cgp-delete-phis | FileCheck %s
+; RUN: llc < %s -mattr=+avx512f -mattr=+avx512vl -mattr=+avx512bw -mattr=+avx512dq -disable-cgp-delete-phis | FileCheck %s
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @func() {
 ; CHECK-LABEL: func:
 ; CHECK:       # %bb.0: # %bb1
 ; CHECK-NEXT:    xorl %eax, %eax

llvm/test/CodeGen/X86/ins_subreg_coalesce-3.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-- | FileCheck %s
+; RUN: llc < %s -mtriple=x86_64-- -disable-cgp-delete-phis | FileCheck %s
 
   %struct.COMPOSITE = type { i8, i16, i16 }
   %struct.FILE = type { ptr, i32, i32, i16, i16, %struct.__sbuf, i32, ptr, ptr, ptr, ptr, ptr, %struct.__sbuf, ptr, i32, [3 x i8], [1 x i8], %struct.__sbuf, i32, i64 }
   %struct.FILE_POS = type { i8, i8, i16, i32 }
   %struct.FIRST_UNION = type { %struct.FILE_POS }
   %struct.FONT_INFO = type { ptr, ptr, ptr, ptr, i32, ptr, ptr, i16, i16, ptr, ptr, ptr, ptr }
   %struct.FOURTH_UNION = type { %struct.STYLE }
   %struct.GAP = type { i8, i8, i16 }

llvm/test/CodeGen/X86/masked_store.ll

; X86-AVX512-NEXT:    retl
   %mask = and <24 x i1> %mask.src, <i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false, i1 true, i1 false>
   call void @llvm.masked.store.v24i32.p0(<24 x i32> %val, ptr %dst, i32 immarg 1, <24 x i1> %mask)
   ret void
 }
 
 ; From https://reviews.llvm.org/rGf8d9097168b7#1165311
 define void @undefshuffle(<8 x i1> %i0, ptr %src, ptr %dst) #0 {
 ; SSE2-LABEL: undefshuffle:
-; SSE2:       ## %bb.0:
+; SSE2:       ## %bb.0: ## %else
 ; SSE2-NEXT:    movaps %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb $1, %al
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    movd %ecx, %xmm0
-; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    pinsrw $1, %ecx, %xmm0
-; SSE2-NEXT:    movl -{{[0-9]+}}(%rsp), %ecx
-; SSE2-NEXT:    testb %al, %al
-; SSE2-NEXT:    pinsrw $2, %ecx, %xmm0
 ; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
-; SSE2-NEXT:    movb $1, %cl
-; SSE2-NEXT:    testb %cl, %cl
+; SSE2-NEXT:    movd %eax, %xmm0
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    pinsrw $1, %eax, %xmm0
+; SSE2-NEXT:    pinsrw $2, -{{[0-9]+}}(%rsp), %xmm0
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
 ; SSE2-NEXT:    pinsrw $3, %eax, %xmm0
-; SSE2-NEXT:    testb %cl, %cl
 ; SSE2-NEXT:    psllw $15, %xmm0
 ; SSE2-NEXT:    packsswb %xmm0, %xmm0
 ; SSE2-NEXT:    pmovmskb %xmm0, %eax
 ; SSE2-NEXT:    testb $1, %al
 ; SSE2-NEXT:    jne LBB32_1
 ; SSE2-NEXT:  ## %bb.2: ## %else23
 ; SSE2-NEXT:    testb $2, %al
 ; SSE2-NEXT:    jne LBB32_3

 ; SSE2-NEXT:    movl $0, 24(%rsi)
 ; SSE2-NEXT:    testb $-128, %al
 ; SSE2-NEXT:    je LBB32_16
 ; SSE2-NEXT:  LBB32_15: ## %cond.store36
 ; SSE2-NEXT:    movl $0, 28(%rsi)
 ; SSE2-NEXT:    retq
 ;
 ; SSE4-LABEL: undefshuffle:
-; SSE4:       ## %bb.0:
-; SSE4-NEXT:    movb $1, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    testb %al, %al
-; SSE4-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
-; SSE4-NEXT:    testb %al, %al
+; SSE4:       ## %bb.0: ## %else
 ; SSE4-NEXT:    psllw $15, %xmm0
+; SSE4-NEXT:    pand {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
 ; SSE4-NEXT:    packsswb %xmm0, %xmm0
 ; SSE4-NEXT:    pmovmskb %xmm0, %eax
 ; SSE4-NEXT:    testb $1, %al
 ; SSE4-NEXT:    jne LBB32_1
 ; SSE4-NEXT:  ## %bb.2: ## %else23
 ; SSE4-NEXT:    testb $2, %al
 ; SSE4-NEXT:    jne LBB32_3
 ; SSE4-NEXT:  LBB32_4: ## %else25

llvm/test/CodeGen/X86/pr49451.ll

 ; X86-NEXT:    # implicit-def: $si
 ; X86-NEXT:    .p2align 4, 0x90
 ; X86-NEXT:  .LBB0_1: # %for.body612
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NEXT:    testb %dl, %dl
 ; X86-NEXT:    je .LBB0_2
 ; X86-NEXT:  # %bb.3: # %if.end1401
 ; X86-NEXT:    # in Loop: Header=BB0_1 Depth=1
-; X86-NEXT:    testb %dl, %dl
 ; X86-NEXT:    addl %eax, %esi
 ; X86-NEXT:    movw %si, s_2
 ; X86-NEXT:    movw %bx, s_0
 ; X86-NEXT:    incl %ecx
 ; X86-NEXT:    incl %ebx
 ; X86-NEXT:    cmpw $73, %cx
 ; X86-NEXT:    jl .LBB0_1
 ; X86-NEXT:  # %bb.4: # %for.body1703
 ; X86-NEXT:  .LBB0_2: # %if.then671
 ;
 ; X64-LABEL: func_6:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movl $23090, %eax # imm = 0x5A32
 ; X64-NEXT:    xorl %ecx, %ecx
 ; X64-NEXT:    # implicit-def: $dx
 ; X64-NEXT:    .p2align 4, 0x90
 ; X64-NEXT:  .LBB0_1: # %for.body612
 ; X64-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X64-NEXT:    testb %cl, %cl
 ; X64-NEXT:    je .LBB0_2
 ; X64-NEXT:  # %bb.3: # %if.end1401
 ; X64-NEXT:    # in Loop: Header=BB0_1 Depth=1
-; X64-NEXT:    testb %cl, %cl
 ; X64-NEXT:    addl %esi, %edx
 ; X64-NEXT:    movw %dx, s_2(%rip)
 ; X64-NEXT:    leal -23090(%rax), %edi
 ; X64-NEXT:    movw %di, s_0(%rip)
 ; X64-NEXT:    incq %rax
 ; X64-NEXT:    leal -23091(%rax), %edi
 ; X64-NEXT:    cmpw $73, %di
 ; X64-NEXT:    jl .LBB0_1

llvm/test/CodeGen/X86/pr53990-incorrect-machine-sink.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-- < %s | FileCheck %s
 
 declare void @clobber()
 
 define void @test(i1 %c, ptr %p, ptr noalias %p2) nounwind {
 ; CHECK-LABEL: test:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushq %rbp
+; CHECK-NEXT:    pushq %r15
 ; CHECK-NEXT:    pushq %r14
 ; CHECK-NEXT:    pushq %rbx
+; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movq %rdx, %rbx
 ; CHECK-NEXT:    movl %edi, %ebp
 ; CHECK-NEXT:    movq (%rsi), %r14
-; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmpq *.LJTI0_0(,%rax,8)
-; CHECK-NEXT:  .LBB0_1: # %split.3
-; CHECK-NEXT:    testb $1, %bpl
-; CHECK-NEXT:    je .LBB0_3
-; CHECK-NEXT:  # %bb.2: # %clobber
-; CHECK-NEXT:    callq clobber@PLT
-; CHECK-NEXT:  .LBB0_3: # %sink
+; CHECK-NEXT:    movb $1, %r15b
+; CHECK-NEXT:    jmp .LBB0_1
+; CHECK-NEXT:    .p2align 4, 0x90
+; CHECK-NEXT:  .LBB0_4: # %sink
+; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    movq %r14, (%rbx)
-; CHECK-NEXT:  .LBB0_4: # %latch
+; CHECK-NEXT:  .LBB0_1: # %loop
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    jmpq *.LJTI0_0(,%rax,8)
+; CHECK-NEXT:    testb %r15b, %r15b
+; CHECK-NEXT:    jne .LBB0_1
+; CHECK-NEXT:  # %bb.2: # %split.3
+; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
+; CHECK-NEXT:    testb $1, %bpl
+; CHECK-NEXT:    je .LBB0_4
+; CHECK-NEXT:  # %bb.3: # %clobber
+; CHECK-NEXT:    # in Loop: Header=BB0_1 Depth=1
+; CHECK-NEXT:    callq clobber@PLT
+; CHECK-NEXT:    jmp .LBB0_4
 entry:
   %val = load i64, ptr %p, align 8
   br label %loop
 
 loop:
   switch i8 undef, label %unreachable [
     i8 0, label %latch
     i8 1, label %split.1

llvm/test/CodeGen/X86/tail-dup-merge-loop-headers.ll

 ; CHECK-NEXT:    pushq %r14
 ; CHECK-NEXT:    pushq %r13
 ; CHECK-NEXT:    pushq %r12
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movl $1, %ebx
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_27
+; CHECK-NEXT:    jne .LBB1_24
 ; CHECK-NEXT:  # %bb.1: # %if.end19
 ; CHECK-NEXT:    movl %esi, %ebp
 ; CHECK-NEXT:    movq %rdi, %r15
 ; CHECK-NEXT:    movl (%rax), %r13d
 ; CHECK-NEXT:    leal (,%r13,4), %ebx
 ; CHECK-NEXT:    movl %ebx, %r12d
 ; CHECK-NEXT:    movl $1, %esi
 ; CHECK-NEXT:    movq %r12, %rdi
 ; CHECK-NEXT:    callq cli_calloc@PLT
 ; CHECK-NEXT:    testl %ebp, %ebp
-; CHECK-NEXT:    je .LBB1_26
+; CHECK-NEXT:    je .LBB1_23
 ; CHECK-NEXT:  # %bb.2: # %if.end19
 ; CHECK-NEXT:    testl %r13d, %r13d
-; CHECK-NEXT:    je .LBB1_26
+; CHECK-NEXT:    je .LBB1_23
 ; CHECK-NEXT:  # %bb.3: # %if.end19
 ; CHECK-NEXT:    movq %rax, %r14
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_26
+; CHECK-NEXT:    jne .LBB1_23
 ; CHECK-NEXT:  # %bb.4: # %if.end19
 ; CHECK-NEXT:    cmpq %r15, %r14
-; CHECK-NEXT:    jb .LBB1_26
+; CHECK-NEXT:    jb .LBB1_23
 ; CHECK-NEXT:  # %bb.5: # %if.end50
 ; CHECK-NEXT:    movq %r14, %rdi
 ; CHECK-NEXT:    movq %r12, %rdx
 ; CHECK-NEXT:    callq memcpy@PLT
 ; CHECK-NEXT:    cmpl $4, %ebx
-; CHECK-NEXT:    jb .LBB1_29
+; CHECK-NEXT:    jb .LBB1_26
 ; CHECK-NEXT:  # %bb.6: # %shared_preheader
-; CHECK-NEXT:    movb $32, %dl
+; CHECK-NEXT:    movb $32, %cl
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    # implicit-def: $rcx
-; CHECK-NEXT:    jmp .LBB1_9
+; CHECK-NEXT:    jmp .LBB1_8
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB1_7: # %merge_predecessor_split
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    movb $32, %dl
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:  .LBB1_8: # %outer_loop_latch
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    movzwl %si, %esi
-; CHECK-NEXT:    decl %esi
-; CHECK-NEXT:    movzwl %si, %esi
-; CHECK-NEXT:    leaq 1(%rcx,%rsi), %rcx
-; CHECK-NEXT:  .LBB1_9: # %outer_loop_header
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
+; CHECK-NEXT:    movb $32, %cl
+; CHECK-NEXT:  .LBB1_8: # %outer_loop_header
 ; CHECK-NEXT:    # =>This Loop Header: Depth=1
-; CHECK-NEXT:    # Child Loop BB1_10 Depth 2
+; CHECK-NEXT:    # Child Loop BB1_9 Depth 2
 ; CHECK-NEXT:    testl %r13d, %r13d
-; CHECK-NEXT:    je .LBB1_19
+; CHECK-NEXT:    je .LBB1_16
 ; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB1_10: # %shared_loop_header
-; CHECK-NEXT:    # Parent Loop BB1_9 Depth=1
+; CHECK-NEXT:  .LBB1_9: # %shared_loop_header
+; CHECK-NEXT:    # Parent Loop BB1_8 Depth=1
 ; CHECK-NEXT:    # => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    testq %r14, %r14
-; CHECK-NEXT:    jne .LBB1_28
-; CHECK-NEXT:  # %bb.11: # %inner_loop_body
-; CHECK-NEXT:    # in Loop: Header=BB1_10 Depth=2
+; CHECK-NEXT:    jne .LBB1_25
+; CHECK-NEXT:  # %bb.10: # %inner_loop_body
+; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=2
 ; CHECK-NEXT:    testl %eax, %eax
-; CHECK-NEXT:    jns .LBB1_10
-; CHECK-NEXT:  # %bb.12: # %if.end96.i
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
+; CHECK-NEXT:    jns .LBB1_9
+; CHECK-NEXT:  # %bb.11: # %if.end96.i
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
 ; CHECK-NEXT:    cmpl $3, %r13d
-; CHECK-NEXT:    jae .LBB1_23
-; CHECK-NEXT:  # %bb.13: # %if.end287.i
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    cmpl $1, %r13d
-; CHECK-NEXT:    setne %dl
+; CHECK-NEXT:    jae .LBB1_20
+; CHECK-NEXT:  # %bb.12: # %if.end287.i
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
 ; CHECK-NEXT:    testb %al, %al
-; CHECK-NEXT:    jne .LBB1_17
-; CHECK-NEXT:  # %bb.14: # %if.end308.i
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
+; CHECK-NEXT:    # implicit-def: $cl
+; CHECK-NEXT:    jne .LBB1_8
+; CHECK-NEXT:  # %bb.13: # %if.end308.i
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
 ; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:    je .LBB1_7
-; CHECK-NEXT:  # %bb.15: # %if.end335.i
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    xorl %edx, %edx
-; CHECK-NEXT:    testb %dl, %dl
-; CHECK-NEXT:    movl $0, %esi
+; CHECK-NEXT:  # %bb.14: # %if.end335.i
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
+; CHECK-NEXT:    xorl %ecx, %ecx
+; CHECK-NEXT:    testb %cl, %cl
 ; CHECK-NEXT:    jne .LBB1_8
-; CHECK-NEXT:  # %bb.16: # %merge_other
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    jmp .LBB1_18
-; CHECK-NEXT:    .p2align 4, 0x90
-; CHECK-NEXT:  .LBB1_17: # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    movb %dl, %sil
-; CHECK-NEXT:    addl $3, %esi
-; CHECK-NEXT:  .LBB1_18: # %outer_loop_latch
-; CHECK-NEXT:    # in Loop: Header=BB1_9 Depth=1
-; CHECK-NEXT:    # implicit-def: $dl
+; CHECK-NEXT:  # %bb.15: # %merge_other
+; CHECK-NEXT:    # in Loop: Header=BB1_8 Depth=1
+; CHECK-NEXT:    # implicit-def: $cl
 ; CHECK-NEXT:    jmp .LBB1_8
-; CHECK-NEXT:  .LBB1_26:
+; CHECK-NEXT:  .LBB1_23:
 ; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    jmp .LBB1_27
-; CHECK-NEXT:  .LBB1_19: # %while.cond.us1412.i
+; CHECK-NEXT:    jmp .LBB1_24
+; CHECK-NEXT:  .LBB1_16: # %while.cond.us1412.i
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:    movl $1, %ebx
-; CHECK-NEXT:    jne .LBB1_21
-; CHECK-NEXT:  # %bb.20: # %while.cond.us1412.i
-; CHECK-NEXT:    decb %dl
-; CHECK-NEXT:    jne .LBB1_27
-; CHECK-NEXT:  .LBB1_21: # %if.end41.us1436.i
-; CHECK-NEXT:  .LBB1_23: # %if.then99.i
+; CHECK-NEXT:    jne .LBB1_18
+; CHECK-NEXT:  # %bb.17: # %while.cond.us1412.i
+; CHECK-NEXT:    decb %cl
+; CHECK-NEXT:    jne .LBB1_24
+; CHECK-NEXT:  .LBB1_18: # %if.end41.us1436.i
+; CHECK-NEXT:  .LBB1_20: # %if.then99.i
 ; CHECK-NEXT:    movq .str.6@GOTPCREL(%rip), %rdi
 ; CHECK-NEXT:    xorl %ebx, %ebx
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    callq cli_dbgmsg@PLT
-; CHECK-NEXT:  .LBB1_27: # %cleanup
+; CHECK-NEXT:  .LBB1_24: # %cleanup
 ; CHECK-NEXT:    movl %ebx, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %r12
 ; CHECK-NEXT:    popq %r13
 ; CHECK-NEXT:    popq %r14
 ; CHECK-NEXT:    popq %r15
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
-; CHECK-NEXT:  .LBB1_28: # %wunpsect.exit.thread.loopexit389
-; CHECK-NEXT:  .LBB1_29: # %wunpsect.exit.thread.loopexit391
+; CHECK-NEXT:  .LBB1_25: # %wunpsect.exit.thread.loopexit389
+; CHECK-NEXT:  .LBB1_26: # %wunpsect.exit.thread.loopexit391
 entry:
   %0 = load i32, i32* undef, align 4
   %mul = shl nsw i32 %0, 2
   br i1 undef, label %if.end19, label %cleanup
 
 if.end19:                                         ; preds = %entry
   %conv = zext i32 %mul to i64
   %call = tail call i8* @cli_calloc(i64 %conv, i64 1)

llvm/test/CodeGen/X86/tail-opts.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -post-RA-scheduler=true | FileCheck %s
+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -post-RA-scheduler=true -disable-cgp-delete-phis | FileCheck %s
 
 declare dso_local void @bar(i32)
 declare dso_local void @car(i32)
 declare dso_local void @dar(i32)
 declare dso_local void @ear(i32)
 declare dso_local void @far(i32)
 declare i1 @qux()
 

llvm/test/CodeGen/X86/x86-shrink-wrapping.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc %s -o - -enable-shrink-wrap=true -pass-remarks-output=%t | FileCheck %s --check-prefix=ENABLE
+; RUN: llc %s -o - -enable-shrink-wrap=true -pass-remarks-output=%t -disable-cgp-delete-phis | FileCheck %s --check-prefix=ENABLE
 ; RUN: cat %t | FileCheck %s --check-prefix=REMARKS
-; RUN: llc %s -o - -enable-shrink-wrap=false | FileCheck %s --check-prefix=DISABLE
+; RUN: llc %s -o - -enable-shrink-wrap=false -disable-cgp-delete-phis | FileCheck %s --check-prefix=DISABLE
 ;
 ; Note: Lots of tests use inline asm instead of regular calls.
 ; This allows to have a better control on what the allocation will do.
 ; Otherwise, we may have spill right in the entry block, defeating
 ; shrink-wrapping. Moreover, some of the inline asm statement (nop)
 ; are here to ensure that the related paths do not end up as critical
 ; edges.
 target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"

llvm/test/Other/codegenprepare-and-debug.ll

 sw.bb29:                                          ; preds = %while.cond, %while.cond
   unreachable
 
 sw.default:                                       ; preds = %while.cond
   unreachable
 
 lbl:                                              ; preds = %sw.bb, %while.cond.lbl_crit_edge
   %var03.1 = phi i32 [ 0, %while.cond.lbl_crit_edge ], [ 3, %sw.bb ]
+  br label %phi_use
+
+phi_use:
+  store i32 %var03.1, ptr %arg0
   unreachable
 }
 
 attributes #0 = { nounwind readnone speculatable }
 
 !llvm.dbg.cu = !{!0}
 !llvm.module.flags = !{!9, !10}
 

llvm/test/Transforms/CodeGenPrepare/X86/computedgoto.ll

bb1:
   indirectbr ptr %newop1, [label %bb0, label %bb1]
 
 exit:
   ret void
 }
 
 ; Make sure we do the right thing for cases where the indirectbr branches to
 ; the block it terminates.
-define void @loop(ptr nocapture readonly %p) {
+define i64 @loop(ptr nocapture readonly %p) {
 ; CHECK-LABEL: @loop(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[DOTSPLIT:%.*]]
 ; CHECK:       bb0:
 ; CHECK-NEXT:    br label [[DOTSPLIT]]
 ; CHECK:       .split:
-; CHECK-NEXT:    [[MERGE:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[BB0:%.*]] ], [ 0, [[BB0_CLONE:%.*]] ]
+; CHECK-NEXT:    [[MERGE:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[BB0:%.*]] ], [ 0, [[ENTRY:%.*]] ]
 ; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds i64, ptr [[P:%.*]], i64 [[MERGE]]
 ; CHECK-NEXT:    store i64 [[MERGE]], ptr [[TMP0]], align 4
 ; CHECK-NEXT:    [[I_NEXT]] = add nuw nsw i64 [[MERGE]], 1
 ; CHECK-NEXT:    [[IDX:%.*]] = srem i64 [[MERGE]], 2
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [2 x ptr], ptr @loop.targets, i64 0, i64 [[IDX]]
 ; CHECK-NEXT:    [[TARGET:%.*]] = load ptr, ptr [[ARRAYIDX]], align 8
 ; CHECK-NEXT:    indirectbr ptr [[TARGET]], [label [[BB0]], label %bb1]
 ; CHECK:       bb1:
-; CHECK-NEXT:    ret void
+; CHECK-NEXT:    ret i64 [[I_NEXT]]
 ;
 entry:
   br label %bb0
 
 bb0:
   %i = phi i64 [ %i.next, %bb0 ], [ 0, %entry ]
   %tmp0 = getelementptr inbounds i64, ptr %p, i64 %i
   store i64 %i, ptr %tmp0, align 4
   %i.next = add nuw nsw i64 %i, 1
   %idx = srem i64 %i, 2
   %arrayidx = getelementptr inbounds [2 x ptr], ptr @loop.targets, i64 0, i64 %idx
   %target = load ptr, ptr %arrayidx, align 8
   indirectbr ptr %target, [label %bb0, label %bb1]
 
 bb1:
-  ret void
+  ret i64 %i.next
 }
 
 ; Don't do anything for cases that contain no phis.
 define void @nophi(ptr %p) {
 ; CHECK-LABEL: @nophi(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[INCDEC_PTR:%.*]] = getelementptr inbounds i32, ptr [[P:%.*]], i64 1
 ; CHECK-NEXT:    [[INITOP:%.*]] = load i32, ptr [[INCDEC_PTR]], align 4

 ; CHECK:       bb0:
 ; CHECK-NEXT:    [[R0:%.*]] = sub i32 [[K]], [[D]]
 ; CHECK-NEXT:    br label [[EXIT:%.*]]
 ; CHECK:       bb1:
 ; CHECK-NEXT:    [[R1:%.*]] = sub i32 [[D]], [[K]]
 ; CHECK-NEXT:    br label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[V:%.*]] = phi i32 [ [[R0]], [[BB0]] ], [ [[R1]], [[BB1:%.*]] ]
-; CHECK-NEXT:    ret i32 0
+; CHECK-NEXT:    ret i32 [[V]]
 ;
 {
 entry:
   %d = add i32 %k, 1
   indirectbr ptr %p, [label %bb0, label %bb1]
 
 bb0:
   %v00 = phi i32 [%k, %entry]
   %v01 = phi i32 [%d, %entry]
   %r0 = sub i32 %v00, %v01
   br label %exit
 
 bb1:
   %v10 = phi i32 [%d, %entry]
   %v11 = phi i32 [%k, %entry]
   %r1 = sub i32 %v10, %v11
   br label %exit
 
 exit:
   %v = phi i32 [%r0, %bb0], [%r1, %bb1]
-  ret i32 0
+  ret i32 %v
 }

llvm/test/Transforms/CodeGenPrepare/X86/delete-assume-dead-code.ll

 }
 
 define i32 @test2(i32 %N) nounwind {
 ; CHECK-LABEL: @test2(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_INC:%.*]], [[HEADER]] ]
-; CHECK-NEXT:    [[IV2:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[IV2_INC:%.*]], [[HEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add i32 [[IV]], 1
-; CHECK-NEXT:    [[IV2_INC]] = add i32 [[IV2]], 1
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i32 [[IV]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[CMP]], label [[EXIT:%.*]], label [[HEADER]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret i32 [[IV]]
 ;
 entry:
   br label %header
 

 }
 
 define i32 @test3(i32 %N) nounwind {
 ; CHECK-LABEL: @test3(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_INC:%.*]], [[HEADER]] ]
-; CHECK-NEXT:    [[IV2:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[IV2_INC:%.*]], [[HEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add i32 [[IV]], 1
-; CHECK-NEXT:    [[IV2_INC]] = add i32 [[IV2]], 1
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i32 [[IV]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[CMP]], label [[EXIT:%.*]], label [[HEADER]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret i32 [[IV]]
 ;
 entry:
   br label %header
 

 
 ; Two assumes case.
 define i32 @test4(i32 %N) nounwind {
 ; CHECK-LABEL: @test4(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_INC:%.*]], [[HEADER]] ]
-; CHECK-NEXT:    [[IV2:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[IV2_INC:%.*]], [[HEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add i32 [[IV]], 1
-; CHECK-NEXT:    [[IV2_INC]] = add i32 [[IV2]], 1
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i32 [[IV]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[CMP]], label [[EXIT:%.*]], label [[HEADER]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret i32 [[IV]]
 ;
 entry:
   br label %header
 

llvm/test/Transforms/CodeGenPrepare/X86/sink-addrmode-base.ll

-; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false -addr-sink-new-phis=true -addr-sink-new-select=true  %s | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-YES
-; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false -addr-sink-new-phis=false -addr-sink-new-select=true %s | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NO
+; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false -addr-sink-new-phis=true -addr-sink-new-select=true -disable-cgp-delete-phis  %s | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-YES
+; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false -addr-sink-new-phis=false -addr-sink-new-select=true -disable-cgp-delete-phis %s | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NO
 target datalayout =
 "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Can we sink for different base if there is no phi for base?
 define i32 @test1(i1 %cond, ptr %b1, ptr %b2) {
 ; CHECK-LABEL: @test1
 entry:

llvm/test/Transforms/CodeGenPrepare/X86/sink-addrmode-two-phi.ll

-; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false  %s | FileCheck %s --check-prefix=CHECK
+; RUN: opt -S -codegenprepare -disable-complex-addr-modes=false -disable-cgp-delete-phis %s | FileCheck %s --check-prefix=CHECK
 target datalayout =
 "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @test() {
 entry:
   %0 = getelementptr inbounds i64, ptr null, i64 undef
   br label %start