[Atomic-Expand] Run SimplifyCFG from Atomic-Expand on CAS loop blocks.

[pravinjagtap]

Summary

There are potential benefits in simplifying
CFG just after atomic-expand pass since
it changes control flow.

On AMDGPU targets, for global FP atomic
operations, atomic-expand
pass emits CAS loop which is not efficient.

To optimize atomics AMDGPU target runs
AMDGPUAtomicOptimizer just before
atomic-expand pass.
Running AMDGPUAtomic Optimzer and
atomic expand introduces new control flow,
therefore, running CFG Simplification allows
better codegen.

AArch64 deals with this by inserting an extra
simplifyCFG pass run, which seems excessive.

[pravinjagtap]

Want to take initial feedback on this approach.

[foad]

What kind of simplifications does it do on CAS loops? Is there a test that shows the effect?

[arsenm]

Needs test that shows changes. Also ideally would show it obviates the need for aarch64-enable-atomic-cfg-tidy

[pravinjagtap]

Added Floating Point tests to showcase the effect of running simplify CFG

[pravinjagtap]

llvm/test/CodeGen/AMDGPU/global_atomics_scan_fadd.ll

2165–2184 ↗

(On Diff #548983)

I am not sure whether this is what we are expecting. None of the existing test-cases need update for this change. I am struggling to demonstrate the actual benefits of running SimplifyCFG of CAS blocks.

[arsenm]

llvm/test/CodeGen/AMDGPU/global_atomics_scan_fadd.ll

2165–2184 ↗

(On Diff #548983)

This needs some pure IR tests in test/Transforms/AtomicExpand. You could start by hacking out the aarch64 option and see what breaks for potentially interesting cases.

Does this only do anything if the atomic is in more complex control flow? Does it only do anything if the dominator tree is precomputed? Do you see more changes if you force the dominator tree to be required?

llvm/test/CodeGen/AMDGPU/global_atomics_scan_fmin.ll

50 ↗

(On Diff #548983)

Something's not right because all this is doing is starting to preserve a block name. Does this only do anything if the atomic is in more complex control flow? Does it only do anything if the dominator tree is precomputed? Do you see more changes if you force the dominator tree to be required?

[pravinjagtap]

Added test to showcase the benefits of running simplifyCFG from atomic-expand

[pravinjagtap]

llvm/test/Transforms/AtomicExpand/AMDGPU/expand-atomic-simplify-cfg-CAS-block.ll

24

Here, we can observe the potential benefits of running simplify CFG. It simplifies the branching.

[pravinjagtap]

llvm/test/Transforms/AtomicExpand/AMDGPU/expand-atomic-simplify-cfg-CAS-block.ll

24

Without this it would have been:

; GFX90A:       atomicrmw.start:	
; GFX90A-NEXT:    [[LOADED:%.*]] = phi float [ [[TMP0]], [[IF]] ], [ [[TMP4:%.*]], [[ATOMICRMW_START]] ]	
; GFX90A-NEXT:    [[NEW:%.*]] = fadd float [[LOADED]], [[IN:%.*]]	
; GFX90A-NEXT:    [[TMP1:%.*]] = bitcast float [[NEW]] to i32	
; GFX90A-NEXT:    [[TMP2:%.*]] = bitcast float [[LOADED]] to i32	
; GFX90A-NEXT:    [[TMP3:%.*]] = cmpxchg ptr addrspace(1) [[OUT]], i32 [[TMP2]], i32 [[TMP1]] seq_cst seq_cst, align 4	
; GFX90A-NEXT:    [[SUCCESS:%.*]] = extractvalue { i32, i1 } [[TMP3]], 1	
; GFX90A-NEXT:    [[NEWLOADED:%.*]] = extractvalue { i32, i1 } [[TMP3]], 0	
; GFX90A-NEXT:    [[TMP4]] = bitcast i32 [[NEWLOADED]] to float	
; GFX90A-NEXT:    br i1 [[SUCCESS]], label [[ATOMICRMW_END:%.*]], label [[ATOMICRMW_START]]	
; GFX90A:       atomicrmw.end:	
; GFX90A-NEXT:    br label [[ENDIF:%.*]]	
; GFX90A:       else:	
; GFX90A-NEXT:    [[TMP5:%.*]] = load float, ptr addrspace(1) [[OUT]], align 4	
; GFX90A-NEXT:    br label [[ATOMICRMW_START2:%.*]]	
; GFX90A:       atomicrmw.start2:	

`

[pravinjagtap]

ping.

[arsenm]

What happens if you remove the aarch64 tidy with this?

llvm/test/CodeGen/AMDGPU/global_atomics_scan_fmin.ll

3196 ↗

(On Diff #549821)

Why were these tests deleted?

llvm/test/Transforms/AtomicExpand/AMDGPU/expand-atomic-simplify-cfg-CAS-block.ll

6

Can you precommit the test?

[pravinjagtap]

llvm/test/CodeGen/AMDGPU/global_atomics_scan_fmin.ll

3196 ↗

(On Diff #549821)

I was trying to pre-commit these tests through D157712

[pravinjagtap]

addressed review comment

[arsenm]

In D157495#4598420, @pravinjagtap wrote:

addressed review comment

Haven't tried to delete the AArch64 atomic tidy?

llvm/lib/CodeGen/AtomicExpandPass.cpp

88–89

This didn't account for the require and preserve dom tree

124

Do you need to track this or can you just clean each one up as it happens?

368

Why is RequireAndPreserveDomTree a cl:opt?

[pravinjagtap]

llvm/lib/CodeGen/AtomicExpandPass.cpp

124

Do you need to track this or can you just clean each one up as it happens?

I am clearing this vector at the beginning itself in runOnFunction

[pravinjagtap]

In D157495#4598667, @arsenm wrote:

In D157495#4598420, @pravinjagtap wrote:

addressed review comment

Haven't tried to delete the AArch64 atomic tidy?

TBH, I am not sure how to exactly achieve this.

[arsenm]

In D157495#4598715, @pravinjagtap wrote:

In D157495#4598667, @arsenm wrote:

In D157495#4598420, @pravinjagtap wrote:

addressed review comment

Haven't tried to delete the AArch64 atomic tidy?

TBH, I am not sure how to exactly achieve this.

Delete the option and run of the pass and see if it's equivalently effective in the existing tests to this

llvm/lib/CodeGen/AtomicExpandPass.cpp

124

That's not what I meant, I mean you performed the expansion and can immediately simplify the block without recording it and treating it like a separate pass

[pravinjagtap]

llvm/lib/CodeGen/AtomicExpandPass.cpp

124

I think, this is a cleaner way compared to simplifying these basic blocks when created. We need to pass inputs argument required for simplifyCFG API from runOnFunction to all the way inside insertRMWCmpXchgLoop through member functions and few helper functions.

[pravinjagtap]

In D157495#4598719, @arsenm wrote:

In D157495#4598715, @pravinjagtap wrote:

In D157495#4598667, @arsenm wrote:

In D157495#4598420, @pravinjagtap wrote:

addressed review comment

Haven't tried to delete the AArch64 atomic tidy?

TBH, I am not sure how to exactly achieve this.

Delete the option and run of the pass and see if it's equivalently effective in the existing tests to this

You mean instead of

simplifyCFG(BB, *TTI, RequireAndPreserveDomTree ? &DTU : nullptr,
            SimplifyCFGOptions()
                .forwardSwitchCondToPhi(true)
                .convertSwitchRangeToICmp(true)
                .convertSwitchToLookupTable(true)
                .needCanonicalLoops(false)
                .hoistCommonInsts(true)
                .sinkCommonInsts(true));

just call simplifyCFG(BB, TTI) ?

[pravinjagtap]

In D157495#4598794, @pravinjagtap wrote:

In D157495#4598719, @arsenm wrote:

In D157495#4598715, @pravinjagtap wrote:

In D157495#4598667, @arsenm wrote:

In D157495#4598420, @pravinjagtap wrote:

addressed review comment

Haven't tried to delete the AArch64 atomic tidy?

TBH, I am not sure how to exactly achieve this.

Delete the option and run of the pass and see if it's equivalently effective in the existing tests to this

You mean instead of

simplifyCFG(BB, *TTI, RequireAndPreserveDomTree ? &DTU : nullptr,
            SimplifyCFGOptions()
                .forwardSwitchCondToPhi(true)
                .convertSwitchRangeToICmp(true)
                .convertSwitchToLookupTable(true)
                .needCanonicalLoops(false)
                .hoistCommonInsts(true)
                .sinkCommonInsts(true));

just call simplifyCFG(BB, TTI) ?

Output is identical without these options for the test in expand-atomic-simplify-cfg-CAS-block.ll.

[pravinjagtap]

Do you want me to update the patch without AArch64 atomic tidy options ? I think, relying on default options of simplifyCFG is good option here.

llvm/lib/CodeGen/AtomicExpandPass.cpp

368

Why is RequireAndPreserveDomTree a cl:opt?

This is based on usage of simplifyCFG in https://github.com/llvm/llvm-project/blob/851c248dfcdbf52ee88e4643e59453fcc13501d5/llvm/lib/Target/AMDGPU/AMDGPUUnifyDivergentExitNodes.cpp#L185

[pravinjagtap]

Switched to default options of SimplifyCFG instead of AArch64 atomic tidy options

[pravinjagtap]

Code clean up

[rovka]

Could you please rephrase the commit message? It's not clear to me what using a "canonical pass" instead of a simplifyCFG pass means.

llvm/lib/CodeGen/AtomicExpandPass.cpp

1533

Why are we only keeping track of these blocks? There seem to be lots of other places in this file that split blocks and create new ones. Shouldn't we call simplifyCFG for all of them?

[pravinjagtap]

llvm/lib/CodeGen/AtomicExpandPass.cpp

1533

Why are we only keeping track of these blocks? There seem to be lots of other places in this file that split blocks and create new ones. Shouldn't we call simplifyCFG for all of them?

Targets can configure this simplification using separate pass run e.g. Aarch64 is running simplifyCFG after atomic expand pass https://github.com/llvm/llvm-project/blob/57c090b2ea03937e7c6a08a594532788d01bb813/llvm/lib/Target/AArch64/AArch64TargetMachine.cpp#L557

We think, having separate pass run will be expensive, therefore, for AMDGPU usecase, we are interested in running it on local changes done by atomic-expand. Do you think, calling simplifyCFG on entire functions makes much more sense ?

[rovka]

llvm/lib/CodeGen/AtomicExpandPass.cpp

1533

We think, having separate pass run will be expensive, therefore, for AMDGPU usecase, we are interested in running it on local changes done by atomic-expand. Do you think, calling simplifyCFG on entire functions makes much more sense ?

Ok, I don't know how much compile time this will save for AMDGPU, so I'll let the other reviewers comment on whether or not we want to teach this pass to clean up after itself. But if we decide that we do want it to clean up (i.e. run simplifyCFG only on the blocks that it has added), I think it should:

1. be consistent about it. Right now it creates basic blocks in several different places, but with your patch it only cleans up some of them. If there's a good reason for this, it should be documented (at least in the commit message if not in the code). If there isn't, then at least leave some FIXMEs for the other cases, so people don't have to scratch their heads while looking through this code.

2. be an opt-in behaviour, kind of like how the SimplifyCFG pass has all those settings you can fiddle with when adding it. AtomicExpand is used by several different backends, not just AArch64, and several of them add a full SimplifyCFG run after it (Arm, Hexagon). That SimplifyCFG run may serve to clean up both after AtomicExpand, but potentially also other passes that run before or in between, so it might not make sense to remove the SimplifyCFG run for them. In those cases, it will be useless for AtomicExpand to invoke its own piecemeal SimplifyCFG, so they should be able to run the "fast and messy" AtomicExpand if they want to.

That's just my 2 cents, maybe @arsenm or @foad have different opinions.

[arsenm]

llvm/lib/CodeGen/AtomicExpandPass.cpp

1533

I don't think the compile time is uniquely expensive for AMDGPU, but I would assume just calling simplifycfg on modified blocks would be simpler (as atomics are rare) than running the full CFG pass after the fact

I think it's odd for simplifycfg to be in the codegen pipeline, so a more targeted application seems better

[pravinjagtap]

Rebased.

[pravinjagtap]

Added comments that documents the motivation for this change.

[arsenm]

I still want to see the impact of removing the aarch64 pass

[pravinjagtap]

Experiment: Want to understand the impact of removing the aarch64 pass

Expecting 18 tests to fail. They are not auto-generatable.

llvm/include/llvm/CodeGen/AtomicExpandUtils.h

 ///     %new_loaded = extractvalue { iN, i1 } %pair, 0
 ///     %success = extractvalue { iN, i1 } %pair, 1
 /// ; End callback produced IR
 ///     br i1 %success, label %atomicrmw.end, label %loop
 /// atomicrmw.end:
 ///     [...]
 ///
 /// Returns true if the containing function was modified.
-bool expandAtomicRMWToCmpXchg(AtomicRMWInst *AI, CreateCmpXchgInstFun CreateCmpXchg);
+bool expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
+                              CreateCmpXchgInstFun CreateCmpXchg,
+                              SmallVector<BasicBlock *> &CmpXchgLoopBlocks);
 
 } // end namespace llvm
 
 #endif // LLVM_CODEGEN_ATOMICEXPANDUTILS_H

llvm/lib/CodeGen/AtomicExpandPass.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/STLFunctionalExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Analysis/InstSimplifyFolder.h"
 #include "llvm/Analysis/OptimizationRemarkEmitter.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/CodeGen/AtomicExpandUtils.h"
 #include "llvm/CodeGen/RuntimeLibcalls.h"
 #include "llvm/CodeGen/TargetLowering.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/CodeGen/ValueTypes.h"
 #include "llvm/IR/Attributes.h"
 #include "llvm/IR/BasicBlock.h"

 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
+#include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/LowerAtomic.h"
 #include <cassert>
 #include <cstdint>
 #include <iterator>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "atomic-expand"
 
 namespace {
 
 class AtomicExpand : public FunctionPass {
   const TargetLowering *TLI = nullptr;
   const DataLayout *DL = nullptr;
 
+  // Atomic-expand pass creates new basic blocks at several places.
+  // The new control flow introduced by this pass can be simplified
+  // by running SimplifyCFG on such basic blocks when the pass finishes.
+  // CmpXchgLoopBlocks holds such basic blocks that
+  // needs simplification and atomic-expand runs SimplifyCFG on these
+  // blocks. For now, this simplification is carried out on CmpXchg Loop
+  // blocks only. Based on other use-cases, blocks that can benefit
+  // from this simplification can be added to this list.
+  SmallVector<BasicBlock *> CmpXchgLoopBlocks;
+
 public:
   static char ID; // Pass identification, replacement for typeid
 
   AtomicExpand() : FunctionPass(ID) {
     initializeAtomicExpandPass(*PassRegistry::getPassRegistry());
   }
 
   bool runOnFunction(Function &F) override;
 
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.addRequired<TargetTransformInfoWrapperPass>();
+  }
+
 private:
   bool bracketInstWithFences(Instruction *I, AtomicOrdering Order);
   IntegerType *getCorrespondingIntegerType(Type *T, const DataLayout &DL);
   LoadInst *convertAtomicLoadToIntegerType(LoadInst *LI);
   bool tryExpandAtomicLoad(LoadInst *LI);
   bool expandAtomicLoadToLL(LoadInst *LI);
   bool expandAtomicLoadToCmpXchg(LoadInst *LI);
   StoreInst *convertAtomicStoreToIntegerType(StoreInst *SI);
private:
   void expandAtomicRMWToMaskedIntrinsic(AtomicRMWInst *AI);
   void expandAtomicCmpXchgToMaskedIntrinsic(AtomicCmpXchgInst *CI);
 
   AtomicCmpXchgInst *convertCmpXchgToIntegerType(AtomicCmpXchgInst *CI);
   static Value *insertRMWCmpXchgLoop(
       IRBuilderBase &Builder, Type *ResultType, Value *Addr, Align AddrAlign,
       AtomicOrdering MemOpOrder, SyncScope::ID SSID,
       function_ref<Value *(IRBuilderBase &, Value *)> PerformOp,
-      CreateCmpXchgInstFun CreateCmpXchg);
+      CreateCmpXchgInstFun CreateCmpXchg,
+      SmallVector<BasicBlock *> &CmpXchgLoopBlocks);
   bool tryExpandAtomicCmpXchg(AtomicCmpXchgInst *CI);
 
   bool expandAtomicCmpXchg(AtomicCmpXchgInst *CI);
   bool isIdempotentRMW(AtomicRMWInst *RMWI);
   bool simplifyIdempotentRMW(AtomicRMWInst *RMWI);
 
   bool expandAtomicOpToLibcall(Instruction *I, unsigned Size, Align Alignment,
                                Value *PointerOperand, Value *ValueOperand,
                                Value *CASExpected, AtomicOrdering Ordering,
                                AtomicOrdering Ordering2,
                                ArrayRef<RTLIB::Libcall> Libcalls);
   void expandAtomicLoadToLibcall(LoadInst *LI);
   void expandAtomicStoreToLibcall(StoreInst *LI);
   void expandAtomicRMWToLibcall(AtomicRMWInst *I);
   void expandAtomicCASToLibcall(AtomicCmpXchgInst *I);
 
   friend bool
   llvm::expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
-                                 CreateCmpXchgInstFun CreateCmpXchg);
+                                 CreateCmpXchgInstFun CreateCmpXchg,
+                                 SmallVector<BasicBlock *> &CmpXchgLoopBlocks);
 };
 
 // IRBuilder to be used for replacement atomic instructions.
 struct ReplacementIRBuilder : IRBuilder<InstSimplifyFolder> {
   // Preserves the DebugLoc from I, and preserves still valid metadata.
   explicit ReplacementIRBuilder(Instruction *I, const DataLayout &DL)
       : IRBuilder(I->getContext(), DL) {
     SetInsertPoint(I);
     this->CollectMetadataToCopy(I, {LLVMContext::MD_pcsections});
   }
 };
 
 } // end anonymous namespace
 
 char AtomicExpand::ID = 0;
 
 char &llvm::AtomicExpandID = AtomicExpand::ID;
 
-INITIALIZE_PASS(AtomicExpand, DEBUG_TYPE, "Expand Atomic instructions", false,
-                false)
-
+INITIALIZE_PASS_BEGIN(AtomicExpand, DEBUG_TYPE, "Expand Atomic instructions",
+                      false, false)
+INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
+INITIALIZE_PASS_END(AtomicExpand, DEBUG_TYPE, "Expand Atomic instructions",
+                    false, false)
 FunctionPass *llvm::createAtomicExpandPass() { return new AtomicExpand(); }
 
 // Helper functions to retrieve the size of atomic instructions.
 static unsigned getAtomicOpSize(LoadInst *LI) {
   const DataLayout &DL = LI->getModule()->getDataLayout();
   return DL.getTypeStoreSize(LI->getType());
 }
 
if (!TPC)
     return false;
 
   auto &TM = TPC->getTM<TargetMachine>();
   const auto *Subtarget = TM.getSubtargetImpl(F);
   if (!Subtarget->enableAtomicExpand())
     return false;
   TLI = Subtarget->getTargetLowering();
   DL = &F.getParent()->getDataLayout();
+  CmpXchgLoopBlocks.clear();
 
   SmallVector<Instruction *, 1> AtomicInsts;
 
   // Changing control-flow while iterating through it is a bad idea, so gather a
   // list of all atomic instructions before we start.
   for (Instruction &I : instructions(F))
     if (I.isAtomic() && !isa<FenceInst>(&I))
       AtomicInsts.push_back(&I);
else if (RMWI) {
           MadeChange = true;
         }
 
         MadeChange |= tryExpandAtomicRMW(RMWI);
       }
     } else if (CASI)
       MadeChange |= tryExpandAtomicCmpXchg(CASI);
   }
+
+  // Run CFG Simplification on CAS loop blocks.
+  auto TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);
+  for (BasicBlock *BB : CmpXchgLoopBlocks)
+    simplifyCFG(BB, *TTI);
+
   return MadeChange;
 }
 
 bool AtomicExpand::bracketInstWithFences(Instruction *I, AtomicOrdering Order) {
   ReplacementIRBuilder Builder(I, *DL);
 
   auto LeadingFence = TLI->emitLeadingFence(Builder, I, Order);
 
   auto TrailingFence = TLI->emitTrailingFence(Builder, I, Order);
   // We have a guard here because not every atomic operation generates a
if (ValueSize < MinCASSize) {
                           : SSNs[AI->getSyncScopeID()];
       OptimizationRemarkEmitter ORE(AI->getFunction());
       ORE.emit([&]() {
         return OptimizationRemark(DEBUG_TYPE, "Passed", AI)
                << "A compare and swap loop was generated for an atomic "
                << AI->getOperationName(AI->getOperation()) << " operation at "
                << MemScope << " memory scope";
       });
-      expandAtomicRMWToCmpXchg(AI, createCmpXchgInstFun);
+      expandAtomicRMWToCmpXchg(AI, createCmpXchgInstFun, CmpXchgLoopBlocks);
     }
     return true;
   }
   case TargetLoweringBase::AtomicExpansionKind::MaskedIntrinsic: {
     expandAtomicRMWToMaskedIntrinsic(AI);
     return true;
   }
   case TargetLoweringBase::AtomicExpansionKind::BitTestIntrinsic: {
auto PerformPartwordOp = [&](IRBuilderBase &Builder, Value *Loaded) {
     return performMaskedAtomicOp(AI->getOperation(), Builder, Loaded,
                                  ValOperand_Shifted, AI->getValOperand(), PMV);
   };
 
   Value *OldResult;
   if (ExpansionKind == TargetLoweringBase::AtomicExpansionKind::CmpXChg) {
     OldResult = insertRMWCmpXchgLoop(Builder, PMV.WordType, PMV.AlignedAddr,
                                      PMV.AlignedAddrAlignment, MemOpOrder, SSID,
-                                     PerformPartwordOp, createCmpXchgInstFun);
+                                     PerformPartwordOp, createCmpXchgInstFun,
+                                     CmpXchgLoopBlocks);
   } else {
     assert(ExpansionKind == TargetLoweringBase::AtomicExpansionKind::LLSC);
     OldResult = insertRMWLLSCLoop(Builder, PMV.WordType, PMV.AlignedAddr,
                                   PMV.AlignedAddrAlignment, MemOpOrder,
                                   PerformPartwordOp);
   }
 
   Value *FinalOldResult = extractMaskedValue(Builder, OldResult, PMV);
bool AtomicExpand::simplifyIdempotentRMW(AtomicRMWInst *RMWI) {
   }
   return false;
 }
 
 Value *AtomicExpand::insertRMWCmpXchgLoop(
     IRBuilderBase &Builder, Type *ResultTy, Value *Addr, Align AddrAlign,
     AtomicOrdering MemOpOrder, SyncScope::ID SSID,
     function_ref<Value *(IRBuilderBase &, Value *)> PerformOp,
-    CreateCmpXchgInstFun CreateCmpXchg) {
+    CreateCmpXchgInstFun CreateCmpXchg,
+    SmallVector<BasicBlock *> &CmpXchgLoopBlocks) {
   LLVMContext &Ctx = Builder.getContext();
   BasicBlock *BB = Builder.GetInsertBlock();
   Function *F = BB->getParent();
 
   // Given: atomicrmw some_op iN* %addr, iN %incr ordering
   //
   // The standard expansion we produce is:
   //     [...]
   //     %init_loaded = load atomic iN* %addr
   //     br label %loop
   // loop:
   //     %loaded = phi iN [ %init_loaded, %entry ], [ %new_loaded, %loop ]
   //     %new = some_op iN %loaded, %incr
   //     %pair = cmpxchg iN* %addr, iN %loaded, iN %new
   //     %new_loaded = extractvalue { iN, i1 } %pair, 0
   //     %success = extractvalue { iN, i1 } %pair, 1
   //     br i1 %success, label %atomicrmw.end, label %loop
   // atomicrmw.end:
   //     [...]
   BasicBlock *ExitBB =
       BB->splitBasicBlock(Builder.GetInsertPoint(), "atomicrmw.end");
+  CmpXchgLoopBlocks.push_back(ExitBB);
   BasicBlock *LoopBB = BasicBlock::Create(Ctx, "atomicrmw.start", F, ExitBB);
-
+  CmpXchgLoopBlocks.push_back(LoopBB);
   // The split call above "helpfully" added a branch at the end of BB (to the
   // wrong place), but we want a load. It's easiest to just remove
   // the branch entirely.
   std::prev(BB->end())->eraseFromParent();
   Builder.SetInsertPoint(BB);
   LoadInst *InitLoaded = Builder.CreateAlignedLoad(ResultTy, Addr, AddrAlign);
   Builder.CreateBr(LoopBB);
 
case TargetLoweringBase::AtomicExpansionKind::MaskedIntrinsic:
     expandAtomicCmpXchgToMaskedIntrinsic(CI);
     return true;
   case TargetLoweringBase::AtomicExpansionKind::NotAtomic:
     return lowerAtomicCmpXchgInst(CI);
   }
 }
 
 // Note: This function is exposed externally by AtomicExpandUtils.h
-bool llvm::expandAtomicRMWToCmpXchg(AtomicRMWInst *AI,
-                                    CreateCmpXchgInstFun CreateCmpXchg) {
+bool llvm::expandAtomicRMWToCmpXchg(
+    AtomicRMWInst *AI, CreateCmpXchgInstFun CreateCmpXchg,
+    SmallVector<BasicBlock *> &CmpXchgLoopBlocks) {
   ReplacementIRBuilder Builder(AI, AI->getModule()->getDataLayout());
   Builder.setIsFPConstrained(
       AI->getFunction()->hasFnAttribute(Attribute::StrictFP));
 
   // FIXME: If FP exceptions are observable, we should force them off for the
   // loop for the FP atomics.
   Value *Loaded = AtomicExpand::insertRMWCmpXchgLoop(
       Builder, AI->getType(), AI->getPointerOperand(), AI->getAlign(),
       AI->getOrdering(), AI->getSyncScopeID(),
       [&](IRBuilderBase &Builder, Value *Loaded) {
         return buildAtomicRMWValue(AI->getOperation(), Builder, Loaded,
                                    AI->getValOperand());
       },
-      CreateCmpXchg);
+      CreateCmpXchg, CmpXchgLoopBlocks);
 
   AI->replaceAllUsesWith(Loaded);
   AI->eraseFromParent();
   return true;
 }
 
 // In order to use one of the sized library calls such as
 // __atomic_fetch_add_4, the alignment must be sufficient, the size
Success = expandAtomicOpToLibcall(
         nullptr, I->getOrdering(), AtomicOrdering::NotAtomic, Libcalls);
 
   // The expansion failed: either there were no libcalls at all for
   // the operation (min/max), or there were only size-specialized
   // libcalls (add/sub/etc) and we needed a generic. So, expand to a
   // CAS libcall, via a CAS loop, instead.
   if (!Success) {
     expandAtomicRMWToCmpXchg(
-        I, [this](IRBuilderBase &Builder, Value *Addr, Value *Loaded,
-                  Value *NewVal, Align Alignment, AtomicOrdering MemOpOrder,
-                  SyncScope::ID SSID, Value *&Success, Value *&NewLoaded) {
+        I,
+        [this](IRBuilderBase &Builder, Value *Addr, Value *Loaded,
           // Create the CAS instruction normally...
           AtomicCmpXchgInst *Pair = Builder.CreateAtomicCmpXchg(
               Addr, Loaded, NewVal, Alignment, MemOpOrder,
               AtomicCmpXchgInst::getStrongestFailureOrdering(MemOpOrder), SSID);
           Success = Builder.CreateExtractValue(Pair, 1, "success");
           NewLoaded = Builder.CreateExtractValue(Pair, 0, "newloaded");
 
           // ...and then expand the CAS into a libcall.
           expandAtomicCASToLibcall(Pair);
-        });
+        },
+        CmpXchgLoopBlocks);
   }
 }
 
 // A helper routine for the above expandAtomic*ToLibcall functions.
 //
 // 'Libcalls' contains an array of enum values for the particular
 // ATOMIC libcalls to be emitted. All of the other arguments besides
 // 'I' are extracted from the Instruction subclass by the

llvm/lib/Target/AArch64/AArch64TargetMachine.cpp

void AArch64PassConfig::addIRPasses() {
   // Always expand atomic operations, we don't deal with atomicrmw or cmpxchg
   // ourselves.
   addPass(createAtomicExpandPass());
 
   // Expand any SVE vector library calls that we can't code generate directly.
   if (EnableSVEIntrinsicOpts && TM->getOptLevel() == CodeGenOpt::Aggressive)
     addPass(createSVEIntrinsicOptsPass());
 
-  // Cmpxchg instructions are often used with a subsequent comparison to
-  // determine whether it succeeded. We can exploit existing control-flow in
-  // ldrex/strex loops to simplify this, but it needs tidying up.
-  if (TM->getOptLevel() != CodeGenOpt::None && EnableAtomicTidy)
-    addPass(createCFGSimplificationPass(SimplifyCFGOptions()
-                                            .forwardSwitchCondToPhi(true)
-                                            .convertSwitchRangeToICmp(true)
-                                            .convertSwitchToLookupTable(true)
-                                            .needCanonicalLoops(false)
-                                            .hoistCommonInsts(true)
-                                            .sinkCommonInsts(true)));
-
   // Run LoopDataPrefetch
   //
   // Run this before LSR to remove the multiplies involved in computing the
   // pointer values N iterations ahead.
   if (TM->getOptLevel() != CodeGenOpt::None) {
     if (EnableLoopDataPrefetch)
       addPass(createLoopDataPrefetchPass());
     if (EnableFalkorHWPFFix)

llvm/test/CodeGen/AArch64/GlobalISel/arm64-atomic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=arm64-apple-ios -global-isel -global-isel-abort=1 -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK-NOLSE,CHECK-NOLSE-O1
 ; RUN: llc < %s -mtriple=arm64-apple-ios -global-isel -global-isel-abort=1 -O0 -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK-NOLSE,CHECK-NOLSE-O0
 ; RUN: llc < %s -mtriple=arm64-apple-ios -global-isel -global-isel-abort=1 -mcpu=apple-a13 -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK-LSE-O1
 ; RUN: llc < %s -mtriple=arm64-apple-ios -global-isel -global-isel-abort=1 -mcpu=apple-a13 -O0 -verify-machineinstrs | FileCheck %s --check-prefixes=CHECK-LSE-O0
 
 define i32 @val_compare_and_swap(ptr %p, i32 %cmp, i32 %new) #0 {
 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB0_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxr w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp w8, w1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB0_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB0_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stxr w9, w2, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB0_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB0_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap_from_load:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:    ldr w9, [x2]
 ; CHECK-NOLSE-O1-NEXT:  LBB1_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxr w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp w8, w1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB1_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB1_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stxr w10, w9, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w10, LBB1_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB1_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 define i32 @val_compare_and_swap_rel(ptr %p, i32 %cmp, i32 %new) #0 {
 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap_rel:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB2_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxr w8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp w8, w1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB2_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB2_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stlxr w9, w2, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB2_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov w0, w8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB2_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 define i64 @val_compare_and_swap_64(ptr %p, i64 %cmp, i64 %new) #0 {
 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap_64:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB3_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldxr x8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp x8, x1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB3_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB3_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stxr w9, x2, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB3_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov x0, x8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB3_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 define i64 @val_compare_and_swap_64_monotonic_seqcst(ptr %p, i64 %cmp, i64 %new) #0 {
 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap_64_monotonic_seqcst:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB4_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxr x8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp x8, x1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB4_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB4_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stlxr w9, x2, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB4_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov x0, x8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB4_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 define i64 @val_compare_and_swap_64_release_acquire(ptr %p, i64 %cmp, i64 %new) #0 {
 ; CHECK-NOLSE-O1-LABEL: val_compare_and_swap_64_release_acquire:
 ; CHECK-NOLSE-O1:       ; %bb.0:
 ; CHECK-NOLSE-O1-NEXT:  LBB5_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldaxr x8, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cmp x8, x1
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB5_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB5_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stlxr w9, x2, [x0]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB5_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3: ; %cmpxchg.end
 ; CHECK-NOLSE-O1-NEXT:    mov x0, x8
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB5_4: ; %cmpxchg.nostore
 ; CHECK-NOLSE-O1-NEXT:    clrex

 ; CHECK-NOLSE-O1-NEXT:    mov x8, x0
 ; CHECK-NOLSE-O1-NEXT:    ; kill: def $w2 killed $w2 def $x2
 ; CHECK-NOLSE-O1-NEXT:  LBB47_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldxrb w0, [x8]
 ; CHECK-NOLSE-O1-NEXT:    and w9, w0, #0xff
 ; CHECK-NOLSE-O1-NEXT:    cmp w9, w1, uxtb
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB47_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB47_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stxrb w9, w2, [x8]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB47_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3:
 ; CHECK-NOLSE-O1-NEXT:    mov w1, #1 ; =0x1
 ; CHECK-NOLSE-O1-NEXT:    ; kill: def $w0 killed $w0 killed $x0
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB47_4: ; %cmpxchg.nostore

 ; CHECK-NOLSE-O1-NEXT:    mov x8, x0
 ; CHECK-NOLSE-O1-NEXT:    ; kill: def $w2 killed $w2 def $x2
 ; CHECK-NOLSE-O1-NEXT:  LBB48_1: ; %cmpxchg.start
 ; CHECK-NOLSE-O1-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NOLSE-O1-NEXT:    ldxrh w0, [x8]
 ; CHECK-NOLSE-O1-NEXT:    and w9, w0, #0xffff
 ; CHECK-NOLSE-O1-NEXT:    cmp w9, w1, uxth
 ; CHECK-NOLSE-O1-NEXT:    b.ne LBB48_4
-; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NOLSE-O1-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NOLSE-O1-NEXT:    ; in Loop: Header=BB48_1 Depth=1
 ; CHECK-NOLSE-O1-NEXT:    stxrh w9, w2, [x8]
 ; CHECK-NOLSE-O1-NEXT:    cbnz w9, LBB48_1
 ; CHECK-NOLSE-O1-NEXT:  ; %bb.3:
 ; CHECK-NOLSE-O1-NEXT:    mov w1, #1 ; =0x1
 ; CHECK-NOLSE-O1-NEXT:    ; kill: def $w0 killed $w0 killed $x0
 ; CHECK-NOLSE-O1-NEXT:    ret
 ; CHECK-NOLSE-O1-NEXT:  LBB48_4: ; %cmpxchg.nostore

llvm/test/CodeGen/AArch64/GlobalISel/arm64-pcsections.ll

define i32 @val_compare_and_swap(ptr %p, i32 %cmp, i32 %new) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $w1, $w2, $x0
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $w8 = LDAXRW renamable $x0, implicit-def $x8, pcsections !0 :: (volatile load (s32) from %ir.p)
   ; CHECK-NEXT:   $wzr = SUBSWrs renamable $w8, renamable $w1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $w1, $w2, $x0, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STXRW renamable $w2, renamable $x0, pcsections !0 :: (volatile store (s32) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define i32 @val_compare_and_swap_from_load(ptr %p, i32 %cmp, ptr %pnew) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $w1, $x0, $x9
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $w8 = LDAXRW renamable $x0, implicit-def $x8, pcsections !0 :: (volatile load (s32) from %ir.p)
   ; CHECK-NEXT:   $wzr = SUBSWrs renamable $w8, renamable $w1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $w1, $x0, $x8, $x9
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w10 = STXRW renamable $w9, renamable $x0, pcsections !0 :: (volatile store (s32) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w10, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define i32 @val_compare_and_swap_rel(ptr %p, i32 %cmp, i32 %new) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $w1, $w2, $x0
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $w8 = LDAXRW renamable $x0, implicit-def $x8, pcsections !0 :: (volatile load (s32) from %ir.p)
   ; CHECK-NEXT:   $wzr = SUBSWrs renamable $w8, renamable $w1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $w1, $w2, $x0, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STLXRW renamable $w2, renamable $x0, pcsections !0 :: (volatile store (s32) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define i64 @val_compare_and_swap_64(ptr %p, i64 %cmp, i64 %new) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $x8 = LDXRX renamable $x0, pcsections !0 :: (volatile load (s64) from %ir.p)
   ; CHECK-NEXT:   $xzr = SUBSXrs renamable $x8, renamable $x1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STXRX renamable $x2, renamable $x0, pcsections !0 :: (volatile store (s64) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define i64 @val_compare_and_swap_64_monotonic_seqcst(ptr %p, i64 %cmp, i64 %new) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $x8 = LDAXRX renamable $x0, pcsections !0 :: (volatile load (s64) from %ir.p)
   ; CHECK-NEXT:   $xzr = SUBSXrs renamable $x8, renamable $x1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STLXRX renamable $x2, renamable $x0, pcsections !0 :: (volatile store (s64) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define i64 @val_compare_and_swap_64_release_acquire(ptr %p, i64 %cmp, i64 %new) {
   ; CHECK-NEXT: bb.1.cmpxchg.start:
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.3(0x04000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $x8 = LDAXRX renamable $x0, pcsections !0 :: (volatile load (s64) from %ir.p)
   ; CHECK-NEXT:   $xzr = SUBSXrs renamable $x8, renamable $x1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.3, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.4(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $x0, $x1, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STLXRX renamable $x2, renamable $x0, pcsections !0 :: (volatile store (s64) into %ir.p)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT:   B %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.cmpxchg.nostore:
define { i8, i1 } @cmpxchg_i8(ptr %ptr, i8 %desired, i8 %new) {
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.4(0x04000000)
   ; CHECK-NEXT:   liveins: $w1, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $w0 = LDXRB renamable $x8, implicit-def $x0, pcsections !0 :: (volatile load (s8) from %ir.ptr)
   ; CHECK-NEXT:   renamable $w9 = ANDWri renamable $w0, 7, pcsections !0
   ; CHECK-NEXT:   dead $wzr = SUBSWrx killed renamable $w9, renamable $w1, 0, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.4, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.3(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $w1, $x0, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STXRB renamable $w2, renamable $x8, pcsections !0 :: (volatile store (s8) into %ir.ptr)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
   ; CHECK-NEXT:   liveins: $x0
define { i16, i1 } @cmpxchg_i16(ptr %ptr, i16 %desired, i16 %new) {
   ; CHECK-NEXT:   successors: %bb.2(0x7c000000), %bb.4(0x04000000)
   ; CHECK-NEXT:   liveins: $w1, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   renamable $w0 = LDXRH renamable $x8, implicit-def $x0, pcsections !0 :: (volatile load (s16) from %ir.ptr)
   ; CHECK-NEXT:   renamable $w9 = ANDWri renamable $w0, 15, pcsections !0
   ; CHECK-NEXT:   dead $wzr = SUBSWrx killed renamable $w9, renamable $w1, 8, implicit-def $nzcv, pcsections !0
   ; CHECK-NEXT:   Bcc 1, %bb.4, implicit killed $nzcv, pcsections !0
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT: bb.2.cmpxchg.trystore:
+  ; CHECK-NEXT: bb.2.cmpxchg.fencedstore:
   ; CHECK-NEXT:   successors: %bb.3(0x04000000), %bb.1(0x7c000000)
   ; CHECK-NEXT:   liveins: $w1, $x0, $x2, $x8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   early-clobber renamable $w9 = STXRH renamable $w2, renamable $x8, pcsections !0 :: (volatile store (s16) into %ir.ptr)
   ; CHECK-NEXT:   CBNZW killed renamable $w9, %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
   ; CHECK-NEXT:   liveins: $x0

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-condbr-lower-tree.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 ; RUN: llc -mtriple aarch64 -stop-after=irtranslator -global-isel -verify-machineinstrs %s -o - 2>&1 | FileCheck %s
 
 declare i32 @bar(...)
 define void @or_cond(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: or_cond
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x20000000), %bb.4(0x60000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
-  ; CHECK:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   G_BRCOND [[ICMP2]](s1), %bb.3
-  ; CHECK:   G_BR %bb.4
-  ; CHECK: bb.4.entry:
-  ; CHECK:   successors: %bb.3(0x2aaaaaab), %bb.2(0x55555555)
-  ; CHECK:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP3]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x20000000), %bb.4(0x60000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP2]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.4
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.4.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x2aaaaaab), %bb.3(0x55555555)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP3]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 0
   %tmp3 = icmp slt i32 %Y, 5
   %tmp4 = or i1 %tmp3, %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 define void @or_cond_select(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: or_cond_select
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x20000000), %bb.4(0x60000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[C2:%[0-9]+]]:_(s1) = G_CONSTANT i1 true
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   [[SELECT:%[0-9]+]]:_(s1) = G_SELECT [[ICMP1]](s1), [[C2]], [[ICMP]]
-  ; CHECK:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   G_BRCOND [[ICMP2]](s1), %bb.3
-  ; CHECK:   G_BR %bb.4
-  ; CHECK: bb.4.entry:
-  ; CHECK:   successors: %bb.3(0x2aaaaaab), %bb.2(0x55555555)
-  ; CHECK:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP3]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x20000000), %bb.4(0x60000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[C2:%[0-9]+]]:_(s1) = G_CONSTANT i1 true
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   [[SELECT:%[0-9]+]]:_(s1) = G_SELECT [[ICMP1]](s1), [[C2]], [[ICMP]]
+  ; CHECK-NEXT:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP2]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.4
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.4.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x2aaaaaab), %bb.3(0x55555555)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP3]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 0
   %tmp3 = icmp slt i32 %Y, 5
   %tmp4 = select i1 %tmp3, i1 true, i1 %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 define void @and_cond(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: and_cond
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.4(0x60000000), %bb.2(0x20000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   [[AND:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP]]
-  ; CHECK:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   G_BRCOND [[ICMP2]](s1), %bb.4
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.4.entry:
-  ; CHECK:   successors: %bb.3(0x55555555), %bb.2(0x2aaaaaab)
-  ; CHECK:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP3]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.4(0x60000000), %bb.3(0x20000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   [[AND:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP2]](s1), %bb.4
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.4.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x55555555), %bb.3(0x2aaaaaab)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP3]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 0
   %tmp3 = icmp slt i32 %Y, 5
   %tmp4 = and i1 %tmp3, %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 define void @and_cond_select(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: and_cond_select
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.4(0x60000000), %bb.2(0x20000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[C2:%[0-9]+]]:_(s1) = G_CONSTANT i1 false
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   [[SELECT:%[0-9]+]]:_(s1) = G_SELECT [[ICMP1]](s1), [[ICMP]], [[C2]]
-  ; CHECK:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   G_BRCOND [[ICMP2]](s1), %bb.4
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.4.entry:
-  ; CHECK:   successors: %bb.3(0x55555555), %bb.2(0x2aaaaaab)
-  ; CHECK:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP3]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.4(0x60000000), %bb.3(0x20000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[C2:%[0-9]+]]:_(s1) = G_CONSTANT i1 false
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   [[SELECT:%[0-9]+]]:_(s1) = G_SELECT [[ICMP1]](s1), [[ICMP]], [[C2]]
+  ; CHECK-NEXT:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP2]](s1), %bb.4
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.4.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x55555555), %bb.3(0x2aaaaaab)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP3]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 0
   %tmp3 = icmp slt i32 %Y, 5
   %tmp4 = select i1 %tmp3, i1 %tmp1, i1 false
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 ; Don't emit two branches for same operands.
 define void @or_cond_same_values_cmp(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: or_cond_same_values_cmp
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
-  ; CHECK:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
-  ; CHECK:   G_BRCOND [[OR]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   G_BRCOND [[OR]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 5
   %tmp3 = icmp slt i32 %X, 5
   %tmp4 = or i1 %tmp3, %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 ; Emit multiple branches for more than 2 cases.
 define void @or_cond_multiple_cases(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: or_cond_multiple_cases
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x10000000), %bb.5(0x70000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY2]](s32), [[C]]
-  ; CHECK:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
-  ; CHECK:   [[OR1:%[0-9]+]]:_(s1) = G_OR [[OR]], [[ICMP2]]
-  ; CHECK:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP3]](s1), %bb.3
-  ; CHECK:   G_BR %bb.5
-  ; CHECK: bb.5.entry:
-  ; CHECK:   successors: %bb.3(0x12492492), %bb.4(0x6db6db6e)
-  ; CHECK:   [[ICMP4:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP4]](s1), %bb.3
-  ; CHECK:   G_BR %bb.4
-  ; CHECK: bb.4.entry:
-  ; CHECK:   successors: %bb.3(0x2aaaaaab), %bb.2(0x55555555)
-  ; CHECK:   [[ICMP5:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY2]](s32), [[C]]
-  ; CHECK:   G_BRCOND [[ICMP5]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x10000000), %bb.5(0x70000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY2]](s32), [[C]]
+  ; CHECK-NEXT:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   [[OR1:%[0-9]+]]:_(s1) = G_OR [[OR]], [[ICMP2]]
+  ; CHECK-NEXT:   [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP3]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.5
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.5.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x12492492), %bb.4(0x6db6db6e)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP4:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP4]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.4
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.4.entry:
+  ; CHECK-NEXT:   successors: %bb.2(0x2aaaaaab), %bb.3(0x55555555)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[ICMP5:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY2]](s32), [[C]]
+  ; CHECK-NEXT:   G_BRCOND [[ICMP5]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 5
   %tmp3 = icmp slt i32 %X, 5
   %tmpZ = icmp eq i32 %Z, 5
   %tmp4 = or i1 %tmp3, %tmp1
   %final = or i1 %tmp4, %tmpZ
   br i1 %final, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 ; (X != null) | (Y != null) --> (X|Y) != 0
 ; Don't emit two branches.
 define void @or_cond_ne_null(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: or_cond_ne_null
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(ne), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(ne), [[COPY1]](s32), [[C]]
-  ; CHECK:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
-  ; CHECK:   G_BRCOND [[OR]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(ne), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(ne), [[COPY1]](s32), [[C]]
+  ; CHECK-NEXT:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   G_BRCOND [[OR]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp ne i32 %X, 0
   %tmp3 = icmp ne i32 %Y, 0
   %tmp4 = or i1 %tmp3, %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 ; If the branch is unpredictable, don't add another branch
 ; regardless of whether they are expensive or not.
 
 define void @unpredictable(i32 %X, i32 %Y, i32 %Z) nounwind {
   ; CHECK-LABEL: name: unpredictable
   ; CHECK: bb.1.entry:
-  ; CHECK:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
-  ; CHECK:   liveins: $w0, $w1, $w2
-  ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
-  ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
-  ; CHECK:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-  ; CHECK:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
-  ; CHECK:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
-  ; CHECK:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
-  ; CHECK:   G_BRCOND [[OR]](s1), %bb.3
-  ; CHECK:   G_BR %bb.2
-  ; CHECK: bb.2.common.ret:
-  ; CHECK:   RET_ReallyLR
-  ; CHECK: bb.3.cond_true:
-  ; CHECK:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
+  ; CHECK-NEXT:   liveins: $w0, $w1, $w2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:_(s32) = COPY $w2
+  ; CHECK-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+  ; CHECK-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+  ; CHECK-NEXT:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[COPY]](s32), [[C]]
+  ; CHECK-NEXT:   [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(slt), [[COPY1]](s32), [[C1]]
+  ; CHECK-NEXT:   [[OR:%[0-9]+]]:_(s1) = G_OR [[ICMP1]], [[ICMP]]
+  ; CHECK-NEXT:   G_BRCOND [[OR]](s1), %bb.2
+  ; CHECK-NEXT:   G_BR %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.cond_true:
+  ; CHECK-NEXT:   TCRETURNdi @bar, 0, csr_aarch64_aapcs, implicit $sp
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.UnifiedReturnBlock:
+  ; CHECK-NEXT:   RET_ReallyLR
 entry:
   %tmp1 = icmp eq i32 %X, 0
   %tmp3 = icmp slt i32 %Y, 5
   %tmp4 = or i1 %tmp3, %tmp1
   br i1 %tmp4, label %cond_true, label %UnifiedReturnBlock, !unpredictable !0
 
 cond_true:
   %tmp5 = tail call i32 (...) @bar( )
   ret void
 
 UnifiedReturnBlock:
   ret void
 }
 
 !0 = !{}

llvm/test/CodeGen/AArch64/GlobalISel/irtranslator-hoisted-constants.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc -mtriple=aarch64-apple-ios -global-isel -stop-after=irtranslator %s -o - | FileCheck %s --check-prefix=TRANSLATED
 ; RUN: llc -mtriple=aarch64-apple-ios -global-isel -stop-before=instruction-select %s -o - | FileCheck %s --check-prefix=PRESELECTION
 ; RUN: llc -mtriple=aarch64-apple-ios -global-isel -stop-after=instruction-select %s -o - | FileCheck %s --check-prefix=POSTSELECTION
 
 ; Check we generate G_CONSTANT_FOLD_BARRIER of constants and don't fold them, since they're
 ; used by constant hoisting to prevent constant folding/propagation.
 
 declare void @callee()
 
 define i32 @test(i32 %a, i1 %c) {
   ; TRANSLATED-LABEL: name: test
   ; TRANSLATED: bb.1.entry:
-  ; TRANSLATED-NEXT:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
+  ; TRANSLATED-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
   ; TRANSLATED-NEXT:   liveins: $w0, $w1
   ; TRANSLATED-NEXT: {{  $}}
   ; TRANSLATED-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
   ; TRANSLATED-NEXT:   [[COPY1:%[0-9]+]]:_(s32) = COPY $w1
   ; TRANSLATED-NEXT:   [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY1]](s32)
   ; TRANSLATED-NEXT:   [[ASSERT_ZEXT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC]], 1
   ; TRANSLATED-NEXT:   [[TRUNC1:%[0-9]+]]:_(s1) = G_TRUNC [[ASSERT_ZEXT]](s8)
   ; TRANSLATED-NEXT:   [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 100000
   ; TRANSLATED-NEXT:   [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
   ; TRANSLATED-NEXT:   [[CONSTANT_FOLD_BARRIER:%[0-9]+]]:_(s32) = G_CONSTANT_FOLD_BARRIER [[C]]
-  ; TRANSLATED-NEXT:   G_BRCOND [[TRUNC1]](s1), %bb.3
-  ; TRANSLATED-NEXT:   G_BR %bb.2
-  ; TRANSLATED-NEXT: {{  $}}
-  ; TRANSLATED-NEXT: bb.2.common.ret:
-  ; TRANSLATED-NEXT:   [[PHI:%[0-9]+]]:_(s32) = G_PHI %7(s32), %bb.3, [[C1]](s32), %bb.1
-  ; TRANSLATED-NEXT:   $w0 = COPY [[PHI]](s32)
-  ; TRANSLATED-NEXT:   RET_ReallyLR implicit $w0
-  ; TRANSLATED-NEXT: {{  $}}
-  ; TRANSLATED-NEXT: bb.3.cont:
-  ; TRANSLATED-NEXT:   successors: %bb.2(0x80000000)
+  ; TRANSLATED-NEXT:   G_BRCOND [[TRUNC1]](s1), %bb.2
+  ; TRANSLATED-NEXT:   G_BR %bb.3
   ; TRANSLATED-NEXT: {{  $}}
+  ; TRANSLATED-NEXT: bb.2.cont:
   ; TRANSLATED-NEXT:   [[ADD:%[0-9]+]]:_(s32) = G_ADD [[COPY]], [[CONSTANT_FOLD_BARRIER]]
   ; TRANSLATED-NEXT:   ADJCALLSTACKDOWN 0, 0, implicit-def $sp, implicit $sp
   ; TRANSLATED-NEXT:   BL @callee, csr_darwin_aarch64_aapcs, implicit-def $lr, implicit $sp
   ; TRANSLATED-NEXT:   ADJCALLSTACKUP 0, 0, implicit-def $sp, implicit $sp
-  ; TRANSLATED-NEXT:   G_BR %bb.2
+  ; TRANSLATED-NEXT:   $w0 = COPY [[ADD]](s32)
+  ; TRANSLATED-NEXT:   RET_ReallyLR implicit $w0
+  ; TRANSLATED-NEXT: {{  $}}
+  ; TRANSLATED-NEXT: bb.3.end:
+  ; TRANSLATED-NEXT:   $w0 = COPY [[C1]](s32)
+  ; TRANSLATED-NEXT:   RET_ReallyLR implicit $w0
+  ;
   ; PRESELECTION-LABEL: name: test
   ; PRESELECTION: bb.1.entry:
-  ; PRESELECTION-NEXT:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
+  ; PRESELECTION-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
   ; PRESELECTION-NEXT:   liveins: $w0, $w1
   ; PRESELECTION-NEXT: {{  $}}
   ; PRESELECTION-NEXT:   [[COPY:%[0-9]+]]:gpr(s32) = COPY $w0
   ; PRESELECTION-NEXT:   [[COPY1:%[0-9]+]]:gpr(s32) = COPY $w1
   ; PRESELECTION-NEXT:   [[TRUNC:%[0-9]+]]:gpr(s8) = G_TRUNC [[COPY1]](s32)
   ; PRESELECTION-NEXT:   [[ASSERT_ZEXT:%[0-9]+]]:gpr(s8) = G_ASSERT_ZEXT [[TRUNC]], 1
-  ; PRESELECTION-NEXT:   [[C:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 0
-  ; PRESELECTION-NEXT:   [[C1:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 100000
-  ; PRESELECTION-NEXT:   [[CONSTANT_FOLD_BARRIER:%[0-9]+]]:gpr(s32) = G_CONSTANT_FOLD_BARRIER [[C1]]
-  ; PRESELECTION-NEXT:   [[C2:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 1
+  ; PRESELECTION-NEXT:   [[C:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 100000
+  ; PRESELECTION-NEXT:   [[CONSTANT_FOLD_BARRIER:%[0-9]+]]:gpr(s32) = G_CONSTANT_FOLD_BARRIER [[C]]
   ; PRESELECTION-NEXT:   [[ANYEXT:%[0-9]+]]:gpr(s32) = G_ANYEXT [[ASSERT_ZEXT]](s8)
-  ; PRESELECTION-NEXT:   [[AND:%[0-9]+]]:gpr(s32) = G_AND [[ANYEXT]], [[C2]]
+  ; PRESELECTION-NEXT:   [[C1:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 1
+  ; PRESELECTION-NEXT:   [[XOR:%[0-9]+]]:gpr(s32) = G_XOR [[ANYEXT]], [[C1]]
+  ; PRESELECTION-NEXT:   [[AND:%[0-9]+]]:gpr(s32) = G_AND [[XOR]], [[C1]]
   ; PRESELECTION-NEXT:   G_BRCOND [[AND]](s32), %bb.3
   ; PRESELECTION-NEXT:   G_BR %bb.2
   ; PRESELECTION-NEXT: {{  $}}
-  ; PRESELECTION-NEXT: bb.2.common.ret:
-  ; PRESELECTION-NEXT:   [[PHI:%[0-9]+]]:gpr(s32) = G_PHI %7(s32), %bb.3, [[C]](s32), %bb.1
-  ; PRESELECTION-NEXT:   $w0 = COPY [[PHI]](s32)
-  ; PRESELECTION-NEXT:   RET_ReallyLR implicit $w0
-  ; PRESELECTION-NEXT: {{  $}}
-  ; PRESELECTION-NEXT: bb.3.cont:
-  ; PRESELECTION-NEXT:   successors: %bb.2(0x80000000)
-  ; PRESELECTION-NEXT: {{  $}}
+  ; PRESELECTION-NEXT: bb.2.cont:
   ; PRESELECTION-NEXT:   [[ADD:%[0-9]+]]:gpr(s32) = G_ADD [[COPY]], [[CONSTANT_FOLD_BARRIER]]
   ; PRESELECTION-NEXT:   ADJCALLSTACKDOWN 0, 0, implicit-def $sp, implicit $sp
   ; PRESELECTION-NEXT:   BL @callee, csr_darwin_aarch64_aapcs, implicit-def $lr, implicit $sp
   ; PRESELECTION-NEXT:   ADJCALLSTACKUP 0, 0, implicit-def $sp, implicit $sp
-  ; PRESELECTION-NEXT:   G_BR %bb.2
+  ; PRESELECTION-NEXT:   $w0 = COPY [[ADD]](s32)
+  ; PRESELECTION-NEXT:   RET_ReallyLR implicit $w0
+  ; PRESELECTION-NEXT: {{  $}}
+  ; PRESELECTION-NEXT: bb.3.end:
+  ; PRESELECTION-NEXT:   [[C2:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 0
+  ; PRESELECTION-NEXT:   $w0 = COPY [[C2]](s32)
+  ; PRESELECTION-NEXT:   RET_ReallyLR implicit $w0
+  ;
   ; POSTSELECTION-LABEL: name: test
   ; POSTSELECTION: bb.1.entry:
-  ; POSTSELECTION-NEXT:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
+  ; POSTSELECTION-NEXT:   successors: %bb.2(0x40000000), %bb.3(0x40000000)
   ; POSTSELECTION-NEXT:   liveins: $w0, $w1
   ; POSTSELECTION-NEXT: {{  $}}
   ; POSTSELECTION-NEXT:   [[COPY:%[0-9]+]]:gpr32 = COPY $w0
   ; POSTSELECTION-NEXT:   [[COPY1:%[0-9]+]]:gpr32 = COPY $w1
-  ; POSTSELECTION-NEXT:   [[COPY2:%[0-9]+]]:gpr32 = COPY $wzr
   ; POSTSELECTION-NEXT:   [[MOVi32imm:%[0-9]+]]:gpr32 = MOVi32imm 100000
-  ; POSTSELECTION-NEXT:   TBNZW [[COPY1]], 0, %bb.3
+  ; POSTSELECTION-NEXT:   TBZW [[COPY1]], 0, %bb.3
   ; POSTSELECTION-NEXT:   B %bb.2
   ; POSTSELECTION-NEXT: {{  $}}
-  ; POSTSELECTION-NEXT: bb.2.common.ret:
-  ; POSTSELECTION-NEXT:   [[PHI:%[0-9]+]]:gpr32 = PHI %7, %bb.3, [[COPY2]], %bb.1
-  ; POSTSELECTION-NEXT:   $w0 = COPY [[PHI]]
-  ; POSTSELECTION-NEXT:   RET_ReallyLR implicit $w0
-  ; POSTSELECTION-NEXT: {{  $}}
-  ; POSTSELECTION-NEXT: bb.3.cont:
-  ; POSTSELECTION-NEXT:   successors: %bb.2(0x80000000)
-  ; POSTSELECTION-NEXT: {{  $}}
+  ; POSTSELECTION-NEXT: bb.2.cont:
   ; POSTSELECTION-NEXT:   [[ADDWrr:%[0-9]+]]:gpr32 = ADDWrr [[COPY]], [[MOVi32imm]]
   ; POSTSELECTION-NEXT:   ADJCALLSTACKDOWN 0, 0, implicit-def $sp, implicit $sp
   ; POSTSELECTION-NEXT:   BL @callee, csr_darwin_aarch64_aapcs, implicit-def $lr, implicit $sp
   ; POSTSELECTION-NEXT:   ADJCALLSTACKUP 0, 0, implicit-def $sp, implicit $sp
-  ; POSTSELECTION-NEXT:   B %bb.2
+  ; POSTSELECTION-NEXT:   $w0 = COPY [[ADDWrr]]
+  ; POSTSELECTION-NEXT:   RET_ReallyLR implicit $w0
+  ; POSTSELECTION-NEXT: {{  $}}
+  ; POSTSELECTION-NEXT: bb.3.end:
+  ; POSTSELECTION-NEXT:   [[COPY2:%[0-9]+]]:gpr32 = COPY $wzr
+  ; POSTSELECTION-NEXT:   $w0 = COPY [[COPY2]]
+  ; POSTSELECTION-NEXT:   RET_ReallyLR implicit $w0
 entry:
   %hc = bitcast i32 100000 to i32
   br i1 %c, label %cont, label %end
 cont:
   %add = add i32 %a, %hc
   call void @callee()
   ret i32 %add
 end:
   ret i32 0
 }
 

llvm/test/CodeGen/AArch64/aarch64-dup-dot-crash.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc -mtriple=arm64-unknown-unknown < %s -o -| FileCheck %s
 
 ; This test covers a case where extract_vector_elt is selected when DUP is
 ; generated. Where it tries to generate a ZextOrTrunc node with floating point
 ; type resulting in a crash.
 ; See https://reviews.llvm.org/D128144#4280024 for context
 define double @dot_product(double %a) {
 ; CHECK-LABEL: dot_product:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    fmov d1, #1.00000000
 ; CHECK-NEXT:    fadd d0, d0, d1
-; CHECK-NEXT:    fadd d0, d0, d1
-; CHECK-NEXT:    movi d1, #0000000000000000
-; CHECK-NEXT:    fadd d0, d0, d1
-; CHECK-NEXT:    fsqrt d0, d0
-; CHECK-NEXT:    fmul d2, d0, d1
-; CHECK-NEXT:    fcmp d0, #0.0
-; CHECK-NEXT:    fcsel d0, d1, d2, gt
+; CHECK-NEXT:    fadd d1, d0, d1
+; CHECK-NEXT:    movi d0, #0000000000000000
+; CHECK-NEXT:    fadd d1, d1, d0
+; CHECK-NEXT:    fsqrt d1, d1
+; CHECK-NEXT:    fcmp d1, #0.0
+; CHECK-NEXT:    b.gt .LBB0_2
+; CHECK-NEXT:  // %bb.1: // %bb.1
+; CHECK-NEXT:    fmul d0, d1, d0
+; CHECK-NEXT:  .LBB0_2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   %fadd = call double @llvm.vector.reduce.fadd.v3f64(double %a, <3 x double> <double 1.000000e+00, double 1.000000e+00, double 0.000000e+00>)
   %sqrt = call double @llvm.sqrt.f64(double %fadd)
   %insert = insertelement <3 x double> zeroinitializer, double %sqrt, i64 0
   %shuffle = shufflevector <3 x double> %insert, <3 x double> zeroinitializer, <3 x i32> zeroinitializer
   %mul = fmul <3 x double> %shuffle, <double 1.000000e+00, double 1.000000e+00, double 0.000000e+00>
   %shuffle.1 = extractelement <3 x double> %mul, i64 0

llvm/test/CodeGen/AArch64/aarch64-fold-lslfast.ll

; CHECK3-NEXT:    ret
   call void @foo()
   store i64 %result, ptr %arrayidx86, align 8
   ret i64 %result
 }
 
 define i64 @multi_use_non_memory(i64 %a, i64 %b) {
 ; CHECK-LABEL: multi_use_non_memory:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    lsl x8, x0, #3
-; CHECK-NEXT:    lsl x9, x1, #3
-; CHECK-NEXT:    cmp x8, x9
-; CHECK-NEXT:    b.lt .LBB3_2
+; CHECK-NEXT:    lsl x0, x0, #3
+; CHECK-NEXT:    lsl x8, x1, #3
+; CHECK-NEXT:    cmp x0, x8
+; CHECK-NEXT:    b.lt .LBB3_4
 ; CHECK-NEXT:  // %bb.1: // %falsebb
-; CHECK-NEXT:    csel x0, x8, x9, gt
+; CHECK-NEXT:    b.le .LBB3_3
+; CHECK-NEXT:  // %bb.2: // %exitbb
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_3: // %endbb
+; CHECK-NEXT:    mov x0, x8
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB3_2: // %truebb
+; CHECK-NEXT:  .LBB3_4: // %truebb
 ; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -16
 ; CHECK-NEXT:    bl foo
 entry:
   %mul1 = shl i64 %a, 3
   %mul2 = shl i64 %b, 3
   %cmp = icmp slt i64 %mul1, %mul2

llvm/test/CodeGen/AArch64/aarch64-matrix-umull-smull.ll

 
 define void @larger_smull(i16* nocapture noundef readonly %x, i16 noundef %y, i32* noalias nocapture noundef writeonly %s, i32 noundef %n) {
 ; CHECK-LABEL: larger_smull:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w3, #1
 ; CHECK-NEXT:    b.lt .LBB3_8
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
 ; CHECK-NEXT:    sxth w8, w1
-; CHECK-NEXT:    cmp w3, #15
+; CHECK-NEXT:    cmp w3, #16
 ; CHECK-NEXT:    mov w9, w3
-; CHECK-NEXT:    b.hi .LBB3_3
+; CHECK-NEXT:    b.hs .LBB3_3
 ; CHECK-NEXT:  // %bb.2:
 ; CHECK-NEXT:    mov x10, xzr
 ; CHECK-NEXT:    b .LBB3_6
 ; CHECK-NEXT:  .LBB3_3: // %vector.ph
 ; CHECK-NEXT:    dup v0.8h, w8
 ; CHECK-NEXT:    and x10, x9, #0xfffffff0
 ; CHECK-NEXT:    add x11, x2, #32
 ; CHECK-NEXT:    add x12, x0, #16
 ; CHECK-NEXT:    mov x13, x10
 ; CHECK-NEXT:  .LBB3_4: // %vector.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp q1, q2, [x12, #-16]
 ; CHECK-NEXT:    subs x13, x13, #16
 ; CHECK-NEXT:    add x12, x12, #32
 ; CHECK-NEXT:    smull2 v3.4s, v0.8h, v1.8h
 ; CHECK-NEXT:    smull v1.4s, v0.4h, v1.4h
 ; CHECK-NEXT:    smull2 v4.4s, v0.8h, v2.8h
 ; CHECK-NEXT:    smull v2.4s, v0.4h, v2.4h
 ; CHECK-NEXT:    stp q1, q3, [x11, #-32]
 ; CHECK-NEXT:    stp q2, q4, [x11], #64
 ; CHECK-NEXT:    b.ne .LBB3_4
 ; CHECK-NEXT:  // %bb.5: // %middle.block
 ; CHECK-NEXT:    cmp x10, x9
 ; CHECK-NEXT:    b.eq .LBB3_8
-; CHECK-NEXT:  .LBB3_6: // %for.body.preheader1
+; CHECK-NEXT:  .LBB3_6: // %for.body.preheader14
 ; CHECK-NEXT:    add x11, x2, x10, lsl #2
 ; CHECK-NEXT:    add x12, x0, x10, lsl #1
 ; CHECK-NEXT:    sub x9, x9, x10
 ; CHECK-NEXT:  .LBB3_7: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldrsh w10, [x12], #2
 ; CHECK-NEXT:    subs x9, x9, #1
 ; CHECK-NEXT:    mul w10, w10, w8

 
 
 define void @larger_umull(i16* nocapture noundef readonly %x, i16 noundef %y, i32* noalias nocapture noundef writeonly %s, i32 noundef %n) {
 ; CHECK-LABEL: larger_umull:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w3, #1
 ; CHECK-NEXT:    b.lt .LBB4_8
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w3, #15
+; CHECK-NEXT:    cmp w3, #16
 ; CHECK-NEXT:    and w8, w1, #0xffff
 ; CHECK-NEXT:    mov w9, w3
-; CHECK-NEXT:    b.hi .LBB4_3
+; CHECK-NEXT:    b.hs .LBB4_3
 ; CHECK-NEXT:  // %bb.2:
 ; CHECK-NEXT:    mov x10, xzr
 ; CHECK-NEXT:    b .LBB4_6
 ; CHECK-NEXT:  .LBB4_3: // %vector.ph
 ; CHECK-NEXT:    dup v0.8h, w8
 ; CHECK-NEXT:    and x10, x9, #0xfffffff0
 ; CHECK-NEXT:    add x11, x2, #32
 ; CHECK-NEXT:    add x12, x0, #16
 ; CHECK-NEXT:    mov x13, x10
 ; CHECK-NEXT:  .LBB4_4: // %vector.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp q1, q2, [x12, #-16]
 ; CHECK-NEXT:    subs x13, x13, #16
 ; CHECK-NEXT:    add x12, x12, #32
 ; CHECK-NEXT:    umull2 v3.4s, v0.8h, v1.8h
 ; CHECK-NEXT:    umull v1.4s, v0.4h, v1.4h
 ; CHECK-NEXT:    umull2 v4.4s, v0.8h, v2.8h
 ; CHECK-NEXT:    umull v2.4s, v0.4h, v2.4h
 ; CHECK-NEXT:    stp q1, q3, [x11, #-32]
 ; CHECK-NEXT:    stp q2, q4, [x11], #64
 ; CHECK-NEXT:    b.ne .LBB4_4
 ; CHECK-NEXT:  // %bb.5: // %middle.block
 ; CHECK-NEXT:    cmp x10, x9
 ; CHECK-NEXT:    b.eq .LBB4_8
-; CHECK-NEXT:  .LBB4_6: // %for.body.preheader1
+; CHECK-NEXT:  .LBB4_6: // %for.body.preheader14
 ; CHECK-NEXT:    add x11, x2, x10, lsl #2
 ; CHECK-NEXT:    add x12, x0, x10, lsl #1
 ; CHECK-NEXT:    sub x9, x9, x10
 ; CHECK-NEXT:  .LBB4_7: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldrh w10, [x12], #2
 ; CHECK-NEXT:    subs x9, x9, #1
 ; CHECK-NEXT:    mul w10, w10, w8

 
 
 define i16 @red_mla_dup_ext_u8_s8_s16(i8* noalias nocapture noundef readonly %A, i8 noundef %B, i32 noundef %n) {
 ; CHECK-LABEL: red_mla_dup_ext_u8_s8_s16:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz w2, .LBB5_3
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
 ; CHECK-NEXT:    sxtb w9, w1
-; CHECK-NEXT:    cmp w2, #15
+; CHECK-NEXT:    cmp w2, #16
 ; CHECK-NEXT:    mov w10, w2
-; CHECK-NEXT:    b.hi .LBB5_4
+; CHECK-NEXT:    b.hs .LBB5_4
 ; CHECK-NEXT:  // %bb.2:
 ; CHECK-NEXT:    mov x11, xzr
 ; CHECK-NEXT:    mov w8, wzr
 ; CHECK-NEXT:    b .LBB5_7
 ; CHECK-NEXT:  .LBB5_3:
 ; CHECK-NEXT:    mov w8, wzr
 ; CHECK-NEXT:    mov w0, w8
 ; CHECK-NEXT:    ret

 ; CHECK-NEXT:    mla v1.8h, v2.8h, v4.8h
 ; CHECK-NEXT:    b.ne .LBB5_5
 ; CHECK-NEXT:  // %bb.6: // %middle.block
 ; CHECK-NEXT:    add v0.8h, v1.8h, v0.8h
 ; CHECK-NEXT:    cmp x11, x10
 ; CHECK-NEXT:    addv h0, v0.8h
 ; CHECK-NEXT:    fmov w8, s0
 ; CHECK-NEXT:    b.eq .LBB5_9
-; CHECK-NEXT:  .LBB5_7: // %for.body.preheader1
+; CHECK-NEXT:  .LBB5_7: // %for.body.preheader17
 ; CHECK-NEXT:    sub x10, x10, x11
 ; CHECK-NEXT:    add x11, x0, x11
 ; CHECK-NEXT:  .LBB5_8: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldrb w12, [x11], #1
 ; CHECK-NEXT:    subs x10, x10, #1
 ; CHECK-NEXT:    madd w8, w12, w9, w8
 ; CHECK-NEXT:    b.ne .LBB5_8

llvm/test/CodeGen/AArch64/addsub.ll

 ; CHECK-NEXT:    b.lt .LBB16_6
 ; CHECK-NEXT:  // %bb.3: // %test4
 ; CHECK-NEXT:    add w11, w9, #3
 ; CHECK-NEXT:    cmp w10, #321
 ; CHECK-NEXT:    str w11, [x8]
 ; CHECK-NEXT:    b.gt .LBB16_6
 ; CHECK-NEXT:  // %bb.4: // %test5
 ; CHECK-NEXT:    add w11, w9, #4
-; CHECK-NEXT:    cmn w10, #443
+; CHECK-NEXT:    cmn w10, #444
 ; CHECK-NEXT:    str w11, [x8]
-; CHECK-NEXT:    b.ge .LBB16_6
+; CHECK-NEXT:    b.gt .LBB16_6
 ; CHECK-NEXT:  // %bb.5: // %test6
 ; CHECK-NEXT:    add w9, w9, #5
 ; CHECK-NEXT:    str w9, [x8]
-; CHECK-NEXT:  .LBB16_6: // %common.ret
+; CHECK-NEXT:  .LBB16_6: // %ret
 ; CHECK-NEXT:    ret
   %val = load i32, ptr @var_i32
   %val2 = load i32, ptr @var2_i32
 
   %cmp_pos_small = icmp ne i32 %val, 4095
   br i1 %cmp_pos_small, label %ret, label %test2
 
 test2:

llvm/test/CodeGen/AArch64/and-mask-removal.ll

 }
 
 define zeroext i1 @test8_0(i8 zeroext %x)  align 2 {
 ; CHECK-LABEL: test8_0:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    add w8, w0, #74
 ; CHECK-NEXT:    and w8, w8, #0xff
 ; CHECK-NEXT:    cmp w8, #236
-; CHECK-NEXT:    cset w0, lo
+; CHECK-NEXT:    b.hs LBB1_2
+; CHECK-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-NEXT:    mov w0, #1 ; =0x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB1_2: ; %ret_false
+; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    ret
 entry:
   %0 = add i8 %x, 74
   %1 = icmp ult i8 %0, -20
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_1(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_1:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    sub w8, w0, #10
 ; CHECK-SD-NEXT:    cmp w8, #89
-; CHECK-SD-NEXT:    cset w0, hi
+; CHECK-SD-NEXT:    b.ls LBB2_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB2_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_1:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #10
 ; CHECK-GI-NEXT:    cmp w8, #90
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB2_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB2_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, 246
   %1 = icmp uge i8 %0, 90
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_2(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_2:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    cmp w0, #208
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB3_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB3_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_2:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #29
 ; CHECK-GI-NEXT:    and w8, w8, #0xff
 ; CHECK-GI-NEXT:    cmp w8, #179
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB3_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB3_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, 227
   %1 = icmp ne i8 %0, 179
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_3(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_3:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    cmp w0, #209
-; CHECK-SD-NEXT:    cset w0, eq
+; CHECK-SD-NEXT:    b.ne LBB4_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB4_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_3:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #55
 ; CHECK-GI-NEXT:    and w8, w8, #0xff
 ; CHECK-GI-NEXT:    cmp w8, #154
-; CHECK-GI-NEXT:    cset w0, eq
+; CHECK-GI-NEXT:    b.ne LBB4_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB4_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, 201
   %1 = icmp eq i8 %0, 154
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_4(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_4:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    cmp w0, #39
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB5_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB5_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_4:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #79
 ; CHECK-GI-NEXT:    and w8, w8, #0xff
 ; CHECK-GI-NEXT:    cmp w8, #216
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB5_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB5_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, -79
   %1 = icmp ne i8 %0, -40
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_5(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_5:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    sub w8, w0, #123
 ; CHECK-SD-NEXT:    cmn w8, #106
-; CHECK-SD-NEXT:    cset w0, hi
+; CHECK-SD-NEXT:    b.ls LBB6_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB6_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_5:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #123
 ; CHECK-GI-NEXT:    cmn w8, #105
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB6_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB6_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, 133
   %1 = icmp uge i8 %0, -105
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_6(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_6:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    sub w8, w0, #58
 ; CHECK-SD-NEXT:    cmp w8, #154
-; CHECK-SD-NEXT:    cset w0, hi
+; CHECK-SD-NEXT:    b.ls LBB7_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB7_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_6:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #58
 ; CHECK-GI-NEXT:    cmp w8, #155
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB7_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB7_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, -58
   %1 = icmp uge i8 %0, 155
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test8_7(i8 zeroext %x)  align 2 {
 ; CHECK-LABEL: test8_7:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    sub w8, w0, #31
 ; CHECK-NEXT:    cmp w8, #124
-; CHECK-NEXT:    cset w0, lo
+; CHECK-NEXT:    b.hs LBB8_2
+; CHECK-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-NEXT:    mov w0, #1 ; =0x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB8_2: ; %ret_false
+; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    ret
 entry:
   %0 = add i8 %x, 225
   %1 = icmp ult i8 %0, 124
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 
 
 define zeroext i1 @test8_8(i8 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test8_8:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    cmp w0, #66
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB9_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB9_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test8_8:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    sub w8, w0, #66
 ; CHECK-GI-NEXT:    cmp w8, #1
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB9_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB9_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i8 %x, 190
   %1 = icmp uge i8 %0, 1
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_0(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_0:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #5086 ; =0x13de
 ; CHECK-SD-NEXT:    cmp w0, w8
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB10_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB10_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_0:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #18547 ; =0x4873
 ; CHECK-GI-NEXT:    mov w9, #23633 ; =0x5c51
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB10_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB10_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, -46989
   %1 = icmp ne i16 %0, -41903
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_2(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_2:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #16882 ; =0x41f2
 ; CHECK-SD-NEXT:    mov w9, #40700 ; =0x9efc
 ; CHECK-SD-NEXT:    add w8, w0, w8
 ; CHECK-SD-NEXT:    cmp w9, w8, uxth
-; CHECK-SD-NEXT:    cset w0, hi
+; CHECK-SD-NEXT:    b.ls LBB11_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB11_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_2:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #16882 ; =0x41f2
 ; CHECK-GI-NEXT:    mov w9, #40699 ; =0x9efb
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB11_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB11_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, 16882
   %1 = icmp ule i16 %0, -24837
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_3(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_3:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #53200 ; =0xcfd0
 ; CHECK-SD-NEXT:    cmp w0, w8
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB12_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB12_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_3:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #29283 ; =0x7263
 ; CHECK-GI-NEXT:    mov w9, #16947 ; =0x4233
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB12_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB12_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, 29283
   %1 = icmp ne i16 %0, 16947
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_4(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_4:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #29985 ; =0x7521
 ; CHECK-SD-NEXT:    mov w9, #15676 ; =0x3d3c
 ; CHECK-SD-NEXT:    add w8, w0, w8
 ; CHECK-SD-NEXT:    cmp w9, w8, uxth
-; CHECK-SD-NEXT:    cset w0, lo
+; CHECK-SD-NEXT:    b.hs LBB13_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB13_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_4:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #29985 ; =0x7521
 ; CHECK-GI-NEXT:    mov w9, #15677 ; =0x3d3d
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, ls
+; CHECK-GI-NEXT:    b.hi LBB13_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB13_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, -35551
   %1 = icmp uge i16 %0, 15677
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_5(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_5:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #23282 ; =0x5af2
 ; CHECK-SD-NEXT:    cmp w0, w8
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB14_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB14_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_5:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #-25214 ; =0xffff9d82
 ; CHECK-GI-NEXT:    mov w9, #63604 ; =0xf874
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB14_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB14_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, -25214
   %1 = icmp ne i16 %0, -1932
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_6(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_6:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #-32194 ; =0xffff823e
 ; CHECK-SD-NEXT:    mov w9, #24320 ; =0x5f00
 ; CHECK-SD-NEXT:    add w8, w0, w8
 ; CHECK-SD-NEXT:    cmp w8, w9
-; CHECK-SD-NEXT:    cset w0, hi
+; CHECK-SD-NEXT:    b.ls LBB15_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB15_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_6:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #-32194 ; =0xffff823e
 ; CHECK-GI-NEXT:    mov w9, #24321 ; =0x5f01
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w8, w9
-; CHECK-GI-NEXT:    cset w0, hs
+; CHECK-GI-NEXT:    b.lo LBB15_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB15_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, -32194
   %1 = icmp uge i16 %0, -41215
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_7(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_7:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #9272 ; =0x2438
 ; CHECK-SD-NEXT:    mov w9, #22619 ; =0x585b
 ; CHECK-SD-NEXT:    add w8, w0, w8
 ; CHECK-SD-NEXT:    cmp w9, w8, uxth
-; CHECK-SD-NEXT:    cset w0, lo
+; CHECK-SD-NEXT:    b.hs LBB16_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB16_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_7:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #9272 ; =0x2438
 ; CHECK-GI-NEXT:    mov w9, #22620 ; =0x585c
 ; CHECK-GI-NEXT:    add w8, w0, w8
 ; CHECK-GI-NEXT:    cmp w9, w8, uxth
-; CHECK-GI-NEXT:    cset w0, ls
+; CHECK-GI-NEXT:    b.hi LBB16_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB16_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, 9272
   %1 = icmp uge i16 %0, -42916
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:
   ret i1 true
 }
 
 define zeroext i1 @test16_8(i16 zeroext %x)  align 2 {
 ; CHECK-SD-LABEL: test16_8:
 ; CHECK-SD:       ; %bb.0: ; %entry
 ; CHECK-SD-NEXT:    mov w8, #4919 ; =0x1337
 ; CHECK-SD-NEXT:    cmp w0, w8
-; CHECK-SD-NEXT:    cset w0, ne
+; CHECK-SD-NEXT:    b.eq LBB17_2
+; CHECK-SD-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-SD-NEXT:    mov w0, #1 ; =0x1
+; CHECK-SD-NEXT:    ret
+; CHECK-SD-NEXT:  LBB17_2: ; %ret_false
+; CHECK-SD-NEXT:    mov w0, wzr
 ; CHECK-SD-NEXT:    ret
 ;
 ; CHECK-GI-LABEL: test16_8:
 ; CHECK-GI:       ; %bb.0: ; %entry
 ; CHECK-GI-NEXT:    mov w8, #6706 ; =0x1a32
 ; CHECK-GI-NEXT:    add w9, w0, #1787
 ; CHECK-GI-NEXT:    cmp w8, w9, uxth
-; CHECK-GI-NEXT:    cset w0, ne
+; CHECK-GI-NEXT:    b.eq LBB17_2
+; CHECK-GI-NEXT:  ; %bb.1: ; %ret_true
+; CHECK-GI-NEXT:    mov w0, #1 ; =0x1
+; CHECK-GI-NEXT:    ret
+; CHECK-GI-NEXT:  LBB17_2: ; %ret_false
+; CHECK-GI-NEXT:    mov w0, wzr
 ; CHECK-GI-NEXT:    ret
 entry:
   %0 = add i16 %x, -63749
   %1 = icmp ne i16 %0, 6706
   br i1 %1, label %ret_true, label %ret_false
 ret_false:
   ret i1 false
 ret_true:

llvm/test/CodeGen/AArch64/andorbrcompare.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-none-elf -verify-machineinstrs %s -o - | FileCheck %s --check-prefixes=CHECK,SDISEL
 ; RUN: llc -mtriple=aarch64-none-elf -global-isel -verify-machineinstrs %s -o - | FileCheck %s --check-prefixes=CHECK,GISEL
 
 declare void @dummy()
 
 define i32 @and_eq_ne_ult(i32 %s0, i32 %s1, i32 %s2, i32 %s3, i32 %s4, i32 %s5, ptr %p) {
 ; SDISEL-LABEL: and_eq_ne_ult:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #0, ne
 ; SDISEL-NEXT:    b.eq .LBB0_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.lo .LBB0_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB0_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_eq_ne_ult:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, eq
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, ne
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB0_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.lo .LBB0_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB0_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp eq i32 %s0, %s1
   %c1 = icmp ne i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp ult i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_ne_ult_ule:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #4, lo
 ; SDISEL-NEXT:    b.ne .LBB1_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.ls .LBB1_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB1_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_ne_ult_ule:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, ne
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, lo
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB1_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.ls .LBB1_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB1_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp ne i32 %s0, %s1
   %c1 = icmp ult i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp ule i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_ult_ule_ugt:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #2, ls
 ; SDISEL-NEXT:    b.lo .LBB2_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.hi .LBB2_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB2_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_ult_ule_ugt:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, lo
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, ls
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB2_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.hi .LBB2_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB2_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp ult i32 %s0, %s1
   %c1 = icmp ule i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp ugt i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_ule_ugt_uge:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #2, hi
 ; SDISEL-NEXT:    b.ls .LBB3_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.hs .LBB3_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB3_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_ule_ugt_uge:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, ls
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, hi
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB3_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.hs .LBB3_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB3_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp ule i32 %s0, %s1
   %c1 = icmp ugt i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp uge i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_ugt_uge_slt:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #0, hs
 ; SDISEL-NEXT:    b.hi .LBB4_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.lt .LBB4_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB4_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_ugt_uge_slt:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, hi
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, hs
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB4_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.lt .LBB4_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB4_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp ugt i32 %s0, %s1
   %c1 = icmp uge i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp slt i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_uge_slt_sle:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #0, lt
 ; SDISEL-NEXT:    b.hs .LBB5_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.le .LBB5_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB5_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_uge_slt_sle:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, hs
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, lt
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB5_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.le .LBB5_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB5_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp uge i32 %s0, %s1
   %c1 = icmp slt i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp sle i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_slt_sle_sgt:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #0, le
 ; SDISEL-NEXT:    b.lt .LBB6_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.gt .LBB6_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB6_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_slt_sle_sgt:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, lt
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, le
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB6_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.gt .LBB6_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB6_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp slt i32 %s0, %s1
   %c1 = icmp sle i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp sgt i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

 ; SDISEL-LABEL: and_sle_sgt_sge:
 ; SDISEL:       // %bb.0: // %entry
 ; SDISEL-NEXT:    cmp w2, w3
 ; SDISEL-NEXT:    ccmp w0, w1, #0, gt
 ; SDISEL-NEXT:    b.le .LBB7_3
 ; SDISEL-NEXT:  // %bb.1: // %entry
 ; SDISEL-NEXT:    cmp w4, w5
 ; SDISEL-NEXT:    b.ge .LBB7_3
-; SDISEL-NEXT:  // %bb.2:
+; SDISEL-NEXT:  // %bb.2: // %else
 ; SDISEL-NEXT:    mov w0, wzr
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  .LBB7_3: // %if
+; SDISEL-NEXT:    mov w8, #1 // =0x1
 ; SDISEL-NEXT:    mov w0, #1 // =0x1
-; SDISEL-NEXT:    str w0, [x6]
+; SDISEL-NEXT:    str w8, [x6]
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: and_sle_sgt_sge:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
 ; GISEL-NEXT:    cset w8, le
 ; GISEL-NEXT:    cmp w2, w3
 ; GISEL-NEXT:    cset w9, gt
 ; GISEL-NEXT:    and w8, w8, w9
 ; GISEL-NEXT:    tbnz w8, #0, .LBB7_3
 ; GISEL-NEXT:  // %bb.1: // %entry
 ; GISEL-NEXT:    cmp w4, w5
-; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    b.ge .LBB7_3
-; GISEL-NEXT:  // %bb.2: // %common.ret
+; GISEL-NEXT:  // %bb.2: // %else
+; GISEL-NEXT:    mov w0, wzr
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  .LBB7_3: // %if
+; GISEL-NEXT:    mov w8, #1 // =0x1
 ; GISEL-NEXT:    mov w0, #1 // =0x1
-; GISEL-NEXT:    str w0, [x6]
+; GISEL-NEXT:    str w8, [x6]
 ; GISEL-NEXT:    ret
 entry:
   %c0 = icmp sle i32 %s0, %s1
   %c1 = icmp sgt i32 %s2, %s3
   %a = and i1 %c0, %c1
   %c2 = icmp sge i32 %s4, %s5
   %o = or i1 %a, %c2
   br i1 %o, label %if, label %else

llvm/test/CodeGen/AArch64/arm64-ccmp.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -debugify-and-strip-all-safe -mcpu=cyclone -verify-machineinstrs -aarch64-enable-ccmp -aarch64-stress-ccmp | FileCheck %s --check-prefixes=CHECK,SDISEL
 ; RUN: llc < %s -debugify-and-strip-all-safe -mcpu=cyclone -verify-machineinstrs -aarch64-enable-ccmp -aarch64-stress-ccmp -global-isel -global-isel-abort=2 | FileCheck %s --check-prefixes=CHECK,GISEL
 target triple = "arm64-apple-ios"
 
 define i32 @single_same(i32 %a, i32 %b) nounwind ssp {
 ; CHECK-LABEL: single_same:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    cmp w0, #5
 ; CHECK-NEXT:    ccmp w1, #17, #4, ne
 ; CHECK-NEXT:    b.ne LBB0_2
 ; CHECK-NEXT:  ; %bb.1: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:  LBB0_2: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 5
   %cmp1 = icmp eq i32 %b, 17
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:

 ; SDISEL-NEXT:    cmp w0, #6
 ; SDISEL-NEXT:    ccmp w1, #17, #0, ge
 ; SDISEL-NEXT:    b.eq LBB1_2
 ; SDISEL-NEXT:  ; %bb.1: ; %if.then
 ; SDISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; SDISEL-NEXT:    bl _foo
 ; SDISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; SDISEL-NEXT:  LBB1_2: ; %if.end
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: single_different:
 ; GISEL:       ; %bb.0: ; %entry
 ; GISEL-NEXT:    cmp w0, #5
 ; GISEL-NEXT:    ccmp w1, #17, #0, gt
 ; GISEL-NEXT:    b.eq LBB1_2
 ; GISEL-NEXT:  ; %bb.1: ; %if.then
 ; GISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; GISEL-NEXT:    bl _foo
 ; GISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; GISEL-NEXT:  LBB1_2: ; %if.end
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 entry:
   %cmp = icmp sle i32 %a, 5
   %cmp1 = icmp ne i32 %b, 17
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:

 ; SDISEL-NEXT:    csinc w8, w8, w1, lt
 ; SDISEL-NEXT:    cmp w8, #16
 ; SDISEL-NEXT:    b.gt LBB2_3
 ; SDISEL-NEXT:  LBB2_2: ; %if.then
 ; SDISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; SDISEL-NEXT:    bl _foo
 ; SDISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; SDISEL-NEXT:  LBB2_3: ; %if.end
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: single_flagclobber:
 ; GISEL:       ; %bb.0: ; %entry
 ; GISEL-NEXT:    cmp w0, #5
 ; GISEL-NEXT:    b.eq LBB2_2
 ; GISEL-NEXT:  ; %bb.1: ; %lor.lhs.false
 ; GISEL-NEXT:    lsl w8, w1, #1
 ; GISEL-NEXT:    cmp w1, #7
 ; GISEL-NEXT:    csinc w8, w8, w1, lt
 ; GISEL-NEXT:    cmp w8, #17
 ; GISEL-NEXT:    b.ge LBB2_3
 ; GISEL-NEXT:  LBB2_2: ; %if.then
 ; GISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; GISEL-NEXT:    bl _foo
 ; GISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; GISEL-NEXT:  LBB2_3: ; %if.end
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 5
   br i1 %cmp, label %if.then, label %lor.lhs.false
 
 lor.lhs.false:                                    ; preds = %entry
   %cmp1 = icmp slt i32 %b, 7
   %mul = shl nsw i32 %b, 1

 ; CHECK-NEXT:    cmp w1, #7
 ; CHECK-NEXT:    csinc w8, w8, w1, lt
 ; CHECK-NEXT:    tbz w8, #3, LBB3_3
 ; CHECK-NEXT:  LBB3_2: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:  LBB3_3: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 5
   br i1 %cmp, label %if.then, label %lor.lhs.false
 
 lor.lhs.false:                                    ; preds = %entry
   %cmp1 = icmp slt i32 %b, 7
   %mul = shl nsw i32 %b, 1

 define i32 @speculate_division(i32 %a, i32 %b) nounwind ssp {
 ; SDISEL-LABEL: speculate_division:
 ; SDISEL:       ; %bb.0: ; %entry
 ; SDISEL-NEXT:    cmp w0, #1
 ; SDISEL-NEXT:    sdiv w8, w1, w0
 ; SDISEL-NEXT:    ccmp w8, #16, #0, ge
 ; SDISEL-NEXT:    b.le LBB4_2
 ; SDISEL-NEXT:  ; %bb.1: ; %if.end
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  LBB4_2: ; %if.then
 ; SDISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; SDISEL-NEXT:    bl _foo
 ; SDISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: speculate_division:
 ; GISEL:       ; %bb.0: ; %entry
 ; GISEL-NEXT:    cmp w0, #0
 ; GISEL-NEXT:    sdiv w8, w1, w0
 ; GISEL-NEXT:    ccmp w8, #17, #0, gt
 ; GISEL-NEXT:    b.lt LBB4_2
 ; GISEL-NEXT:  ; %bb.1: ; %if.end
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  LBB4_2: ; %if.then
 ; GISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; GISEL-NEXT:    bl _foo
 ; GISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, 0
   br i1 %cmp, label %land.lhs.true, label %if.end
 
 land.lhs.true:
   %div = sdiv i32 %b, %a
   %cmp1 = icmp slt i32 %div, 17

 ; SDISEL:       ; %bb.0: ; %entry
 ; SDISEL-NEXT:    cmp w0, #1
 ; SDISEL-NEXT:    scvtf s1, w0
 ; SDISEL-NEXT:    fdiv s0, s0, s1
 ; SDISEL-NEXT:    fmov s1, #17.00000000
 ; SDISEL-NEXT:    fccmp s0, s1, #8, ge
 ; SDISEL-NEXT:    b.ge LBB5_2
 ; SDISEL-NEXT:  ; %bb.1: ; %if.end
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ; SDISEL-NEXT:  LBB5_2: ; %if.then
 ; SDISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; SDISEL-NEXT:    bl _foo
 ; SDISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; SDISEL-NEXT:    mov w0, #7
+; SDISEL-NEXT:    mov w0, #7 ; =0x7
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: single_fcmp:
 ; GISEL:       ; %bb.0: ; %entry
 ; GISEL-NEXT:    cmp w0, #0
 ; GISEL-NEXT:    scvtf s1, w0
 ; GISEL-NEXT:    fdiv s0, s0, s1
 ; GISEL-NEXT:    fmov s1, #17.00000000
 ; GISEL-NEXT:    fccmp s0, s1, #8, gt
 ; GISEL-NEXT:    b.ge LBB5_2
 ; GISEL-NEXT:  ; %bb.1: ; %if.end
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 ; GISEL-NEXT:  LBB5_2: ; %if.then
 ; GISEL-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; GISEL-NEXT:    bl _foo
 ; GISEL-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; GISEL-NEXT:    mov w0, #7
+; GISEL-NEXT:    mov w0, #7 ; =0x7
 ; GISEL-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, 0
   br i1 %cmp, label %land.lhs.true, label %if.end
 
 land.lhs.true:
   %conv = sitofp i32 %a to float
   %div = fdiv float %b, %conv

 ; CHECK-NEXT:    cmp w0, #0
 ; CHECK-NEXT:    ccmp w1, #17, #0, ne
 ; CHECK-NEXT:    b.eq LBB7_2
 ; CHECK-NEXT:  ; %bb.1: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:  LBB7_2: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 0
   %cmp1 = icmp ne i32 %b, 17
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:

 ; CHECK-LABEL: immediate_range:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    cmp w0, #5
 ; CHECK-NEXT:    b.eq LBB8_3
 ; CHECK-NEXT:  ; %bb.1: ; %entry
 ; CHECK-NEXT:    cmp w1, #32
 ; CHECK-NEXT:    b.eq LBB8_3
 ; CHECK-NEXT:  ; %bb.2: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB8_3: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 5
   %cmp1 = icmp eq i32 %b, 32
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:

 ; CHECK-NEXT:    cmp w0, #0
 ; CHECK-NEXT:    ccmp w1, #0, #0, ne
 ; CHECK-NEXT:    b.eq LBB9_2
 ; CHECK-NEXT:  ; %bb.1: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:  LBB9_2: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 0
   %cmp1 = icmp ne i32 %b, 0
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:

 ; CHECK-NEXT:    cmp w0, #0
 ; CHECK-NEXT:    ccmp w1, #0, #4, ne
 ; CHECK-NEXT:    b.ne LBB10_2
 ; CHECK-NEXT:  ; %bb.1: ; %if.then
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:  LBB10_2: ; %if.end
-; CHECK-NEXT:    mov w0, #7
+; CHECK-NEXT:    mov w0, #7 ; =0x7
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp eq i32 %a, 0
   %cmp1 = icmp eq i32 %b, 0
   %or.cond = or i1 %cmp, %cmp1
   br i1 %or.cond, label %if.then, label %if.end
 
 if.then:
   %call = tail call i32 @foo() nounwind
   br label %if.end
 
 if.end:
   ret i32 7
 }
 declare i32 @foo()
 
 %str1 = type { %str2 }
 %str2 = type { [24 x i8], ptr, i32, ptr, i32, [4 x i8], ptr, ptr, ptr, ptr, ptr, ptr, ptr, ptr, ptr, ptr, i8, ptr, ptr, ptr }
 
 ; Test case distilled from 126.gcc.
 ; The phi in sw.bb.i.i gets multiple operands for the %entry predecessor.
 define void @build_modify_expr() nounwind ssp {
-; CHECK-LABEL: build_modify_expr:
-; CHECK:       ; %bb.0: ; %entry
-; CHECK-NEXT:    ret
+; SDISEL-LABEL: build_modify_expr:
+; SDISEL:       ; %bb.0: ; %entry
+; SDISEL-NEXT:    cmp w8, #37
+; SDISEL-NEXT:    mov w8, #1 ; =0x1
+; SDISEL-NEXT:    lsl x8, x8, xzr
+; SDISEL-NEXT:    mov x9, #31 ; =0x1f
+; SDISEL-NEXT:    movk x9, #48, lsl #32
+; SDISEL-NEXT:    and x8, x8, x9
+; SDISEL-NEXT:    ccmp x8, #0, #4, ls
+; SDISEL-NEXT:    b.eq LBB11_2
+; SDISEL-NEXT:  ; %bb.1: ; %if.end85
+; SDISEL-NEXT:    ret
+; SDISEL-NEXT:  LBB11_2: ; %sw.bb.i.i.preheader
+; SDISEL-NEXT:    ; implicit-def: $x8
+; SDISEL-NEXT:  LBB11_3: ; %sw.bb.i.i
+; SDISEL-NEXT:    ; =>This Inner Loop Header: Depth=1
+; SDISEL-NEXT:    ldr x8, [x8, #32]
+; SDISEL-NEXT:    b LBB11_3
+;
+; GISEL-LABEL: build_modify_expr:
+; GISEL:       ; %bb.0: ; %entry
+; GISEL-NEXT:    ; implicit-def: $x8
+; GISEL-NEXT:    cmp w8, #37
+; GISEL-NEXT:    b.hi LBB11_3
+; GISEL-NEXT:  ; %bb.1: ; %entry
+; GISEL-NEXT:    mov w9, #1 ; =0x1
+; GISEL-NEXT:    ; implicit-def: $x8
+; GISEL-NEXT:    tbz w9, #0, LBB11_3
+; GISEL-NEXT:  ; %bb.2: ; %if.end85
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  LBB11_3: ; %sw.bb.i.i
+; GISEL-NEXT:    ; =>This Inner Loop Header: Depth=1
+; GISEL-NEXT:    ldr x8, [x8, #32]
+; GISEL-NEXT:    b LBB11_3
 entry:
   switch i32 undef, label %sw.bb.i.i [
     i32 69, label %if.end85
     i32 70, label %if.end85
     i32 71, label %if.end85
     i32 72, label %if.end85
     i32 73, label %if.end85
     i32 105, label %if.end85

 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp w1, #5
 ; SDISEL-NEXT:    ccmp w0, w1, #0, ne
 ; SDISEL-NEXT:    csel x0, x2, x3, lt
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: select_and:
 ; GISEL:       ; %bb.0:
-; GISEL-NEXT:    mov w8, #5
+; GISEL-NEXT:    mov w8, #5 ; =0x5
 ; GISEL-NEXT:    cmp w8, w1
 ; GISEL-NEXT:    ccmp w0, w1, #0, ne
 ; GISEL-NEXT:    csel x0, x2, x3, lt
 ; GISEL-NEXT:    ret
   %1 = icmp slt i32 %w0, %w1
   %2 = icmp ne i32 5, %w1
   %3 = and i1 %1, %2
   %sel = select i1 %3, i64 %x2, i64 %x3
   ret i64 %sel
 }
 
 define i64 @select_or(i32 %w0, i32 %w1, i64 %x2, i64 %x3) {
 ; SDISEL-LABEL: select_or:
 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp w1, #5
 ; SDISEL-NEXT:    ccmp w0, w1, #8, eq
 ; SDISEL-NEXT:    csel x0, x2, x3, lt
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: select_or:
 ; GISEL:       ; %bb.0:
-; GISEL-NEXT:    mov w8, #5
+; GISEL-NEXT:    mov w8, #5 ; =0x5
 ; GISEL-NEXT:    cmp w8, w1
 ; GISEL-NEXT:    ccmp w0, w1, #8, eq
 ; GISEL-NEXT:    csel x0, x2, x3, lt
 ; GISEL-NEXT:    ret
   %1 = icmp slt i32 %w0, %w1
   %2 = icmp ne i32 5, %w1
   %3 = or i1 %1, %2
   %sel = select i1 %3, i64 %x2, i64 %x3
   ret i64 %sel
 }
 
 define float @select_or_float(i32 %w0, i32 %w1, float %x2, float %x3) {
 ; SDISEL-LABEL: select_or_float:
 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp w1, #5
 ; SDISEL-NEXT:    ccmp w0, w1, #8, eq
 ; SDISEL-NEXT:    fcsel s0, s0, s1, lt
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: select_or_float:
 ; GISEL:       ; %bb.0:
-; GISEL-NEXT:    mov w8, #5
+; GISEL-NEXT:    mov w8, #5 ; =0x5
 ; GISEL-NEXT:    cmp w8, w1
 ; GISEL-NEXT:    ccmp w0, w1, #8, eq
 ; GISEL-NEXT:    fcsel s0, s0, s1, lt
 ; GISEL-NEXT:    ret
   %1 = icmp slt i32 %w0, %w1
   %2 = icmp ne i32 5, %w1
   %3 = or i1 %1, %2
   %sel = select i1 %3, float %x2,float %x3
   ret float %sel
 }
 
 define i64 @gccbug(i64 %x0, i64 %x1) {
 ; SDISEL-LABEL: gccbug:
 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp x0, #2
 ; SDISEL-NEXT:    ccmp x0, #4, #4, ne
 ; SDISEL-NEXT:    ccmp x1, #0, #0, eq
-; SDISEL-NEXT:    mov w8, #1
+; SDISEL-NEXT:    mov w8, #1 ; =0x1
 ; SDISEL-NEXT:    cinc x0, x8, eq
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: gccbug:
 ; GISEL:       ; %bb.0:
-; GISEL-NEXT:    mov w8, #2
+; GISEL-NEXT:    mov w8, #2 ; =0x2
 ; GISEL-NEXT:    cmp x0, #2
 ; GISEL-NEXT:    ccmp x0, #4, #4, ne
 ; GISEL-NEXT:    ccmp x1, #0, #0, eq
 ; GISEL-NEXT:    csinc x0, x8, xzr, eq
 ; GISEL-NEXT:    ret
   %cmp0 = icmp eq i64 %x1, 0
   %cmp1 = icmp eq i64 %x0, 2
   %cmp2 = icmp eq i64 %x0, 4
; CHECK-NEXT:    ret
   %sel = select i1 %and, i32 %v1, i32 %v2
   ret i32 %sel
 }
 
 define i32 @select_andor32(i32 %v1, i32 %v2, i32 %v3) {
 ; SDISEL-LABEL: select_andor32:
 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp w1, w2
-; SDISEL-NEXT:    mov w8, #32
+; SDISEL-NEXT:    mov w8, #32 ; =0x20
 ; SDISEL-NEXT:    ccmp w0, w8, #4, lt
 ; SDISEL-NEXT:    ccmp w0, w1, #0, eq
 ; SDISEL-NEXT:    csel w0, w0, w1, eq
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: select_andor32:
 ; GISEL:       ; %bb.0:
-; GISEL-NEXT:    mov w8, #32
+; GISEL-NEXT:    mov w8, #32 ; =0x20
 ; GISEL-NEXT:    cmp w1, w2
 ; GISEL-NEXT:    ccmp w0, w8, #4, lt
 ; GISEL-NEXT:    ccmp w0, w1, #0, eq
 ; GISEL-NEXT:    csel w0, w0, w1, eq
 ; GISEL-NEXT:    ret
   %c0 = icmp eq i32 %v1, %v2
   %c1 = icmp sge i32 %v2, %v3
   %c2 = icmp eq i32 %v1, 32

 ; sequence.
 define i32 @select_noccmp3(i32 %v0, i32 %v1, i32 %v2) {
 ; SDISEL-LABEL: select_noccmp3:
 ; SDISEL:       ; %bb.0:
 ; SDISEL-NEXT:    cmp w0, #0
 ; SDISEL-NEXT:    ccmp w0, #13, #0, ge
 ; SDISEL-NEXT:    cset w8, gt
 ; SDISEL-NEXT:    cmp w0, #22
-; SDISEL-NEXT:    mov w9, #44
+; SDISEL-NEXT:    mov w9, #44 ; =0x2c
 ; SDISEL-NEXT:    ccmp w0, w9, #0, ge
 ; SDISEL-NEXT:    csel w8, wzr, w8, le
 ; SDISEL-NEXT:    cmp w0, #99
-; SDISEL-NEXT:    mov w9, #77
+; SDISEL-NEXT:    mov w9, #77 ; =0x4d
 ; SDISEL-NEXT:    ccmp w0, w9, #4, ne
 ; SDISEL-NEXT:    cset w9, eq
 ; SDISEL-NEXT:    tst w8, w9
 ; SDISEL-NEXT:    csel w0, w1, w2, ne
 ; SDISEL-NEXT:    ret
 ;
 ; GISEL-LABEL: select_noccmp3:
 ; GISEL:       ; %bb.0:

llvm/test/CodeGen/AArch64/arm64-csel.ll

entry:
   %sub3 = sub nsw i32 0, %sub
   %cond = select i1 %cmp, i32 %sub, i32 %sub3
   ret i32 %cond
 }
 
 ; make sure we can handle branch instruction in optimizeCompare.
 define i32@foo6(i32 %a, i32 %b) nounwind ssp {
 ; CHECK-LABEL: foo6:
-; CHECK:       // %bb.0: // %common.ret
-; CHECK-NEXT:    subs w8, w0, w1
-; CHECK-NEXT:    csinc w0, w8, wzr, le
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    sub w0, w0, w1
+; CHECK-NEXT:    cmp w0, #1
+; CHECK-NEXT:    b.lt .LBB5_2
+; CHECK-NEXT:  // %bb.1: // %l.if
+; CHECK-NEXT:    mov w0, #1 // =0x1
+; CHECK-NEXT:  .LBB5_2: // %l.else
 ; CHECK-NEXT:    ret
   %sub = sub nsw i32 %a, %b
   %cmp = icmp sgt i32 %sub, 0
   br i1 %cmp, label %l.if, label %l.else
 
 l.if:
   ret i32 1
 
 l.else:
   ret i32 %sub
 }
 
 ; If CPSR is used multiple times and V flag is used, we don't remove cmp.
 define i32 @foo7(i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: foo7:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    subs w8, w0, w1
-; CHECK-NEXT:    cneg w9, w8, mi
-; CHECK-NEXT:    cmn w8, #1
-; CHECK-NEXT:    csel w10, w9, w0, lt
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csel w0, w10, w9, ge
+; CHECK-NEXT:    subs w9, w0, w1
+; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    cneg w0, w9, mi
+; CHECK-NEXT:    tbnz w9, #31, .LBB6_2
+; CHECK-NEXT:  // %bb.1: // %if.then
+; CHECK-NEXT:    cmn w9, #1
+; CHECK-NEXT:    csel w0, w0, w8, lt
+; CHECK-NEXT:  .LBB6_2: // %if.else
 ; CHECK-NEXT:    ret
 entry:
   %sub = sub nsw i32 %a, %b
   %cmp = icmp sgt i32 %sub, -1
   %sub3 = sub nsw i32 0, %sub
   %cond = select i1 %cmp, i32 %sub, i32 %sub3
   br i1 %cmp, label %if.then, label %if.else
 

llvm/test/CodeGen/AArch64/arm64-instruction-mix-remarks.ll

 ; CHECK-LABEL: foo:
 ; CHECK:       ; %bb.0: ; %entry
 ; CHECK-NEXT:    ldr w9, [x0]
 ; CHECK-NEXT:    mov x8, x0
 ; CHECK-NEXT:    add w0, w9, w1
 ; CHECK-NEXT:    add x9, x0, x2
 ; CHECK-NEXT:    sub x9, x9, #244, lsl #12 ; =999424
 ; CHECK-NEXT:    cmp x9, #575
-; CHECK-NEXT:    b.eq LBB0_2
-; CHECK-NEXT:  ; %bb.1: ; %else
+; CHECK-NEXT:    b.ne LBB0_2
+; CHECK-NEXT:  ; %bb.1: ; %then
+; CHECK-NEXT:    ; kill: def $w0 killed $w0 killed $x0
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB0_2: ; %else
 ; CHECK-NEXT:    mul w9, w0, w1
 ; CHECK-NEXT:    mul w0, w9, w1
 ; CHECK-NEXT:    mov w9, #10 ; =0xa
 ; CHECK-NEXT:    str w9, [x8]
-; CHECK-NEXT:  LBB0_2: ; %common.ret
-; CHECK-NEXT:    ; kill: def $w0 killed $w0 killed $x0
 ; CHECK-NEXT:    ret
 entry:
   %l = load i32, ptr %ptr, !dbg !4
   %add = add i32 %l, %x, !dbg !4
   %add.ext = zext i32 %add to i64, !dbg !4
   %add.64 = add i64 %add.ext, %y, !dbg !4
   %c = icmp eq i64 %add.64, 999999, !dbg !4
   br i1 %c, label %then, label %else, !dbg !4

llvm/test/CodeGen/AArch64/arm64-neon-v1i1-setcc.ll

; CHECK-NEXT:    ret
   %tst = icmp eq i64 %lhs, %rhs
   %evil = insertelement <1 x i1> undef, i1 %tst, i32 0
   %res = select <1 x i1> %evil, <1 x i64> zeroinitializer, <1 x i64> %v3
   ret <1 x i64> %res
 }
 
 define i32 @test_br_extr_cmp(<1 x i64> %v1, <1 x i64> %v2) {
 ; CHECK-LABEL: test_br_extr_cmp:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    // kill: def $d1 killed $d1 def $q1
 ; CHECK-NEXT:    // kill: def $d0 killed $d0 def $q0
 ; CHECK-NEXT:    fmov x8, d1
 ; CHECK-NEXT:    fmov x9, d0
 ; CHECK-NEXT:    cmp x9, x8
-; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    b.ne .LBB6_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    mov w0, #1 // =0x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB6_2: // %if.then
+; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    ret
   %1 = icmp eq <1 x i64> %v1, %v2
   %2 = extractelement <1 x i1> %1, i32 0
   br i1 %2, label %if.end, label %if.then
 
 if.then:
   ret i32 0;
 
 if.end:
   ret i32 1;
 }

llvm/test/CodeGen/AArch64/arm64-shrink-wrapping.ll

 ; ENABLE-NEXT:    stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
 ; ENABLE-NEXT:    stp x29, x30, [sp, #16] ; 16-byte Folded Spill
 ; ENABLE-NEXT:    add x29, sp, #16
 ; ENABLE-NEXT:    .cfi_def_cfa w29, 16
 ; ENABLE-NEXT:    .cfi_offset w30, -8
 ; ENABLE-NEXT:    .cfi_offset w29, -16
 ; ENABLE-NEXT:    .cfi_offset w19, -24
 ; ENABLE-NEXT:    .cfi_offset w20, -32
-; ENABLE-NEXT:    cbnz wzr, LBB11_3
+; ENABLE-NEXT:    cbnz wzr, LBB11_5
 ; ENABLE-NEXT:  ; %bb.1: ; %if.then
 ; ENABLE-NEXT:    sub x8, sp, #16
 ; ENABLE-NEXT:    mov sp, x8
 ; ENABLE-NEXT:    mov w9, wzr
-; ENABLE-NEXT:  LBB11_2: ; %for.body
-; ENABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    ; InlineAsm Start
 ; ENABLE-NEXT:    mov x10, #0 ; =0x0
 ; ENABLE-NEXT:    ; InlineAsm End
-; ENABLE-NEXT:    add w10, w10, w9
+; ENABLE-NEXT:    b LBB11_3
+; ENABLE-NEXT:  LBB11_2: ; %body2
+; ENABLE-NEXT:    ; in Loop: Header=BB11_3 Depth=1
+; ENABLE-NEXT:    ; InlineAsm Start
+; ENABLE-NEXT:    nop
+; ENABLE-NEXT:    ; InlineAsm End
 ; ENABLE-NEXT:    mov w9, #1 ; =0x1
-; ENABLE-NEXT:    str w10, [x8]
+; ENABLE-NEXT:  LBB11_3: ; %for.body
+; ENABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
+; ENABLE-NEXT:    add w9, w10, w9
+; ENABLE-NEXT:    str w9, [x8]
+; ENABLE-NEXT:    cbnz wzr, LBB11_2
+; ENABLE-NEXT:  ; %bb.4: ; %body1
+; ENABLE-NEXT:    ; in Loop: Header=BB11_3 Depth=1
 ; ENABLE-NEXT:    ; InlineAsm Start
 ; ENABLE-NEXT:    nop
 ; ENABLE-NEXT:    ; InlineAsm End
-; ENABLE-NEXT:    b LBB11_2
-; ENABLE-NEXT:  LBB11_3: ; %if.end
+; ENABLE-NEXT:    b LBB11_3
+; ENABLE-NEXT:  LBB11_5: ; %if.end
 ; ENABLE-NEXT:    sub sp, x29, #16
 ; ENABLE-NEXT:    ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
 ; ENABLE-NEXT:    ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
 ; ENABLE-NEXT:    ret
 ;
 ; DISABLE-LABEL: infiniteloop2:
 ; DISABLE:       ; %bb.0: ; %entry
 ; DISABLE-NEXT:    stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
 ; DISABLE-NEXT:    stp x29, x30, [sp, #16] ; 16-byte Folded Spill
 ; DISABLE-NEXT:    add x29, sp, #16
 ; DISABLE-NEXT:    .cfi_def_cfa w29, 16
 ; DISABLE-NEXT:    .cfi_offset w30, -8
 ; DISABLE-NEXT:    .cfi_offset w29, -16
 ; DISABLE-NEXT:    .cfi_offset w19, -24
 ; DISABLE-NEXT:    .cfi_offset w20, -32
-; DISABLE-NEXT:    cbnz wzr, LBB11_3
+; DISABLE-NEXT:    cbnz wzr, LBB11_5
 ; DISABLE-NEXT:  ; %bb.1: ; %if.then
 ; DISABLE-NEXT:    sub x8, sp, #16
 ; DISABLE-NEXT:    mov sp, x8
 ; DISABLE-NEXT:    mov w9, wzr
-; DISABLE-NEXT:  LBB11_2: ; %for.body
-; DISABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    ; InlineAsm Start
 ; DISABLE-NEXT:    mov x10, #0 ; =0x0
 ; DISABLE-NEXT:    ; InlineAsm End
-; DISABLE-NEXT:    add w10, w10, w9
+; DISABLE-NEXT:    b LBB11_3
+; DISABLE-NEXT:  LBB11_2: ; %body2
+; DISABLE-NEXT:    ; in Loop: Header=BB11_3 Depth=1
+; DISABLE-NEXT:    ; InlineAsm Start
+; DISABLE-NEXT:    nop
+; DISABLE-NEXT:    ; InlineAsm End
 ; DISABLE-NEXT:    mov w9, #1 ; =0x1
-; DISABLE-NEXT:    str w10, [x8]
+; DISABLE-NEXT:  LBB11_3: ; %for.body
+; DISABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
+; DISABLE-NEXT:    add w9, w10, w9
+; DISABLE-NEXT:    str w9, [x8]
+; DISABLE-NEXT:    cbnz wzr, LBB11_2
+; DISABLE-NEXT:  ; %bb.4: ; %body1
+; DISABLE-NEXT:    ; in Loop: Header=BB11_3 Depth=1
 ; DISABLE-NEXT:    ; InlineAsm Start
 ; DISABLE-NEXT:    nop
 ; DISABLE-NEXT:    ; InlineAsm End
-; DISABLE-NEXT:    b LBB11_2
-; DISABLE-NEXT:  LBB11_3: ; %if.end
+; DISABLE-NEXT:    b LBB11_3
+; DISABLE-NEXT:  LBB11_5: ; %if.end
 ; DISABLE-NEXT:    sub sp, x29, #16
 ; DISABLE-NEXT:    ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
 ; DISABLE-NEXT:    ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
 ; DISABLE-NEXT:    ret
 entry:
   br i1 undef, label %if.then, label %if.end
 
 if.then:

 if.end:
   ret void
 }
 
 ; Another infinite loop test this time with two nested infinite loop.
 define void @infiniteloop3() {
 ; ENABLE-LABEL: infiniteloop3:
 ; ENABLE:       ; %bb.0: ; %entry
-; ENABLE-NEXT:    cbnz wzr, LBB12_5
-; ENABLE-NEXT:  ; %bb.1: ; %loop2a.preheader
+; ENABLE-NEXT:    cbnz wzr, LBB12_2
+; ENABLE-NEXT:  ; %bb.1: ; %body
+; ENABLE-NEXT:    cbnz wzr, LBB12_6
+; ENABLE-NEXT:  LBB12_2: ; %loop2a.preheader
 ; ENABLE-NEXT:    mov x8, xzr
 ; ENABLE-NEXT:    mov x9, xzr
 ; ENABLE-NEXT:    mov x11, xzr
-; ENABLE-NEXT:    b LBB12_3
-; ENABLE-NEXT:  LBB12_2: ; %loop2b
-; ENABLE-NEXT:    ; in Loop: Header=BB12_3 Depth=1
+; ENABLE-NEXT:    b LBB12_4
+; ENABLE-NEXT:  LBB12_3: ; %loop2b
+; ENABLE-NEXT:    ; in Loop: Header=BB12_4 Depth=1
 ; ENABLE-NEXT:    str x10, [x11]
 ; ENABLE-NEXT:    mov x11, x10
-; ENABLE-NEXT:  LBB12_3: ; %loop1
+; ENABLE-NEXT:  LBB12_4: ; %loop1
 ; ENABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; ENABLE-NEXT:    mov x10, x9
 ; ENABLE-NEXT:    ldr x9, [x8]
-; ENABLE-NEXT:    cbnz x8, LBB12_2
-; ENABLE-NEXT:  ; %bb.4: ; in Loop: Header=BB12_3 Depth=1
+; ENABLE-NEXT:    cbnz x8, LBB12_3
+; ENABLE-NEXT:  ; %bb.5: ; in Loop: Header=BB12_4 Depth=1
 ; ENABLE-NEXT:    mov x8, x10
 ; ENABLE-NEXT:    mov x11, x10
-; ENABLE-NEXT:    b LBB12_3
-; ENABLE-NEXT:  LBB12_5: ; %end
+; ENABLE-NEXT:    b LBB12_4
+; ENABLE-NEXT:  LBB12_6: ; %end
 ; ENABLE-NEXT:    ret
 ;
 ; DISABLE-LABEL: infiniteloop3:
 ; DISABLE:       ; %bb.0: ; %entry
-; DISABLE-NEXT:    cbnz wzr, LBB12_5
-; DISABLE-NEXT:  ; %bb.1: ; %loop2a.preheader
+; DISABLE-NEXT:    cbnz wzr, LBB12_2
+; DISABLE-NEXT:  ; %bb.1: ; %body
+; DISABLE-NEXT:    cbnz wzr, LBB12_6
+; DISABLE-NEXT:  LBB12_2: ; %loop2a.preheader
 ; DISABLE-NEXT:    mov x8, xzr
 ; DISABLE-NEXT:    mov x9, xzr
 ; DISABLE-NEXT:    mov x11, xzr
-; DISABLE-NEXT:    b LBB12_3
-; DISABLE-NEXT:  LBB12_2: ; %loop2b
-; DISABLE-NEXT:    ; in Loop: Header=BB12_3 Depth=1
+; DISABLE-NEXT:    b LBB12_4
+; DISABLE-NEXT:  LBB12_3: ; %loop2b
+; DISABLE-NEXT:    ; in Loop: Header=BB12_4 Depth=1
 ; DISABLE-NEXT:    str x10, [x11]
 ; DISABLE-NEXT:    mov x11, x10
-; DISABLE-NEXT:  LBB12_3: ; %loop1
+; DISABLE-NEXT:  LBB12_4: ; %loop1
 ; DISABLE-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; DISABLE-NEXT:    mov x10, x9
 ; DISABLE-NEXT:    ldr x9, [x8]
-; DISABLE-NEXT:    cbnz x8, LBB12_2
-; DISABLE-NEXT:  ; %bb.4: ; in Loop: Header=BB12_3 Depth=1
+; DISABLE-NEXT:    cbnz x8, LBB12_3
+; DISABLE-NEXT:  ; %bb.5: ; in Loop: Header=BB12_4 Depth=1
 ; DISABLE-NEXT:    mov x8, x10
 ; DISABLE-NEXT:    mov x11, x10
-; DISABLE-NEXT:    b LBB12_3
-; DISABLE-NEXT:  LBB12_5: ; %end
+; DISABLE-NEXT:    b LBB12_4
+; DISABLE-NEXT:  LBB12_6: ; %end
 ; DISABLE-NEXT:    ret
 entry:
   br i1 undef, label %loop2a, label %body
 
 body:                                             ; preds = %entry
   br i1 undef, label %loop2a, label %end
 
 loop1:                                            ; preds = %loop2a, %loop2b

llvm/test/CodeGen/AArch64/arm64-xaluo.ll

 
 ;
 ; Check the use of the overflow bit in combination with a branch instruction.
 ;
 define zeroext i1 @saddo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: saddo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn w0, w1
-; SDAG-NEXT:    cset w0, vc
+; SDAG-NEXT:    b.vc .LBB69_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB69_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: saddo.br.i32:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn w0, w1
+; FAST-NEXT:    b.vc .LBB69_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB69_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, vs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: saddo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn w0, w1
-; GISEL-NEXT:    cset w8, vs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.vc .LBB69_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB69_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.sadd.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @saddo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: saddo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn x0, x1
-; SDAG-NEXT:    cset w0, vc
+; SDAG-NEXT:    b.vc .LBB70_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB70_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: saddo.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn x0, x1
+; FAST-NEXT:    b.vc .LBB70_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB70_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, vs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: saddo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn x0, x1
-; GISEL-NEXT:    cset w8, vs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.vc .LBB70_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB70_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.sadd.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @uaddo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: uaddo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn w0, w1
-; SDAG-NEXT:    cset w0, lo
+; SDAG-NEXT:    b.lo .LBB71_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB71_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: uaddo.br.i32:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn w0, w1
+; FAST-NEXT:    b.lo .LBB71_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB71_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, hs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: uaddo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn w0, w1
-; GISEL-NEXT:    cset w8, hs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.lo .LBB71_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB71_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.uadd.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @uaddo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: uaddo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn x0, x1
-; SDAG-NEXT:    cset w0, lo
+; SDAG-NEXT:    b.lo .LBB72_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB72_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: uaddo.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn x0, x1
+; FAST-NEXT:    b.lo .LBB72_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB72_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, hs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: uaddo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn x0, x1
-; GISEL-NEXT:    cset w8, hs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.lo .LBB72_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB72_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.uadd.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @ssubo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: ssubo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmp w0, w1
-; SDAG-NEXT:    cset w0, vc
+; SDAG-NEXT:    b.vc .LBB73_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB73_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: ssubo.br.i32:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmp w0, w1
+; FAST-NEXT:    b.vc .LBB73_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB73_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, vs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: ssubo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
-; GISEL-NEXT:    cset w8, vs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.vc .LBB73_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB73_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.ssub.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @ssubo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: ssubo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmp x0, x1
-; SDAG-NEXT:    cset w0, vc
+; SDAG-NEXT:    b.vc .LBB74_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB74_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: ssubo.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmp x0, x1
+; FAST-NEXT:    b.vc .LBB74_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB74_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, vs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: ssubo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp x0, x1
-; GISEL-NEXT:    cset w8, vs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.vc .LBB74_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB74_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.ssub.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @usubo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: usubo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmp w0, w1
-; SDAG-NEXT:    cset w0, hs
+; SDAG-NEXT:    b.hs .LBB75_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB75_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: usubo.br.i32:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmp w0, w1
+; FAST-NEXT:    b.hs .LBB75_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB75_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, lo
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: usubo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp w0, w1
-; GISEL-NEXT:    cset w8, lo
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.hs .LBB75_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB75_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.usub.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @usubo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: usubo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmp x0, x1
-; SDAG-NEXT:    cset w0, hs
+; SDAG-NEXT:    b.hs .LBB76_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB76_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: usubo.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmp x0, x1
+; FAST-NEXT:    b.hs .LBB76_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB76_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, lo
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: usubo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmp x0, x1
-; GISEL-NEXT:    cset w8, lo
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.hs .LBB76_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB76_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.usub.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @smulo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: smulo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    smull x8, w0, w1
 ; SDAG-NEXT:    cmp x8, w8, sxtw
-; SDAG-NEXT:    cset w0, eq
+; SDAG-NEXT:    b.eq .LBB77_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB77_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: smulo.br.i32:
 ; FAST:       // %bb.0: // %entry
-; FAST-NEXT:    smull x9, w0, w1
+; FAST-NEXT:    smull x8, w0, w1
+; FAST-NEXT:    cmp x8, w8, sxtw
+; FAST-NEXT:    b.eq .LBB77_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB77_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cmp x9, w9, sxtw
-; FAST-NEXT:    cset w9, ne
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: smulo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    smull x8, w0, w1
 ; GISEL-NEXT:    mul w9, w0, w1
 ; GISEL-NEXT:    asr x8, x8, #32
 ; GISEL-NEXT:    cmp w8, w9, asr #31
-; GISEL-NEXT:    cset w8, ne
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.eq .LBB77_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB77_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.smul.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @smulo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: smulo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    mul x8, x0, x1
 ; SDAG-NEXT:    smulh x9, x0, x1
 ; SDAG-NEXT:    cmp x9, x8, asr #63
-; SDAG-NEXT:    cset w0, eq
+; SDAG-NEXT:    b.eq .LBB78_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB78_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: smulo.br.i64:
 ; FAST:       // %bb.0: // %entry
-; FAST-NEXT:    mul x9, x0, x1
+; FAST-NEXT:    mul x8, x0, x1
+; FAST-NEXT:    smulh x9, x0, x1
+; FAST-NEXT:    cmp x9, x8, asr #63
+; FAST-NEXT:    b.eq .LBB78_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB78_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    smulh x10, x0, x1
-; FAST-NEXT:    cmp x10, x9, asr #63
-; FAST-NEXT:    cset w9, ne
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: smulo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    smulh x8, x0, x1
 ; GISEL-NEXT:    mul x9, x0, x1
 ; GISEL-NEXT:    cmp x8, x9, asr #63
-; GISEL-NEXT:    cset w8, ne
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.eq .LBB78_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB78_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.smul.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @smulo2.br.i64(i64 %v1) {
 ; SDAG-LABEL: smulo2.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn x0, x0
-; SDAG-NEXT:    cset w0, vc
+; SDAG-NEXT:    b.vc .LBB79_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB79_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: smulo2.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn x0, x0
+; FAST-NEXT:    b.vc .LBB79_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB79_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, vs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: smulo2.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn x0, x0
-; GISEL-NEXT:    cset w8, vs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.vc .LBB79_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB79_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.smul.with.overflow.i64(i64 %v1, i64 2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @umulo.br.i32(i32 %v1, i32 %v2) {
 ; SDAG-LABEL: umulo.br.i32:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    umull x8, w0, w1
 ; SDAG-NEXT:    tst x8, #0xffffffff00000000
-; SDAG-NEXT:    cset w0, eq
+; SDAG-NEXT:    b.eq .LBB80_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB80_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: umulo.br.i32:
 ; FAST:       // %bb.0: // %entry
-; FAST-NEXT:    umull x9, w0, w1
+; FAST-NEXT:    umull x8, w0, w1
+; FAST-NEXT:    tst x8, #0xffffffff00000000
+; FAST-NEXT:    b.eq .LBB80_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB80_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    tst x9, #0xffffffff00000000
-; FAST-NEXT:    cset w9, ne
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: umulo.br.i32:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    umull x8, w0, w1
 ; GISEL-NEXT:    lsr x8, x8, #32
 ; GISEL-NEXT:    cmp w8, #0
-; GISEL-NEXT:    cset w8, ne
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.eq .LBB80_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB80_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i32, i1} @llvm.umul.with.overflow.i32(i32 %v1, i32 %v2)
   %val = extractvalue {i32, i1} %t, 0
   %obit = extractvalue {i32, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @umulo.br.i64(i64 %v1, i64 %v2) {
 ; SDAG-LABEL: umulo.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    umulh x8, x0, x1
-; SDAG-NEXT:    cmp xzr, x8
-; SDAG-NEXT:    cset w0, eq
+; SDAG-NEXT:    cbz x8, .LBB81_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB81_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: umulo.br.i64:
 ; FAST:       // %bb.0: // %entry
-; FAST-NEXT:    umulh x9, x0, x1
+; FAST-NEXT:    umulh x8, x0, x1
+; FAST-NEXT:    cmp xzr, x8
+; FAST-NEXT:    b.eq .LBB81_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB81_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cmp xzr, x9
-; FAST-NEXT:    cset w9, ne
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: umulo.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    umulh x8, x0, x1
 ; GISEL-NEXT:    cmp x8, #0
-; GISEL-NEXT:    cset w8, ne
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.eq .LBB81_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB81_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.umul.with.overflow.i64(i64 %v1, i64 %v2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:
   ret i1 false
 
 continue:
   ret i1 true
 }
 
 define zeroext i1 @umulo2.br.i64(i64 %v1) {
 ; SDAG-LABEL: umulo2.br.i64:
 ; SDAG:       // %bb.0: // %entry
 ; SDAG-NEXT:    cmn x0, x0
-; SDAG-NEXT:    cset w0, lo
+; SDAG-NEXT:    b.lo .LBB82_2
+; SDAG-NEXT:  // %bb.1: // %overflow
+; SDAG-NEXT:    mov w0, wzr
+; SDAG-NEXT:    ret
+; SDAG-NEXT:  .LBB82_2: // %continue
+; SDAG-NEXT:    mov w0, #1 // =0x1
 ; SDAG-NEXT:    ret
 ;
 ; FAST-LABEL: umulo2.br.i64:
 ; FAST:       // %bb.0: // %entry
 ; FAST-NEXT:    cmn x0, x0
+; FAST-NEXT:    b.lo .LBB82_2
+; FAST-NEXT:  // %bb.1: // %overflow
+; FAST-NEXT:    and w0, wzr, #0x1
+; FAST-NEXT:    ret
+; FAST-NEXT:  .LBB82_2: // %continue
 ; FAST-NEXT:    mov w8, #1 // =0x1
-; FAST-NEXT:    cset w9, hs
-; FAST-NEXT:    bic w8, w8, w9
 ; FAST-NEXT:    and w0, w8, #0x1
 ; FAST-NEXT:    ret
 ;
 ; GISEL-LABEL: umulo2.br.i64:
 ; GISEL:       // %bb.0: // %entry
 ; GISEL-NEXT:    cmn x0, x0
-; GISEL-NEXT:    cset w8, hs
-; GISEL-NEXT:    eor w0, w8, #0x1
+; GISEL-NEXT:    b.lo .LBB82_2
+; GISEL-NEXT:  // %bb.1: // %overflow
+; GISEL-NEXT:    mov w0, wzr
+; GISEL-NEXT:    ret
+; GISEL-NEXT:  .LBB82_2: // %continue
+; GISEL-NEXT:    mov w0, #1 // =0x1
 ; GISEL-NEXT:    ret
 entry:
   %t = call {i64, i1} @llvm.umul.with.overflow.i64(i64 %v1, i64 2)
   %val = extractvalue {i64, i1} %t, 0
   %obit = extractvalue {i64, i1} %t, 1
   br i1 %obit, label %overflow, label %continue
 
 overflow:

llvm/test/CodeGen/AArch64/atomic-ops-msvc.ll

 ; CHECK-NEXT:    and w8, w0, #0xff
 ; CHECK-NEXT:    adrp x9, var8
 ; CHECK-NEXT:    add x9, x9, :lo12:var8
 ; CHECK-NEXT:  .LBB40_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxrb w0, [x9]
 ; CHECK-NEXT:    cmp w0, w8
 ; CHECK-NEXT:    b.ne .LBB40_4
-; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
+; CHECK-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; CHECK-NEXT:    // in Loop: Header=BB40_1 Depth=1
 ; CHECK-NEXT:    stxrb w10, w1, [x9]
 ; CHECK-NEXT:    cbnz w10, .LBB40_1
 ; CHECK-NEXT:  // %bb.3: // %cmpxchg.end
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB40_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    clrex

 ; CHECK-NEXT:    and w8, w0, #0xffff
 ; CHECK-NEXT:    adrp x9, var16
 ; CHECK-NEXT:    add x9, x9, :lo12:var16
 ; CHECK-NEXT:  .LBB41_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxrh w0, [x9]
 ; CHECK-NEXT:    cmp w0, w8
 ; CHECK-NEXT:    b.ne .LBB41_4
-; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
+; CHECK-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; CHECK-NEXT:    // in Loop: Header=BB41_1 Depth=1
 ; CHECK-NEXT:    stlxrh w10, w1, [x9]
 ; CHECK-NEXT:    cbnz w10, .LBB41_1
 ; CHECK-NEXT:  // %bb.3: // %cmpxchg.success
 ; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB41_4: // %cmpxchg.nostore

 ; CHECK-NEXT:    mov w8, w0
 ; CHECK-NEXT:    adrp x9, var32
 ; CHECK-NEXT:    add x9, x9, :lo12:var32
 ; CHECK-NEXT:  .LBB42_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldxr w0, [x9]
 ; CHECK-NEXT:    cmp w0, w8
 ; CHECK-NEXT:    b.ne .LBB42_4
-; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
+; CHECK-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; CHECK-NEXT:    // in Loop: Header=BB42_1 Depth=1
 ; CHECK-NEXT:    stlxr w10, w1, [x9]
 ; CHECK-NEXT:    cbnz w10, .LBB42_1
 ; CHECK-NEXT:  // %bb.3: // %cmpxchg.end
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  .LBB42_4: // %cmpxchg.nostore
 ; CHECK-NEXT:    clrex

 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x9, var64
 ; CHECK-NEXT:    add x9, x9, :lo12:var64
 ; CHECK-NEXT:  .LBB43_1: // %cmpxchg.start
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldxr x8, [x9]
 ; CHECK-NEXT:    cmp x8, x0
 ; CHECK-NEXT:    b.ne .LBB43_3
-; CHECK-NEXT:  // %bb.2: // %cmpxchg.trystore
+; CHECK-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; CHECK-NEXT:    // in Loop: Header=BB43_1 Depth=1
 ; CHECK-NEXT:    stxr w10, x1, [x9]
 ; CHECK-NEXT:    cbnz w10, .LBB43_1
 ; CHECK-NEXT:    b .LBB43_4
 ; CHECK-NEXT:  .LBB43_3: // %cmpxchg.nostore
 ; CHECK-NEXT:    clrex
 ; CHECK-NEXT:  .LBB43_4: // %cmpxchg.end
 ; CHECK-NEXT:    adrp x9, var64

llvm/test/CodeGen/AArch64/atomic-ops.ll

 ; INLINE_ATOMICS-NEXT:    and w8, w0, #0xff
 ; INLINE_ATOMICS-NEXT:    adrp x9, var8
 ; INLINE_ATOMICS-NEXT:    add x9, x9, :lo12:var8
 ; INLINE_ATOMICS-NEXT:  .LBB40_1: // %cmpxchg.start
 ; INLINE_ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
 ; INLINE_ATOMICS-NEXT:    ldaxrb w0, [x9]
 ; INLINE_ATOMICS-NEXT:    cmp w0, w8
 ; INLINE_ATOMICS-NEXT:    b.ne .LBB40_4
-; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.trystore
+; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; INLINE_ATOMICS-NEXT:    // in Loop: Header=BB40_1 Depth=1
 ; INLINE_ATOMICS-NEXT:    stxrb w10, w1, [x9]
 ; INLINE_ATOMICS-NEXT:    cbnz w10, .LBB40_1
 ; INLINE_ATOMICS-NEXT:  // %bb.3: // %cmpxchg.end
 ; INLINE_ATOMICS-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; INLINE_ATOMICS-NEXT:    ret
 ; INLINE_ATOMICS-NEXT:  .LBB40_4: // %cmpxchg.nostore
 ; INLINE_ATOMICS-NEXT:    clrex

 ; INLINE_ATOMICS-NEXT:    and w8, w0, #0xffff
 ; INLINE_ATOMICS-NEXT:    adrp x9, var16
 ; INLINE_ATOMICS-NEXT:    add x9, x9, :lo12:var16
 ; INLINE_ATOMICS-NEXT:  .LBB41_1: // %cmpxchg.start
 ; INLINE_ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
 ; INLINE_ATOMICS-NEXT:    ldaxrh w0, [x9]
 ; INLINE_ATOMICS-NEXT:    cmp w0, w8
 ; INLINE_ATOMICS-NEXT:    b.ne .LBB41_4
-; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.trystore
+; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; INLINE_ATOMICS-NEXT:    // in Loop: Header=BB41_1 Depth=1
 ; INLINE_ATOMICS-NEXT:    stlxrh w10, w1, [x9]
 ; INLINE_ATOMICS-NEXT:    cbnz w10, .LBB41_1
 ; INLINE_ATOMICS-NEXT:  // %bb.3: // %cmpxchg.end
 ; INLINE_ATOMICS-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; INLINE_ATOMICS-NEXT:    ret
 ; INLINE_ATOMICS-NEXT:  .LBB41_4: // %cmpxchg.nostore
 ; INLINE_ATOMICS-NEXT:    clrex

 ; INLINE_ATOMICS-NEXT:    mov w8, w0
 ; INLINE_ATOMICS-NEXT:    adrp x9, var32
 ; INLINE_ATOMICS-NEXT:    add x9, x9, :lo12:var32
 ; INLINE_ATOMICS-NEXT:  .LBB42_1: // %cmpxchg.start
 ; INLINE_ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
 ; INLINE_ATOMICS-NEXT:    ldxr w0, [x9]
 ; INLINE_ATOMICS-NEXT:    cmp w0, w8
 ; INLINE_ATOMICS-NEXT:    b.ne .LBB42_4
-; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.trystore
+; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; INLINE_ATOMICS-NEXT:    // in Loop: Header=BB42_1 Depth=1
 ; INLINE_ATOMICS-NEXT:    stlxr w10, w1, [x9]
 ; INLINE_ATOMICS-NEXT:    cbnz w10, .LBB42_1
 ; INLINE_ATOMICS-NEXT:  // %bb.3: // %cmpxchg.end
 ; INLINE_ATOMICS-NEXT:    // kill: def $w0 killed $w0 killed $x0
 ; INLINE_ATOMICS-NEXT:    ret
 ; INLINE_ATOMICS-NEXT:  .LBB42_4: // %cmpxchg.nostore
 ; INLINE_ATOMICS-NEXT:    clrex

 ; INLINE_ATOMICS:       // %bb.0:
 ; INLINE_ATOMICS-NEXT:    adrp x9, var64
 ; INLINE_ATOMICS-NEXT:    add x9, x9, :lo12:var64
 ; INLINE_ATOMICS-NEXT:  .LBB43_1: // %cmpxchg.start
 ; INLINE_ATOMICS-NEXT:    // =>This Inner Loop Header: Depth=1
 ; INLINE_ATOMICS-NEXT:    ldxr x8, [x9]
 ; INLINE_ATOMICS-NEXT:    cmp x8, x0
 ; INLINE_ATOMICS-NEXT:    b.ne .LBB43_3
-; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.trystore
+; INLINE_ATOMICS-NEXT:  // %bb.2: // %cmpxchg.fencedstore
 ; INLINE_ATOMICS-NEXT:    // in Loop: Header=BB43_1 Depth=1
 ; INLINE_ATOMICS-NEXT:    stxr w10, x1, [x9]
 ; INLINE_ATOMICS-NEXT:    cbnz w10, .LBB43_1
 ; INLINE_ATOMICS-NEXT:    b .LBB43_4
 ; INLINE_ATOMICS-NEXT:  .LBB43_3: // %cmpxchg.nostore
 ; INLINE_ATOMICS-NEXT:    clrex
 ; INLINE_ATOMICS-NEXT:  .LBB43_4: // %cmpxchg.end
 ; INLINE_ATOMICS-NEXT:    adrp x9, var64

llvm/test/CodeGen/AArch64/branch-relax-alignment.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-darwin -aarch64-bcc-offset-bits=4 -align-all-nofallthru-blocks=4 < %s | FileCheck %s
 
 ; Long branch is assumed because the block has a higher alignment
 ; requirement than the function.
 
 define i32 @invert_bcc_block_align_higher_func(i32 %x, i32 %y) align 4 #0 {
 ; CHECK-LABEL: invert_bcc_block_align_higher_func:
-; CHECK:       ; %bb.0: ; %common.ret
-; CHECK-NEXT:    mov w8, #9 ; =0x9
+; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    cmp w0, w1
-; CHECK-NEXT:    mov w9, #42 ; =0x2a
-; CHECK-NEXT:    cset w0, ne
-; CHECK-NEXT:    csel w8, w9, w8, eq
+; CHECK-NEXT:    b.eq LBB0_1
+; CHECK-NEXT:    b LBB0_2
+; CHECK-NEXT:  LBB0_1: ; %bb1
+; CHECK-NEXT:    mov w8, #42 ; =0x2a
+; CHECK-NEXT:    mov w0, wzr
+; CHECK-NEXT:    str w8, [x8]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:    .p2align 4
+; CHECK-NEXT:  LBB0_2: ; %bb2
+; CHECK-NEXT:    mov w8, #9 ; =0x9
+; CHECK-NEXT:    mov w0, #1 ; =0x1
 ; CHECK-NEXT:    str w8, [x8]
 ; CHECK-NEXT:    ret
   %1 = icmp eq i32 %x, %y
   br i1 %1, label %bb1, label %bb2
 
 bb2:
   store volatile i32 9, ptr undef
   ret i32 1
 
 bb1:
   store volatile i32 42, ptr undef
   ret i32 0
 }
 
 attributes #0 = { nounwind }

llvm/test/CodeGen/AArch64/branch-relax-asm.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-ios7.0 -disable-block-placement -aarch64-tbz-offset-bits=4 -o - %s | FileCheck %s
 define i32 @test_asm_length(i32 %in) {
   ; It would be more natural to use just one "tbnz %false" here, but if the
   ; number of instructions in the asm is counted reasonably, that block is out
   ; of the limited range we gave tbz. So branch relaxation has to invert the
   ; condition.
 ; CHECK-LABEL: test_asm_length:
 ; CHECK:       ; %bb.0:
-; CHECK-NEXT:    tbz w0, #0, LBB0_2
-; CHECK-NEXT:  ; %bb.1:
-; CHECK-NEXT:    mov w0, wzr
-; CHECK-NEXT:    ret
-; CHECK-NEXT:  LBB0_2: ; %true
-; CHECK-NEXT:    mov w0, #4
+; CHECK-NEXT:    tbz w0, #0, LBB0_1
+; CHECK-NEXT:    b LBB0_2
+; CHECK-NEXT:  LBB0_1: ; %true
+; CHECK-NEXT:    mov w0, #4 ; =0x4
 ; CHECK-NEXT:    ; InlineAsm Start
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    ; InlineAsm End
 ; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB0_2: ; %false
+; CHECK-NEXT:    mov w0, wzr
+; CHECK-NEXT:    ret
   %val = and i32 %in, 1
   %tst = icmp eq i32 %val, 0
   br i1 %tst, label %true, label %false
 
 true:
   call void asm sideeffect "nop\0A\09nop\0A\09nop\0A\09nop\0A\09nop\0A\09nop", ""()
   ret i32 4
 
 false:
   ret i32 0
 }

llvm/test/CodeGen/AArch64/branch-relax-bcc.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-darwin -aarch64-bcc-offset-bits=3 < %s | FileCheck %s
 
 define i32 @invert_bcc(float %x, float %y) #0 {
 ; CHECK-LABEL: invert_bcc:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    fcmp s0, s1
-; CHECK-NEXT:    mov w0, wzr
-; CHECK-NEXT:    mov w8, #42 ; =0x2a
-; CHECK-NEXT:    b.pl LBB0_3
+; CHECK-NEXT:    b.ne LBB0_3
 ; CHECK-NEXT:    b LBB0_2
 ; CHECK-NEXT:  LBB0_3:
-; CHECK-NEXT:    b.gt LBB0_2
-; CHECK-NEXT:  ; %bb.1: ; %common.ret
-; CHECK-NEXT:    str w8, [x8]
-; CHECK-NEXT:    ret
-; CHECK-NEXT:  LBB0_2: ; %bb2
-; CHECK-NEXT:    mov w0, #1 ; =0x1
+; CHECK-NEXT:    b.vc LBB0_1
+; CHECK-NEXT:    b LBB0_2
+; CHECK-NEXT:  LBB0_1: ; %bb2
 ; CHECK-NEXT:    mov w8, #9 ; =0x9
+; CHECK-NEXT:    mov w0, #1 ; =0x1
 ; CHECK-NEXT:    ; InlineAsm Start
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    ; InlineAsm End
 ; CHECK-NEXT:    str w8, [x8]
 ; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB0_2: ; %bb1
+; CHECK-NEXT:    mov w8, #42 ; =0x2a
+; CHECK-NEXT:    mov w0, wzr
+; CHECK-NEXT:    str w8, [x8]
+; CHECK-NEXT:    ret
   %1 = fcmp ueq float %x, %y
   br i1 %1, label %bb1, label %bb2
 
 bb2:
   call void asm sideeffect
     "nop
      nop",
     ""() #0

llvm/test/CodeGen/AArch64/branch-relax-cbz.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-darwin -aarch64-cbz-offset-bits=3 < %s | FileCheck %s
 
 
 define void @split_block_no_fallthrough(i64 %val) #0 {
 ; CHECK-LABEL: split_block_no_fallthrough:
 ; CHECK:       ; %bb.0: ; %bb
 ; CHECK-NEXT:    cmn x0, #5
 ; CHECK-NEXT:    b.le LBB0_3
 ; CHECK-NEXT:  ; %bb.1: ; %b3
 ; CHECK-NEXT:    ldr w8, [x8]
 ; CHECK-NEXT:    cbnz w8, LBB0_2
 ; CHECK-NEXT:    b LBB0_4
-; CHECK-NEXT:  LBB0_2: ; %common.ret
+; CHECK-NEXT:  LBB0_2: ; %b8
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB0_3: ; %b2
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    mov w0, #93 ; =0x5d
 ; CHECK-NEXT:    bl _extfunc
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-NEXT:    cbz w0, LBB0_4
 ; CHECK-NEXT:    b LBB0_2

llvm/test/CodeGen/AArch64/cgp-usubo.ll

 
 ; Verify insertion point for multi-BB.
 
 declare void @call(i1)
 
 define i1 @usubo_ult_sub_dominates_i64(i64 %x, i64 %y, ptr %p, i1 %cond) nounwind {
 ; CHECK-LABEL: usubo_ult_sub_dominates_i64:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    tbz w3, #0, .LBB7_2
+; CHECK-NEXT:    tbz w3, #0, .LBB7_3
 ; CHECK-NEXT:  // %bb.1: // %t
 ; CHECK-NEXT:    subs x8, x0, x1
-; CHECK-NEXT:    cset w3, lo
 ; CHECK-NEXT:    str x8, [x2]
-; CHECK-NEXT:  .LBB7_2: // %common.ret
+; CHECK-NEXT:    tbz w3, #0, .LBB7_3
+; CHECK-NEXT:  // %bb.2: // %end
+; CHECK-NEXT:    cset w0, lo
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB7_3: // %f
 ; CHECK-NEXT:    and w0, w3, #0x1
 ; CHECK-NEXT:    ret
 entry:
   br i1 %cond, label %t, label %f
 
 t:
   %s = sub i64 %x, %y
   store i64 %s, ptr %p
   br i1 %cond, label %end, label %f
 
 f:
   ret i1 %cond
 
 end:
   %ov = icmp ult i64 %x, %y
   ret i1 %ov
 }
 
 define i1 @usubo_ult_cmp_dominates_i64(i64 %x, i64 %y, ptr %p, i1 %cond) nounwind {
 ; CHECK-LABEL: usubo_ult_cmp_dominates_i64:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    stp x30, x23, [sp, #-48]! // 16-byte Folded Spill
+; CHECK-NEXT:    str x30, [sp, #-48]! // 8-byte Folded Spill
 ; CHECK-NEXT:    stp x20, x19, [sp, #32] // 16-byte Folded Spill
-; CHECK-NEXT:    mov w19, w3
+; CHECK-NEXT:    mov w20, w3
 ; CHECK-NEXT:    stp x22, x21, [sp, #16] // 16-byte Folded Spill
 ; CHECK-NEXT:    tbz w3, #0, .LBB8_3
 ; CHECK-NEXT:  // %bb.1: // %t
 ; CHECK-NEXT:    cmp x0, x1
-; CHECK-NEXT:    mov x23, x0
-; CHECK-NEXT:    mov x20, x2
-; CHECK-NEXT:    cset w21, lo
-; CHECK-NEXT:    mov x22, x1
-; CHECK-NEXT:    mov w0, w21
+; CHECK-NEXT:    mov x22, x0
+; CHECK-NEXT:    mov x19, x2
+; CHECK-NEXT:    cset w0, lo
+; CHECK-NEXT:    mov x21, x1
 ; CHECK-NEXT:    bl call
-; CHECK-NEXT:    subs x8, x23, x22
+; CHECK-NEXT:    subs x8, x22, x21
 ; CHECK-NEXT:    b.hs .LBB8_3
 ; CHECK-NEXT:  // %bb.2: // %end
-; CHECK-NEXT:    mov w19, w21
-; CHECK-NEXT:    str x8, [x20]
-; CHECK-NEXT:  .LBB8_3: // %common.ret
-; CHECK-NEXT:    and w0, w19, #0x1
+; CHECK-NEXT:    cset w0, lo
+; CHECK-NEXT:    str x8, [x19]
+; CHECK-NEXT:    b .LBB8_4
+; CHECK-NEXT:  .LBB8_3: // %f
+; CHECK-NEXT:    and w0, w20, #0x1
+; CHECK-NEXT:  .LBB8_4: // %f
 ; CHECK-NEXT:    ldp x20, x19, [sp, #32] // 16-byte Folded Reload
 ; CHECK-NEXT:    ldp x22, x21, [sp, #16] // 16-byte Folded Reload
-; CHECK-NEXT:    ldp x30, x23, [sp], #48 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #48 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
 entry:
   br i1 %cond, label %t, label %f
 
 t:
   %ov = icmp ult i64 %x, %y
   call void @call(i1 %ov)
   br i1 %ov, label %end, label %f

llvm/test/CodeGen/AArch64/check-sign-bit-before-extension.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=aarch64-gnu-linux -o - | FileCheck %s
 
 ; These tests make sure that the `cmp` instruction is rendered with an
 ; instruction that checks the sign bit of the original unextended data
 ; (%in) instead of the sign bit of the sign extended one that is
 ; created by the type legalization process.
 ;
 ; The tests are subdivided in tests that determine the sign bit
 ; looking through a `sign_extend_inreg` and tests that determine the
 ; sign bit looking through a `sign_extend`.
 
 define i32 @f_i8_sign_extend_inreg(i8 %in, i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: f_i8_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    sxtb w8, w0
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csel w8, w1, w2, ge
-; CHECK-NEXT:    add w0, w8, w0, uxtb
+; CHECK-NEXT:    and w8, w0, #0xff
+; CHECK-NEXT:    tbnz w0, #7, .LBB0_2
+; CHECK-NEXT:  // %bb.1: // %A
+; CHECK-NEXT:    add w0, w8, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB0_2: // %B
+; CHECK-NEXT:    add w0, w8, w2
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i8 %in, -1
   %ext = zext i8 %in to i32
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i32 %ext, %a
   ret i32 %retA
 
 B:
   %retB = add i32 %ext, %b
   ret i32 %retB
 }
 
 define i32 @f_i16_sign_extend_inreg(i16 %in, i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: f_i16_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    sxth w8, w0
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csel w8, w1, w2, ge
-; CHECK-NEXT:    add w0, w8, w0, uxth
+; CHECK-NEXT:    and w8, w0, #0xffff
+; CHECK-NEXT:    tbnz w0, #15, .LBB1_2
+; CHECK-NEXT:  // %bb.1: // %A
+; CHECK-NEXT:    add w0, w8, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB1_2: // %B
+; CHECK-NEXT:    add w0, w8, w2
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i16 %in, -1
   %ext = zext i16 %in to i32
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i32 %ext, %a
   ret i32 %retA
 
 B:
   %retB = add i32 %ext, %b
   ret i32 %retB
 }
 
 define i64 @f_i32_sign_extend_inreg(i32 %in, i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: f_i32_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    csel x8, x1, x2, ge
-; CHECK-NEXT:    add x0, x8, w0, uxtw
+; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    tbnz w0, #31, .LBB2_2
+; CHECK-NEXT:  // %bb.1: // %A
+; CHECK-NEXT:    add x0, x8, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB2_2: // %B
+; CHECK-NEXT:    add x0, x8, x2
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %in, -1
   %ext = zext i32 %in to i64
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i64 %ext, %a
   ret i64 %retA
 
 B:
   %retB = add i64 %ext, %b
   ret i64 %retB
 }
 
 define i32 @g_i8_sign_extend_inreg(i8 %in, i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: g_i8_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    sxtb w8, w0
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csel w8, w1, w2, lt
-; CHECK-NEXT:    add w0, w8, w0, uxtb
+; CHECK-NEXT:    and w8, w0, #0xff
+; CHECK-NEXT:    tbnz w0, #7, .LBB3_2
+; CHECK-NEXT:  // %bb.1: // %B
+; CHECK-NEXT:    add w0, w8, w2
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_2: // %A
+; CHECK-NEXT:    add w0, w8, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp slt i8 %in, 0
   %ext = zext i8 %in to i32
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i32 %ext, %a
   ret i32 %retA
 
 B:
   %retB = add i32 %ext, %b
   ret i32 %retB
 }
 
 define i32 @g_i16_sign_extend_inreg(i16 %in, i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: g_i16_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    sxth w8, w0
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csel w8, w1, w2, lt
-; CHECK-NEXT:    add w0, w8, w0, uxth
+; CHECK-NEXT:    and w8, w0, #0xffff
+; CHECK-NEXT:    tbnz w0, #15, .LBB4_2
+; CHECK-NEXT:  // %bb.1: // %B
+; CHECK-NEXT:    add w0, w8, w2
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB4_2: // %A
+; CHECK-NEXT:    add w0, w8, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp slt i16 %in, 0
   %ext = zext i16 %in to i32
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i32 %ext, %a
   ret i32 %retA
 
 B:
   %retB = add i32 %ext, %b
   ret i32 %retB
 }
 
 define i64 @g_i32_sign_extend_inreg(i32 %in, i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: g_i32_sign_extend_inreg:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    csel x8, x1, x2, lt
-; CHECK-NEXT:    add x0, x8, w0, uxtw
+; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    tbnz w0, #31, .LBB5_2
+; CHECK-NEXT:  // %bb.1: // %B
+; CHECK-NEXT:    add x0, x8, x2
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB5_2: // %A
+; CHECK-NEXT:    add x0, x8, x1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp slt i32 %in, 0
   %ext = zext i32 %in to i64
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i64 %ext, %a
   ret i64 %retA
 
 B:
   %retB = add i64 %ext, %b
   ret i64 %retB
 }
 
 define i64 @f_i32_sign_extend_i64(i32 %in, i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: f_i32_sign_extend_i64:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    // kill: def $w0 killed $w0 def $x0
-; CHECK-NEXT:    sxtw x8, w0
-; CHECK-NEXT:    cmp x8, #0
-; CHECK-NEXT:    csel x8, x1, x2, ge
-; CHECK-NEXT:    add x0, x8, w0, uxtw
+; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    tbnz w0, #31, .LBB6_2
+; CHECK-NEXT:  // %bb.1: // %A
+; CHECK-NEXT:    add x0, x8, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB6_2: // %B
+; CHECK-NEXT:    add x0, x8, x2
 ; CHECK-NEXT:    ret
 entry:
   %inext = sext i32 %in to i64
   %cmp = icmp sgt i64 %inext, -1
   %ext = zext i32 %in to i64
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i64 %ext, %a
   ret i64 %retA
 
 B:
   %retB = add i64 %ext, %b
   ret i64 %retB
 }
 
 define i64 @g_i32_sign_extend_i64(i32 %in, i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: g_i32_sign_extend_i64:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    // kill: def $w0 killed $w0 def $x0
-; CHECK-NEXT:    sxtw x8, w0
-; CHECK-NEXT:    cmp x8, #0
-; CHECK-NEXT:    csel x8, x1, x2, lt
-; CHECK-NEXT:    add x0, x8, w0, uxtw
+; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    tbnz w0, #31, .LBB7_2
+; CHECK-NEXT:  // %bb.1: // %B
+; CHECK-NEXT:    add x0, x8, x2
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB7_2: // %A
+; CHECK-NEXT:    add x0, x8, x1
 ; CHECK-NEXT:    ret
 entry:
   %inext = sext i32 %in to i64
   %cmp = icmp slt i64 %inext, 0
   %ext = zext i32 %in to i64
   br i1 %cmp, label %A, label %B
 
 A:
   %retA = add i64 %ext, %a
   ret i64 %retA
 
 B:
   %retB = add i64 %ext, %b
   ret i64 %retB
 }

llvm/test/CodeGen/AArch64/cmp-frameindex.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64 %s -o - | FileCheck %s
 
 define void @test_frameindex_cmp() {
 ; CHECK-LABEL: test_frameindex_cmp:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -16
 ; CHECK-NEXT:    cmn sp, #12
 ; CHECK-NEXT:    b.eq .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %bb1
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB0_2: // %common.ret
+; CHECK-NEXT:  .LBB0_2: // %bb2
 ; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %stack = alloca i8
   %stack.int = ptrtoint ptr %stack to i64
   %cmp = icmp ne i64 %stack.int, 0
   br i1 %cmp, label %bb1, label %bb2
 
 bb1:
   call void @bar()
   ret void
 
 bb2:
   ret void
 }
 
 declare void @bar()

llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-ios7.0 -o - %s | FileCheck %s
 ; RUN: llc -mtriple=aarch64-apple-ios7.0 -mattr=+outline-atomics -o - %s | FileCheck %s --check-prefix=OUTLINE-ATOMICS
 
 define i32 @test_return(ptr %p, i32 %oldval, i32 %newval) {
 ; CHECK-LABEL: test_return:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:  LBB0_1: ; %cmpxchg.start
 ; CHECK-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxr w8, [x0]
 ; CHECK-NEXT:    cmp w8, w1
 ; CHECK-NEXT:    b.ne LBB0_4
-; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NEXT:    ; in Loop: Header=BB0_1 Depth=1
 ; CHECK-NEXT:    stlxr w8, w2, [x0]
 ; CHECK-NEXT:    cbnz w8, LBB0_1
 ; CHECK-NEXT:  ; %bb.3:
 ; CHECK-NEXT:    mov w0, #1 ; =0x1
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB0_4: ; %cmpxchg.nostore
 ; CHECK-NEXT:    mov w0, wzr

 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    and w8, w1, #0xff
 ; CHECK-NEXT:    ; kill: def $w2 killed $w2 def $x2
 ; CHECK-NEXT:  LBB1_1: ; %cmpxchg.start
 ; CHECK-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxrb w9, [x0]
 ; CHECK-NEXT:    cmp w9, w8
 ; CHECK-NEXT:    b.ne LBB1_4
-; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NEXT:    ; in Loop: Header=BB1_1 Depth=1
 ; CHECK-NEXT:    stlxrb w9, w2, [x0]
 ; CHECK-NEXT:    cbnz w9, LBB1_1
 ; CHECK-NEXT:  ; %bb.3:
 ; CHECK-NEXT:    mov w8, #1 ; =0x1
 ; CHECK-NEXT:    eor w0, w8, #0x1
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB1_4: ; %cmpxchg.nostore

 
 define void @test_conditional(ptr %p, i32 %oldval, i32 %newval) {
 ; CHECK-LABEL: test_conditional:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:  LBB2_1: ; %cmpxchg.start
 ; CHECK-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxr w8, [x0]
 ; CHECK-NEXT:    cmp w8, w1
-; CHECK-NEXT:    b.ne LBB2_4
-; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NEXT:    b.ne LBB2_5
+; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NEXT:    ; in Loop: Header=BB2_1 Depth=1
 ; CHECK-NEXT:    stlxr w8, w2, [x0]
 ; CHECK-NEXT:    cbnz w8, LBB2_1
-; CHECK-NEXT:  ; %bb.3: ; %true
-; CHECK-NEXT:    b _bar
-; CHECK-NEXT:  LBB2_4: ; %cmpxchg.nostore
-; CHECK-NEXT:    clrex
+; CHECK-NEXT:  ; %bb.3:
+; CHECK-NEXT:    mov w8, #1 ; =0x1
+; CHECK-NEXT:    cbnz w8, LBB2_6
+; CHECK-NEXT:  LBB2_4: ; %false
 ; CHECK-NEXT:    b _baz
+; CHECK-NEXT:  LBB2_5: ; %cmpxchg.nostore
+; CHECK-NEXT:    mov w8, wzr
+; CHECK-NEXT:    clrex
+; CHECK-NEXT:    cbz w8, LBB2_4
+; CHECK-NEXT:  LBB2_6: ; %true
+; CHECK-NEXT:    b _bar
 ;
 ; OUTLINE-ATOMICS-LABEL: test_conditional:
 ; OUTLINE-ATOMICS:       ; %bb.0:
 ; OUTLINE-ATOMICS-NEXT:    stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    stp x29, x30, [sp, #16] ; 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 32
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w29, -16

 ; CHECK-NEXT:    mov x19, x2
 ; CHECK-NEXT:    mov w20, w1
 ; CHECK-NEXT:    mov w21, w0
 ; CHECK-NEXT:  LBB3_1: ; %cmpxchg.start
 ; CHECK-NEXT:    ; =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldaxr w8, [x19]
 ; CHECK-NEXT:    cmp w8, w21
 ; CHECK-NEXT:    b.ne LBB3_4
-; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.trystore
+; CHECK-NEXT:  ; %bb.2: ; %cmpxchg.fencedstore
 ; CHECK-NEXT:    ; in Loop: Header=BB3_1 Depth=1
 ; CHECK-NEXT:    stlxr w8, w20, [x19]
 ; CHECK-NEXT:    cbnz w8, LBB3_1
 ; CHECK-NEXT:  ; %bb.3:
 ; CHECK-NEXT:    mov w8, #1 ; =0x1
 ; CHECK-NEXT:    b LBB3_5
 ; CHECK-NEXT:  LBB3_4: ; %cmpxchg.nostore
 ; CHECK-NEXT:    mov w8, wzr
 ; CHECK-NEXT:    clrex
-; CHECK-NEXT:  LBB3_5: ; %for.cond.preheader
-; CHECK-NEXT:    mov w22, #2 ; =0x2
-; CHECK-NEXT:  LBB3_6: ; %for.cond
+; CHECK-NEXT:  LBB3_5: ; %cmpxchg.end
+; CHECK-NEXT:    mov w22, #4 ; =0x4
+; CHECK-NEXT:    b LBB3_7
+; CHECK-NEXT:  LBB3_6: ; %if.end
+; CHECK-NEXT:    ; in Loop: Header=BB3_7 Depth=1
+; CHECK-NEXT:    sub x22, x22, #4
+; CHECK-NEXT:  LBB3_7: ; %for.cond
 ; CHECK-NEXT:    ; =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cbz w22, LBB3_9
-; CHECK-NEXT:  ; %bb.7: ; %for.body
-; CHECK-NEXT:    ; in Loop: Header=BB3_6 Depth=1
-; CHECK-NEXT:    sub w22, w22, #1
+; CHECK-NEXT:    cmn w22, #4
+; CHECK-NEXT:    b.eq LBB3_10
+; CHECK-NEXT:  ; %bb.8: ; %for.body
+; CHECK-NEXT:    ; in Loop: Header=BB3_7 Depth=1
+; CHECK-NEXT:    ldr w10, [x19, x22]
 ; CHECK-NEXT:    orr w9, w21, w20
-; CHECK-NEXT:    ldr w10, [x19, w22, sxtw #2]
 ; CHECK-NEXT:    cmp w9, w10
 ; CHECK-NEXT:    b.eq LBB3_6
-; CHECK-NEXT:  ; %bb.8: ; %if.then
-; CHECK-NEXT:    ; in Loop: Header=BB3_6 Depth=1
-; CHECK-NEXT:    sxtw x8, w22
-; CHECK-NEXT:    str w9, [x19, x8, lsl #2]
+; CHECK-NEXT:  ; %bb.9: ; %if.then
+; CHECK-NEXT:    ; in Loop: Header=BB3_7 Depth=1
+; CHECK-NEXT:    str w9, [x19, x22]
 ; CHECK-NEXT:    bl _foo
 ; CHECK-NEXT:    mov w8, wzr
 ; CHECK-NEXT:    b LBB3_6
-; CHECK-NEXT:  LBB3_9: ; %for.cond.cleanup
+; CHECK-NEXT:  LBB3_10: ; %for.cond.cleanup
 ; CHECK-NEXT:    ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
 ; CHECK-NEXT:    and w0, w8, #0x1
 ; CHECK-NEXT:    ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
 ; CHECK-NEXT:    ldp x22, x21, [sp], #48 ; 16-byte Folded Reload
 ; CHECK-NEXT:    ret
 ;
 ; OUTLINE-ATOMICS-LABEL: test_conditional2:
 ; OUTLINE-ATOMICS:       ; %bb.0: ; %entry
 ; OUTLINE-ATOMICS-NEXT:    stp x22, x21, [sp, #-48]! ; 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    stp x20, x19, [sp, #16] ; 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    stp x29, x30, [sp, #32] ; 16-byte Folded Spill
 ; OUTLINE-ATOMICS-NEXT:    .cfi_def_cfa_offset 48
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w30, -8
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w29, -16
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w19, -24
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w20, -32
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w21, -40
 ; OUTLINE-ATOMICS-NEXT:    .cfi_offset w22, -48
 ; OUTLINE-ATOMICS-NEXT:    mov x19, x2
 ; OUTLINE-ATOMICS-NEXT:    mov w20, w1
 ; OUTLINE-ATOMICS-NEXT:    mov w21, w0
 ; OUTLINE-ATOMICS-NEXT:    bl ___aarch64_cas4_acq_rel
 ; OUTLINE-ATOMICS-NEXT:    cmp w0, w21
-; OUTLINE-ATOMICS-NEXT:    mov w22, #2 ; =0x2
+; OUTLINE-ATOMICS-NEXT:    mov w22, #4 ; =0x4
 ; OUTLINE-ATOMICS-NEXT:    cset w8, eq
-; OUTLINE-ATOMICS-NEXT:  LBB3_1: ; %for.cond
+; OUTLINE-ATOMICS-NEXT:    b LBB3_2
+; OUTLINE-ATOMICS-NEXT:  LBB3_1: ; %if.end
+; OUTLINE-ATOMICS-NEXT:    ; in Loop: Header=BB3_2 Depth=1
+; OUTLINE-ATOMICS-NEXT:    sub x22, x22, #4
+; OUTLINE-ATOMICS-NEXT:  LBB3_2: ; %for.cond
 ; OUTLINE-ATOMICS-NEXT:    ; =>This Inner Loop Header: Depth=1
-; OUTLINE-ATOMICS-NEXT:    cbz w22, LBB3_4
-; OUTLINE-ATOMICS-NEXT:  ; %bb.2: ; %for.body
-; OUTLINE-ATOMICS-NEXT:    ; in Loop: Header=BB3_1 Depth=1
-; OUTLINE-ATOMICS-NEXT:    sub w22, w22, #1
+; OUTLINE-ATOMICS-NEXT:    cmn w22, #4
+; OUTLINE-ATOMICS-NEXT:    b.eq LBB3_5
+; OUTLINE-ATOMICS-NEXT:  ; %bb.3: ; %for.body
+; OUTLINE-ATOMICS-NEXT:    ; in Loop: Header=BB3_2 Depth=1
+; OUTLINE-ATOMICS-NEXT:    ldr w10, [x19, x22]
 ; OUTLINE-ATOMICS-NEXT:    orr w9, w21, w20
-; OUTLINE-ATOMICS-NEXT:    ldr w10, [x19, w22, sxtw #2]
 ; OUTLINE-ATOMICS-NEXT:    cmp w9, w10
 ; OUTLINE-ATOMICS-NEXT:    b.eq LBB3_1
-; OUTLINE-ATOMICS-NEXT:  ; %bb.3: ; %if.then
-; OUTLINE-ATOMICS-NEXT:    ; in Loop: Header=BB3_1 Depth=1
-; OUTLINE-ATOMICS-NEXT:    sxtw x8, w22
-; OUTLINE-ATOMICS-NEXT:    str w9, [x19, x8, lsl #2]
+; OUTLINE-ATOMICS-NEXT:  ; %bb.4: ; %if.then
+; OUTLINE-ATOMICS-NEXT:    ; in Loop: Header=BB3_2 Depth=1
+; OUTLINE-ATOMICS-NEXT:    str w9, [x19, x22]
 ; OUTLINE-ATOMICS-NEXT:    bl _foo
 ; OUTLINE-ATOMICS-NEXT:    mov w8, wzr
 ; OUTLINE-ATOMICS-NEXT:    b LBB3_1
-; OUTLINE-ATOMICS-NEXT:  LBB3_4: ; %for.cond.cleanup
+; OUTLINE-ATOMICS-NEXT:  LBB3_5: ; %for.cond.cleanup
 ; OUTLINE-ATOMICS-NEXT:    ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    and w0, w8, #0x1
 ; OUTLINE-ATOMICS-NEXT:    ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ldp x22, x21, [sp], #48 ; 16-byte Folded Reload
 ; OUTLINE-ATOMICS-NEXT:    ret
 entry:
   %pair = cmpxchg ptr %c, i32 %a, i32 %b seq_cst seq_cst
   %success = extractvalue { i32, i1 } %pair, 1

llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll

 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    mov w0, #-1 // =0xffffffff
 ; CHECK-NEXT:    bl yoo
 ; CHECK-NEXT:    cmp w19, #0
 ; CHECK-NEXT:    mov w1, #2 // =0x2
 ; CHECK-NEXT:    fmov d8, d0
 ; CHECK-NEXT:    cinc w0, w19, gt
 ; CHECK-NEXT:    bl xoo
-; CHECK-NEXT:    fmov d0, #-1.00000000
 ; CHECK-NEXT:    fcmp d8, #0.0
+; CHECK-NEXT:    b.gt .LBB9_5
+; CHECK-NEXT:  // %bb.4: // %cond.false12
+; CHECK-NEXT:    fmov d0, #-1.00000000
+; CHECK-NEXT:    fadd d8, d8, d0
+; CHECK-NEXT:  .LBB9_5: // %cond.end14
+; CHECK-NEXT:    fmov d0, d8
 ; CHECK-NEXT:    fmov d1, #-2.00000000
-; CHECK-NEXT:    fadd d0, d8, d0
-; CHECK-NEXT:    fcsel d0, d8, d0, gt
 ; CHECK-NEXT:    bl woo
 ; CHECK-NEXT:    ldp x30, x19, [sp, #16] // 16-byte Folded Reload
 ; CHECK-NEXT:    mov w0, #4 // =0x4
 ; CHECK-NEXT:    ldr d8, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-NEXT:    .cfi_restore w19
 ; CHECK-NEXT:    .cfi_restore w30
 ; CHECK-NEXT:    .cfi_restore b8

 
 return:                                           ; preds = %land.lhs.true, %cond.end14
   %retval.0 = phi i32 [ 4, %cond.end14 ], [ 3, %land.lhs.true ]
   ret i32 %retval.0
 }
 
 define void @cmp_shifted(i32 %in, i32 %lhs, i32 %rhs) #0 {
 ; CHECK-LABEL: cmp_shifted:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -16
-; CHECK-NEXT:    mov w8, #42 // =0x2a
-; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    mov w9, #128 // =0x80
-; CHECK-NEXT:    csinc w8, w8, wzr, gt
 ; CHECK-NEXT:    cmp w0, #2, lsl #12 // =8192
-; CHECK-NEXT:    csel w0, w9, w8, ge
+; CHECK-NEXT:    b.lt .LBB10_2
+; CHECK-NEXT:  // %bb.1: // %true
+; CHECK-NEXT:    mov w0, #128 // =0x80
+; CHECK-NEXT:    b .LBB10_5
+; CHECK-NEXT:  .LBB10_2: // %false
+; CHECK-NEXT:    cmp w0, #1
+; CHECK-NEXT:    b.lt .LBB10_4
+; CHECK-NEXT:  // %bb.3: // %truer
+; CHECK-NEXT:    mov w0, #42 // =0x2a
+; CHECK-NEXT:    b .LBB10_5
+; CHECK-NEXT:  .LBB10_4: // %falser
+; CHECK-NEXT:    mov w0, #1 // =0x1
+; CHECK-NEXT:  .LBB10_5: // %true
 ; CHECK-NEXT:    bl zoo
 ; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-NEXT:    .cfi_restore w30
 ; CHECK-NEXT:    ret
 ; [...]
 
   %tst_low = icmp sgt i32 %in, 8191

llvm/test/CodeGen/AArch64/cond-br-tuning.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -debugify-and-strip-all-safe < %s -O3 -mtriple=aarch64-eabi -verify-machineinstrs | FileCheck %s
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-linaro-linux-gnueabi"
 
 ; CMN is an alias of ADDS.
 
 define void @test_add_cbz(i32 %a, i32 %b, ptr %ptr) {
 ; CHECK-LABEL: test_add_cbz:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    cmn w0, w1
-; CHECK-NEXT:    cset w8, eq
+; CHECK-NEXT:    b.eq .LBB0_2
+; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str wzr, [x2]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB0_2: // %L2
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str w8, [x2]
 ; CHECK-NEXT:    ret
   %c = add nsw i32 %a, %b
   %d = icmp ne i32 %c, 0
   br i1 %d, label %L1, label %L2
 L1:
   store i32 0, ptr %ptr, align 4
   ret void
 L2:
   store i32 1, ptr %ptr, align 4
   ret void
 }
 
 define void @test_add_cbz_multiple_use(i32 %a, i32 %b, ptr %ptr) {
 ; CHECK-LABEL: test_add_cbz_multiple_use:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adds w8, w0, w1
-; CHECK-NEXT:    mov w9, #10 // =0xa
-; CHECK-NEXT:    csel w8, w9, w8, ne
+; CHECK-NEXT:    b.eq .LBB1_2
+; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    mov w8, #10 // =0xa
+; CHECK-NEXT:  .LBB1_2: // %L2
 ; CHECK-NEXT:    str w8, [x2]
 ; CHECK-NEXT:    ret
   %c = add nsw i32 %a, %b
   %d = icmp ne i32 %c, 0
   br i1 %d, label %L1, label %L2
 L1:
   store i32 10, ptr %ptr, align 4
   ret void
 L2:
   store i32 %c, ptr %ptr, align 4
   ret void
 }
 
 define void @test_add_cbz_64(i64 %a, i64 %b, ptr %ptr) {
 ; CHECK-LABEL: test_add_cbz_64:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    cmn x0, x1
-; CHECK-NEXT:    cset w8, eq
+; CHECK-NEXT:    b.eq .LBB2_2
+; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str xzr, [x2]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB2_2: // %L2
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str x8, [x2]
 ; CHECK-NEXT:    ret
   %c = add nsw i64 %a, %b
   %d = icmp ne i64 %c, 0
   br i1 %d, label %L1, label %L2
 L1:
   store i64 0, ptr %ptr, align 4
   ret void
 L2:
   store i64 1, ptr %ptr, align 4
   ret void
 }
 
 define void @test_and_cbz(i32 %a, ptr %ptr) {
 ; CHECK-LABEL: test_and_cbz:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    tst w0, #0x6
-; CHECK-NEXT:    cset w8, eq
+; CHECK-NEXT:    b.eq .LBB3_2
+; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str wzr, [x1]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_2: // %L2
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str w8, [x1]
 ; CHECK-NEXT:    ret
   %c = and i32 %a, 6
   %d = icmp ne i32 %c, 0
   br i1 %d, label %L1, label %L2
 L1:
   store i32 0, ptr %ptr, align 4
   ret void
 L2:
   store i32 1, ptr %ptr, align 4
   ret void
 }
 
 define void @test_bic_cbnz(i32 %a, i32 %b, ptr %ptr) {
 ; CHECK-LABEL: test_bic_cbnz:
-; CHECK:       // %bb.0: // %common.ret
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    bics wzr, w1, w0
-; CHECK-NEXT:    cset w8, ne
+; CHECK-NEXT:    b.ne .LBB4_2
+; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str wzr, [x2]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB4_2: // %L2
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str w8, [x2]
 ; CHECK-NEXT:    ret
   %c = and i32 %a, %b
   %d = icmp eq i32 %c, %b
   br i1 %d, label %L1, label %L2
 L1:
   store i32 0, ptr %ptr, align 4
   ret void

llvm/test/CodeGen/AArch64/csr-split.ll

 define dso_local signext i32 @test2(ptr %p1) local_unnamed_addr uwtable  {
 ; CHECK-LABEL: test2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w19, -8
 ; CHECK-NEXT:    .cfi_offset w30, -16
 ; CHECK-NEXT:    .cfi_remember_state
-; CHECK-NEXT:    cbz x0, .LBB1_3
-; CHECK-NEXT:  // %bb.1: // %entry
+; CHECK-NEXT:    cbz x0, .LBB1_2
+; CHECK-NEXT:  // %bb.1: // %if.end
 ; CHECK-NEXT:    adrp x8, a
 ; CHECK-NEXT:    mov x19, x0
 ; CHECK-NEXT:    ldrsw x8, [x8, :lo12:a]
 ; CHECK-NEXT:    cmp x8, x0
-; CHECK-NEXT:    b.ne .LBB1_3
-; CHECK-NEXT:  // %bb.2: // %if.then2
-; CHECK-NEXT:    bl callVoid
-; CHECK-NEXT:    mov x0, x19
+; CHECK-NEXT:    b.eq .LBB1_3
+; CHECK-NEXT:  .LBB1_2: // %return
+; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; CHECK-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-NEXT:    .cfi_restore w19
 ; CHECK-NEXT:    .cfi_restore w30
-; CHECK-NEXT:    b callNonVoid
-; CHECK-NEXT:  .LBB1_3: // %return
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB1_3: // %if.then2
 ; CHECK-NEXT:    .cfi_restore_state
-; CHECK-NEXT:    mov w0, wzr
+; CHECK-NEXT:    bl callVoid
+; CHECK-NEXT:    mov x0, x19
 ; CHECK-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; CHECK-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-NEXT:    .cfi_restore w19
 ; CHECK-NEXT:    .cfi_restore w30
-; CHECK-NEXT:    ret
+; CHECK-NEXT:    b callNonVoid
 ;
 ; CHECK-APPLE-LABEL: test2:
 ; CHECK-APPLE:       ; %bb.0: ; %entry
 ; CHECK-APPLE-NEXT:    stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
 ; CHECK-APPLE-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-APPLE-NEXT:    stp x29, x30, [sp, #16] ; 16-byte Folded Spill
 ; CHECK-APPLE-NEXT:    .cfi_offset w30, -8
 ; CHECK-APPLE-NEXT:    .cfi_offset w29, -16
 ; CHECK-APPLE-NEXT:    .cfi_offset w19, -24
 ; CHECK-APPLE-NEXT:    .cfi_offset w20, -32
 ; CHECK-APPLE-NEXT:    .cfi_remember_state
-; CHECK-APPLE-NEXT:    cbz x0, LBB1_3
-; CHECK-APPLE-NEXT:  ; %bb.1: ; %entry
+; CHECK-APPLE-NEXT:    cbz x0, LBB1_2
+; CHECK-APPLE-NEXT:  ; %bb.1: ; %if.end
 ; CHECK-APPLE-NEXT:  Lloh2:
 ; CHECK-APPLE-NEXT:    adrp x8, _a@PAGE
 ; CHECK-APPLE-NEXT:    mov x19, x0
 ; CHECK-APPLE-NEXT:  Lloh3:
 ; CHECK-APPLE-NEXT:    ldrsw x8, [x8, _a@PAGEOFF]
 ; CHECK-APPLE-NEXT:    cmp x8, x0
-; CHECK-APPLE-NEXT:    b.ne LBB1_3
-; CHECK-APPLE-NEXT:  ; %bb.2: ; %if.then2
-; CHECK-APPLE-NEXT:    bl _callVoid
+; CHECK-APPLE-NEXT:    b.eq LBB1_3
+; CHECK-APPLE-NEXT:  LBB1_2: ; %return
 ; CHECK-APPLE-NEXT:    ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
-; CHECK-APPLE-NEXT:    mov x0, x19
+; CHECK-APPLE-NEXT:    mov w0, wzr
 ; CHECK-APPLE-NEXT:    ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
 ; CHECK-APPLE-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-APPLE-NEXT:    .cfi_restore w30
 ; CHECK-APPLE-NEXT:    .cfi_restore w29
 ; CHECK-APPLE-NEXT:    .cfi_restore w19
 ; CHECK-APPLE-NEXT:    .cfi_restore w20
-; CHECK-APPLE-NEXT:    b _callNonVoid
-; CHECK-APPLE-NEXT:  LBB1_3: ; %return
+; CHECK-APPLE-NEXT:    ret
+; CHECK-APPLE-NEXT:  LBB1_3: ; %if.then2
 ; CHECK-APPLE-NEXT:    .cfi_restore_state
+; CHECK-APPLE-NEXT:    bl _callVoid
 ; CHECK-APPLE-NEXT:    ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
-; CHECK-APPLE-NEXT:    mov w0, wzr
+; CHECK-APPLE-NEXT:    mov x0, x19
 ; CHECK-APPLE-NEXT:    ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
 ; CHECK-APPLE-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-APPLE-NEXT:    .cfi_restore w30
 ; CHECK-APPLE-NEXT:    .cfi_restore w29
 ; CHECK-APPLE-NEXT:    .cfi_restore w19
 ; CHECK-APPLE-NEXT:    .cfi_restore w20
-; CHECK-APPLE-NEXT:    ret
+; CHECK-APPLE-NEXT:    b _callNonVoid
 ; CHECK-APPLE-NEXT:    .loh AdrpLdr Lloh2, Lloh3
 entry:
   %tobool = icmp eq ptr %p1, null
   br i1 %tobool, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %0 = load i32, ptr @a, align 4, !tbaa !2
   %conv = sext i32 %0 to i64

llvm/test/CodeGen/AArch64/fast-isel-branch-cond-split.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-apple-darwin -fast-isel -fast-isel-abort=1 -verify-machineinstrs < %s | FileCheck %s
 
 define i64 @test_or(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_or:
 ; CHECK:       ; %bb.0: ; %bb1
-; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    cbnz w0, LBB0_2
+; CHECK-NEXT:  LBB0_1: ; %bb3
 ; CHECK-NEXT:    mov x0, xzr
-; CHECK-NEXT:    cbnz w8, LBB0_2
-; CHECK-NEXT:  LBB0_1: ; %common.ret
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB0_2: ; %bb1.cond.split
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cbz w1, LBB0_1
 ; CHECK-NEXT:  ; %bb.3: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload

 bb4:
   %2 = call i64 @bar()
   ret i64 %2
 }
 
 define i64 @test_or_select(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_or_select:
 ; CHECK:       ; %bb.0: ; %bb1
-; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    cbnz w0, LBB1_2
+; CHECK-NEXT:  LBB1_1: ; %bb3
 ; CHECK-NEXT:    mov x0, xzr
-; CHECK-NEXT:    cbnz w8, LBB1_2
-; CHECK-NEXT:  LBB1_1: ; %common.ret
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB1_2: ; %bb1.cond.split
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cbz w1, LBB1_1
 ; CHECK-NEXT:  ; %bb.3: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload

 bb4:
   %2 = call i64 @bar()
   ret i64 %2
 }
 
 define i64 @test_and(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_and:
 ; CHECK:       ; %bb.0: ; %bb1
-; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    cbnz w0, LBB2_2
+; CHECK-NEXT:  LBB2_1: ; %bb3
 ; CHECK-NEXT:    mov x0, xzr
-; CHECK-NEXT:    cbnz w8, LBB2_2
-; CHECK-NEXT:  LBB2_1: ; %common.ret
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB2_2: ; %bb1.cond.split
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cbz w1, LBB2_1
 ; CHECK-NEXT:  ; %bb.3: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload

 bb4:
   %2 = call i64 @bar()
   ret i64 %2
 }
 
 define i64 @test_and_select(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_and_select:
 ; CHECK:       ; %bb.0: ; %bb1
-; CHECK-NEXT:    mov w8, w0
+; CHECK-NEXT:    cbnz w0, LBB3_2
+; CHECK-NEXT:  LBB3_1: ; %bb3
 ; CHECK-NEXT:    mov x0, xzr
-; CHECK-NEXT:    cbnz w8, LBB3_2
-; CHECK-NEXT:  LBB3_1: ; %common.ret
 ; CHECK-NEXT:    ret
 ; CHECK-NEXT:  LBB3_2: ; %bb1.cond.split
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cbz w1, LBB3_1
 ; CHECK-NEXT:  ; %bb.3: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload

 }
 
 ; If the branch is unpredictable, don't add another branch.
 
 define i64 @test_or_unpredictable(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_or_unpredictable:
 ; CHECK:       ; %bb.0: ; %bb1
 ; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cset w8, eq
 ; CHECK-NEXT:    cmp w1, #0
 ; CHECK-NEXT:    cset w9, eq
 ; CHECK-NEXT:    orr w8, w8, w9
 ; CHECK-NEXT:    tbnz w8, #0, LBB4_2
 ; CHECK-NEXT:  ; %bb.1: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; CHECK-NEXT:  LBB4_2: ; %common.ret
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB4_2: ; %bb3
+; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    ret
 bb1:
   %0 = icmp eq i32 %a, 0
   %1 = icmp eq i32 %b, 0
   %or.cond = or i1 %0, %1
   br i1 %or.cond, label %bb3, label %bb4, !unpredictable !2
 
 bb3:
   ret i64 0
 
 bb4:
   %2 = call i64 @bar()
   ret i64 %2
 }
 
 define i64 @test_and_unpredictable(i32 %a, i32 %b) {
 ; CHECK-LABEL: test_and_unpredictable:
 ; CHECK:       ; %bb.0: ; %bb1
 ; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    cset w8, ne
 ; CHECK-NEXT:    cmp w1, #0
 ; CHECK-NEXT:    cset w9, ne
 ; CHECK-NEXT:    and w8, w8, w9
 ; CHECK-NEXT:    tbz w8, #0, LBB5_2
 ; CHECK-NEXT:  ; %bb.1: ; %bb4
 ; CHECK-NEXT:    stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -8
 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    bl _bar
 ; CHECK-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
-; CHECK-NEXT:  LBB5_2: ; %common.ret
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  LBB5_2: ; %bb3
+; CHECK-NEXT:    mov x0, xzr
 ; CHECK-NEXT:    ret
 bb1:
   %0 = icmp ne i32 %a, 0
   %1 = icmp ne i32 %b, 0
   %or.cond = and i1 %0, %1
   br i1 %or.cond, label %bb4, label %bb3, !unpredictable !2
 
 bb3:

llvm/test/CodeGen/AArch64/implicit-null-check.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -verify-machineinstrs -O3 -mtriple=aarch64-unknown-unknown -enable-implicit-null-checks | FileCheck %s
 
 ; Basic test for implicit null check conversion - this is analogous to the
 ; file with the same name in the X86 tree, but adjusted to remove patterns
 ; related to memory folding of arithmetic (since aarch64 doesn't), and add
 ; a couple of aarch64 specific tests.
 
 define i32 @imp_null_check_load_fallthrough(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_fallthrough:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp0:
 ; CHECK-NEXT:    ldr w0, [x0] // on-fault: .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB0_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB0_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  not_null:
   %t = load i32, ptr %x
   ret i32 %t
 
 is_null:
   ret i32 42
 }
 
 
 define i32 @imp_null_check_load_reorder(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_reorder:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp1:
 ; CHECK-NEXT:    ldr w0, [x0] // on-fault: .LBB1_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB1_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB1_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %t = load i32, ptr %x
   ret i32 %t
 }
 
 define i32 @imp_null_check_unordered_load(ptr %x) {
 ; CHECK-LABEL: imp_null_check_unordered_load:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp2:
 ; CHECK-NEXT:    ldr w0, [x0] // on-fault: .LBB2_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB2_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB2_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %t = load atomic i32, ptr %x unordered, align 4
   ret i32 %t
 }
 
 
 ; TODO: Can be converted into implicit check.
 ;; Probably could be implicit, but we're conservative for now
 define i32 @imp_null_check_seq_cst_load(ptr %x) {
 ; CHECK-LABEL: imp_null_check_seq_cst_load:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB3_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ldar w0, [x0]
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB3_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB3_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %t = load atomic i32, ptr %x seq_cst, align 4
   ret i32 %t
 }
 
 ;; Might be memory mapped IO, so can't rely on fault behavior
 define i32 @imp_null_check_volatile_load(ptr %x) {
 ; CHECK-LABEL: imp_null_check_volatile_load:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB4_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ldr w0, [x0]
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB4_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB4_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %t = load volatile i32, ptr %x, align 4
   ret i32 %t
 }
 
 
 define i8 @imp_null_check_load_i8(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_i8:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp3:
 ; CHECK-NEXT:    ldrb w0, [x0] // on-fault: .LBB5_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB5_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB5_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i8 42
 
  not_null:
   %t = load i8, ptr %x
   ret i8 %t
 }
 
 define i256 @imp_null_check_load_i256(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_i256:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB6_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
+; CHECK-NEXT:    ldp x8, x1, [x0]
 ; CHECK-NEXT:    ldp x2, x3, [x0, #16]
-; CHECK-NEXT:    ldp x0, x1, [x0]
+; CHECK-NEXT:    mov x0, x8
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB6_2:
+; CHECK-NEXT:  .LBB6_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    mov x1, xzr
 ; CHECK-NEXT:    mov x2, xzr
 ; CHECK-NEXT:    mov x3, xzr
-; CHECK-NEXT:    mov w0, #42
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i256 42
 
  not_null:
   %t = load i256, ptr %x
   ret i256 %t
 }
 
 
 
 define i32 @imp_null_check_gep_load(ptr %x) {
 ; CHECK-LABEL: imp_null_check_gep_load:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp4:
 ; CHECK-NEXT:    ldr w0, [x0, #128] // on-fault: .LBB7_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB7_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB7_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %x.gep = getelementptr i32, ptr %x, i32 32
   %t = load i32, ptr %x.gep
   ret i32 %t
 }
 
 define i32 @imp_null_check_add_result(ptr %x, i32 %p) {
 ; CHECK-LABEL: imp_null_check_add_result:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:  .Ltmp5:
 ; CHECK-NEXT:    ldr w8, [x0] // on-fault: .LBB8_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    add w0, w8, w1
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB8_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB8_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 

 ; CHECK-LABEL: imp_null_check_hoist_over_udiv:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB9_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    udiv w8, w1, w2
 ; CHECK-NEXT:    ldr w9, [x0]
 ; CHECK-NEXT:    add w0, w9, w8
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB9_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB9_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %p1 = udiv i32 %a, %b
   %t = load i32, ptr %x
   %res = add i32 %t, %p1
   ret i32 %res
 }
 
 
 ; TODO: We should be able to hoist this - we can on x86, why isn't this
 ; working for aarch64?  Aliasing?
 define i32 @imp_null_check_hoist_over_unrelated_load(ptr %x, ptr %y, ptr %z) {
 ; CHECK-LABEL: imp_null_check_hoist_over_unrelated_load:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    cbz x0, .LBB10_2
+; CHECK-NEXT:  .Ltmp6:
+; CHECK-NEXT:    ldr w0, [x0] // on-fault: .LBB10_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ldr w8, [x1]
-; CHECK-NEXT:    ldr w0, [x0]
 ; CHECK-NEXT:    str w8, [x2]
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB10_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB10_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %t0 = load i32, ptr %y
   %t1 = load i32, ptr %x
   store i32 %t0, ptr %z
   ret i32 %t1
 }
 
 define i32 @imp_null_check_gep_load_with_use_dep(ptr %x, i32 %a) {
 ; CHECK-LABEL: imp_null_check_gep_load_with_use_dep:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:  .Ltmp6:
+; CHECK-NEXT:  .Ltmp7:
 ; CHECK-NEXT:    ldr w8, [x0] // on-fault: .LBB11_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    add w9, w0, w1
 ; CHECK-NEXT:    add w8, w9, w8
 ; CHECK-NEXT:    add w0, w8, #4
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB11_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB11_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 

 define i32 @imp_null_check_load_fence1(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_fence1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB12_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    dmb ishld
 ; CHECK-NEXT:    ldr w0, [x0]
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB12_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB12_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
 entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
 is_null:
   ret i32 42
 
 not_null:
   fence acquire
   %t = load i32, ptr %x
   ret i32 %t
 }
 
 ;; TODO: We could handle this case as we can lift the fence into the
 ;; previous block before the conditional without changing behavior.
 define i32 @imp_null_check_load_fence2(ptr %x) {
 ; CHECK-LABEL: imp_null_check_load_fence2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB13_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    dmb ish
 ; CHECK-NEXT:    ldr w0, [x0]
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB13_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB13_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
 entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
 is_null:
   ret i32 42
 
 not_null:
   fence seq_cst
   %t = load i32, ptr %x
   ret i32 %t
 }
 
 ; TODO: We can fold to implicit null here, not sure why this isn't working
 define void @imp_null_check_store(ptr %x) {
 ; CHECK-LABEL: imp_null_check_store:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB14_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
-; CHECK-NEXT:    mov w8, #1
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str w8, [x0]
-; CHECK-NEXT:  .LBB14_2: // %common.ret
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB14_2: // %is_null
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret void
 
  not_null:
   store i32 1, ptr %x
   ret void
 }
 
 ;; TODO: can be implicit
 define void @imp_null_check_unordered_store(ptr %x) {
 ; CHECK-LABEL: imp_null_check_unordered_store:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cbz x0, .LBB15_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
-; CHECK-NEXT:    mov w8, #1
+; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    str w8, [x0]
-; CHECK-NEXT:  .LBB15_2: // %common.ret
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB15_2: // %is_null
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret void
 
  not_null:
   store atomic i32 1, ptr %x unordered, align 4
   ret void
 }
 
 define i32 @imp_null_check_neg_gep_load(ptr %x) {
 ; CHECK-LABEL: imp_null_check_neg_gep_load:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:  .Ltmp7:
+; CHECK-NEXT:  .Ltmp8:
 ; CHECK-NEXT:    ldur w0, [x0, #-128] // on-fault: .LBB16_2
 ; CHECK-NEXT:  // %bb.1: // %not_null
 ; CHECK-NEXT:    ret
-; CHECK-NEXT:  .LBB16_2:
-; CHECK-NEXT:    mov w0, #42
+; CHECK-NEXT:  .LBB16_2: // %is_null
+; CHECK-NEXT:    mov w0, #42 // =0x2a
 ; CHECK-NEXT:    ret
  entry:
   %c = icmp eq ptr %x, null
   br i1 %c, label %is_null, label %not_null, !make.implicit !0
 
  is_null:
   ret i32 42
 
  not_null:
   %x.gep = getelementptr i32, ptr %x, i32 -32
   %t = load i32, ptr %x.gep
   ret i32 %t
 }
 
 !0 = !{}

llvm/test/CodeGen/AArch64/large-stack-cmp.ll

 ; CHECK-NEXT:    adds x8, sp, #1, lsl #12 ; =4096
 ; CHECK-NEXT:    cmn x8, #32
 ; CHECK-NEXT:    b.eq LBB0_2
 ; CHECK-NEXT:  ; %bb.1: ; %false
 ; CHECK-NEXT:    bl _baz
 ; CHECK-NEXT:    b LBB0_3
 ; CHECK-NEXT:  LBB0_2: ; %true
 ; CHECK-NEXT:    bl _bar
-; CHECK-NEXT:  LBB0_3: ; %common.ret
+; CHECK-NEXT:  LBB0_3: ; %true
 ; CHECK-NEXT:    add sp, sp, #1, lsl #12 ; =4096
 ; CHECK-NEXT:    add sp, sp, #80
 ; CHECK-NEXT:    ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
 ; CHECK-NEXT:    ldp x28, x27, [sp], #32 ; 16-byte Folded Reload
 ; CHECK-NEXT:    ret
 
 %var = alloca i32, i32 12
   %var2 = alloca i32, i32 1030

llvm/test/CodeGen/AArch64/ldst-opt-after-block-placement.ll

 ; CHECK-NEXT:    b.lt .LBB0_3
 ; CHECK-NEXT:    b .LBB0_4
 ; CHECK-NEXT:  .LBB0_2: // %if.else
 ; CHECK-NEXT:    ldp x8, x9, [x1]
 ; CHECK-NEXT:    cmp x8, x9
 ; CHECK-NEXT:    b.ge .LBB0_4
 ; CHECK-NEXT:  .LBB0_3: // %exit1
 ; CHECK-NEXT:    str xzr, [x1, #8]
-; CHECK-NEXT:  .LBB0_4: // %common.ret
+; CHECK-NEXT:  .LBB0_4: // %exit2
 ; CHECK-NEXT:    ret
 entry:
   br i1 %cond, label %if.then, label %if.else
 
 if.then:
   %0 = getelementptr inbounds i64, ptr %ptr, i64 2
   %1 = load i64, ptr %0, align 8
   store i64 0, ptr %0, align 8

llvm/test/CodeGen/AArch64/ldst-paired-aliasing.ll

 ; CHECK-LABEL: main:
 ; CHECK:       // %bb.0: // %for.body.lr.ph.i.i.i.i.i.i63
 ; CHECK-NEXT:    sub sp, sp, #112
 ; CHECK-NEXT:    str x30, [sp, #96] // 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 112
 ; CHECK-NEXT:    .cfi_offset w30, -16
 ; CHECK-NEXT:    bl _Z5setupv
 ; CHECK-NEXT:    movi v0.4s, #1
-; CHECK-NEXT:    mov w9, #1
+; CHECK-NEXT:    mov w9, #1 // =0x1
 ; CHECK-NEXT:    add x0, sp, #48
 ; CHECK-NEXT:    mov x1, sp
 ; CHECK-NEXT:    str xzr, [sp, #80]
 ; CHECK-NEXT:    str w9, [sp, #80]
 ; CHECK-NEXT:    stp q0, q0, [sp, #48]
 ; CHECK-NEXT:    ldr w8, [sp, #48]
 ; CHECK-NEXT:    cmp w8, #1
 ; CHECK-NEXT:    b.ne .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %for.inc
 ; CHECK-NEXT:    bl f
 ; CHECK-NEXT:    b .LBB0_3
 ; CHECK-NEXT:  .LBB0_2: // %if.then
 ; CHECK-NEXT:    bl f2
-; CHECK-NEXT:  .LBB0_3: // %common.ret
+; CHECK-NEXT:  .LBB0_3: // %for.inc
 ; CHECK-NEXT:    ldr x30, [sp, #96] // 8-byte Folded Reload
 ; CHECK-NEXT:    mov w0, wzr
 ; CHECK-NEXT:    add sp, sp, #112
 ; CHECK-NEXT:    ret
 
 
 for.body.lr.ph.i.i.i.i.i.i63:
   %b1 = alloca [10 x i32], align 16

llvm/test/CodeGen/AArch64/logical_shifted_reg.ll

 ; CHECK-NEXT:    ldr x9, [x8]
 ; CHECK-NEXT:    ldr x10, [x10]
 ; CHECK-NEXT:    tst x9, x10
 ; CHECK-NEXT:    b.gt .LBB2_4
 ; CHECK-NEXT:  // %bb.1: // %test2
 ; CHECK-NEXT:    tst x9, x10, lsl #63
 ; CHECK-NEXT:    b.lt .LBB2_4
 ; CHECK-NEXT:  // %bb.2: // %test3
-; CHECK-NEXT:    and x10, x9, x10, asr #12
-; CHECK-NEXT:    cmp x10, #1
-; CHECK-NEXT:    b.ge .LBB2_4
+; CHECK-NEXT:    tst x9, x10, asr #12
+; CHECK-NEXT:    b.gt .LBB2_4
 ; CHECK-NEXT:  // %bb.3: // %other_exit
 ; CHECK-NEXT:    str x9, [x8]
-; CHECK-NEXT:  .LBB2_4: // %common.ret
+; CHECK-NEXT:  .LBB2_4: // %ret
 ; CHECK-NEXT:    ret
   %val1 = load i64, ptr @var1_64
   %val2 = load i64, ptr @var2_64
 
   %simple_and = and i64 %val1, %val2
   %tst1 = icmp sgt i64 %simple_and, 0
   br i1 %tst1, label %ret, label %test2, !prof !1
 

llvm/test/CodeGen/AArch64/machine-combiner-copy.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=aarch64-linux-gnuabi -mattr=+fullfp16 -O3 | FileCheck %s
 
 define void @fma_dup_f16(ptr noalias nocapture noundef readonly %A, half noundef %B, ptr noalias nocapture noundef %C, i32 noundef %n) {
 ; CHECK-LABEL: fma_dup_f16:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    // kill: def $h0 killed $h0 def $q0
 ; CHECK-NEXT:    cbz w2, .LBB0_8
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w2, #15
+; CHECK-NEXT:    cmp w2, #16
 ; CHECK-NEXT:    mov w8, w2
-; CHECK-NEXT:    b.hi .LBB0_3
+; CHECK-NEXT:    b.hs .LBB0_3
 ; CHECK-NEXT:  // %bb.2:
 ; CHECK-NEXT:    mov x9, xzr
 ; CHECK-NEXT:    b .LBB0_6
 ; CHECK-NEXT:  .LBB0_3: // %vector.ph
 ; CHECK-NEXT:    and x9, x8, #0xfffffff0
 ; CHECK-NEXT:    add x10, x1, #16
 ; CHECK-NEXT:    add x11, x0, #16
 ; CHECK-NEXT:    mov x12, x9
 ; CHECK-NEXT:  .LBB0_4: // %vector.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp q1, q3, [x11, #-16]
 ; CHECK-NEXT:    subs x12, x12, #16
 ; CHECK-NEXT:    ldp q2, q4, [x10, #-16]
 ; CHECK-NEXT:    add x11, x11, #32
 ; CHECK-NEXT:    fmla v2.8h, v1.8h, v0.h[0]
 ; CHECK-NEXT:    fmla v4.8h, v3.8h, v0.h[0]
 ; CHECK-NEXT:    stp q2, q4, [x10, #-16]
 ; CHECK-NEXT:    add x10, x10, #32
 ; CHECK-NEXT:    b.ne .LBB0_4
 ; CHECK-NEXT:  // %bb.5: // %middle.block
 ; CHECK-NEXT:    cmp x9, x8
 ; CHECK-NEXT:    b.eq .LBB0_8
-; CHECK-NEXT:  .LBB0_6: // %for.body.preheader1
+; CHECK-NEXT:  .LBB0_6: // %for.body.preheader14
 ; CHECK-NEXT:    lsl x10, x9, #1
 ; CHECK-NEXT:    sub x8, x8, x9
 ; CHECK-NEXT:    add x9, x1, x10
 ; CHECK-NEXT:    add x10, x0, x10
 ; CHECK-NEXT:  .LBB0_7: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldr h1, [x10], #2
 ; CHECK-NEXT:    ldr h2, [x9]

llvm/test/CodeGen/AArch64/merge-store-dependency.ll

 ; A53-NEXT:    .cfi_def_cfa_offset 16
 ; A53-NEXT:    .cfi_offset w19, -8
 ; A53-NEXT:    .cfi_offset w30, -16
 ; A53-NEXT:    .cfi_remember_state
 ; A53-NEXT:    movi v0.2d, #0000000000000000
 ; A53-NEXT:    mov x8, x0
 ; A53-NEXT:    mov x19, x8
 ; A53-NEXT:    mov w0, w1
-; A53-NEXT:    mov w9, #256
+; A53-NEXT:    mov w9, #256 // =0x100
 ; A53-NEXT:    stp x2, x3, [x8, #32]
 ; A53-NEXT:    mov x2, x8
 ; A53-NEXT:    str q0, [x19, #16]!
 ; A53-NEXT:    str w1, [x19]
-; A53-NEXT:    mov w1, #4
+; A53-NEXT:    mov w1, #4 // =0x4
 ; A53-NEXT:    str q0, [x8]
 ; A53-NEXT:    strh w9, [x8, #24]
 ; A53-NEXT:    str wzr, [x8, #20]
 ; A53-NEXT:    bl fcntl
+; A53-NEXT:    ldr w8, [x19]
+; A53-NEXT:    tbnz w8, #31, .LBB0_6
+; A53-NEXT:  // %bb.1: // %while.body.i.preheader
 ; A53-NEXT:    adrp x9, gv0
 ; A53-NEXT:    add x9, x9, :lo12:gv0
 ; A53-NEXT:    cmp x19, x9
-; A53-NEXT:    b.eq .LBB0_4
-; A53-NEXT:  // %bb.1:
-; A53-NEXT:    ldr w8, [x19]
+; A53-NEXT:    b.eq .LBB0_5
+; A53-NEXT:  // %bb.2: // %while.body.i.split.ver.us.preheader
 ; A53-NEXT:    ldr w9, [x9]
 ; A53-NEXT:    .p2align 4, , 8
-; A53-NEXT:  .LBB0_2: // %while.body.i.split.ver.us
+; A53-NEXT:  .LBB0_3: // %while.body.i.split.ver.us
 ; A53-NEXT:    // =>This Inner Loop Header: Depth=1
 ; A53-NEXT:    lsl w9, w9, #1
 ; A53-NEXT:    cmp w9, w8
-; A53-NEXT:    b.le .LBB0_2
-; A53-NEXT:  // %bb.3: // %while.end.i
+; A53-NEXT:    b.le .LBB0_3
+; A53-NEXT:  // %bb.4: // %while.end.i
 ; A53-NEXT:    bl foo
 ; A53-NEXT:    adrp x8, gv1
 ; A53-NEXT:    str x0, [x8, :lo12:gv1]
 ; A53-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
 ; A53-NEXT:    .cfi_def_cfa_offset 0
 ; A53-NEXT:    .cfi_restore w19
 ; A53-NEXT:    .cfi_restore w30
 ; A53-NEXT:    ret
 ; A53-NEXT:    .p2align 4, , 8
-; A53-NEXT:  .LBB0_4: // %while.body.i.split
+; A53-NEXT:  .LBB0_5: // %while.body.i.split
 ; A53-NEXT:    // =>This Inner Loop Header: Depth=1
 ; A53-NEXT:    .cfi_restore_state
-; A53-NEXT:    b .LBB0_4
+; A53-NEXT:    b .LBB0_5
+; A53-NEXT:  .LBB0_6: // %if.then.i
 entry:
   tail call void @llvm.memset.p0.i64(ptr align 8 %fde, i8 0, i64 40, i1 false)
   %state = getelementptr inbounds %struct1, ptr %fde, i64 0, i32 4
   store i16 256, ptr %state, align 8
   %fd1 = getelementptr inbounds %struct1, ptr %fde, i64 0, i32 2
   store i32 %fd, ptr %fd1, align 8
   %force_eof = getelementptr inbounds %struct1, ptr %fde, i64 0, i32 3
   store i32 0, ptr %force_eof, align 4

llvm/test/CodeGen/AArch64/optimize-cond-branch.ll

 ; CHECK-LABEL: func:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:    cbnz w8, .LBB0_3
 ; CHECK-NEXT:  // %bb.1: // %b1
 ; CHECK-NEXT:    cbz wzr, .LBB0_4
 ; CHECK-NEXT:  // %bb.2: // %b3
 ; CHECK-NEXT:    ldr w8, [x8]
-; CHECK-NEXT:    and w0, w8, #0x100
-; CHECK-NEXT:    cbz w0, .LBB0_5
-; CHECK-NEXT:  .LBB0_3: // %common.ret.sink.split
+; CHECK-NEXT:    tbz w8, #8, .LBB0_5
+; CHECK-NEXT:  .LBB0_3: // %b7
 ; CHECK-NEXT:    b extfunc
 ; CHECK-NEXT:  .LBB0_4: // %b2
 ; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    .cfi_offset w30, -16
 ; CHECK-NEXT:    bl extfunc
 ; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-NEXT:    .cfi_restore w30
 ; CHECK-NEXT:    cbnz w0, .LBB0_3
-; CHECK-NEXT:  .LBB0_5: // %common.ret
+; CHECK-NEXT:  .LBB0_5: // %b8
 ; CHECK-NEXT:    ret
   %c0 = icmp sgt i64 0, 0
   br i1 %c0, label %b1, label %b6
 
 b1:
   br i1 undef, label %b3, label %b2
 
 b2:

llvm/test/CodeGen/AArch64/pr51476.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-unknown-linux-gnu < %s | FileCheck %s
 
 define void @test(i8 %arg) nounwind {
 ; CHECK-LABEL: test:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
-; CHECK-NEXT:    and w9, w0, #0xff
-; CHECK-NEXT:    mov w8, #1
-; CHECK-NEXT:    cmp w9, #1
+; CHECK-NEXT:    and w8, w0, #0xff
+; CHECK-NEXT:    cmp w8, #1
 ; CHECK-NEXT:    cset w0, ne
 ; CHECK-NEXT:    strb w0, [sp, #12]
-; CHECK-NEXT:    cbz w8, .LBB0_2
+; CHECK-NEXT:    cbnz wzr, .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %do_call
 ; CHECK-NEXT:    bl unknown
-; CHECK-NEXT:  .LBB0_2: // %common.ret
+; CHECK-NEXT:  .LBB0_2: // %exit
 ; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %tmp = alloca i8
   %cmp1 = icmp ne i8 %arg, 1
   %zext = zext i1 %cmp1 to i8
   store i8 %zext, ptr %tmp
   %zext2 = load i8, ptr %tmp
   %cmp2 = icmp eq i8 %zext2, 3

llvm/test/CodeGen/AArch64/rm_redundant_cmp.ll

 define void @test_i16_2cmp_signed_1() {
 ; CHECK-LABEL: test_i16_2cmp_signed_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_s_i8_i16
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_s_i8_i16]
 ; CHECK-NEXT:    ldrsh w9, [x8, #2]
 ; CHECK-NEXT:    ldrsh w10, [x8, #4]
 ; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.lt .LBB0_2
-; CHECK-NEXT:  // %bb.1: // %if.end8.sink.split
+; CHECK-NEXT:    b.gt .LBB0_2
+; CHECK-NEXT:  // %bb.1: // %if.else
+; CHECK-NEXT:    b.ne .LBB0_3
+; CHECK-NEXT:  .LBB0_2: // %if.then7
 ; CHECK-NEXT:    strh w9, [x8]
-; CHECK-NEXT:  .LBB0_2: // %if.end8
+; CHECK-NEXT:  .LBB0_3: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i16, ptr getelementptr inbounds (%struct.s_signed_i16, ptr @cost_s_i8_i16, i64 0, i32 1), align 2
   %1 = load i16, ptr getelementptr inbounds (%struct.s_signed_i16, ptr @cost_s_i8_i16, i64 0, i32 2), align 2
   %cmp = icmp sgt i16 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
if.end8:                                          ; preds = %if.else, %if.then7, %if.then
   ret void
 }
 
 define void @test_i16_2cmp_signed_2() {
 ; CHECK-LABEL: test_i16_2cmp_signed_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_s_i8_i16
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_s_i8_i16]
-; CHECK-NEXT:    ldrsh w9, [x8, #2]
-; CHECK-NEXT:    ldrsh w10, [x8, #4]
-; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.gt .LBB1_2
-; CHECK-NEXT:  // %bb.1: // %if.else
-; CHECK-NEXT:    mov w9, w10
-; CHECK-NEXT:    b.ge .LBB1_3
-; CHECK-NEXT:  .LBB1_2: // %if.end8.sink.split
+; CHECK-NEXT:    ldrsh w10, [x8, #2]
+; CHECK-NEXT:    ldrsh w9, [x8, #4]
+; CHECK-NEXT:    cmp w10, w9
+; CHECK-NEXT:    b.le .LBB1_2
+; CHECK-NEXT:  // %bb.1: // %if.then
+; CHECK-NEXT:    strh w10, [x8]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB1_2: // %if.else
+; CHECK-NEXT:    b.ge .LBB1_4
+; CHECK-NEXT:  // %bb.3: // %if.then7
 ; CHECK-NEXT:    strh w9, [x8]
-; CHECK-NEXT:  .LBB1_3: // %if.end8
+; CHECK-NEXT:  .LBB1_4: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i16, ptr getelementptr inbounds (%struct.s_signed_i16, ptr @cost_s_i8_i16, i64 0, i32 1), align 2
   %1 = load i16, ptr getelementptr inbounds (%struct.s_signed_i16, ptr @cost_s_i8_i16, i64 0, i32 2), align 2
   %cmp = icmp sgt i16 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry

 define void @test_i16_2cmp_unsigned_1() {
 ; CHECK-LABEL: test_i16_2cmp_unsigned_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_u_i16
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_u_i16]
 ; CHECK-NEXT:    ldrh w9, [x8, #2]
 ; CHECK-NEXT:    ldrh w10, [x8, #4]
 ; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.lo .LBB2_2
-; CHECK-NEXT:  // %bb.1: // %if.end8.sink.split
+; CHECK-NEXT:    b.hi .LBB2_2
+; CHECK-NEXT:  // %bb.1: // %if.else
+; CHECK-NEXT:    b.ne .LBB2_3
+; CHECK-NEXT:  .LBB2_2: // %if.then7
 ; CHECK-NEXT:    strh w9, [x8]
-; CHECK-NEXT:  .LBB2_2: // %if.end8
+; CHECK-NEXT:  .LBB2_3: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i16, ptr getelementptr inbounds (%struct.s_unsigned_i16, ptr @cost_u_i16, i64 0, i32 1), align 2
   %1 = load i16, ptr getelementptr inbounds (%struct.s_unsigned_i16, ptr @cost_u_i16, i64 0, i32 2), align 2
   %cmp = icmp ugt i16 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
if.end8:                                          ; preds = %if.else, %if.then7, %if.then
   ret void
 }
 
 define void @test_i16_2cmp_unsigned_2() {
 ; CHECK-LABEL: test_i16_2cmp_unsigned_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_u_i16
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_u_i16]
-; CHECK-NEXT:    ldrh w9, [x8, #2]
-; CHECK-NEXT:    ldrh w10, [x8, #4]
-; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.hi .LBB3_2
-; CHECK-NEXT:  // %bb.1: // %if.else
-; CHECK-NEXT:    mov w9, w10
-; CHECK-NEXT:    b.hs .LBB3_3
-; CHECK-NEXT:  .LBB3_2: // %if.end8.sink.split
+; CHECK-NEXT:    ldrh w10, [x8, #2]
+; CHECK-NEXT:    ldrh w9, [x8, #4]
+; CHECK-NEXT:    cmp w10, w9
+; CHECK-NEXT:    b.ls .LBB3_2
+; CHECK-NEXT:  // %bb.1: // %if.then
+; CHECK-NEXT:    strh w10, [x8]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_2: // %if.else
+; CHECK-NEXT:    b.hs .LBB3_4
+; CHECK-NEXT:  // %bb.3: // %if.then7
 ; CHECK-NEXT:    strh w9, [x8]
-; CHECK-NEXT:  .LBB3_3: // %if.end8
+; CHECK-NEXT:  .LBB3_4: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i16, ptr getelementptr inbounds (%struct.s_unsigned_i16, ptr @cost_u_i16, i64 0, i32 1), align 2
   %1 = load i16, ptr getelementptr inbounds (%struct.s_unsigned_i16, ptr @cost_u_i16, i64 0, i32 2), align 2
   %cmp = icmp ugt i16 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry

 define void @test_i8_2cmp_signed_1() {
 ; CHECK-LABEL: test_i8_2cmp_signed_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_s
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_s]
 ; CHECK-NEXT:    ldrsb w9, [x8, #1]
 ; CHECK-NEXT:    ldrsb w10, [x8, #2]
 ; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.lt .LBB4_2
-; CHECK-NEXT:  // %bb.1: // %if.end8.sink.split
+; CHECK-NEXT:    b.gt .LBB4_2
+; CHECK-NEXT:  // %bb.1: // %if.else
+; CHECK-NEXT:    b.ne .LBB4_3
+; CHECK-NEXT:  .LBB4_2: // %if.then7
 ; CHECK-NEXT:    strb w9, [x8]
-; CHECK-NEXT:  .LBB4_2: // %if.end8
+; CHECK-NEXT:  .LBB4_3: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i8, ptr getelementptr inbounds (%struct.s_signed_i8, ptr @cost_s, i64 0, i32 1), align 2
   %1 = load i8, ptr getelementptr inbounds (%struct.s_signed_i8, ptr @cost_s, i64 0, i32 2), align 2
   %cmp = icmp sgt i8 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
if.end8:                                          ; preds = %if.else, %if.then7, %if.then
   ret void
 }
 
 define void @test_i8_2cmp_signed_2() {
 ; CHECK-LABEL: test_i8_2cmp_signed_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_s
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_s]
-; CHECK-NEXT:    ldrsb w9, [x8, #1]
-; CHECK-NEXT:    ldrsb w10, [x8, #2]
-; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.gt .LBB5_2
-; CHECK-NEXT:  // %bb.1: // %if.else
-; CHECK-NEXT:    mov w9, w10
-; CHECK-NEXT:    b.ge .LBB5_3
-; CHECK-NEXT:  .LBB5_2: // %if.end8.sink.split
+; CHECK-NEXT:    ldrsb w10, [x8, #1]
+; CHECK-NEXT:    ldrsb w9, [x8, #2]
+; CHECK-NEXT:    cmp w10, w9
+; CHECK-NEXT:    b.le .LBB5_2
+; CHECK-NEXT:  // %bb.1: // %if.then
+; CHECK-NEXT:    strb w10, [x8]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB5_2: // %if.else
+; CHECK-NEXT:    b.ge .LBB5_4
+; CHECK-NEXT:  // %bb.3: // %if.then7
 ; CHECK-NEXT:    strb w9, [x8]
-; CHECK-NEXT:  .LBB5_3: // %if.end8
+; CHECK-NEXT:  .LBB5_4: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i8, ptr getelementptr inbounds (%struct.s_signed_i8, ptr @cost_s, i64 0, i32 1), align 2
   %1 = load i8, ptr getelementptr inbounds (%struct.s_signed_i8, ptr @cost_s, i64 0, i32 2), align 2
   %cmp = icmp sgt i8 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry

 define void @test_i8_2cmp_unsigned_1() {
 ; CHECK-LABEL: test_i8_2cmp_unsigned_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_u_i8
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_u_i8]
 ; CHECK-NEXT:    ldrb w9, [x8, #1]
 ; CHECK-NEXT:    ldrb w10, [x8, #2]
 ; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.lo .LBB6_2
-; CHECK-NEXT:  // %bb.1: // %if.end8.sink.split
+; CHECK-NEXT:    b.hi .LBB6_2
+; CHECK-NEXT:  // %bb.1: // %if.else
+; CHECK-NEXT:    b.ne .LBB6_3
+; CHECK-NEXT:  .LBB6_2: // %if.then7
 ; CHECK-NEXT:    strb w9, [x8]
-; CHECK-NEXT:  .LBB6_2: // %if.end8
+; CHECK-NEXT:  .LBB6_3: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i8, ptr getelementptr inbounds (%struct.s_unsigned_i8, ptr @cost_u_i8, i64 0, i32 1), align 2
   %1 = load i8, ptr getelementptr inbounds (%struct.s_unsigned_i8, ptr @cost_u_i8, i64 0, i32 2), align 2
   %cmp = icmp ugt i8 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
if.end8:                                          ; preds = %if.else, %if.then7, %if.then
   ret void
 }
 
 define void @test_i8_2cmp_unsigned_2() {
 ; CHECK-LABEL: test_i8_2cmp_unsigned_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    adrp x8, :got:cost_u_i8
 ; CHECK-NEXT:    ldr x8, [x8, :got_lo12:cost_u_i8]
-; CHECK-NEXT:    ldrb w9, [x8, #1]
-; CHECK-NEXT:    ldrb w10, [x8, #2]
-; CHECK-NEXT:    cmp w9, w10
-; CHECK-NEXT:    b.hi .LBB7_2
-; CHECK-NEXT:  // %bb.1: // %if.else
-; CHECK-NEXT:    mov w9, w10
-; CHECK-NEXT:    b.hs .LBB7_3
-; CHECK-NEXT:  .LBB7_2: // %if.end8.sink.split
+; CHECK-NEXT:    ldrb w10, [x8, #1]
+; CHECK-NEXT:    ldrb w9, [x8, #2]
+; CHECK-NEXT:    cmp w10, w9
+; CHECK-NEXT:    b.ls .LBB7_2
+; CHECK-NEXT:  // %bb.1: // %if.then
+; CHECK-NEXT:    strb w10, [x8]
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB7_2: // %if.else
+; CHECK-NEXT:    b.hs .LBB7_4
+; CHECK-NEXT:  // %bb.3: // %if.then7
 ; CHECK-NEXT:    strb w9, [x8]
-; CHECK-NEXT:  .LBB7_3: // %if.end8
+; CHECK-NEXT:  .LBB7_4: // %if.end8
 ; CHECK-NEXT:    ret
 entry:
   %0 = load i8, ptr getelementptr inbounds (%struct.s_unsigned_i8, ptr @cost_u_i8, i64 0, i32 1), align 2
   %1 = load i8, ptr getelementptr inbounds (%struct.s_unsigned_i8, ptr @cost_u_i8, i64 0, i32 2), align 2
   %cmp = icmp ugt i8 %0, %1
   br i1 %cmp, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry

llvm/test/CodeGen/AArch64/signbit-test.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=aarch64-- | FileCheck %s
 
 define i64 @test_clear_mask_i64_i32(i64 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i64_i32:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    csel x0, x8, x0, ge
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x80000000
+; CHECK-NEXT:    csel x0, x0, x8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 2147483648
   %r = icmp eq i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i64 @test_set_mask_i64_i32(i64 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i64_i32:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    tst x0, #0x80000000
-; CHECK-NEXT:    csel x0, x8, x0, ne
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x80000000
+; CHECK-NEXT:    csel x0, x0, x8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 2147483648
   %r = icmp ne i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i64 @test_clear_mask_i64_i16(i64 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i64_i16:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    tst x0, #0x8000
-; CHECK-NEXT:    csel x0, x8, x0, eq
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x8000
+; CHECK-NEXT:    csel x0, x0, x8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 32768
   %r = icmp eq i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i64 @test_set_mask_i64_i16(i64 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i64_i16:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    tst x0, #0x8000
-; CHECK-NEXT:    csel x0, x8, x0, ne
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x8000
+; CHECK-NEXT:    csel x0, x0, x8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 32768
   %r = icmp ne i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i64 @test_clear_mask_i64_i8(i64 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i64_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    tst x0, #0x80
-; CHECK-NEXT:    csel x0, x8, x0, eq
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x80
+; CHECK-NEXT:    csel x0, x0, x8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 128
   %r = icmp eq i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i64 @test_set_mask_i64_i8(i64 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i64_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
-; CHECK-NEXT:    tst x0, #0x80
-; CHECK-NEXT:    csel x0, x8, x0, ne
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    tst w0, #0x80
+; CHECK-NEXT:    csel x0, x0, x8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i64 %x, 128
   %r = icmp ne i64 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i64 [ %x, %entry], [ 42, %t]
   ret i64 %ret
 }
 
 define i32 @test_clear_mask_i32_i16(i32 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i32_i16:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
 ; CHECK-NEXT:    tst w0, #0x8000
-; CHECK-NEXT:    csel w0, w8, w0, eq
+; CHECK-NEXT:    csel w0, w0, w8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i32 %x, 32768
   %r = icmp eq i32 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i32 [ %x, %entry], [ 42, %t]
   ret i32 %ret
 }
 
 define i32 @test_set_mask_i32_i16(i32 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i32_i16:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
 ; CHECK-NEXT:    tst w0, #0x8000
-; CHECK-NEXT:    csel w0, w8, w0, ne
+; CHECK-NEXT:    csel w0, w0, w8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i32 %x, 32768
   %r = icmp ne i32 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i32 [ %x, %entry], [ 42, %t]
   ret i32 %ret
 }
 
 define i32 @test_clear_mask_i32_i8(i32 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i32_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
 ; CHECK-NEXT:    tst w0, #0x80
-; CHECK-NEXT:    csel w0, w8, w0, eq
+; CHECK-NEXT:    csel w0, w0, w8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i32 %x, 128
   %r = icmp eq i32 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i32 [ %x, %entry], [ 42, %t]
   ret i32 %ret
 }
 
 define i32 @test_set_mask_i32_i8(i32 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i32_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
 ; CHECK-NEXT:    tst w0, #0x80
-; CHECK-NEXT:    csel w0, w8, w0, ne
+; CHECK-NEXT:    csel w0, w0, w8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i32 %x, 128
   %r = icmp ne i32 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i32 [ %x, %entry], [ 42, %t]
   ret i32 %ret
 }
 
 define i16 @test_clear_mask_i16_i8(i16 %x) nounwind {
 ; CHECK-LABEL: test_clear_mask_i16_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    and w9, w0, #0xffff
 ; CHECK-NEXT:    tst w0, #0x80
-; CHECK-NEXT:    csel w0, w8, w0, eq
+; CHECK-NEXT:    csel w0, w9, w8, ne
 ; CHECK-NEXT:    ret
 entry:
   %a = and i16 %x, 128
   %r = icmp eq i16 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i16 [ %x, %entry], [ 42, %t]
   ret i16 %ret
 }
 
 define i16 @test_set_mask_i16_i8(i16 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i16_i8:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    and w9, w0, #0xffff
 ; CHECK-NEXT:    tst w0, #0x80
-; CHECK-NEXT:    csel w0, w8, w0, ne
+; CHECK-NEXT:    csel w0, w9, w8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i16 %x, 128
   %r = icmp ne i16 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i16 [ %x, %entry], [ 42, %t]
   ret i16 %ret
 }
 
 define i16 @test_set_mask_i16_i7(i16 %x) nounwind {
 ; CHECK-LABEL: test_set_mask_i16_i7:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #42
+; CHECK-NEXT:    mov w8, #42 // =0x2a
+; CHECK-NEXT:    and w9, w0, #0xffff
 ; CHECK-NEXT:    tst w0, #0x40
-; CHECK-NEXT:    csel w0, w8, w0, ne
+; CHECK-NEXT:    csel w0, w9, w8, eq
 ; CHECK-NEXT:    ret
 entry:
   %a = and i16 %x, 64
   %r = icmp ne i16 %a, 0
   br i1 %r, label %t, label %f
 t:
   br label %f
 f:
   %ret = phi i16 [ %x, %entry], [ 42, %t]
   ret i16 %ret
 }

llvm/test/CodeGen/AArch64/statepoint-call-lowering.ll

 ; CHECK-NEXT:    str x0, [sp, #8]
 ; CHECK-NEXT:    bl return_i1
 ; CHECK-NEXT:  .Ltmp8:
 ; CHECK-NEXT:    tbz w20, #0, .LBB8_2
 ; CHECK-NEXT:  // %bb.1: // %left
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    ldr x0, [sp, #8]
 ; CHECK-NEXT:    bl consume
-; CHECK-NEXT:    b .LBB8_3
-; CHECK-NEXT:  .LBB8_2:
-; CHECK-NEXT:    mov w19, #1 // =0x1
-; CHECK-NEXT:  .LBB8_3: // %common.ret
 ; CHECK-NEXT:    and w0, w19, #0x1
+; CHECK-NEXT:    b .LBB8_3
+; CHECK-NEXT:  .LBB8_2: // %right
+; CHECK-NEXT:    mov w0, #1 // =0x1
+; CHECK-NEXT:  .LBB8_3: // %right
 ; CHECK-NEXT:    ldp x20, x19, [sp, #16] // 16-byte Folded Reload
 ; CHECK-NEXT:    ldr x30, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
 entry:
   %safepoint_token = tail call token (i64, i32, ptr, i32, i32, ...) @llvm.experimental.gc.statepoint.p0(i64 0, i32 0, ptr elementtype(i1 ()) @return_i1, i32 0, i32 0, i32 0, i32 0) ["gc-live" (ptr addrspace(1) %a)]
   br i1 %external_cond, label %left, label %right
 
 left:

llvm/test/CodeGen/AArch64/sve-breakdown-scalable-vectortype.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64--linux-gnu -mattr=+sve < %s | FileCheck %s
 
 ; Test that scalable vectors that are a multiple of the legal vector size
 ; can be properly broken down into part vectors.
 
 declare aarch64_sve_vector_pcs void @bar()
 
 ;
 ; Vectors twice the size
 ;
 
 define <vscale x 32 x i8> @wide_32i8(i1 %b, <vscale x 16 x i8> %legal, <vscale x 32 x i8> %illegal) nounwind {
 ; CHECK-LABEL: wide_32i8:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB0_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB0_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 32 x i8> undef
 L2:
   ret <vscale x 32 x i8> %illegal
 }
 
 define <vscale x 16 x i16> @wide_16i16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 16 x i16> %illegal) nounwind {
 ; CHECK-LABEL: wide_16i16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB1_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB1_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB1_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 16 x i16> undef
 L2:
   ret <vscale x 16 x i16> %illegal
 }
 
 define <vscale x 8 x i32> @wide_8i32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 8 x i32> %illegal) nounwind {
 ; CHECK-LABEL: wide_8i32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB2_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB2_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB2_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 8 x i32> undef
 L2:
   ret <vscale x 8 x i32> %illegal
 }
 
 define <vscale x 4 x i64> @wide_4i64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 4 x i64> %illegal) nounwind {
 ; CHECK-LABEL: wide_4i64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB3_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB3_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 4 x i64> undef
 L2:
   ret <vscale x 4 x i64> %illegal
 }
 
 define <vscale x 16 x half> @wide_16f16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 16 x half> %illegal) nounwind {
 ; CHECK-LABEL: wide_16f16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB4_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB4_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB4_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 16 x half> undef
 L2:
   ret <vscale x 16 x half> %illegal
 }
 
 define <vscale x 8 x float> @wide_8f32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 8 x float> %illegal) nounwind {
 ; CHECK-LABEL: wide_8f32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB5_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB5_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB5_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 8 x float> undef
 L2:
   ret <vscale x 8 x float> %illegal
 }
 
 define <vscale x 4 x double> @wide_4f64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 4 x double> %illegal) nounwind {
 ; CHECK-LABEL: wide_4f64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-2
-; CHECK-NEXT:    str z9, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z1.d
-; CHECK-NEXT:    str z8, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z2.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB6_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB6_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z9.d
-; CHECK-NEXT:    mov z1.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #2
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB6_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 4 x double> undef
 L2:
   ret <vscale x 4 x double> %illegal
 }
 
 ;
 ; Vectors three times the size
 ;
 
 define <vscale x 48 x i8> @wide_48i8(i1 %b, <vscale x 16 x i8> %legal, <vscale x 48 x i8> %illegal) nounwind {
 ; CHECK-LABEL: wide_48i8:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB7_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB7_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB7_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 48 x i8> undef
 L2:
   ret <vscale x 48 x i8> %illegal
 }
 
 define <vscale x 24 x i16> @wide_24i16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 24 x i16> %illegal) nounwind {
 ; CHECK-LABEL: wide_24i16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB8_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB8_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB8_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 24 x i16> undef
 L2:
   ret <vscale x 24 x i16> %illegal
 }
 
 define <vscale x 12 x i32> @wide_12i32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 12 x i32> %illegal) nounwind {
 ; CHECK-LABEL: wide_12i32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB9_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB9_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB9_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 12 x i32> undef
 L2:
   ret <vscale x 12 x i32> %illegal
 }
 
 define <vscale x 6 x i64> @wide_6i64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 6 x i64> %illegal) nounwind {
 ; CHECK-LABEL: wide_6i64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB10_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB10_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB10_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 6 x i64> undef
 L2:
   ret <vscale x 6 x i64> %illegal
 }
 
 define <vscale x 24 x half> @wide_24f16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 24 x half> %illegal) nounwind {
 ; CHECK-LABEL: wide_24f16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB11_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB11_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB11_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 24 x half> undef
 L2:
   ret <vscale x 24 x half> %illegal
 }
 
 define <vscale x 12 x float> @wide_12f32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 12 x float> %illegal) nounwind {
 ; CHECK-LABEL: wide_12f32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB12_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB12_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB12_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 12 x float> undef
 L2:
   ret <vscale x 12 x float> %illegal
 }
 
 define <vscale x 6 x double> @wide_6f64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 6 x double> %illegal) nounwind {
 ; CHECK-LABEL: wide_6f64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-3
-; CHECK-NEXT:    str z10, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z1.d
-; CHECK-NEXT:    str z9, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z2.d
-; CHECK-NEXT:    str z8, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z3.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB13_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB13_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z10.d
-; CHECK-NEXT:    mov z1.d, z9.d
-; CHECK-NEXT:    ldr z10, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z8.d
-; CHECK-NEXT:    ldr z9, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #3
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB13_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 6 x double> undef
 L2:
   ret <vscale x 6 x double> %illegal
 }
 
 ;
 ; Vectors four times the size
 ;
 
 define <vscale x 64 x i8> @wide_64i8(i1 %b, <vscale x 16 x i8> %legal, <vscale x 64 x i8> %illegal) nounwind {
 ; CHECK-LABEL: wide_64i8:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB14_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB14_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB14_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 64 x i8> undef
 L2:
   ret <vscale x 64 x i8> %illegal
 }
 
 define <vscale x 32 x i16> @wide_32i16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 32 x i16> %illegal) nounwind {
 ; CHECK-LABEL: wide_32i16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB15_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB15_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB15_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 32 x i16> undef
 L2:
   ret <vscale x 32 x i16> %illegal
 }
 
 define <vscale x 16 x i32> @wide_16i32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 16 x i32> %illegal) nounwind {
 ; CHECK-LABEL: wide_16i32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB16_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB16_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB16_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 16 x i32> undef
 L2:
   ret <vscale x 16 x i32> %illegal
 }
 
 define <vscale x 8 x i64> @wide_8i64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 8 x i64> %illegal) nounwind {
 ; CHECK-LABEL: wide_8i64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB17_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB17_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB17_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 8 x i64> undef
 L2:
   ret <vscale x 8 x i64> %illegal
 }
 
 define <vscale x 32 x half> @wide_32f16(i1 %b, <vscale x 16 x i8> %legal, <vscale x 32 x half> %illegal) nounwind {
 ; CHECK-LABEL: wide_32f16:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB18_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB18_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB18_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 32 x half> undef
 L2:
   ret <vscale x 32 x half> %illegal
 }
 
 define <vscale x 16 x float> @wide_16f32(i1 %b, <vscale x 16 x i8> %legal, <vscale x 16 x float> %illegal) nounwind {
 ; CHECK-LABEL: wide_16f32:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB19_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB19_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB19_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 16 x float> undef
 L2:
   ret <vscale x 16 x float> %illegal
 }
 
 define <vscale x 8 x double> @wide_8f64(i1 %b, <vscale x 16 x i8> %legal, <vscale x 8 x double> %illegal) nounwind {
 ; CHECK-LABEL: wide_8f64:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
-; CHECK-NEXT:    addvl sp, sp, #-4
-; CHECK-NEXT:    str z11, [sp] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z11.d, z1.d
-; CHECK-NEXT:    str z10, [sp, #1, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z10.d, z2.d
-; CHECK-NEXT:    str z9, [sp, #2, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z9.d, z3.d
-; CHECK-NEXT:    str z8, [sp, #3, mul vl] // 16-byte Folded Spill
-; CHECK-NEXT:    mov z8.d, z4.d
 ; CHECK-NEXT:    tbz w0, #0, .LBB20_2
 ; CHECK-NEXT:  // %bb.1: // %L1
+; CHECK-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    bl bar
-; CHECK-NEXT:  .LBB20_2: // %common.ret
-; CHECK-NEXT:    mov z0.d, z11.d
-; CHECK-NEXT:    mov z1.d, z10.d
-; CHECK-NEXT:    ldr z11, [sp] // 16-byte Folded Reload
-; CHECK-NEXT:    mov z2.d, z9.d
-; CHECK-NEXT:    mov z3.d, z8.d
-; CHECK-NEXT:    ldr z10, [sp, #1, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z9, [sp, #2, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    ldr z8, [sp, #3, mul vl] // 16-byte Folded Reload
-; CHECK-NEXT:    addvl sp, sp, #4
-; CHECK-NEXT:    ldp x29, x30, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB20_2: // %L2
+; CHECK-NEXT:    mov z0.d, z1.d
+; CHECK-NEXT:    mov z1.d, z2.d
+; CHECK-NEXT:    mov z2.d, z3.d
+; CHECK-NEXT:    mov z3.d, z4.d
 ; CHECK-NEXT:    ret
   br i1 %b, label %L1, label %L2
 L1:
   call aarch64_sve_vector_pcs void @bar()
   ret <vscale x 8 x double> undef
 L2:
   ret <vscale x 8 x double> %illegal
 }

llvm/test/CodeGen/AArch64/sve-lsr-scaled-index-addressing-mode.ll

 ; ASM-NEXT:  .LBB0_1: // %loop
 ; ASM-NEXT:    // =>This Inner Loop Header: Depth=1
 ; ASM-NEXT:    ld1h { z1.h }, p0/z, [x0, x8, lsl #1]
 ; ASM-NEXT:    add z1.h, z1.h, z0.h
 ; ASM-NEXT:    st1h { z1.h }, p0, [x1, x8, lsl #1]
 ; ASM-NEXT:    add x8, x8, x9
 ; ASM-NEXT:    cmp x8, #1024
 ; ASM-NEXT:    b.ne .LBB0_1
-; ASM-NEXT:  // %bb.2: // %exit
+; ASM-NEXT:  // %bb.2: // %loop.exit
 ; ASM-NEXT:    ret
 entry:
   br label %loop.ph
 
 loop.ph:
   %p_vec.splatinsert = insertelement <vscale x 8 x i16> undef, i16 3, i32 0
   %p_vec.splat = shufflevector <vscale x 8 x i16> %p_vec.splatinsert, <vscale x 8 x i16> undef, <vscale x 8 x i32> zeroinitializer
   %vscale = call i64 @llvm.vscale.i64()

 ; ASM-NEXT:  .LBB1_1: // %loop
 ; ASM-NEXT:    // =>This Inner Loop Header: Depth=1
 ; ASM-NEXT:    ld1h { z1.h }, p0/z, [x0, x8, lsl #1]
 ; ASM-NEXT:    add z1.h, z1.h, z0.h
 ; ASM-NEXT:    st1h { z1.h }, p0, [x1, x8, lsl #1]
 ; ASM-NEXT:    add x8, x8, x9
 ; ASM-NEXT:    cmp x2, x8
 ; ASM-NEXT:    b.ne .LBB1_1
-; ASM-NEXT:  // %bb.2: // %exit
+; ASM-NEXT:  // %bb.2: // %loop.exit
 ; ASM-NEXT:    ret
 entry:
   br label %loop.ph
 
 loop.ph:
   %p_vec.splatinsert = insertelement <vscale x 8 x i16> undef, i16 3, i32 0
   %p_vec.splat = shufflevector <vscale x 8 x i16> %p_vec.splatinsert, <vscale x 8 x i16> undef, <vscale x 8 x i32> zeroinitializer
   %ptrue_vec.splatinsert = insertelement <vscale x 8 x i1> undef, i1 true, i32 0

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-subvector.ll

 ; ensuring insert/extract sequences are not folded away.
 
 target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-linux-gnu"
 
 ; i8
 define void @subvector_v4i8(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4i8:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ptrue p0.h, vl4
 ; CHECK-NEXT:    ld1b { z0.h }, p0/z, [x0]
 ; CHECK-NEXT:    st1b { z0.h }, p0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x i8>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x i8> %a, ptr %out
   ret void
 }
 
 define void @subvector_v8i8(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v8i8:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr d0, [x0]
 ; CHECK-NEXT:    str d0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <8 x i8>, ptr %in
   br label %bb1
 
 bb1:
   store <8 x i8> %a, ptr %out
   ret void
 }
 
 define void @subvector_v16i8(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v16i8:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <16 x i8>, ptr %in
   br label %bb1
 
 bb1:
   store <16 x i8> %a, ptr %out
   ret void
 }
 
 define void @subvector_v32i8(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v32i8:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <32 x i8>, ptr %in
   br label %bb1
 
 bb1:
   store <32 x i8> %a, ptr %out
   ret void
 }
 
 ; i16
 define void @subvector_v2i16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2i16:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    sub sp, sp, #16
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    ldrh w8, [x0, #2]
 ; CHECK-NEXT:    ptrue p0.s, vl2
 ; CHECK-NEXT:    str w8, [sp, #12]
 ; CHECK-NEXT:    ldrh w8, [x0]
 ; CHECK-NEXT:    str w8, [sp, #8]
 ; CHECK-NEXT:    ldr d0, [sp, #8]
 ; CHECK-NEXT:    st1h { z0.s }, p0, [x1]
 ; CHECK-NEXT:    add sp, sp, #16
 ; CHECK-NEXT:    ret
   %a = load <2 x i16>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x i16> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4i16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4i16:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr d0, [x0]
 ; CHECK-NEXT:    str d0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x i16>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x i16> %a, ptr %out
   ret void
 }
 
 define void @subvector_v8i16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v8i16:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <8 x i16>, ptr %in
   br label %bb1
 
 bb1:
   store <8 x i16> %a, ptr %out
   ret void
 }
 
 define void @subvector_v16i16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v16i16:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <16 x i16>, ptr %in
   br label %bb1
 
 bb1:
   store <16 x i16> %a, ptr %out
   ret void
 }
 
 ; i32
 define void @subvector_v2i32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2i32:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr d0, [x0]
 ; CHECK-NEXT:    str d0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <2 x i32>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x i32> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4i32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4i32:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x i32>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x i32> %a, ptr %out
   ret void
 }
 
 define void @subvector_v8i32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v8i32:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <8 x i32>, ptr %in
   br label %bb1
 
 bb1:
   store <8 x i32> %a, ptr %out
   ret void
 }
 
 ; i64
 define void @subvector_v2i64(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2i64:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <2 x i64>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x i64> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4i64(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4i64:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x i64>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x i64> %a, ptr %out
   ret void
 }
 
 ; f16
 define void @subvector_v2f16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2f16:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldr w8, [x0]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldr s0, [x0]
+; CHECK-NEXT:    fmov w8, s0
 ; CHECK-NEXT:    str w8, [x1]
 ; CHECK-NEXT:    ret
   %a = load <2 x half>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x half> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4f16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4f16:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr d0, [x0]
 ; CHECK-NEXT:    str d0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x half>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x half> %a, ptr %out
   ret void
 }
 
 define void @subvector_v8f16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v8f16:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <8 x half>, ptr %in
   br label %bb1
 
 bb1:
   store <8 x half> %a, ptr %out
   ret void
 }
 
 define void @subvector_v16f16(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v16f16:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <16 x half>, ptr %in
   br label %bb1
 
 bb1:
   store <16 x half> %a, ptr %out
   ret void
 }
 
 ; f32
 define void @subvector_v2f32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2f32:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr d0, [x0]
 ; CHECK-NEXT:    str d0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <2 x float>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x float> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4f32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4f32:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x float>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x float> %a, ptr %out
   ret void
 }
 
 define void @subvector_v8f32(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v8f32:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <8 x float>,ptr %in
   br label %bb1
 
 bb1:
   store <8 x float> %a, ptr %out
   ret void
 }
 
 ; f64
 define void @subvector_v2f64(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v2f64:
-; CHECK:       // %bb.0: // %bb1
+; CHECK:       // %bb.0:
 ; CHECK-NEXT:    ldr q0, [x0]
 ; CHECK-NEXT:    str q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <2 x double>, ptr %in
   br label %bb1
 
 bb1:
   store <2 x double> %a, ptr %out
   ret void
 }
 
 define void @subvector_v4f64(ptr %in, ptr %out) {
 ; CHECK-LABEL: subvector_v4f64:
-; CHECK:       // %bb.0: // %bb1
-; CHECK-NEXT:    ldp q0, q1, [x0]
-; CHECK-NEXT:    stp q0, q1, [x1]
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldp q1, q0, [x0]
+; CHECK-NEXT:    stp q1, q0, [x1]
 ; CHECK-NEXT:    ret
   %a = load <4 x double>, ptr %in
   br label %bb1
 
 bb1:
   store <4 x double> %a, ptr %out
   ret void
 }

llvm/test/CodeGen/AArch64/swifterror.ll

 ; CHECK-APPLE-NEXT:    mov x29, sp
 ; CHECK-APPLE-NEXT:    .cfi_def_cfa w29, 16
 ; CHECK-APPLE-NEXT:    .cfi_offset w30, -8
 ; CHECK-APPLE-NEXT:    .cfi_offset w29, -16
 ; CHECK-APPLE-NEXT:    cbz w0, LBB3_2
 ; CHECK-APPLE-NEXT:  ; %bb.1: ; %gen_error
 ; CHECK-APPLE-NEXT:    mov w0, #16 ; =0x10
 ; CHECK-APPLE-NEXT:    bl _malloc
-; CHECK-APPLE-NEXT:    mov x21, x0
 ; CHECK-APPLE-NEXT:    fmov s0, #1.00000000
 ; CHECK-APPLE-NEXT:    mov w8, #1 ; =0x1
+; CHECK-APPLE-NEXT:    mov x21, x0
 ; CHECK-APPLE-NEXT:    strb w8, [x0, #8]
 ; CHECK-APPLE-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-APPLE-NEXT:    ret
-; CHECK-APPLE-NEXT:  LBB3_2:
+; CHECK-APPLE-NEXT:  LBB3_2: ; %normal
 ; CHECK-APPLE-NEXT:    movi d0, #0000000000000000
 ; CHECK-APPLE-NEXT:    ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
 ; CHECK-APPLE-NEXT:    ret
 ;
 ; CHECK-O0-AARCH64-LABEL: foo_if:
 ; CHECK-O0-AARCH64:       ; %bb.0: ; %entry
 ; CHECK-O0-AARCH64-NEXT:    sub sp, sp, #32
 ; CHECK-O0-AARCH64-NEXT:    stp x29, x30, [sp, #16] ; 16-byte Folded Spill

llvm/test/CodeGen/AArch64/tbl-loops.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64 < %s | FileCheck %s
 
 define void @loop1(ptr noalias nocapture noundef writeonly %dst, ptr nocapture noundef readonly %data, i32 noundef %width) {
 ; CHECK-LABEL: loop1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    subs w8, w2, #1
 ; CHECK-NEXT:    b.lt .LBB0_8
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w8, #6
-; CHECK-NEXT:    b.hi .LBB0_3
+; CHECK-NEXT:    cmp w8, #7
+; CHECK-NEXT:    b.hs .LBB0_3
 ; CHECK-NEXT:  // %bb.2:
 ; CHECK-NEXT:    mov w10, wzr
 ; CHECK-NEXT:    mov x8, x1
 ; CHECK-NEXT:    mov x9, x0
 ; CHECK-NEXT:    b .LBB0_6
 ; CHECK-NEXT:  .LBB0_3: // %vector.ph
 ; CHECK-NEXT:    add x11, x8, #1
 ; CHECK-NEXT:    mov w8, #1132396544 // =0x437f0000

 ; CHECK-NEXT:    xtn v2.8b, v2.8h
 ; CHECK-NEXT:    mov v1.s[1], v2.s[0]
 ; CHECK-NEXT:    stur d1, [x12, #-4]
 ; CHECK-NEXT:    add x12, x12, #8
 ; CHECK-NEXT:    b.ne .LBB0_4
 ; CHECK-NEXT:  // %bb.5: // %middle.block
 ; CHECK-NEXT:    cmp x11, x10
 ; CHECK-NEXT:    b.eq .LBB0_8
-; CHECK-NEXT:  .LBB0_6: // %for.body.preheader1
+; CHECK-NEXT:  .LBB0_6: // %for.body.preheader21
 ; CHECK-NEXT:    movi d0, #0000000000000000
 ; CHECK-NEXT:    sub w10, w2, w10
 ; CHECK-NEXT:    mov w11, #1132396544 // =0x437f0000
 ; CHECK-NEXT:  .LBB0_7: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    fmov s2, w11
 ; CHECK-NEXT:    ldr s1, [x8], #4
 ; CHECK-NEXT:    fcmp s1, s2

 }
 
 define void @loop2(ptr noalias nocapture noundef writeonly %dst, ptr nocapture noundef readonly %data, i32 noundef %width) {
 ; CHECK-LABEL: loop2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    subs w8, w2, #1
 ; CHECK-NEXT:    b.lt .LBB1_7
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w8, #2
-; CHECK-NEXT:    b.ls .LBB1_4
+; CHECK-NEXT:    cmp w8, #3
+; CHECK-NEXT:    b.lo .LBB1_4
 ; CHECK-NEXT:  // %bb.2: // %vector.memcheck
 ; CHECK-NEXT:    ubfiz x9, x8, #1, #32
 ; CHECK-NEXT:    add x9, x9, #2
 ; CHECK-NEXT:    add x10, x1, x9, lsl #2
 ; CHECK-NEXT:    cmp x10, x0
 ; CHECK-NEXT:    b.ls .LBB1_8
 ; CHECK-NEXT:  // %bb.3: // %vector.memcheck
 ; CHECK-NEXT:    add x9, x0, x9
 ; CHECK-NEXT:    cmp x9, x1
 ; CHECK-NEXT:    b.ls .LBB1_8
 ; CHECK-NEXT:  .LBB1_4:
 ; CHECK-NEXT:    mov w10, wzr
 ; CHECK-NEXT:    mov x8, x1
 ; CHECK-NEXT:    mov x9, x0
-; CHECK-NEXT:  .LBB1_5: // %for.body.preheader1
+; CHECK-NEXT:  .LBB1_5: // %for.body.preheader35
 ; CHECK-NEXT:    movi d0, #0000000000000000
 ; CHECK-NEXT:    sub w10, w2, w10
 ; CHECK-NEXT:    mov w11, #1132396544 // =0x437f0000
 ; CHECK-NEXT:  .LBB1_6: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp s1, s3, [x8], #8
 ; CHECK-NEXT:    fmov s2, w11
 ; CHECK-NEXT:    fcmp s1, s2

 }
 
 define void @loop3(ptr noalias nocapture noundef writeonly %dst, ptr nocapture noundef readonly %data, i32 noundef %width) {
 ; CHECK-LABEL: loop3:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    subs w8, w2, #1
 ; CHECK-NEXT:    b.lt .LBB2_9
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w8, #2
-; CHECK-NEXT:    b.ls .LBB2_6
+; CHECK-NEXT:    cmp w8, #3
+; CHECK-NEXT:    b.lo .LBB2_6
 ; CHECK-NEXT:  // %bb.2: // %vector.memcheck
 ; CHECK-NEXT:    add x9, x8, w8, uxtw #1
 ; CHECK-NEXT:    add x9, x9, #3
 ; CHECK-NEXT:    add x10, x1, x9, lsl #2
 ; CHECK-NEXT:    add x9, x0, x9
 ; CHECK-NEXT:    cmp x10, x0
 ; CHECK-NEXT:    ccmp x9, x1, #0, hi
 ; CHECK-NEXT:    b.hi .LBB2_6

 ; CHECK-NEXT:  // %bb.5: // %middle.block
 ; CHECK-NEXT:    cmp x11, x10
 ; CHECK-NEXT:    b.ne .LBB2_7
 ; CHECK-NEXT:    b .LBB2_9
 ; CHECK-NEXT:  .LBB2_6:
 ; CHECK-NEXT:    mov w10, wzr
 ; CHECK-NEXT:    mov x8, x1
 ; CHECK-NEXT:    mov x9, x0
-; CHECK-NEXT:  .LBB2_7: // %for.body.preheader1
+; CHECK-NEXT:  .LBB2_7: // %for.body.preheader46
 ; CHECK-NEXT:    movi d0, #0000000000000000
 ; CHECK-NEXT:    sub w10, w2, w10
 ; CHECK-NEXT:    mov w11, #1132396544 // =0x437f0000
 ; CHECK-NEXT:  .LBB2_8: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp s1, s3, [x8]
 ; CHECK-NEXT:    fmov s2, w11
 ; CHECK-NEXT:    fcmp s1, s2

 }
 
 define void @loop4(ptr noalias nocapture noundef writeonly %dst, ptr nocapture noundef readonly %data, i32 noundef %width) {
 ; CHECK-LABEL: loop4:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    subs w8, w2, #1
 ; CHECK-NEXT:    b.lt .LBB3_7
 ; CHECK-NEXT:  // %bb.1: // %for.body.preheader
-; CHECK-NEXT:    cmp w8, #2
-; CHECK-NEXT:    b.ls .LBB3_4
+; CHECK-NEXT:    cmp w8, #3
+; CHECK-NEXT:    b.lo .LBB3_4
 ; CHECK-NEXT:  // %bb.2: // %vector.memcheck
 ; CHECK-NEXT:    ubfiz x9, x8, #2, #32
 ; CHECK-NEXT:    add x9, x9, #4
 ; CHECK-NEXT:    add x10, x1, x9, lsl #2
 ; CHECK-NEXT:    cmp x10, x0
 ; CHECK-NEXT:    b.ls .LBB3_8
 ; CHECK-NEXT:  // %bb.3: // %vector.memcheck
 ; CHECK-NEXT:    add x9, x0, x9
 ; CHECK-NEXT:    cmp x9, x1
 ; CHECK-NEXT:    b.ls .LBB3_8
 ; CHECK-NEXT:  .LBB3_4:
 ; CHECK-NEXT:    mov w10, wzr
 ; CHECK-NEXT:    mov x8, x1
 ; CHECK-NEXT:    mov x9, x0
-; CHECK-NEXT:  .LBB3_5: // %for.body.preheader1
+; CHECK-NEXT:  .LBB3_5: // %for.body.preheader57
 ; CHECK-NEXT:    movi d0, #0000000000000000
 ; CHECK-NEXT:    sub w10, w2, w10
 ; CHECK-NEXT:    mov w11, #1132396544 // =0x437f0000
 ; CHECK-NEXT:  .LBB3_6: // %for.body
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldp s1, s3, [x8]
 ; CHECK-NEXT:    fmov s2, w11
 ; CHECK-NEXT:    fcmp s1, s2

llvm/test/CodeGen/AArch64/typepromotion-phisret.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64 %s -o - | FileCheck %s
 
 define void @phi_feeding_phi_args(i8 %a, i8 %b) {
 ; CHECK-LABEL: phi_feeding_phi_args:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    and w8, w0, #0xff
 ; CHECK-NEXT:    and w9, w1, #0xff
 ; CHECK-NEXT:    cmp w8, w9
 ; CHECK-NEXT:    csel w8, w8, w9, hi
 ; CHECK-NEXT:  .LBB0_1: // %loop
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    lsl w9, w8, #1
 ; CHECK-NEXT:    sub w10, w8, #2
-; CHECK-NEXT:    cmp w8, #254
-; CHECK-NEXT:    csel w8, w10, w9, lo
+; CHECK-NEXT:    cmp w8, #253
+; CHECK-NEXT:    csel w8, w9, w10, hi
 ; CHECK-NEXT:    cmp w8, #255
 ; CHECK-NEXT:    b.ne .LBB0_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   %0 = icmp ugt i8 %a, %b
   br i1 %0, label %preheader, label %empty
 

 ; CHECK-LABEL: phi_feeding_phi_zeroext_args:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w0, w1
 ; CHECK-NEXT:    csel w8, w0, w1, hi
 ; CHECK-NEXT:  .LBB1_1: // %loop
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    lsl w9, w8, #1
 ; CHECK-NEXT:    sub w10, w8, #2
-; CHECK-NEXT:    cmp w8, #254
-; CHECK-NEXT:    csel w8, w10, w9, lo
+; CHECK-NEXT:    cmp w8, #253
+; CHECK-NEXT:    csel w8, w9, w10, hi
 ; CHECK-NEXT:    cmp w8, #255
 ; CHECK-NEXT:    b.ne .LBB1_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   %0 = icmp ugt i8 %a, %b
   br i1 %0, label %preheader, label %empty
 

 exit:                                             ; preds = %if.end
   ret void
 }
 
 define void @phi_i16() {
 ; CHECK-LABEL: phi_i16:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov w8, wzr
-; CHECK-NEXT:    mov w9, #1 // =0x1
 ; CHECK-NEXT:  .LBB2_1: // %loop
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmp w8, #128
-; CHECK-NEXT:    cinc w10, w9, lo
-; CHECK-NEXT:    add w8, w8, w10
+; CHECK-NEXT:    cmp w8, #127
+; CHECK-NEXT:    add w9, w8, #2
+; CHECK-NEXT:    csinc w8, w9, w8, ls
 ; CHECK-NEXT:    cmp w8, #253
 ; CHECK-NEXT:    b.lo .LBB2_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   br label %loop
 
 loop:                                             ; preds = %if.end, %entry

 exit:                                             ; preds = %if.end
   ret void
 }
 
 define i8 @ret_i8() {
 ; CHECK-LABEL: ret_i8:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov w0, wzr
-; CHECK-NEXT:    mov w8, #1 // =0x1
 ; CHECK-NEXT:  .LBB3_1: // %loop
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmp w0, #128
-; CHECK-NEXT:    cinc w9, w8, lo
-; CHECK-NEXT:    add w0, w0, w9
+; CHECK-NEXT:    cmp w0, #127
+; CHECK-NEXT:    add w8, w0, #2
+; CHECK-NEXT:    csinc w0, w8, w0, ls
 ; CHECK-NEXT:    cmp w0, #252
 ; CHECK-NEXT:    b.hi .LBB3_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   br label %loop
 
 loop:                                             ; preds = %if.end, %entry

 exit:                                             ; preds = %if.end
   ret i8 %inc2
 }
 
 define i16 @phi_multiple_undefs(i16 zeroext %arg) {
 ; CHECK-LABEL: phi_multiple_undefs:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    mov w8, wzr
-; CHECK-NEXT:    mov w9, #1 // =0x1
 ; CHECK-NEXT:  .LBB4_1: // %loop
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmp w8, #128
-; CHECK-NEXT:    cinc w10, w9, lo
-; CHECK-NEXT:    add w8, w8, w10
+; CHECK-NEXT:    cmp w8, #127
+; CHECK-NEXT:    add w9, w8, #2
+; CHECK-NEXT:    csinc w8, w9, w8, ls
 ; CHECK-NEXT:    cmp w8, #253
 ; CHECK-NEXT:    b.lo .LBB4_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret
 entry:
   br label %loop
 
 loop:                                             ; preds = %if.end, %entry

llvm/test/CodeGen/AArch64/use-cr-result-of-dom-icmp-st.ll

 
 target datalayout = "e-m:e-i64:64-n32:64"
 
 define i64 @ll_a_op_b__2(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_op_b__2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    lsl x8, x0, x1
 ; CHECK-NEXT:    cmn x8, #2
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB0_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB0_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i64 %a, %b
   %cmp = icmp sgt i64 %shl, -2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i64 %shl, -2
   %mul = select i1 %cmp2, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_op_b__1(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_op_b__1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    lsl x8, x0, x1
+; CHECK-NEXT:    tbnz x8, #63, .LBB1_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB1_2: // %if.end
 ; CHECK-NEXT:    cmn x8, #1
-; CHECK-NEXT:    csinc x9, x1, xzr, eq
-; CHECK-NEXT:    cmp x8, #0
-; CHECK-NEXT:    mul x9, x9, x0
-; CHECK-NEXT:    csel x0, x1, x9, ge
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i64 %a, %b
   %cmp = icmp sgt i64 %shl, -1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i64 %shl, -1
   %mul = select i1 %cmp2, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_op_b_0(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_op_b_0:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    lsl x8, x0, x1
 ; CHECK-NEXT:    cmp x8, #0
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB2_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB2_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i64 %a, %b
   %cmp = icmp sgt i64 %shl, 0
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i64 %shl, 0
   %mul = select i1 %cmp2, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_op_b_1(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_op_b_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    lsl x8, x0, x1
 ; CHECK-NEXT:    cmp x8, #1
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB3_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB3_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i64 %a, %b
   %cmp = icmp sgt i64 %shl, 1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i64 %shl, 1
   %mul = select i1 %cmp2, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_op_b_2(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_op_b_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    lsl x8, x0, x1
 ; CHECK-NEXT:    cmp x8, #2
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB4_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB4_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i64 %a, %b
   %cmp = icmp sgt i64 %shl, 2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i64 %shl, 2
   %mul = select i1 %cmp2, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a__2(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a__2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmn x0, #2
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB5_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB5_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i64 %a, -2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i64 %a, -2
   %mul = select i1 %cmp1, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a__1(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a__1:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    tbnz x0, #63, .LBB6_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB6_2: // %if.end
 ; CHECK-NEXT:    cmn x0, #1
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    cmp x0, #0
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, ge
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i64 %a, -1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i64 %a, -1
   %mul = select i1 %cmp1, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_0(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_0:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp x0, #0
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB7_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB7_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i64 %a, 0
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i64 %a, 0
   %mul = select i1 %cmp1, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_1(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp x0, #1
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB8_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB8_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i64 %a, 1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i64 %a, 1
   %mul = select i1 %cmp1, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @ll_a_2(i64 %a, i64 %b) {
 ; CHECK-LABEL: ll_a_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp x0, #2
-; CHECK-NEXT:    csinc x8, x1, xzr, eq
-; CHECK-NEXT:    mul x8, x8, x0
-; CHECK-NEXT:    csel x0, x1, x8, gt
+; CHECK-NEXT:    b.le .LBB9_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    mov x0, x1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB9_2: // %if.end
+; CHECK-NEXT:    csinc x8, x1, xzr, ge
+; CHECK-NEXT:    mul x0, x8, x0
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i64 %a, 2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i64 %a, 2
   %mul = select i1 %cmp1, i64 %b, i64 1
   %spec.select = mul nsw i64 %mul, %a
   ret i64 %spec.select
 
 return:                                           ; preds = %entry
   ret i64 %b
 }
 
 define i64 @i_a_op_b__2(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_op_b__2:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    lsl w8, w0, w1
 ; CHECK-NEXT:    cmn w8, #2
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    b.gt .LBB10_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB10_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i32 %a, %b
   %cmp = icmp sgt i32 %shl, -2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i32 %shl, -2
   %mul = select i1 %cmp2, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_op_b__1(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_op_b__1:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    lsl w8, w0, w1
+; CHECK-NEXT:    tbnz w8, #31, .LBB11_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    sxtw x0, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB11_2: // %if.end
 ; CHECK-NEXT:    cmn w8, #1
-; CHECK-NEXT:    csinc w9, w1, wzr, eq
-; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    mul w9, w9, w0
-; CHECK-NEXT:    csel w8, w1, w9, ge
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i32 %a, %b
   %cmp = icmp sgt i32 %shl, -1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i32 %shl, -1
   %mul = select i1 %cmp2, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_op_b_0(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_op_b_0:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    lsl w8, w0, w1
 ; CHECK-NEXT:    cmp w8, #0
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    b.le .LBB12_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    sxtw x0, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB12_2: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i32 %a, %b
   %cmp = icmp sgt i32 %shl, 0
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i32 %shl, 0
   %mul = select i1 %cmp2, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_op_b_1(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_op_b_1:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    lsl w8, w0, w1
 ; CHECK-NEXT:    cmp w8, #1
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    b.gt .LBB13_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB13_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i32 %a, %b
   %cmp = icmp sgt i32 %shl, 1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i32 %shl, 1
   %mul = select i1 %cmp2, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_op_b_2(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_op_b_2:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    lsl w8, w0, w1
 ; CHECK-NEXT:    cmp w8, #2
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    b.gt .LBB14_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB14_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %shl = shl i32 %a, %b
   %cmp = icmp sgt i32 %shl, 2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp2 = icmp eq i32 %shl, 2
   %mul = select i1 %cmp2, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a__2(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a__2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmn w0, #2
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
+; CHECK-NEXT:    b.gt .LBB15_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB15_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, -2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i32 %a, -2
   %mul = select i1 %cmp1, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a__1(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a__1:
 ; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
+; CHECK-NEXT:    tbnz w0, #31, .LBB16_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    sxtw x0, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB16_2: // %if.end
 ; CHECK-NEXT:    cmn w0, #1
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, ge
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, -1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i32 %a, -1
   %mul = select i1 %cmp1, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_0(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_0:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w0, #0
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
+; CHECK-NEXT:    b.le .LBB17_2
+; CHECK-NEXT:  // %bb.1: // %return
+; CHECK-NEXT:    sxtw x0, w1
+; CHECK-NEXT:    ret
+; CHECK-NEXT:  .LBB17_2: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, 0
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i32 %a, 0
   %mul = select i1 %cmp1, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_1(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w0, #1
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
+; CHECK-NEXT:    b.gt .LBB18_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB18_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, 1
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i32 %a, 1
   %mul = select i1 %cmp1, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }
 
 define i64 @i_a_2(i32 signext %a, i32 signext %b) {
 ; CHECK-LABEL: i_a_2:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    cmp w0, #2
-; CHECK-NEXT:    csinc w8, w1, wzr, eq
-; CHECK-NEXT:    mul w8, w8, w0
-; CHECK-NEXT:    csel w8, w1, w8, gt
-; CHECK-NEXT:    sxtw x0, w8
+; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
+; CHECK-NEXT:    b.gt .LBB19_2
+; CHECK-NEXT:  // %bb.1: // %if.end
+; CHECK-NEXT:    csinc w8, w1, wzr, ge
+; CHECK-NEXT:    mul w1, w8, w0
+; CHECK-NEXT:  .LBB19_2: // %return
+; CHECK-NEXT:    sxtw x0, w1
 ; CHECK-NEXT:    ret
 entry:
   %cmp = icmp sgt i32 %a, 2
   br i1 %cmp, label %return, label %if.end
 
 if.end:                                           ; preds = %entry
   %cmp1 = icmp eq i32 %a, 2
   %mul = select i1 %cmp1, i32 %b, i32 1
   %spec.select = mul nsw i32 %mul, %a
   br label %return
 
 return:                                           ; preds = %if.end, %entry
   %retval.0.in = phi i32 [ %b, %entry ], [ %spec.select, %if.end ]
   %retval.0 = sext i32 %retval.0.in to i64
   ret i64 %retval.0
 }

llvm/test/Transforms/AtomicExpand/AMDGPU/expand-atomic-simplify-cfg-CAS-block.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S -mtriple=amdgcn-amd-amdhsa -mcpu=gfx90a -atomic-expand %s | FileCheck -check-prefix=GFX90A %s
 
 declare i32 @llvm.amdgcn.workitem.id.x()
 
 define amdgpu_kernel void @divergent_cfg(ptr addrspace(1) %out, float %in) #0 {
 ; GFX90A-LABEL: @divergent_cfg(
 ; GFX90A-NEXT:  entry:
 ; GFX90A-NEXT:    [[TID:%.*]] = call i32 @llvm.amdgcn.workitem.id.x()
 ; GFX90A-NEXT:    [[D_CMP:%.*]] = icmp ult i32 [[TID]], 16
 ; GFX90A-NEXT:    br i1 [[D_CMP]], label [[IF:%.*]], label [[ELSE:%.*]]
 ; GFX90A:       if:
 ; GFX90A-NEXT:    [[TMP0:%.*]] = load float, ptr addrspace(1) [[OUT:%.*]], align 4
 ; GFX90A-NEXT:    br label [[ATOMICRMW_START:%.*]]
 ; GFX90A:       atomicrmw.start:
 ; GFX90A-NEXT:    [[LOADED:%.*]] = phi float [ [[TMP0]], [[IF]] ], [ [[TMP4:%.*]], [[ATOMICRMW_START]] ]
 ; GFX90A-NEXT:    [[NEW:%.*]] = fadd float [[LOADED]], [[IN:%.*]]
 ; GFX90A-NEXT:    [[TMP1:%.*]] = bitcast float [[NEW]] to i32
 ; GFX90A-NEXT:    [[TMP2:%.*]] = bitcast float [[LOADED]] to i32
 ; GFX90A-NEXT:    [[TMP3:%.*]] = cmpxchg ptr addrspace(1) [[OUT]], i32 [[TMP2]], i32 [[TMP1]] seq_cst seq_cst, align 4
 ; GFX90A-NEXT:    [[SUCCESS:%.*]] = extractvalue { i32, i1 } [[TMP3]], 1
 ; GFX90A-NEXT:    [[NEWLOADED:%.*]] = extractvalue { i32, i1 } [[TMP3]], 0
 ; GFX90A-NEXT:    [[TMP4]] = bitcast i32 [[NEWLOADED]] to float
-; GFX90A-NEXT:    br i1 [[SUCCESS]], label [[ATOMICRMW_END:%.*]], label [[ATOMICRMW_START]]
-; GFX90A:       atomicrmw.end:
-; GFX90A-NEXT:    br label [[ENDIF:%.*]]
+; GFX90A-NEXT:    br i1 [[SUCCESS]], label [[ENDIF:%.*]], label [[ATOMICRMW_START]]
 ; GFX90A:       else:
 ; GFX90A-NEXT:    [[TMP5:%.*]] = load float, ptr addrspace(1) [[OUT]], align 4
 ; GFX90A-NEXT:    br label [[ATOMICRMW_START2:%.*]]
 ; GFX90A:       atomicrmw.start2:
 ; GFX90A-NEXT:    [[LOADED3:%.*]] = phi float [ [[TMP5]], [[ELSE]] ], [ [[TMP9:%.*]], [[ATOMICRMW_START2]] ]
 ; GFX90A-NEXT:    [[NEW4:%.*]] = fadd float [[LOADED3]], [[IN]]
 ; GFX90A-NEXT:    [[TMP6:%.*]] = bitcast float [[NEW4]] to i32
 ; GFX90A-NEXT:    [[TMP7:%.*]] = bitcast float [[LOADED3]] to i32
 ; GFX90A-NEXT:    [[TMP8:%.*]] = cmpxchg ptr addrspace(1) [[OUT]], i32 [[TMP7]], i32 [[TMP6]] seq_cst seq_cst, align 4
 ; GFX90A-NEXT:    [[SUCCESS5:%.*]] = extractvalue { i32, i1 } [[TMP8]], 1
 ; GFX90A-NEXT:    [[NEWLOADED6:%.*]] = extractvalue { i32, i1 } [[TMP8]], 0
 ; GFX90A-NEXT:    [[TMP9]] = bitcast i32 [[NEWLOADED6]] to float
-; GFX90A-NEXT:    br i1 [[SUCCESS5]], label [[ATOMICRMW_END1:%.*]], label [[ATOMICRMW_START2]]
-; GFX90A:       atomicrmw.end1:
-; GFX90A-NEXT:    br label [[ENDIF]]
+; GFX90A-NEXT:    br i1 [[SUCCESS5]], label [[ENDIF]], label [[ATOMICRMW_START2]]
 ; GFX90A:       endif:
-; GFX90A-NEXT:    [[COMBINE:%.*]] = phi float [ [[TMP4]], [[ATOMICRMW_END]] ], [ [[TMP9]], [[ATOMICRMW_END1]] ]
+; GFX90A-NEXT:    [[COMBINE:%.*]] = phi float [ [[TMP4]], [[ATOMICRMW_START]] ], [ [[TMP9]], [[ATOMICRMW_START2]] ]
 ; GFX90A-NEXT:    store float [[COMBINE]], ptr addrspace(1) [[OUT]], align 4
 ; GFX90A-NEXT:    ret void
 ;
 entry:
   %tid = call i32 @llvm.amdgcn.workitem.id.x()
   %d_cmp = icmp ult i32 %tid, 16
   br i1 %d_cmp, label %if, label %else
 

llvm/test/Transforms/CodeGenPrepare/AArch64/large-offset-gep.ll

 while_end:
   ret void
 }
 
 define void @test3(ptr %s1, ptr %s2, i1 %cond, i32 %n) {
 ; CHECK-LABEL: test3:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    tst w2, #0x1
-; CHECK-NEXT:    csel x9, x1, x0, ne
+; CHECK-NEXT:    csel x9, x0, x1, eq
 ; CHECK-NEXT:    cbz x9, .LBB2_3
 ; CHECK-NEXT:  // %bb.1: // %while_cond.preheader
 ; CHECK-NEXT:    mov w10, #40000 // =0x9c40
 ; CHECK-NEXT:    mov w8, wzr
 ; CHECK-NEXT:    add x9, x9, x10
 ; CHECK-NEXT:    cmp w8, w3
 ; CHECK-NEXT:    b.ge .LBB2_3
 ; CHECK-NEXT:  .LBB2_2: // %while_body

llvm/test/Transforms/LoopStrengthReduce/AArch64/lsr-pre-inc-offset-check.ll

 ; when building a bootstrap build on an AArch64 machine, where the
 ; preferred addresing mode is preindexed.
 
 
 %"Type" = type <{[166 x [338 x i8]]}>
 define void @test_lsr_pre_inc_offset_check(ptr %p) {
 ; CHECK-LABEL: test_lsr_pre_inc_offset_check:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    mov w8, #165
+; CHECK-NEXT:    mov w8, #165 // =0xa5
 ; CHECK-NEXT:    add x9, x0, #339
-; CHECK-NEXT:    mov w10, #2
-; CHECK-NEXT:  .LBB0_1: // %main
+; CHECK-NEXT:    mov w10, #2 // =0x2
+; CHECK-NEXT:  .LBB0_1: // %if.then
 ; CHECK-NEXT:    // =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    str wzr, [x9]
 ; CHECK-NEXT:    subs x8, x8, #1
 ; CHECK-NEXT:    strb w10, [x9, #1]
 ; CHECK-NEXT:    add x9, x9, #338
 ; CHECK-NEXT:    b.ne .LBB0_1
 ; CHECK-NEXT:  // %bb.2: // %exit
 ; CHECK-NEXT:    ret