[WIP][ScalarEvolution] Merge howManyGreaterThans with howManyLessThans.

[efriedma]

Summary

They're basically the same function, so let's not implement it twice. And they've fallen out of sync a bit: howManyLessThans has more features.

Still need to investigate:

1. We're replacing smin/umin expressions with less efficient smax/umax expressions. Not sure how we're supposed to canonicalize here.
2. The isLoopEntryGuardedByCond check is failing in places where it used to succeed.

[mkazantsev]

LGTM, but I think we can remove more copy-paste (see comment). Can go as follow-up.

llvm/lib/Analysis/ScalarEvolution.cpp

11813

Instead of this, can we just have smth like

ICmpInst::Predicate Cond = IsSigned ? ICmpInst::ICMP_SLT : ICmpInst::ICMP_ULT;
if (Invert)
  Cond = getSwappedPredicate(Cond);

? This piece still lppks copy-pastish.

[nikic]

There's a minor compile-time impact for this refactoring: https://llvm-compile-time-tracker.com/compare.php?from=57006d2f6d96d8a6836ae901218ed615071b3b8e&to=eca6ad2e31d754420fb114872bfc4060a698155e&stat=instructions It's okay, but you might want to cut down on the number of repeated/redundant getNotSCEV operations you perform. (They're not cached and known to be somewhat expensive.)

llvm/lib/Analysis/ScalarEvolution.cpp

11831

I'm confused by what you're doing here. getNotSCEV has a fold that does ~umin(~x, ~y) to umax(x, y), so the structure you're using here should get undone?

[reames]

Several level of comments.

First, the LT handling is substantially more powerful than the GT handling. e.g. for unknown strides. Given that, this is definitely NOT nfc, and should have accompanying test changes.

Second, I'd prefer this not go in until the underlying LT miscompile we've been discussing is addressed. The GT code handles this case differently, and I'm not sure that pushing GT through this code is safe.

Third, I'd suggest an alternate interface. Please remove the IsSigned and Invert parameters, and just pass through the condition. These two properties can be derived inside the method.

[efriedma]

llvm/lib/Analysis/ScalarEvolution.cpp

11831

getNotSCEV(RHS) and getNotSCEV(Start) will likely be simplified.

[efriedma]

Address review comments. (Still going to look at adding more tests.)

[efriedma]

Note to self: differences between old and new gt codepath that need tests:

1. Now compute max trip count even if RHS is not invariant.
2. Now more aggressive with strides that are not known positive (although this codepath currently has issues...)
3. Now use isUBOnWrap() for stride greater than one.
4. Now compute MaxOrZero.

[efriedma]

Added tests. A couple more cleanups.

[reames]

llvm/lib/Analysis/ScalarEvolution.cpp

11669

For sanity sake, please actually set Pred here so that if we use it below, we get what we expect.

11812

I'd prefer to make it more powerful if possible.

A couple ideas, not sure if these work out or not:

• Use SimplifyICmpOperands to canonicalize operands to isBasicBlockEntryGuardedByCond.
• Teach isLoopEntryGuardedByCond to undo your not(lhs/rhs) transform.

[efriedma]

llvm/lib/Analysis/ScalarEvolution.cpp

11812

There's actually another layer of complexity here on the main branch: if LHS/RHS are pointers, we have ptrtoint expressions floating around here, which isLoopEntryGuardedByCond also doesn't handle well.

After D105510 is merged, I can try messing with isLoopEntryGuardedByCond etc.

[efriedma]

Show simplified version. Not ready to merge.

[reames]

@efriedma Just to note, I think we've hit the point where howManyLessThans isn't getting any more stable in the near future, and howManyGreaterThans is at least as buggy. If you wanted to rebase this, I'd be up for reviewing with intent to land in near future.

[reames]

Pending action by author, please request review once ready for more discussion. Just getting this off my active review queue.

[lebedev.ri]

This review seems to be stuck/dead, consider abandoning if no longer relevant.

llvm/lib/Analysis/ScalarEvolution.cpp

if (RHS->getType()->isPointerTy()) {
       if (isa<SCEVCouldNotCompute>(RHS))
         return RHS;
     }
     ExitLimit EL = howFarToNonZero(getMinusSCEV(LHS, RHS), L);
     if (EL.hasAnyInfo()) return EL;
     break;
   }
   case ICmpInst::ICMP_SLT:
-  case ICmpInst::ICMP_ULT: {                    // while (X < Y)
-    ExitLimit EL =
-        howManyLessThans(LHS, RHS, L, Pred, ControlsExit, AllowPredicates);
-    if (EL.hasAnyInfo()) return EL;
-    break;
-  }
+  case ICmpInst::ICMP_ULT:
   case ICmpInst::ICMP_SGT:
-  case ICmpInst::ICMP_UGT: {                    // while (X > Y)
-    bool IsSigned = Pred == ICmpInst::ICMP_SGT;
+  case ICmpInst::ICMP_UGT: {
+    // while (X > Y) or while (X < Y)
     ExitLimit EL =
-        howManyGreaterThans(LHS, RHS, L, IsSigned, ControlsExit,
-                            AllowPredicates);
+        howManyLessThans(LHS, RHS, L, Pred, ControlsExit, AllowPredicates);
     if (EL.hasAnyInfo()) return EL;
     break;
   }
   default:
     break;
   }
 
   auto *ExhaustiveCount =

 }
 
 ScalarEvolution::ExitLimit
 ScalarEvolution::howManyLessThans(const SCEV *LHS, const SCEV *RHS,
                                   const Loop *L, ICmpInst::Predicate Pred,
                                   bool ControlsExit, bool AllowPredicates) {
   SmallPtrSet<const SCEVPredicate *, 4> Predicates;
 
-  assert(ICmpInst::isLT(Pred) && "Unexpected pred");
+  assert((ICmpInst::isGT(Pred) || ICmpInst::isLT(Pred)) && "Unexpected pred");
   bool IsSigned = ICmpInst::isSigned(Pred);
 
   const SCEVAddRecExpr *IV = dyn_cast<SCEVAddRecExpr>(LHS);
   bool PredicatedIV = false;
 
   if (!IV && AllowPredicates) {
     // Try to make this an AddRec using runtime tests, in the first X
     // iterations of this loop, where X is the SCEV expression found by the
     // algorithm below.
     IV = convertSCEVToAddRecWithPredicates(LHS, L, Predicates);
     PredicatedIV = true;
   }
 
   // Avoid weird loops
   if (!IV || IV->getLoop() != L || !IV->isAffine())
     return getCouldNotCompute();
 
+  // Force IV/RHS to integer types, so we can do arithmetic.
+  if (IV->getType()->isPointerTy()) {
+    const SCEV *IVCast = getLosslessPtrToIntExpr(IV);
+    if (isa<SCEVCouldNotCompute>(IVCast))
+      return IVCast;
+    IV = cast<SCEVAddRecExpr>(IVCast);
+  }
+  if (RHS->getType()->isPointerTy()) {
+    RHS = getLosslessPtrToIntExpr(RHS);
+    if (isa<SCEVCouldNotCompute>(RHS))
+      return RHS;
+  }
+
+  // If this is a greater-than comparison, invert the LHS/RHS to make it a
+  // less-than comparison.
+  if (ICmpInst::isGT(Pred)) {
+    IV = cast<SCEVAddRecExpr>(getNotSCEV(IV));
+    RHS = getNotSCEV(RHS);
+    Pred = ICmpInst::getSwappedPredicate(Pred);
+  }
+
   auto WrapType = IsSigned ? SCEV::FlagNSW : SCEV::FlagNUW;
   bool NoWrap = ControlsExit && IV->getNoWrapFlags(WrapType);
 
   const SCEV *Stride = IV->getStepRecurrence(*this);
 
   bool PositiveStride = isKnownPositive(Stride);
 
   // Avoid negative or zero stride values.
if (!PositiveStride) {
     // conditions exploit NoWrapFlags, allowing to optimize in presence of
     // undefined behaviors like the case of C language.
     if (canIVOverflowOnLT(RHS, Stride, IsSigned) && !isUBOnWrap())
       return getCouldNotCompute();
   }
 
   const SCEV *Start = IV->getStart();
 
-  // Preserve pointer-typed Start/RHS to pass to isLoopEntryGuardedByCond.
-  // Use integer-typed versions for actual computation.
-  const SCEV *OrigStart = Start;
-  const SCEV *OrigRHS = RHS;
-  if (Start->getType()->isPointerTy()) {
-    Start = getLosslessPtrToIntExpr(Start);
-    if (isa<SCEVCouldNotCompute>(Start))
-      return Start;
-  }
-  if (RHS->getType()->isPointerTy()) {
-    RHS = getLosslessPtrToIntExpr(RHS);
-    if (isa<SCEVCouldNotCompute>(RHS))
-      return RHS;
-  }
-
   // When the RHS is not invariant, we do not know the end bound of the loop and
   // cannot calculate the ExactBECount needed by ExitLimit. However, we can
   // calculate the MaxBECount, given the start, stride and max value for the end
   // bound of the loop (RHS), and the fact that IV does not overflow (which is
   // checked above).
   if (!isLoopInvariant(RHS, L)) {
     const SCEV *MaxBECount = computeMaxBECountForLT(
         Start, Stride, RHS, getTypeSizeInBits(LHS->getType()), IsSigned);
ScalarEvolution::howManyLessThans(const SCEV *LHS, const SCEV *RHS,
   // If the loop entry is guarded by the result of the backedge test of the
   // first loop iteration, then we know the backedge will be taken at least
   // once and so the backedge taken count is as above. If not then we use the
   // expression (max(End,Start)-Start)/Stride to describe the backedge count,
   // as if the backedge is taken at least once max(End,Start) is End and so the
   // result is as above, and if not max(End,Start) is Start so we get a backedge
   // count of zero.
   const SCEV *BECount;
-  if (isLoopEntryGuardedByCond(L, Pred, getMinusSCEV(OrigStart, Stride),
-                               OrigRHS))
+  if (isLoopEntryGuardedByCond(L, Pred, getMinusSCEV(Start, Stride), RHS))
     BECount = BECountIfBackedgeTaken;
   else {
     const SCEV *End;
     // If we know that RHS >= Start in the context of loop, then we know that
     // max(RHS, Start) = RHS at this point.
-    if (isLoopEntryGuardedByCond(L, ICmpInst::getInversePredicate(Pred),
-                                 OrigRHS, OrigStart))
+    if (isLoopEntryGuardedByCond(L, ICmpInst::getInversePredicate(Pred), RHS,
+                                 Start))
       End = RHS;
     else
       End = IsSigned ? getSMaxExpr(RHS, Start) : getUMaxExpr(RHS, Start);
     BECount = computeBECount(IsSigned, Start, End, Stride);
   }
 
   const SCEV *MaxBECount;
   bool MaxOrZero = false;
   if (isa<SCEVConstant>(BECount))
     MaxBECount = BECount;
   else if (isa<SCEVConstant>(BECountIfBackedgeTaken)) {
     // If we know exactly how many times the backedge will be taken if it's
     // taken at least once, then the backedge count will either be that or
     // zero.
     MaxBECount = BECountIfBackedgeTaken;
     MaxOrZero = true;
   } else {
     MaxBECount = computeMaxBECountForLT(
         Start, Stride, RHS, getTypeSizeInBits(LHS->getType()), IsSigned);
   }
 
   if (isa<SCEVCouldNotCompute>(MaxBECount) &&
       !isa<SCEVCouldNotCompute>(BECount))
     MaxBECount = getConstant(getUnsignedRangeMax(BECount));

llvm/test/Analysis/ScalarEvolution/lt-overflow.ll

 ; CHECK: Loop %for.body: Unpredictable backedge-taken count.
 ; CHECK: Determining loop execution counts for: @test_non_invariant_rhs
 ; CHECK: Loop %for.body: Unpredictable backedge-taken count.
 ; CHECK: Determining loop execution counts for: @test_abnormal_exit
 ; CHECK: Loop %for.body: Unpredictable backedge-taken count.
 ; CHECK: Determining loop execution counts for: @test_other_exit
 ; CHECK: Loop %for.body: <multiple exits> Unpredictable backedge-taken count.
 ; CHECK: Determining loop execution counts for: @test_gt
-; CHECK: Loop %for.body: Unpredictable backedge-taken count.
+; CHECK: Loop %for.body: backedge-taken count is ((((1 + (-1 * %S)) umax (-1 + (-1 * %N))) + %S) /u 2)
 
 define void @test(i32 %N) mustprogress {
 entry:
   br label %for.body
 
 for.body:
   %iv = phi i32 [ %iv.next, %for.body ], [ 0, %entry ]
   %iv.next = add i32 %iv, 2

llvm/test/Analysis/ScalarEvolution/max-trip-count.ll

 loop.exit:
   ret void
 }
 
 ; sgt with negative stride
 define void @changing_end_bound7(i32 %start, i32* %n_addr, i32* %addr) {
 ; CHECK-LABEL: Determining loop execution counts for: @changing_end_bound7
 ; CHECK: Loop %loop: Unpredictable backedge-taken count.
-; CHECK: Loop %loop: Unpredictable max backedge-taken count.
+; CHECK: Loop %loop: max backedge-taken count is -1
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ %start, %entry ], [ %iv.next, %loop ]
   %acc = phi i32 [ 0, %entry ], [ %acc.next, %loop ]
   %val = load atomic i32, i32* %addr unordered, align 4
   fence acquire

llvm/test/Analysis/ScalarEvolution/predicated-trip-count.ll

 ; true.
 
 ; CHECK: Classifying expressions for: @test2
 
 ; CHECK:      %i.0.ext = sext i16 %i.0 to i32
 ; CHECK-NEXT:    -->  (sext i16 {%Start,+,-1}<%bb3> to i32)
 ; CHECK:       Loop %bb3: Unpredictable backedge-taken count.
 ; CHECK-NEXT:  Loop %bb3: Unpredictable max backedge-taken count.
-; CHECK-NEXT:  Loop %bb3: Predicated backedge-taken count is (1 + (sext i16 %Start to i32) + (-1 * ((1 + (sext i16 %Start to i32))<nsw> smin %M)))
+; CHECK-NEXT:  Loop %bb3: Predicated backedge-taken count is (2 + (sext i16 %Start to i32) + ((-2 + (-1 * (sext i16 %Start to i32))<nsw>)<nsw> smax (-1 + (-1 * %M))))
 ; CHECK-NEXT:  Predicates:
 ; CHECK-NEXT:    {%Start,+,-1}<%bb3> Added Flags: <nssw>
 
 define void @test2(i32 %N, i32 %M, i16 %Start) {
 entry:
         br label %bb3
 
 bb:             ; preds = %bb3

llvm/test/Analysis/ScalarEvolution/smin-smax-folds.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
 ; RUN: opt -analyze -enable-new-pm=0 -scalar-evolution < %s | FileCheck %s
 ; RUN: opt -disable-output "-passes=print<scalar-evolution>" < %s 2>&1 | FileCheck %s
 
 ; Tests for smin & smax folds.
 
 ; Test case from PR1614.
 define i32 @test_PR1614(i32 %a, i32 %b, i32 %c) {
 ; CHECK-LABEL: 'test_PR1614'
-; CHECK: -->  (%a smax %b)
-; CHECK: -->  (%a smax %b smax %c)
-; CHECK-NOT: smax
+; CHECK-NEXT:  Classifying expressions for: @test_PR1614
+; CHECK-NEXT:    %B = select i1 %A, i32 %a, i32 %b
+; CHECK-NEXT:    --> (%a smax %b) U: full-set S: full-set
+; CHECK-NEXT:    %D = select i1 %C, i32 %B, i32 %c
+; CHECK-NEXT:    --> (%a smax %b smax %c) U: full-set S: full-set
+; CHECK-NEXT:  Determining loop execution counts for: @test_PR1614
+;
 
   %A = icmp sgt i32 %a, %b
   %B = select i1 %A, i32 %a, i32 %b
   %C = icmp sle i32 %c, %B
   %D = select i1 %C, i32 %B, i32 %c
   ret i32 %D
 }
 
for.cond.cleanup.loopexit:
   br label %for.cond.cleanup
 
 for.cond.cleanup:
   ret void
 }
 
 define void @smin_to_smax(i32 %n) {
 ; FIXME: ((-1 * (0 smin %n)) + %n)  is actually just  (0 smax %n)
-
 ; CHECK-LABEL: 'smin_to_smax'
 ; CHECK-NEXT:  Classifying expressions for: @smin_to_smax
 ; CHECK-NEXT:    %i.011 = phi i32 [ %n, %for.body.lr.ph ], [ %dec, %for.body ]
-; CHECK-NEXT:    --> {%n,+,-1}<nsw><%for.body> U: full-set S: full-set Exits: (0 smin %n) LoopDispositions: { %for.body: Computable }
+; CHECK-NEXT:    --> {%n,+,-1}<nsw><%for.body> U: full-set S: full-set Exits: (-1 + (-1 * (-1 smax (-1 + (-1 * %n))))<nsw>)<nsw> LoopDispositions: { %for.body: Computable }
 ; CHECK-NEXT:    %dec = add nsw i32 %i.011, -1
-; CHECK-NEXT:    --> {(-1 + %n),+,-1}<nw><%for.body> U: full-set S: full-set Exits: (-1 + (0 smin %n)) LoopDispositions: { %for.body: Computable }
+; CHECK-NEXT:    --> {(-1 + %n),+,-1}<nw><%for.body> U: full-set S: full-set Exits: (-2 + (-1 * (-1 smax (-1 + (-1 * %n))))<nsw>) LoopDispositions: { %for.body: Computable }
 ; CHECK-NEXT:  Determining loop execution counts for: @smin_to_smax
-; CHECK-NEXT:  Loop %for.body: backedge-taken count is ((-1 * (0 smin %n)) + %n)
+; CHECK-NEXT:  Loop %for.body: backedge-taken count is (1 + (-1 smax (-1 + (-1 * %n))) + %n)
 ; CHECK-NEXT:  Loop %for.body: max backedge-taken count is 2147483647
-; CHECK-NEXT:  Loop %for.body: Predicated backedge-taken count is ((-1 * (0 smin %n)) + %n)
+; CHECK-NEXT:  Loop %for.body: Predicated backedge-taken count is (1 + (-1 smax (-1 + (-1 * %n))) + %n)
 ; CHECK-NEXT:   Predicates:
 ; CHECK:       Loop %for.body: Trip multiple is 1
 ;
 entry:
   br label %for.body.lr.ph
 
 for.body.lr.ph:
   br label %for.body
for.cond.cleanup.loopexit:
   br label %for.cond.cleanup
 
 for.cond.cleanup:
   ret void
 }
 
 ; The information from the loop guard can be used to simplify the trip count expression.
 define void @smax_simplify_with_guard(i32 %start, i32 %n) {
-; CHECK-LABEL:  'smax_simplify_with_guard'
+; CHECK-LABEL: 'smax_simplify_with_guard'
 ; CHECK-NEXT:  Classifying expressions for: @smax_simplify_with_guard
 ; CHECK-NEXT:    %k.0.i26 = phi i32 [ %start, %loop.ph ], [ %inc.i, %loop ]
-; CHECK-NEXT:    -->  {%start,+,1}<nsw><%loop> U: full-set S: full-set      Exits: %n     LoopDispositions: { %loop: Computable }
+; CHECK-NEXT:    --> {%start,+,1}<nsw><%loop> U: full-set S: full-set Exits: %n LoopDispositions: { %loop: Computable }
 ; CHECK-NEXT:    %inc.i = add nsw i32 %k.0.i26, 1
-; CHECK-NEXT:    -->  {(1 + %start),+,1}<nw><%loop> U: full-set S: full-set     Exits: (1 + %n)       LoopDispositions: { %loop: Computable }
+; CHECK-NEXT:    --> {(1 + %start),+,1}<nw><%loop> U: full-set S: full-set Exits: (1 + %n) LoopDispositions: { %loop: Computable }
 ; CHECK-NEXT:  Determining loop execution counts for: @smax_simplify_with_guard
 ; CHECK-NEXT:  Loop %loop: backedge-taken count is ((-1 * %start) + %n)
 ; CHECK-NEXT:  Loop %loop: max backedge-taken count is -1
-; CHECK-NEXT:  Loop %loop: Predicated backedge-taken count is ((-1 * %start) +  %n)
+; CHECK-NEXT:  Loop %loop: Predicated backedge-taken count is ((-1 * %start) + %n)
 ; CHECK-NEXT:   Predicates:
 ; CHECK:       Loop %loop: Trip multiple is 1
+;
 entry:
   %guard = icmp sge i32 %n, %start
   br i1 %guard, label %loop.ph, label %exit
 
 loop.ph:
   br label %loop
 
 loop:
   %k.0.i26 = phi i32 [ %start, %loop.ph ], [ %inc.i, %loop ]
   %inc.i = add nsw i32 %k.0.i26, 1
   %cmp26.not.i.not = icmp slt i32 %k.0.i26, %n
   br i1 %cmp26.not.i.not, label %loop, label %exit
 
 exit:
   ret void
 }

llvm/test/Analysis/ScalarEvolution/trip-count13.ll

 
 loop:
   %iv = phi i8 [ %start, %entry ], [ %iv.inc, %loop ]
   %iv.inc = add nsw i8 %iv, -1
   %iv.cmp = icmp sgt i8 %iv, %rhs
   br i1 %iv.cmp, label %loop, label %leave
 
 ; CHECK-LABEL: Determining loop execution counts for: @s_2
-; CHECK-NEXT: Loop %loop: backedge-taken count is ((-1 * ((-100 + %start) smin %start)) + %start)
-; CHECK-NEXT: Loop %loop: max backedge-taken count is -1
+; CHECK-NEXT: Loop %loop: backedge-taken count is (1 + ((99 + (-1 * %start)) smax (-1 + (-1 * %start))) + %start)
+; CHECK-NEXT: Loop %loop: max backedge-taken count is 100, actual taken count either this or zero.
 
 leave:
   ret void
 }

llvm/test/Analysis/ScalarEvolution/trip-count3.ll

 
 declare i32 @fclose(%struct.FILE* nocapture) nounwind
 
 declare void @sha_stream(%struct.SHA_INFO* nocapture, %struct.FILE* nocapture) nounwind
 
 define void @sha_stream_bb3_2E_i(%struct.SHA_INFO* %sha_info, i8* %data1, i32, i8** %buffer_addr.0.i.out, i32* %count_addr.0.i.out) nounwind {
 ; CHECK-LABEL: 'sha_stream_bb3_2E_i'
 ; CHECK-NEXT:  Determining loop execution counts for: @sha_stream_bb3_2E_i
-; CHECK-NEXT:  Loop %bb3.i: backedge-taken count is ((63 + (-1 * (63 smin %0)) + %0) /u 64)
+; CHECK-NEXT:  Loop %bb3.i: backedge-taken count is ((64 + (-64 smax (-1 + (-1 * %0))) + %0) /u 64)
 ; CHECK-NEXT:  Loop %bb3.i: max backedge-taken count is 33554431
-; CHECK-NEXT:  Loop %bb3.i: Predicated backedge-taken count is ((63 + (-1 * (63 smin %0)) + %0) /u 64)
+; CHECK-NEXT:  Loop %bb3.i: Predicated backedge-taken count is ((64 + (-64 smax (-1 + (-1 * %0))) + %0) /u 64)
 ; CHECK-NEXT:   Predicates:
 ; CHECK:       Loop %bb3.i: Trip multiple is 1
 ;
 newFuncRoot:
   br label %bb3.i
 
 sha_update.exit.exitStub:                         ; preds = %bb3.i
   store i8* %buffer_addr.0.i, i8** %buffer_addr.0.i.out

llvm/test/CodeGen/Thumb2/LowOverheadLoops/exitcount.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -verify-machineinstrs -tail-predication=enabled -o - %s | FileCheck %s
 %struct.SpeexPreprocessState_ = type { i32, i32, half*, half* }
 
 define void @foo(%struct.SpeexPreprocessState_* nocapture readonly %st, i16* %x) {
 ; CHECK-LABEL: foo:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    ldrd r12, r2, [r0]
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    ldrd r12, lr, [r0]
 ; CHECK-NEXT:    ldrd r4, r3, [r0, #8]
-; CHECK-NEXT:    rsb r12, r12, r2, lsl #1
+; CHECK-NEXT:    rsb r12, r12, lr, lsl #1
 ; CHECK-NEXT:    dlstp.16 lr, r12
 ; CHECK-NEXT:  .LBB0_1: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrh.u16 q0, [r3], #16
 ; CHECK-NEXT:    vstrh.16 q0, [r4], #16
 ; CHECK-NEXT:    letp lr, .LBB0_1
 ; CHECK-NEXT:  @ %bb.2: @ %do.end
 ; CHECK-NEXT:    ldr r2, [r0]
 ; CHECK-NEXT:    ldr r0, [r0, #8]
 ; CHECK-NEXT:    vmov.i16 q0, #0x1800
 ; CHECK-NEXT:    add.w r0, r0, r12, lsl #1
 ; CHECK-NEXT:    dlstp.16 lr, r2
 ; CHECK-NEXT:  .LBB0_3: @ %do.body6
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrh.u16 q1, [r1], #16
 ; CHECK-NEXT:    vcvt.f16.s16 q1, q1
 ; CHECK-NEXT:    vmul.f16 q1, q1, q0
 ; CHECK-NEXT:    vstrh.16 q1, [r0], #16
 ; CHECK-NEXT:    letp lr, .LBB0_3
 ; CHECK-NEXT:  @ %bb.4: @ %do.end13
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %ps_size = getelementptr inbounds %struct.SpeexPreprocessState_, %struct.SpeexPreprocessState_* %st, i32 0, i32 1
   %0 = load i32, i32* %ps_size, align 4
   %mul = shl nsw i32 %0, 1
   %frame_size = getelementptr inbounds %struct.SpeexPreprocessState_, %struct.SpeexPreprocessState_* %st, i32 0, i32 0
   %1 = load i32, i32* %frame_size, align 4
   %sub = sub nsw i32 %mul, %1
   %inbuf = getelementptr inbounds %struct.SpeexPreprocessState_, %struct.SpeexPreprocessState_* %st, i32 0, i32 3

llvm/test/CodeGen/Thumb2/LowOverheadLoops/lsr-profitable-chain.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O3 -tail-predication=enabled -mtriple=thumbv8.1m.main -mattr=+mve,+mve.fp %s -o - | FileCheck %s
 
 target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
 target triple = "thumbv8.1m-arm-none-eabi"
 
 ; Tests that LSR will not interfere with the VCTP intrinsic,
 ; and that this loop will correctly become tail-predicated.
 
 define arm_aapcs_vfpcc float @vctpi32(float* %0, i32 %1) {
 ; CHECK-LABEL: vctpi32:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmvn.i32 q1, #0x1f
 ; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    movs r2, #0
+; CHECK-NEXT:    movs r4, #0
 ; CHECK-NEXT:    vadd.i32 q1, q3, q1
-; CHECK-NEXT:    subs r3, r1, #1
-; CHECK-NEXT:    vidup.u32 q2, r2, #8
+; CHECK-NEXT:    vidup.u32 q2, r4, #8
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vadd.i32 q1, q2, r0
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    dlstp.32 lr, r3
+; CHECK-NEXT:    subs r0, r1, #1
+; CHECK-NEXT:    dlstp.32 lr, r0
 ; CHECK-NEXT:  .LBB0_1: @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q2, [q1, #32]!
 ; CHECK-NEXT:    vadd.f32 q0, q0, q2
 ; CHECK-NEXT:    letp lr, .LBB0_1
 ; CHECK-NEXT:  @ %bb.2:
 ; CHECK-NEXT:    bl vecAddAcrossF32Mve
 ; CHECK-NEXT:    vmov s0, r0
 ; CHECK-NEXT:    vcvt.f32.s32 s0, s0
 ; CHECK-NEXT:    vabs.f32 s0, s0
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
   %3 = tail call { <4 x i32>, i32 } @llvm.arm.mve.vidup.v4i32(i32 0, i32 8)
   %4 = extractvalue { <4 x i32>, i32 } %3, 0
   %5 = add nsw i32 %1, -1
   %6 = ptrtoint float* %0 to i32
   %7 = insertelement <4 x i32> undef, i32 %6, i32 0
   %8 = add <4 x i32> %7, <i32 -32, i32 undef, i32 undef, i32 undef>
   %9 = shufflevector <4 x i32> %8, <4 x i32> undef, <4 x i32> zeroinitializer
   %10 = add <4 x i32> %4, %9

llvm/test/CodeGen/Thumb2/LowOverheadLoops/mov-operand.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -verify-machineinstrs -tail-predication=enabled -o - %s | FileCheck %s
 
 define arm_aapcs_vfpcc void @arm_var_f32_mve(float* %pSrc, i32 %blockSize, float* nocapture %pResult) {
 ; CHECK-LABEL: arm_var_f32_mve:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    mov r12, r0
+; CHECK-NEXT:    mov r4, r0
 ; CHECK-NEXT:    dlstp.32 lr, r1
 ; CHECK-NEXT:  .LBB0_1: @ %do.body.i
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vldrw.u32 q1, [r12], #16
+; CHECK-NEXT:    vldrw.u32 q1, [r4], #16
 ; CHECK-NEXT:    vadd.f32 q0, q0, q1
 ; CHECK-NEXT:    letp lr, .LBB0_1
 ; CHECK-NEXT:  @ %bb.2: @ %arm_mean_f32_mve.exit
 ; CHECK-NEXT:    vmov s4, r1
 ; CHECK-NEXT:    vadd.f32 s0, s3, s3
 ; CHECK-NEXT:    vcvt.f32.u32 s4, s4
 ; CHECK-NEXT:    vdiv.f32 s0, s0, s4
-; CHECK-NEXT:    vmov r12, s0
+; CHECK-NEXT:    vmov r4, s0
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
 ; CHECK-NEXT:    dlstp.32 lr, r1
 ; CHECK-NEXT:  .LBB0_3: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q1, [r0], #16
-; CHECK-NEXT:    vsub.f32 q1, q1, r12
+; CHECK-NEXT:    vsub.f32 q1, q1, r4
 ; CHECK-NEXT:    vfma.f32 q0, q1, q1
 ; CHECK-NEXT:    letp lr, .LBB0_3
 ; CHECK-NEXT:  @ %bb.4: @ %do.end
 ; CHECK-NEXT:    subs r0, r1, #1
 ; CHECK-NEXT:    vadd.f32 s0, s3, s3
 ; CHECK-NEXT:    vmov s2, r0
 ; CHECK-NEXT:    vcvt.f32.u32 s2, s2
 ; CHECK-NEXT:    vdiv.f32 s0, s0, s2
 ; CHECK-NEXT:    vstr s0, [r2]
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   br label %do.body.i
 
 do.body.i:                                        ; preds = %entry, %do.body.i
   %blkCnt.0.i = phi i32 [ %sub.i, %do.body.i ], [ %blockSize, %entry ]
   %sumVec.0.i = phi <4 x float> [ %3, %do.body.i ], [ zeroinitializer, %entry ]
   %pSrc.addr.0.i = phi float* [ %add.ptr.i, %do.body.i ], [ %pSrc, %entry ]
   %0 = tail call <4 x i1> @llvm.arm.mve.vctp32(i32 %blkCnt.0.i)

llvm/test/CodeGen/Thumb2/LowOverheadLoops/predicated-liveout-unknown-lanes.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -O3 -tail-predication=force-enabled-no-reductions %s -o - | FileCheck %s
 
 define arm_aapcs_vfpcc <4 x float> @arm_max_no_idx_f32_mve(float* %pSrc, i32 %blockSize, float* nocapture %pResult) {
 ; CHECK-LABEL: arm_max_no_idx_f32_mve:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    subs r2, r1, #4
-; CHECK-NEXT:    movw r3, #0
+; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    movt r3, #65408
 ; CHECK-NEXT:    vdup.32 q0, r3
 ; CHECK-NEXT:    dlstp.32 lr, r1
 ; CHECK-NEXT:  .LBB0_1: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q1, [r0], #16
 ; CHECK-NEXT:    vmaxnm.f32 q0, q1, q0
 ; CHECK-NEXT:    letp lr, .LBB0_1

llvm/test/CodeGen/Thumb2/LowOverheadLoops/remat-vctp.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main -mattr=+mve.fp -enable-arm-maskedgatscat=false %s -o - | FileCheck %s
 
 define void @remat_vctp(i32* %arg, i32* %arg1, i32* %arg2, i32* %arg3, i32* %arg4, i16 zeroext %arg5) {
 ; CHECK-LABEL: remat_vctp:
 ; CHECK:       @ %bb.0: @ %bb
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    ldrd r5, r12, [sp, #80]
+; CHECK-NEXT:    ldrd r4, r12, [sp, #80]
 ; CHECK-NEXT:    vmvn.i32 q0, #0x80000000
 ; CHECK-NEXT:    vmov.i32 q1, #0x3f
 ; CHECK-NEXT:    vmov.i32 q2, #0x1
 ; CHECK-NEXT:    dlstp.32 lr, r12
 ; CHECK-NEXT:  .LBB0_1: @ %bb6
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q4, [r1], #16
 ; CHECK-NEXT:    vabs.s32 q5, q4
 ; CHECK-NEXT:    vcls.s32 q3, q5
 ; CHECK-NEXT:    vshl.u32 q5, q5, q3
 ; CHECK-NEXT:    vadd.i32 q3, q3, q2
 ; CHECK-NEXT:    vshr.u32 q6, q5, #24
 ; CHECK-NEXT:    vand q6, q6, q1
-; CHECK-NEXT:    vldrw.u32 q7, [r5, q6, uxtw #2]
+; CHECK-NEXT:    vldrw.u32 q7, [r4, q6, uxtw #2]
 ; CHECK-NEXT:    vqrdmulh.s32 q6, q7, q5
 ; CHECK-NEXT:    vqsub.s32 q6, q0, q6
 ; CHECK-NEXT:    vqrdmulh.s32 q6, q7, q6
 ; CHECK-NEXT:    vqshl.s32 q6, q6, #1
 ; CHECK-NEXT:    vqrdmulh.s32 q5, q6, q5
 ; CHECK-NEXT:    vqsub.s32 q5, q0, q5
 ; CHECK-NEXT:    vqrdmulh.s32 q5, q6, q5
 ; CHECK-NEXT:    vqshl.s32 q5, q5, #1

 
 bb44:                                             ; preds = %bb6
   ret void
 }
 
 define void @dont_remat_predicated_vctp(i32* %arg, i32* %arg1, i32* %arg2, i32* %arg3, i32* %arg4, i16 zeroext %arg5, i32 %conv.mask) {
 ; CHECK-LABEL: dont_remat_predicated_vctp:
 ; CHECK:       @ %bb.0: @ %bb
-; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    sub sp, #8
-; CHECK-NEXT:    ldrd r6, r12, [sp, #88]
-; CHECK-NEXT:    movs r4, #4
-; CHECK-NEXT:    cmp.w r12, #4
+; CHECK-NEXT:    ldrd r4, r12, [sp, #88]
+; CHECK-NEXT:    mvn lr, #4
+; CHECK-NEXT:    mvn.w r5, r12
+; CHECK-NEXT:    cmn.w r5, #5
+; CHECK-NEXT:    csinv r5, lr, r12, le
 ; CHECK-NEXT:    vmvn.i32 q0, #0x80000000
-; CHECK-NEXT:    csel r5, r12, r4, lt
+; CHECK-NEXT:    add r5, r12
 ; CHECK-NEXT:    vmov.i32 q1, #0x3f
-; CHECK-NEXT:    sub.w r5, r12, r5
-; CHECK-NEXT:    vmov.i32 q2, #0x1
-; CHECK-NEXT:    add.w lr, r5, #3
+; CHECK-NEXT:    add.w lr, r5, #4
 ; CHECK-NEXT:    movs r5, #1
+; CHECK-NEXT:    vmov.i32 q2, #0x1
 ; CHECK-NEXT:    add.w lr, r5, lr, lsr #2
+; CHECK-NEXT:    movs r5, #4
 ; CHECK-NEXT:  .LBB1_1: @ %bb6
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vctp.32 r12
 ; CHECK-NEXT:    sub.w r12, r12, #4
 ; CHECK-NEXT:    vpst
-; CHECK-NEXT:    vctpt.32 r4
+; CHECK-NEXT:    vctpt.32 r5
 ; CHECK-NEXT:    vstr p0, [sp, #4] @ 4-byte Spill
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vldrwt.u32 q4, [r1], #16
 ; CHECK-NEXT:    vabs.s32 q5, q4
 ; CHECK-NEXT:    vcls.s32 q3, q5
 ; CHECK-NEXT:    vshl.u32 q5, q5, q3
 ; CHECK-NEXT:    vadd.i32 q3, q3, q2
 ; CHECK-NEXT:    vshr.u32 q6, q5, #24
 ; CHECK-NEXT:    vand q6, q6, q1
-; CHECK-NEXT:    vldrw.u32 q7, [r6, q6, uxtw #2]
+; CHECK-NEXT:    vldrw.u32 q7, [r4, q6, uxtw #2]
 ; CHECK-NEXT:    vqrdmulh.s32 q6, q7, q5
 ; CHECK-NEXT:    vqsub.s32 q6, q0, q6
 ; CHECK-NEXT:    vqrdmulh.s32 q6, q7, q6
 ; CHECK-NEXT:    vqshl.s32 q6, q6, #1
 ; CHECK-NEXT:    vqrdmulh.s32 q5, q6, q5
 ; CHECK-NEXT:    vqsub.s32 q5, q0, q5
 ; CHECK-NEXT:    vqrdmulh.s32 q5, q6, q5
 ; CHECK-NEXT:    vqshl.s32 q5, q5, #1
 ; CHECK-NEXT:    vpt.s32 lt, q4, zr
 ; CHECK-NEXT:    vnegt.s32 q5, q5
 ; CHECK-NEXT:    vldr p0, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vldrwt.u32 q4, [r0], #16
 ; CHECK-NEXT:    vqrdmulh.s32 q4, q4, q5
 ; CHECK-NEXT:    vpstt
 ; CHECK-NEXT:    vstrwt.32 q4, [r2], #16
 ; CHECK-NEXT:    vstrwt.32 q3, [r3], #16
 ; CHECK-NEXT:    le lr, .LBB1_1
 ; CHECK-NEXT:  @ %bb.2: @ %bb44
 ; CHECK-NEXT:    add sp, #8
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, r5, r6, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 bb:
   %i = zext i16 %arg5 to i32
   br label %bb6
 
 bb6:                                              ; preds = %bb6, %bb
   %i7 = phi i32* [ %arg3, %bb ], [ %i38, %bb6 ]
   %i8 = phi i32 [ %i, %bb ], [ %i42, %bb6 ]
   %i9 = phi i32* [ %arg2, %bb ], [ %i41, %bb6 ]

llvm/test/CodeGen/Thumb2/LowOverheadLoops/unpredload.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=armv8.1m.main -mattr=+mve -tail-predication=enabled --verify-machineinstrs %s -o - | FileCheck %s
 
 define void @arm_cmplx_mag_squared_q15_mve(i16* %pSrc, i16* %pDst, i32 %blockSize) {
 ; CHECK-LABEL: arm_cmplx_mag_squared_q15_mve:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    subs.w r3, r2, #8
 ; CHECK-NEXT:    dlstp.16 lr, r2
 ; CHECK-NEXT:  .LBB0_1: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vld20.16 {q0, q1}, [r0]
 ; CHECK-NEXT:    vld21.16 {q0, q1}, [r0]!
 ; CHECK-NEXT:    vmulh.s16 q2, q1, q1
 ; CHECK-NEXT:    vmulh.s16 q0, q0, q0
 ; CHECK-NEXT:    vqadd.s16 q0, q0, q2

 do.end:                                           ; preds = %do.body
   ret void
 }
 
 define i32 @bad(i32* readonly %x, i32* nocapture readonly %y, i32 %n) {
 ; CHECK-LABEL: bad:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    mov r3, r2
-; CHECK-NEXT:    cmp r2, #4
-; CHECK-NEXT:    it ge
-; CHECK-NEXT:    movge r3, #4
-; CHECK-NEXT:    subs r3, r2, r3
-; CHECK-NEXT:    add.w r12, r3, #3
+; CHECK-NEXT:    mvns r3, r2
+; CHECK-NEXT:    mvn r12, #4
+; CHECK-NEXT:    cmn.w r3, #5
+; CHECK-NEXT:    csinv r3, r12, r2, le
+; CHECK-NEXT:    add r3, r2
+; CHECK-NEXT:    add.w r12, r3, #4
 ; CHECK-NEXT:    movs r3, #1
 ; CHECK-NEXT:    add.w r3, r3, r12, lsr #2
 ; CHECK-NEXT:    mov.w r12, #0
 ; CHECK-NEXT:    dls lr, r3
 ; CHECK-NEXT:  .LBB1_1: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vctp.32 r2
 ; CHECK-NEXT:    vpst

llvm/test/CodeGen/Thumb2/LowOverheadLoops/while-loops.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-eabi -mattr=+mve --verify-machineinstrs %s -o - | FileCheck %s
 
 ; Tail predicated so we use DLSTP
 define void @simple(i32* nocapture readonly %x, i32* nocapture readnone %y, i32* nocapture %z, i32 %m, i32 %n) {
 ; CHECK-LABEL: simple:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    ldr r1, [sp, #8]
 ; CHECK-NEXT:    mov r12, r3
 ; CHECK-NEXT:    adds r3, r1, #3
 ; CHECK-NEXT:    lsrs r3, r3, #2
 ; CHECK-NEXT:    beq .LBB0_3
 ; CHECK-NEXT:  @ %bb.1: @ %do.body.preheader
 ; CHECK-NEXT:    dlstp.32 lr, r1
 ; CHECK-NEXT:  .LBB0_2: @ %do.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [r0], #16
 ; CHECK-NEXT:    vaddva.s32 r12, q0
 ; CHECK-NEXT:    letp lr, .LBB0_2
 ; CHECK-NEXT:  .LBB0_3: @ %if.end
 ; CHECK-NEXT:    str.w r12, [r2]
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %add = add i32 %n, 3
   %div = lshr i32 %add, 2
   %cmp.not = icmp eq i32 %div, 0
   br i1 %cmp.not, label %if.end, label %do.body
 
 do.body:                                          ; preds = %entry, %do.body
   %n.addr.0 = phi i32 [ %sub, %do.body ], [ %n, %entry ]
if.end:                                           ; preds = %do.body, %entry
   store i32 %s.1, i32* %z, align 4
   ret void
 }
 
 ; Tail predicated so we use DLSTP
 define void @nested(i32* nocapture readonly %x, i32* nocapture readnone %y, i32* nocapture %z, i32 %m, i32 %n) {
 ; CHECK-LABEL: nested:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
 ; CHECK-NEXT:    cbz r3, .LBB1_8
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr.w r12, [sp, #24]
+; CHECK-NEXT:    ldr.w r12, [sp, #28]
 ; CHECK-NEXT:    movs r1, #0
 ; CHECK-NEXT:    b .LBB1_4
 ; CHECK-NEXT:  .LBB1_2: @ in Loop: Header=BB1_4 Depth=1
 ; CHECK-NEXT:    mov r4, r3
 ; CHECK-NEXT:  .LBB1_3: @ %if.end
 ; CHECK-NEXT:    @ in Loop: Header=BB1_4 Depth=1
 ; CHECK-NEXT:    str.w r4, [r2, r1, lsl #2]
 ; CHECK-NEXT:    adds r1, #1
 ; CHECK-NEXT:    cmp r1, r3
 ; CHECK-NEXT:    beq .LBB1_8
 ; CHECK-NEXT:  .LBB1_4: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB1_6 Depth 2
-; CHECK-NEXT:    add.w r6, r12, #3
-; CHECK-NEXT:    lsrs r7, r6, #2
+; CHECK-NEXT:    add.w r7, r12, #3
+; CHECK-NEXT:    lsrs r6, r7, #2
 ; CHECK-NEXT:    beq .LBB1_2
 ; CHECK-NEXT:  @ %bb.5: @ %do.body.preheader
 ; CHECK-NEXT:    @ in Loop: Header=BB1_4 Depth=1
-; CHECK-NEXT:    bic r5, r6, #3
 ; CHECK-NEXT:    mov r4, r3
-; CHECK-NEXT:    add.w r8, r0, r5, lsl #2
+; CHECK-NEXT:    bic r6, r7, #3
+; CHECK-NEXT:    add.w r9, r0, r6, lsl #2
 ; CHECK-NEXT:    dlstp.32 lr, r12
 ; CHECK-NEXT:  .LBB1_6: @ %do.body
 ; CHECK-NEXT:    @ Parent Loop BB1_4 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vldrw.u32 q0, [r0], #16
 ; CHECK-NEXT:    vaddva.s32 r4, q0
 ; CHECK-NEXT:    letp lr, .LBB1_6
 ; CHECK-NEXT:  @ %bb.7: @ %if.end.loopexit
 ; CHECK-NEXT:    @ in Loop: Header=BB1_4 Depth=1
-; CHECK-NEXT:    sub.w r12, r12, r5
-; CHECK-NEXT:    mov r0, r8
+; CHECK-NEXT:    sub.w r12, r12, r6
+; CHECK-NEXT:    mov r0, r9
 ; CHECK-NEXT:    b .LBB1_3
 ; CHECK-NEXT:  .LBB1_8: @ %for.cond.cleanup
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
 entry:
   %cmp20.not = icmp eq i32 %m, 0
   br i1 %cmp20.not, label %for.cond.cleanup, label %for.body
 
 for.cond.cleanup:                                 ; preds = %if.end, %entry
   ret void
 
 for.body:                                         ; preds = %entry, %if.end

llvm/test/CodeGen/Thumb2/mve-float16regloops.ll

 }
 
 
 define void @fir(%struct.arm_fir_instance_f32* nocapture readonly %S, half* nocapture readonly %pSrc, half* nocapture %pDst, i32 %blockSize) {
 ; CHECK-LABEL: fir:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
-; CHECK-NEXT:    .pad #24
-; CHECK-NEXT:    sub sp, #24
+; CHECK-NEXT:    .pad #28
+; CHECK-NEXT:    sub sp, #28
 ; CHECK-NEXT:    cmp r3, #8
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
 ; CHECK-NEXT:    blo.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.1: @ %entry
 ; CHECK-NEXT:    lsrs.w r12, r3, #2
 ; CHECK-NEXT:    beq.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.2: @ %while.body.lr.ph
 ; CHECK-NEXT:    ldrh r4, [r0]
 ; CHECK-NEXT:    movs r1, #1
-; CHECK-NEXT:    ldrd r5, r3, [r0, #4]
+; CHECK-NEXT:    ldrd r7, r3, [r0, #4]
 ; CHECK-NEXT:    sub.w r0, r4, #8
-; CHECK-NEXT:    add.w r7, r0, r0, lsr #29
-; CHECK-NEXT:    and r0, r0, #7
-; CHECK-NEXT:    asrs r6, r7, #3
-; CHECK-NEXT:    cmp r6, #1
+; CHECK-NEXT:    add.w r6, r0, r0, lsr #29
+; CHECK-NEXT:    asrs r5, r6, #3
+; CHECK-NEXT:    cmp r5, #1
 ; CHECK-NEXT:    it gt
-; CHECK-NEXT:    asrgt r1, r7, #3
-; CHECK-NEXT:    add.w r7, r5, r4, lsl #1
+; CHECK-NEXT:    asrgt r1, r6, #3
 ; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
-; CHECK-NEXT:    subs r1, r7, #2
-; CHECK-NEXT:    rsbs r7, r4, #0
-; CHECK-NEXT:    str r7, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    add.w r7, r3, #16
-; CHECK-NEXT:    str r4, [sp, #12] @ 4-byte Spill
-; CHECK-NEXT:    str r7, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    str r0, [sp, #16] @ 4-byte Spill
-; CHECK-NEXT:    b .LBB16_5
-; CHECK-NEXT:  .LBB16_3: @ %for.end
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    wls lr, r0, .LBB16_4
-; CHECK-NEXT:    b .LBB16_9
-; CHECK-NEXT:  .LBB16_4: @ %while.end
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    and r1, r0, #7
+; CHECK-NEXT:    mvn r6, #1
+; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    mvns r0, r1
+; CHECK-NEXT:    cmn.w r0, #2
+; CHECK-NEXT:    csinv r0, r6, r1, le
+; CHECK-NEXT:    str r4, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    add r0, r1
+; CHECK-NEXT:    adds r0, #2
+; CHECK-NEXT:    str r0, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r7, r4, lsl #1
+; CHECK-NEXT:    subs r1, r0, #2
+; CHECK-NEXT:    rsbs r0, r4, #0
+; CHECK-NEXT:    str r0, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r3, #16
+; CHECK-NEXT:    str r0, [sp, #8] @ 4-byte Spill
+; CHECK-NEXT:    b .LBB16_4
+; CHECK-NEXT:  .LBB16_3: @ %while.end
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    subs.w r12, r12, #1
 ; CHECK-NEXT:    vstrb.8 q0, [r2], #8
-; CHECK-NEXT:    add.w r0, r5, r0, lsl #1
-; CHECK-NEXT:    add.w r5, r0, #8
+; CHECK-NEXT:    add.w r0, r7, r0, lsl #1
+; CHECK-NEXT:    add.w r7, r0, #8
 ; CHECK-NEXT:    beq.w .LBB16_12
-; CHECK-NEXT:  .LBB16_5: @ %while.body
+; CHECK-NEXT:  .LBB16_4: @ %while.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB16_7 Depth 2
+; CHECK-NEXT:    @ Child Loop BB16_6 Depth 2
 ; CHECK-NEXT:    @ Child Loop BB16_10 Depth 2
-; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #24] @ 4-byte Reload
 ; CHECK-NEXT:    ldrh.w lr, [r3, #14]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0], #8
-; CHECK-NEXT:    ldrh.w r8, [r3, #12]
-; CHECK-NEXT:    ldrh r7, [r3, #10]
-; CHECK-NEXT:    ldrh r4, [r3, #8]
-; CHECK-NEXT:    ldrh r6, [r3, #6]
-; CHECK-NEXT:    ldrh.w r9, [r3, #4]
+; CHECK-NEXT:    ldrh.w r9, [r3, #12]
+; CHECK-NEXT:    ldrh r4, [r3, #10]
+; CHECK-NEXT:    ldrh r6, [r3, #8]
+; CHECK-NEXT:    ldrh r5, [r3, #6]
+; CHECK-NEXT:    ldrh.w r10, [r3, #4]
 ; CHECK-NEXT:    ldrh.w r11, [r3, #2]
-; CHECK-NEXT:    ldrh.w r10, [r3]
+; CHECK-NEXT:    ldrh.w r8, [r3]
 ; CHECK-NEXT:    vstrb.8 q0, [r1], #8
-; CHECK-NEXT:    vldrw.u32 q0, [r5]
-; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
-; CHECK-NEXT:    adds r0, r5, #2
+; CHECK-NEXT:    vldrw.u32 q0, [r7]
+; CHECK-NEXT:    str r0, [sp, #24] @ 4-byte Spill
+; CHECK-NEXT:    adds r0, r7, #2
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vmul.f16 q0, q0, r10
-; CHECK-NEXT:    adds r0, r5, #6
+; CHECK-NEXT:    vmul.f16 q0, q0, r8
+; CHECK-NEXT:    adds r0, r7, #6
 ; CHECK-NEXT:    vfma.f16 q0, q1, r11
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #4]
-; CHECK-NEXT:    vfma.f16 q0, q1, r9
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #4]
+; CHECK-NEXT:    vfma.f16 q0, q1, r10
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    add.w r0, r5, #10
+; CHECK-NEXT:    add.w r0, r7, #10
+; CHECK-NEXT:    vfma.f16 q0, q1, r5
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #8]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r6
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #8]
-; CHECK-NEXT:    vfma.f16 q0, q1, r4
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    add.w r0, r5, #14
-; CHECK-NEXT:    vfma.f16 q0, q1, r7
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #12]
-; CHECK-NEXT:    adds r5, #16
-; CHECK-NEXT:    vfma.f16 q0, q1, r8
+; CHECK-NEXT:    add.w r0, r7, #14
+; CHECK-NEXT:    vfma.f16 q0, q1, r4
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #12]
+; CHECK-NEXT:    adds r7, #16
+; CHECK-NEXT:    vfma.f16 q0, q1, r9
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
 ; CHECK-NEXT:    vfma.f16 q0, q1, lr
 ; CHECK-NEXT:    cmp r0, #16
-; CHECK-NEXT:    blo .LBB16_8
-; CHECK-NEXT:  @ %bb.6: @ %for.body.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
+; CHECK-NEXT:    blo .LBB16_7
+; CHECK-NEXT:  @ %bb.5: @ %for.body.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    dls lr, r0
-; CHECK-NEXT:    ldr r6, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:  .LBB16_7: @ %for.body
-; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
+; CHECK-NEXT:    ldr r5, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:  .LBB16_6: @ %for.body
+; CHECK-NEXT:    @ Parent Loop BB16_4 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldrh r0, [r6], #16
-; CHECK-NEXT:    vldrw.u32 q1, [r5]
-; CHECK-NEXT:    adds r4, r5, #2
+; CHECK-NEXT:    ldrh r0, [r5], #16
+; CHECK-NEXT:    vldrw.u32 q1, [r7]
+; CHECK-NEXT:    adds r4, r7, #2
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-14]
-; CHECK-NEXT:    adds r4, r5, #6
+; CHECK-NEXT:    ldrh r0, [r5, #-14]
+; CHECK-NEXT:    adds r4, r7, #6
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-12]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #4]
+; CHECK-NEXT:    ldrh r0, [r5, #-12]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #4]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-10]
-; CHECK-NEXT:    add.w r4, r5, #10
+; CHECK-NEXT:    ldrh r0, [r5, #-10]
+; CHECK-NEXT:    add.w r4, r7, #10
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-8]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #8]
+; CHECK-NEXT:    ldrh r0, [r5, #-8]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #8]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-6]
-; CHECK-NEXT:    ldrh r4, [r6, #-2]
+; CHECK-NEXT:    ldrh r0, [r5, #-6]
+; CHECK-NEXT:    ldrh r4, [r5, #-2]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-4]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #12]
+; CHECK-NEXT:    ldrh r0, [r5, #-4]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #12]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    add.w r0, r5, #14
+; CHECK-NEXT:    add.w r0, r7, #14
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    adds r5, #16
+; CHECK-NEXT:    adds r7, #16
 ; CHECK-NEXT:    vfma.f16 q0, q1, r4
-; CHECK-NEXT:    le lr, .LBB16_7
-; CHECK-NEXT:    b .LBB16_3
-; CHECK-NEXT:  .LBB16_8: @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r6, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:    b .LBB16_3
-; CHECK-NEXT:  .LBB16_9: @ %while.body76.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    le lr, .LBB16_6
+; CHECK-NEXT:    b .LBB16_8
+; CHECK-NEXT:  .LBB16_7: @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r5, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:  .LBB16_8: @ %for.end
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    beq.w .LBB16_3
+; CHECK-NEXT:  @ %bb.9: @ %while.body76.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    dls lr, r0
+; CHECK-NEXT:    mov r0, r7
 ; CHECK-NEXT:  .LBB16_10: @ %while.body76
-; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB16_4 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldrh r4, [r6], #2
+; CHECK-NEXT:    ldrh r4, [r5], #2
 ; CHECK-NEXT:    vldrh.u16 q1, [r0], #2
 ; CHECK-NEXT:    vfma.f16 q0, q1, r4
 ; CHECK-NEXT:    le lr, .LBB16_10
 ; CHECK-NEXT:  @ %bb.11: @ %while.end.loopexit
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    add.w r5, r5, r0, lsl #1
-; CHECK-NEXT:    b .LBB16_4
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    add.w r7, r7, r0, lsl #1
+; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_12: @ %if.end
-; CHECK-NEXT:    add sp, #24
+; CHECK-NEXT:    add sp, #28
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %pState1 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 1
   %0 = load half*, half** %pState1, align 4
   %pCoeffs2 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 2
   %1 = load half*, half** %pCoeffs2, align 4
   %numTaps3 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 0
   %2 = load i16, i16* %numTaps3, align 4

llvm/test/CodeGen/Thumb2/mve-float32regloops.ll

 ; CHECK-NEXT:    cmp r3, #8
 ; CHECK-NEXT:    blo.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.1: @ %entry
 ; CHECK-NEXT:    lsrs.w r12, r3, #2
 ; CHECK-NEXT:    beq.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.2: @ %while.body.lr.ph
 ; CHECK-NEXT:    ldrh r6, [r0]
 ; CHECK-NEXT:    movs r5, #1
-; CHECK-NEXT:    ldrd r4, r10, [r0, #4]
+; CHECK-NEXT:    ldrd r8, r9, [r0, #4]
 ; CHECK-NEXT:    sub.w r0, r6, #8
 ; CHECK-NEXT:    add.w r3, r0, r0, lsr #29
-; CHECK-NEXT:    and r0, r0, #7
 ; CHECK-NEXT:    asrs r7, r3, #3
 ; CHECK-NEXT:    cmp r7, #1
+; CHECK-NEXT:    and r7, r0, #7
 ; CHECK-NEXT:    it gt
 ; CHECK-NEXT:    asrgt r5, r3, #3
-; CHECK-NEXT:    add.w r3, r4, r6, lsl #2
-; CHECK-NEXT:    sub.w r9, r3, #4
-; CHECK-NEXT:    rsbs r3, r6, #0
-; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
-; CHECK-NEXT:    add.w r3, r10, #32
-; CHECK-NEXT:    str r5, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    mvn r3, #1
+; CHECK-NEXT:    str r5, [sp] @ 4-byte Spill
+; CHECK-NEXT:    mvns r0, r7
+; CHECK-NEXT:    cmn.w r0, #2
+; CHECK-NEXT:    csinv r0, r3, r7, le
+; CHECK-NEXT:    str r7, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    add r0, r7
 ; CHECK-NEXT:    str r6, [sp, #16] @ 4-byte Spill
-; CHECK-NEXT:    str r3, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
-; CHECK-NEXT:    b .LBB16_5
-; CHECK-NEXT:  .LBB16_3: @ %for.end
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
-; CHECK-NEXT:    ldrd r0, r9, [sp, #20] @ 8-byte Folded Reload
-; CHECK-NEXT:    wls lr, r0, .LBB16_4
-; CHECK-NEXT:    b .LBB16_9
-; CHECK-NEXT:  .LBB16_4: @ %while.end
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
+; CHECK-NEXT:    adds r0, #2
+; CHECK-NEXT:    str r0, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r8, r6, lsl #2
+; CHECK-NEXT:    sub.w r11, r0, #4
+; CHECK-NEXT:    rsbs r0, r6, #0
+; CHECK-NEXT:    str r0, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r9, #32
+; CHECK-NEXT:    str r0, [sp, #8] @ 4-byte Spill
+; CHECK-NEXT:    b .LBB16_4
+; CHECK-NEXT:  .LBB16_3: @ %while.end
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    subs.w r12, r12, #1
+; CHECK-NEXT:    ldrd r11, r1, [sp, #24] @ 8-byte Folded Reload
 ; CHECK-NEXT:    vstrb.8 q0, [r2], #16
-; CHECK-NEXT:    add.w r0, r4, r0, lsl #2
-; CHECK-NEXT:    add.w r4, r0, #16
+; CHECK-NEXT:    add.w r0, r8, r0, lsl #2
+; CHECK-NEXT:    add.w r8, r0, #16
 ; CHECK-NEXT:    beq .LBB16_12
-; CHECK-NEXT:  .LBB16_5: @ %while.body
+; CHECK-NEXT:  .LBB16_4: @ %while.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB16_7 Depth 2
+; CHECK-NEXT:    @ Child Loop BB16_6 Depth 2
 ; CHECK-NEXT:    @ Child Loop BB16_10 Depth 2
-; CHECK-NEXT:    add.w lr, r10, #8
+; CHECK-NEXT:    add.w r7, r9, #8
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
-; CHECK-NEXT:    ldrd r3, r7, [r10]
-; CHECK-NEXT:    ldm.w lr, {r0, r5, r6, lr}
-; CHECK-NEXT:    ldrd r11, r8, [r10, #24]
-; CHECK-NEXT:    vstrb.8 q0, [r9], #16
-; CHECK-NEXT:    vldrw.u32 q0, [r4], #32
-; CHECK-NEXT:    strd r9, r1, [sp, #24] @ 8-byte Folded Spill
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-28]
+; CHECK-NEXT:    ldrd r3, r6, [r9]
+; CHECK-NEXT:    ldm r7, {r0, r5, r7}
+; CHECK-NEXT:    ldrd r4, lr, [r9, #20]
+; CHECK-NEXT:    ldr.w r10, [r9, #28]
+; CHECK-NEXT:    vstrb.8 q0, [r11], #16
+; CHECK-NEXT:    vldrw.u32 q0, [r8], #32
+; CHECK-NEXT:    strd r11, r1, [sp, #24] @ 8-byte Folded Spill
+; CHECK-NEXT:    vldrw.u32 q1, [r8, #-28]
 ; CHECK-NEXT:    vmul.f32 q0, q0, r3
-; CHECK-NEXT:    vldrw.u32 q6, [r4, #-24]
-; CHECK-NEXT:    vldrw.u32 q4, [r4, #-20]
-; CHECK-NEXT:    vfma.f32 q0, q1, r7
-; CHECK-NEXT:    vldrw.u32 q5, [r4, #-16]
+; CHECK-NEXT:    vldrw.u32 q6, [r8, #-24]
+; CHECK-NEXT:    vldrw.u32 q4, [r8, #-20]
+; CHECK-NEXT:    vfma.f32 q0, q1, r6
+; CHECK-NEXT:    vldrw.u32 q5, [r8, #-16]
 ; CHECK-NEXT:    vfma.f32 q0, q6, r0
-; CHECK-NEXT:    vldrw.u32 q2, [r4, #-12]
+; CHECK-NEXT:    vldrw.u32 q2, [r8, #-12]
 ; CHECK-NEXT:    vfma.f32 q0, q4, r5
-; CHECK-NEXT:    vldrw.u32 q3, [r4, #-8]
-; CHECK-NEXT:    vfma.f32 q0, q5, r6
+; CHECK-NEXT:    vldrw.u32 q3, [r8, #-8]
+; CHECK-NEXT:    vfma.f32 q0, q5, r7
 ; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    vfma.f32 q0, q2, lr
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-4]
-; CHECK-NEXT:    vfma.f32 q0, q3, r11
+; CHECK-NEXT:    vfma.f32 q0, q2, r4
+; CHECK-NEXT:    vldrw.u32 q1, [r8, #-4]
+; CHECK-NEXT:    vfma.f32 q0, q3, lr
 ; CHECK-NEXT:    cmp r0, #16
-; CHECK-NEXT:    vfma.f32 q0, q1, r8
-; CHECK-NEXT:    blo .LBB16_8
-; CHECK-NEXT:  @ %bb.6: @ %for.body.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    vfma.f32 q0, q1, r10
+; CHECK-NEXT:    blo .LBB16_7
+; CHECK-NEXT:  @ %bb.5: @ %for.body.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    dls lr, r0
-; CHECK-NEXT:    ldr r7, [sp, #8] @ 4-byte Reload
-; CHECK-NEXT:  .LBB16_7: @ %for.body
-; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
+; CHECK-NEXT:    ldr r6, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:  .LBB16_6: @ %for.body
+; CHECK-NEXT:    @ Parent Loop BB16_4 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldm.w r7, {r0, r3, r5, r6, r8, r11}
-; CHECK-NEXT:    vldrw.u32 q1, [r4], #32
-; CHECK-NEXT:    vldrw.u32 q6, [r4, #-24]
-; CHECK-NEXT:    vldrw.u32 q4, [r4, #-20]
+; CHECK-NEXT:    ldm.w r6, {r0, r3, r4, r5, r7, r10, r11}
+; CHECK-NEXT:    vldrw.u32 q1, [r8], #32
+; CHECK-NEXT:    vldrw.u32 q6, [r8, #-24]
+; CHECK-NEXT:    vldrw.u32 q4, [r8, #-20]
 ; CHECK-NEXT:    vfma.f32 q0, q1, r0
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-28]
-; CHECK-NEXT:    vldrw.u32 q5, [r4, #-16]
-; CHECK-NEXT:    vldrw.u32 q2, [r4, #-12]
+; CHECK-NEXT:    vldrw.u32 q1, [r8, #-28]
+; CHECK-NEXT:    vldrw.u32 q5, [r8, #-16]
+; CHECK-NEXT:    vldrw.u32 q2, [r8, #-12]
 ; CHECK-NEXT:    vfma.f32 q0, q1, r3
-; CHECK-NEXT:    ldrd r9, r1, [r7, #24]
-; CHECK-NEXT:    vfma.f32 q0, q6, r5
-; CHECK-NEXT:    vldrw.u32 q3, [r4, #-8]
-; CHECK-NEXT:    vfma.f32 q0, q4, r6
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-4]
-; CHECK-NEXT:    vfma.f32 q0, q5, r8
-; CHECK-NEXT:    adds r7, #32
-; CHECK-NEXT:    vfma.f32 q0, q2, r11
-; CHECK-NEXT:    vfma.f32 q0, q3, r9
+; CHECK-NEXT:    vldrw.u32 q3, [r8, #-8]
+; CHECK-NEXT:    vfma.f32 q0, q6, r4
+; CHECK-NEXT:    ldr r1, [r6, #28]
+; CHECK-NEXT:    vfma.f32 q0, q4, r5
+; CHECK-NEXT:    vldrw.u32 q1, [r8, #-4]
+; CHECK-NEXT:    vfma.f32 q0, q5, r7
+; CHECK-NEXT:    adds r6, #32
+; CHECK-NEXT:    vfma.f32 q0, q2, r10
+; CHECK-NEXT:    vfma.f32 q0, q3, r11
 ; CHECK-NEXT:    vfma.f32 q0, q1, r1
-; CHECK-NEXT:    le lr, .LBB16_7
-; CHECK-NEXT:    b .LBB16_3
-; CHECK-NEXT:  .LBB16_8: @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r7, [sp, #8] @ 4-byte Reload
-; CHECK-NEXT:    b .LBB16_3
-; CHECK-NEXT:  .LBB16_9: @ %while.body76.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    mov r3, r4
+; CHECK-NEXT:    le lr, .LBB16_6
+; CHECK-NEXT:    b .LBB16_8
+; CHECK-NEXT:  .LBB16_7: @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r6, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:  .LBB16_8: @ %for.end
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    beq .LBB16_3
+; CHECK-NEXT:  @ %bb.9: @ %while.body76.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
+; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    mov r3, r8
+; CHECK-NEXT:    dls lr, r0
 ; CHECK-NEXT:  .LBB16_10: @ %while.body76
-; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB16_4 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldr r0, [r7], #4
+; CHECK-NEXT:    ldr r0, [r6], #4
 ; CHECK-NEXT:    vldrw.u32 q1, [r3], #4
 ; CHECK-NEXT:    vfma.f32 q0, q1, r0
 ; CHECK-NEXT:    le lr, .LBB16_10
 ; CHECK-NEXT:  @ %bb.11: @ %while.end.loopexit
-; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
+; CHECK-NEXT:    @ in Loop: Header=BB16_4 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
-; CHECK-NEXT:    add.w r4, r4, r0, lsl #2
-; CHECK-NEXT:    b .LBB16_4
+; CHECK-NEXT:    add.w r8, r8, r0, lsl #2
+; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_12: @ %if.end
 ; CHECK-NEXT:    add sp, #32
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %pState1 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 1
   %0 = load float*, float** %pState1, align 4

llvm/test/Transforms/HardwareLoops/ARM/simple-do.ll

 while.end:
   %res.0.lcssa = phi i32 [ 0, %entry ], [ %add, %while.end.loopexit ]
   ret i32 %res.0.lcssa
 }
 
 ; CHECK-LABEL: do_dec2
 
 ; CHECK: entry:
-; CHECK: [[ROUND:%[^ ]+]] = add i32 %n, 1
-; CHECK: [[SMIN:%[^ ]+]] = call i32 @llvm.smin.i32(i32 %n, i32 2)
-; CHECK: [[SUB:%[^ ]+]] = sub i32 [[ROUND]], [[SMIN]]
-; CHECK: [[HALVE:%[^ ]+]] = lshr i32 [[SUB]], 1
+; CHECK: [[HALVE:%[^ ]+]] = lshr i32 {{.*}}, 1
 ; CHECK: [[COUNT:%[^ ]+]] = add nuw i32 [[HALVE]], 1
 
 ; CHECK: while.body.lr.ph:
 ; CHECK: [[START:%[^ ]+]] = call i32 @llvm.start.loop.iterations.i32(i32 [[COUNT]])
 ; CHECK: br label %while.body
 
 ; CHECK: [[REM:%[^ ]+]] = phi i32 [ [[START]], %while.body.lr.ph ], [ [[LOOP_DEC:%[^ ]+]], %while.body ]
 ; CHECK: [[LOOP_DEC]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[REM]], i32 1)

llvm/test/Transforms/HardwareLoops/scalar-while.ll

 }
 
 define void @while_gt(i32 %i, i32 %N, i32* nocapture %A) {
 ; CHECK-DEC-LABEL: @while_gt(
 ; CHECK-DEC-NEXT:  entry:
 ; CHECK-DEC-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-DEC-NEXT:    br i1 [[CMP4]], label [[WHILE_BODY_PREHEADER:%.*]], label [[WHILE_END:%.*]]
 ; CHECK-DEC:       while.body.preheader:
-; CHECK-DEC-NEXT:    [[TMP0:%.*]] = sub i32 [[I]], [[N]]
-; CHECK-DEC-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP0]])
+; CHECK-DEC-NEXT:    [[TMP0:%.*]] = sub i32 0, [[I]]
+; CHECK-DEC-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-DEC-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-DEC-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-DEC-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 1
+; CHECK-DEC-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-DEC-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-DEC:       while.body:
 ; CHECK-DEC-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-DEC-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-DEC-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-DEC-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-DEC-NEXT:    [[TMP1:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-DEC-NEXT:    br i1 [[TMP1]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-DEC-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-DEC-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-DEC:       while.end:
 ; CHECK-DEC-NEXT:    ret void
 ;
 ; CHECK-PHI-LABEL: @while_gt(
 ; CHECK-PHI-NEXT:  entry:
 ; CHECK-PHI-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-PHI-NEXT:    br i1 [[CMP4]], label [[WHILE_BODY_PREHEADER:%.*]], label [[WHILE_END:%.*]]
 ; CHECK-PHI:       while.body.preheader:
-; CHECK-PHI-NEXT:    [[TMP0:%.*]] = sub i32 [[I]], [[N]]
-; CHECK-PHI-NEXT:    [[TMP1:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP0]])
+; CHECK-PHI-NEXT:    [[TMP0:%.*]] = sub i32 0, [[I]]
+; CHECK-PHI-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-PHI-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-PHI-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-PHI-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 1
+; CHECK-PHI-NEXT:    [[TMP4:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-PHI-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-PHI:       while.body:
 ; CHECK-PHI-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
-; CHECK-PHI-NEXT:    [[TMP2:%.*]] = phi i32 [ [[TMP1]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP3:%.*]], [[WHILE_BODY]] ]
+; CHECK-PHI-NEXT:    [[TMP5:%.*]] = phi i32 [ [[TMP4]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP6:%.*]], [[WHILE_BODY]] ]
 ; CHECK-PHI-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-PHI-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-PHI-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-PHI-NEXT:    [[TMP3]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP2]], i32 1)
-; CHECK-PHI-NEXT:    [[TMP4:%.*]] = icmp ne i32 [[TMP3]], 0
-; CHECK-PHI-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-PHI-NEXT:    [[TMP6]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP5]], i32 1)
+; CHECK-PHI-NEXT:    [[TMP7:%.*]] = icmp ne i32 [[TMP6]], 0
+; CHECK-PHI-NEXT:    br i1 [[TMP7]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-PHI:       while.end:
 ; CHECK-PHI-NEXT:    ret void
 ;
 ; CHECK-NESTED-LABEL: @while_gt(
 ; CHECK-NESTED-NEXT:  entry:
 ; CHECK-NESTED-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-NESTED-NEXT:    br i1 [[CMP4]], label [[WHILE_BODY_PREHEADER:%.*]], label [[WHILE_END:%.*]]
 ; CHECK-NESTED:       while.body.preheader:
-; CHECK-NESTED-NEXT:    [[TMP0:%.*]] = sub i32 [[I]], [[N]]
-; CHECK-NESTED-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP0]])
+; CHECK-NESTED-NEXT:    [[TMP0:%.*]] = sub i32 0, [[I]]
+; CHECK-NESTED-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-NESTED-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-NESTED-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-NESTED-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 1
+; CHECK-NESTED-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-NESTED-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-NESTED:       while.body:
 ; CHECK-NESTED-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NESTED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-NESTED-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-NESTED-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-NESTED-NEXT:    [[TMP1:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-NESTED-NEXT:    br i1 [[TMP1]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-NESTED-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-NESTED-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-NESTED:       while.end:
 ; CHECK-NESTED-NEXT:    ret void
 ;
 ; CHECK-GUARD-LABEL: @while_gt(
 ; CHECK-GUARD-NEXT:  entry:
 ; CHECK-GUARD-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[I:%.*]], [[N:%.*]]
-; CHECK-GUARD-NEXT:    [[TMP0:%.*]] = sub i32 [[I]], [[N]]
 ; CHECK-GUARD-NEXT:    br i1 [[CMP4]], label [[WHILE_BODY_PREHEADER:%.*]], label [[WHILE_END:%.*]]
 ; CHECK-GUARD:       while.body.preheader:
-; CHECK-GUARD-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP0]])
+; CHECK-GUARD-NEXT:    [[TMP0:%.*]] = sub i32 0, [[I]]
+; CHECK-GUARD-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-GUARD-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-GUARD-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-GUARD-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 1
+; CHECK-GUARD-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-GUARD-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-GUARD:       while.body:
 ; CHECK-GUARD-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-GUARD-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-GUARD-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-GUARD-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-GUARD-NEXT:    [[TMP1:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-GUARD-NEXT:    br i1 [[TMP1]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-GUARD-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-GUARD-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-GUARD:       while.end:
 ; CHECK-GUARD-NEXT:    ret void
 ;
 ; CHECK-PHIGUARD-LABEL: @while_gt(
 ; CHECK-PHIGUARD-NEXT:  entry:
 ; CHECK-PHIGUARD-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[I:%.*]], [[N:%.*]]
-; CHECK-PHIGUARD-NEXT:    [[TMP0:%.*]] = sub i32 [[I]], [[N]]
 ; CHECK-PHIGUARD-NEXT:    br i1 [[CMP4]], label [[WHILE_BODY_PREHEADER:%.*]], label [[WHILE_END:%.*]]
 ; CHECK-PHIGUARD:       while.body.preheader:
-; CHECK-PHIGUARD-NEXT:    [[TMP1:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP0]])
+; CHECK-PHIGUARD-NEXT:    [[TMP0:%.*]] = sub i32 0, [[I]]
+; CHECK-PHIGUARD-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-PHIGUARD-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-PHIGUARD-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-PHIGUARD-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 1
+; CHECK-PHIGUARD-NEXT:    [[TMP4:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-PHIGUARD-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-PHIGUARD:       while.body:
 ; CHECK-PHIGUARD-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
-; CHECK-PHIGUARD-NEXT:    [[TMP2:%.*]] = phi i32 [ [[TMP1]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP3:%.*]], [[WHILE_BODY]] ]
+; CHECK-PHIGUARD-NEXT:    [[TMP5:%.*]] = phi i32 [ [[TMP4]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP6:%.*]], [[WHILE_BODY]] ]
 ; CHECK-PHIGUARD-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-PHIGUARD-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-PHIGUARD-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-PHIGUARD-NEXT:    [[TMP3]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP2]], i32 1)
-; CHECK-PHIGUARD-NEXT:    [[TMP4:%.*]] = icmp ne i32 [[TMP3]], 0
-; CHECK-PHIGUARD-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-PHIGUARD-NEXT:    [[TMP6]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP5]], i32 1)
+; CHECK-PHIGUARD-NEXT:    [[TMP7:%.*]] = icmp ne i32 [[TMP6]], 0
+; CHECK-PHIGUARD-NEXT:    br i1 [[TMP7]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-PHIGUARD:       while.end:
 ; CHECK-PHIGUARD-NEXT:    ret void
 ;
 entry:
   %cmp4 = icmp sgt i32 %i, %N
   br i1 %cmp4, label %while.body, label %while.end
 
 while.body:

 }
 
 define void @while_gte(i32 %i, i32 %N, i32* nocapture %A) {
 ; CHECK-DEC-LABEL: @while_gte(
 ; CHECK-DEC-NEXT:  entry:
 ; CHECK-DEC-NEXT:    [[CMP4:%.*]] = icmp slt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-DEC-NEXT:    br i1 [[CMP4]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK-DEC:       while.body.preheader:
-; CHECK-DEC-NEXT:    [[TMP0:%.*]] = add i32 [[I]], 1
-; CHECK-DEC-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[N]]
-; CHECK-DEC-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP1]])
+; CHECK-DEC-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[I]]
+; CHECK-DEC-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-DEC-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-DEC-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-DEC-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 2
+; CHECK-DEC-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-DEC-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-DEC:       while.body:
 ; CHECK-DEC-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-DEC-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-DEC-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-DEC-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-DEC-NEXT:    [[TMP2:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-DEC-NEXT:    br i1 [[TMP2]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-DEC-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-DEC-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-DEC:       while.end:
 ; CHECK-DEC-NEXT:    ret void
 ;
 ; CHECK-PHI-LABEL: @while_gte(
 ; CHECK-PHI-NEXT:  entry:
 ; CHECK-PHI-NEXT:    [[CMP4:%.*]] = icmp slt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-PHI-NEXT:    br i1 [[CMP4]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK-PHI:       while.body.preheader:
-; CHECK-PHI-NEXT:    [[TMP0:%.*]] = add i32 [[I]], 1
-; CHECK-PHI-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[N]]
-; CHECK-PHI-NEXT:    [[TMP2:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP1]])
+; CHECK-PHI-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[I]]
+; CHECK-PHI-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-PHI-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-PHI-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-PHI-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 2
+; CHECK-PHI-NEXT:    [[TMP4:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-PHI-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-PHI:       while.body:
 ; CHECK-PHI-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
-; CHECK-PHI-NEXT:    [[TMP3:%.*]] = phi i32 [ [[TMP2]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP4:%.*]], [[WHILE_BODY]] ]
+; CHECK-PHI-NEXT:    [[TMP5:%.*]] = phi i32 [ [[TMP4]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP6:%.*]], [[WHILE_BODY]] ]
 ; CHECK-PHI-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-PHI-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-PHI-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-PHI-NEXT:    [[TMP4]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP3]], i32 1)
-; CHECK-PHI-NEXT:    [[TMP5:%.*]] = icmp ne i32 [[TMP4]], 0
-; CHECK-PHI-NEXT:    br i1 [[TMP5]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-PHI-NEXT:    [[TMP6]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP5]], i32 1)
+; CHECK-PHI-NEXT:    [[TMP7:%.*]] = icmp ne i32 [[TMP6]], 0
+; CHECK-PHI-NEXT:    br i1 [[TMP7]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-PHI:       while.end:
 ; CHECK-PHI-NEXT:    ret void
 ;
 ; CHECK-NESTED-LABEL: @while_gte(
 ; CHECK-NESTED-NEXT:  entry:
 ; CHECK-NESTED-NEXT:    [[CMP4:%.*]] = icmp slt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-NESTED-NEXT:    br i1 [[CMP4]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK-NESTED:       while.body.preheader:
-; CHECK-NESTED-NEXT:    [[TMP0:%.*]] = add i32 [[I]], 1
-; CHECK-NESTED-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[N]]
-; CHECK-NESTED-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP1]])
+; CHECK-NESTED-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[I]]
+; CHECK-NESTED-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-NESTED-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-NESTED-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-NESTED-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 2
+; CHECK-NESTED-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-NESTED-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-NESTED:       while.body:
 ; CHECK-NESTED-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NESTED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-NESTED-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-NESTED-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-NESTED-NEXT:    [[TMP2:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-NESTED-NEXT:    br i1 [[TMP2]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-NESTED-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-NESTED-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-NESTED:       while.end:
 ; CHECK-NESTED-NEXT:    ret void
 ;
 ; CHECK-GUARD-LABEL: @while_gte(
 ; CHECK-GUARD-NEXT:  entry:
 ; CHECK-GUARD-NEXT:    [[CMP4:%.*]] = icmp slt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-GUARD-NEXT:    br i1 [[CMP4]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK-GUARD:       while.body.preheader:
-; CHECK-GUARD-NEXT:    [[TMP0:%.*]] = add i32 [[I]], 1
-; CHECK-GUARD-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[N]]
-; CHECK-GUARD-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP1]])
+; CHECK-GUARD-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[I]]
+; CHECK-GUARD-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-GUARD-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-GUARD-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-GUARD-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 2
+; CHECK-GUARD-NEXT:    call void @llvm.set.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-GUARD-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-GUARD:       while.body:
 ; CHECK-GUARD-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-GUARD-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-GUARD-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-GUARD-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-GUARD-NEXT:    [[TMP2:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
-; CHECK-GUARD-NEXT:    br i1 [[TMP2]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-GUARD-NEXT:    [[TMP4:%.*]] = call i1 @llvm.loop.decrement.i32(i32 1)
+; CHECK-GUARD-NEXT:    br i1 [[TMP4]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-GUARD:       while.end:
 ; CHECK-GUARD-NEXT:    ret void
 ;
 ; CHECK-PHIGUARD-LABEL: @while_gte(
 ; CHECK-PHIGUARD-NEXT:  entry:
 ; CHECK-PHIGUARD-NEXT:    [[CMP4:%.*]] = icmp slt i32 [[I:%.*]], [[N:%.*]]
 ; CHECK-PHIGUARD-NEXT:    br i1 [[CMP4]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK-PHIGUARD:       while.body.preheader:
-; CHECK-PHIGUARD-NEXT:    [[TMP0:%.*]] = add i32 [[I]], 1
-; CHECK-PHIGUARD-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[N]]
-; CHECK-PHIGUARD-NEXT:    [[TMP2:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP1]])
+; CHECK-PHIGUARD-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[I]]
+; CHECK-PHIGUARD-NEXT:    [[TMP1:%.*]] = sub i32 -1, [[N]]
+; CHECK-PHIGUARD-NEXT:    [[SMAX:%.*]] = call i32 @llvm.smax.i32(i32 [[TMP0]], i32 [[TMP1]])
+; CHECK-PHIGUARD-NEXT:    [[TMP2:%.*]] = add i32 [[I]], [[SMAX]]
+; CHECK-PHIGUARD-NEXT:    [[TMP3:%.*]] = add i32 [[TMP2]], 2
+; CHECK-PHIGUARD-NEXT:    [[TMP4:%.*]] = call i32 @llvm.start.loop.iterations.i32(i32 [[TMP3]])
 ; CHECK-PHIGUARD-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK-PHIGUARD:       while.body:
 ; CHECK-PHIGUARD-NEXT:    [[I_ADDR_05:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[I]], [[WHILE_BODY_PREHEADER]] ]
-; CHECK-PHIGUARD-NEXT:    [[TMP3:%.*]] = phi i32 [ [[TMP2]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP4:%.*]], [[WHILE_BODY]] ]
+; CHECK-PHIGUARD-NEXT:    [[TMP5:%.*]] = phi i32 [ [[TMP4]], [[WHILE_BODY_PREHEADER]] ], [ [[TMP6:%.*]], [[WHILE_BODY]] ]
 ; CHECK-PHIGUARD-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i32 [[I_ADDR_05]]
 ; CHECK-PHIGUARD-NEXT:    store i32 [[I_ADDR_05]], i32* [[ARRAYIDX]], align 4
 ; CHECK-PHIGUARD-NEXT:    [[DEC]] = add nsw i32 [[I_ADDR_05]], -1
-; CHECK-PHIGUARD-NEXT:    [[TMP4]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP3]], i32 1)
-; CHECK-PHIGUARD-NEXT:    [[TMP5:%.*]] = icmp ne i32 [[TMP4]], 0
-; CHECK-PHIGUARD-NEXT:    br i1 [[TMP5]], label [[WHILE_BODY]], label [[WHILE_END]]
+; CHECK-PHIGUARD-NEXT:    [[TMP6]] = call i32 @llvm.loop.decrement.reg.i32(i32 [[TMP5]], i32 1)
+; CHECK-PHIGUARD-NEXT:    [[TMP7:%.*]] = icmp ne i32 [[TMP6]], 0
+; CHECK-PHIGUARD-NEXT:    br i1 [[TMP7]], label [[WHILE_BODY]], label [[WHILE_END]]
 ; CHECK-PHIGUARD:       while.end:
 ; CHECK-PHIGUARD-NEXT:    ret void
 ;
 entry:
   %cmp4 = icmp slt i32 %i, %N
   br i1 %cmp4, label %while.end, label %while.body
 
 while.body:

llvm/test/Transforms/IndVarSimplify/2011-11-01-lftrptr.ll

 define i8 @testnullptrptr(i8* %buf, i8* %end) nounwind {
 ; PTR64-LABEL: @testnullptrptr(
 ; PTR64-NEXT:    [[END1:%.*]] = ptrtoint i8* [[END:%.*]] to i64
 ; PTR64-NEXT:    br label [[LOOPGUARD:%.*]]
 ; PTR64:       loopguard:
 ; PTR64-NEXT:    [[GUARD:%.*]] = icmp ult i8* null, [[END]]
 ; PTR64-NEXT:    br i1 [[GUARD]], label [[PREHEADER:%.*]], label [[EXIT:%.*]]
 ; PTR64:       preheader:
-; PTR64-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* null, i64 [[END1]]
+; PTR64-NEXT:    [[UMAX:%.*]] = call i64 @llvm.umax.i64(i64 [[END1]], i64 1)
+; PTR64-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* null, i64 [[UMAX]]
 ; PTR64-NEXT:    br label [[LOOP:%.*]]
 ; PTR64:       loop:
 ; PTR64-NEXT:    [[P_01_US_US:%.*]] = phi i8* [ null, [[PREHEADER]] ], [ [[GEP:%.*]], [[LOOP]] ]
 ; PTR64-NEXT:    [[GEP]] = getelementptr inbounds i8, i8* [[P_01_US_US]], i64 1
 ; PTR64-NEXT:    [[SNEXT:%.*]] = load i8, i8* [[GEP]], align 1
 ; PTR64-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[GEP]], [[SCEVGEP]]
 ; PTR64-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
 ; PTR64:       exit.loopexit:
 ; PTR64-NEXT:    [[SNEXT_LCSSA:%.*]] = phi i8 [ [[SNEXT]], [[LOOP]] ]
 ; PTR64-NEXT:    br label [[EXIT]]
 ; PTR64:       exit:
 ; PTR64-NEXT:    [[RET:%.*]] = phi i8 [ 0, [[LOOPGUARD]] ], [ [[SNEXT_LCSSA]], [[EXIT_LOOPEXIT]] ]
 ; PTR64-NEXT:    ret i8 [[RET]]
 ;
 ; PTR32-LABEL: @testnullptrptr(
 ; PTR32-NEXT:    [[END1:%.*]] = ptrtoint i8* [[END:%.*]] to i32
 ; PTR32-NEXT:    br label [[LOOPGUARD:%.*]]
 ; PTR32:       loopguard:
 ; PTR32-NEXT:    [[GUARD:%.*]] = icmp ult i8* null, [[END]]
 ; PTR32-NEXT:    br i1 [[GUARD]], label [[PREHEADER:%.*]], label [[EXIT:%.*]]
 ; PTR32:       preheader:
-; PTR32-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* null, i32 [[END1]]
+; PTR32-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[END1]], i32 1)
+; PTR32-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* null, i32 [[UMAX]]
 ; PTR32-NEXT:    br label [[LOOP:%.*]]
 ; PTR32:       loop:
 ; PTR32-NEXT:    [[P_01_US_US:%.*]] = phi i8* [ null, [[PREHEADER]] ], [ [[GEP:%.*]], [[LOOP]] ]
 ; PTR32-NEXT:    [[GEP]] = getelementptr inbounds i8, i8* [[P_01_US_US]], i64 1
 ; PTR32-NEXT:    [[SNEXT:%.*]] = load i8, i8* [[GEP]], align 1
 ; PTR32-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[GEP]], [[SCEVGEP]]
 ; PTR32-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
 ; PTR32:       exit.loopexit:

 
 exit:
   %ret = phi i8 [0, %loopguard], [%snext, %loop]
   ret i8 %ret
 }
 
 define i8 @testptrptr(i8* %buf, i8* %end) nounwind {
 ; PTR64-LABEL: @testptrptr(
-; PTR64-NEXT:    [[BUF2:%.*]] = ptrtoint i8* [[BUF:%.*]] to i64
-; PTR64-NEXT:    [[END1:%.*]] = ptrtoint i8* [[END:%.*]] to i64
 ; PTR64-NEXT:    br label [[LOOPGUARD:%.*]]
 ; PTR64:       loopguard:
-; PTR64-NEXT:    [[GUARD:%.*]] = icmp ult i8* [[BUF]], [[END]]
+; PTR64-NEXT:    [[GUARD:%.*]] = icmp ult i8* [[BUF:%.*]], [[END:%.*]]
 ; PTR64-NEXT:    br i1 [[GUARD]], label [[PREHEADER:%.*]], label [[EXIT:%.*]]
 ; PTR64:       preheader:
-; PTR64-NEXT:    [[TMP1:%.*]] = sub i64 [[END1]], [[BUF2]]
-; PTR64-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* [[BUF]], i64 [[TMP1]]
 ; PTR64-NEXT:    br label [[LOOP:%.*]]
 ; PTR64:       loop:
 ; PTR64-NEXT:    [[P_01_US_US:%.*]] = phi i8* [ [[BUF]], [[PREHEADER]] ], [ [[GEP:%.*]], [[LOOP]] ]
 ; PTR64-NEXT:    [[GEP]] = getelementptr inbounds i8, i8* [[P_01_US_US]], i64 1
 ; PTR64-NEXT:    [[SNEXT:%.*]] = load i8, i8* [[GEP]], align 1
-; PTR64-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[GEP]], [[SCEVGEP]]
-; PTR64-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
+; PTR64-NEXT:    [[CMP:%.*]] = icmp ult i8* [[GEP]], [[END]]
+; PTR64-NEXT:    br i1 [[CMP]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
 ; PTR64:       exit.loopexit:
 ; PTR64-NEXT:    [[SNEXT_LCSSA:%.*]] = phi i8 [ [[SNEXT]], [[LOOP]] ]
 ; PTR64-NEXT:    br label [[EXIT]]
 ; PTR64:       exit:
 ; PTR64-NEXT:    [[RET:%.*]] = phi i8 [ 0, [[LOOPGUARD]] ], [ [[SNEXT_LCSSA]], [[EXIT_LOOPEXIT]] ]
 ; PTR64-NEXT:    ret i8 [[RET]]
 ;
 ; PTR32-LABEL: @testptrptr(
-; PTR32-NEXT:    [[BUF2:%.*]] = ptrtoint i8* [[BUF:%.*]] to i32
-; PTR32-NEXT:    [[END1:%.*]] = ptrtoint i8* [[END:%.*]] to i32
 ; PTR32-NEXT:    br label [[LOOPGUARD:%.*]]
 ; PTR32:       loopguard:
-; PTR32-NEXT:    [[GUARD:%.*]] = icmp ult i8* [[BUF]], [[END]]
+; PTR32-NEXT:    [[GUARD:%.*]] = icmp ult i8* [[BUF:%.*]], [[END:%.*]]
 ; PTR32-NEXT:    br i1 [[GUARD]], label [[PREHEADER:%.*]], label [[EXIT:%.*]]
 ; PTR32:       preheader:
-; PTR32-NEXT:    [[TMP1:%.*]] = sub i32 [[END1]], [[BUF2]]
-; PTR32-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, i8* [[BUF]], i32 [[TMP1]]
 ; PTR32-NEXT:    br label [[LOOP:%.*]]
 ; PTR32:       loop:
 ; PTR32-NEXT:    [[P_01_US_US:%.*]] = phi i8* [ [[BUF]], [[PREHEADER]] ], [ [[GEP:%.*]], [[LOOP]] ]
 ; PTR32-NEXT:    [[GEP]] = getelementptr inbounds i8, i8* [[P_01_US_US]], i64 1
 ; PTR32-NEXT:    [[SNEXT:%.*]] = load i8, i8* [[GEP]], align 1
-; PTR32-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[GEP]], [[SCEVGEP]]
-; PTR32-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
+; PTR32-NEXT:    [[CMP:%.*]] = icmp ult i8* [[GEP]], [[END]]
+; PTR32-NEXT:    br i1 [[CMP]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
 ; PTR32:       exit.loopexit:
 ; PTR32-NEXT:    [[SNEXT_LCSSA:%.*]] = phi i8 [ [[SNEXT]], [[LOOP]] ]
 ; PTR32-NEXT:    br label [[EXIT]]
 ; PTR32:       exit:
 ; PTR32-NEXT:    [[RET:%.*]] = phi i8 [ 0, [[LOOPGUARD]] ], [ [[SNEXT_LCSSA]], [[EXIT_LOOPEXIT]] ]
 ; PTR32-NEXT:    ret i8 [[RET]]
 ;
   br label %loopguard

llvm/test/Transforms/IndVarSimplify/ARM/code-size.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -mtriple=thumbv8m.main -indvars -S < %s | FileCheck %s --check-prefix=CHECK-V8M
 ; RUN: opt -mtriple=thumbv8a -indvars -S < %s | FileCheck %s --check-prefix=CHECK-V8A
 
 define i32 @remove_loop(i32 %size) #0 {
 ; CHECK-V8M-LABEL: @remove_loop(
 ; CHECK-V8M-NEXT:  entry:
-; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[SIZE:%.*]], 31
-; CHECK-V8M-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[SIZE]], i32 31)
-; CHECK-V8M-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[UMIN]]
-; CHECK-V8M-NEXT:    [[TMP2:%.*]] = lshr i32 [[TMP1]], 5
-; CHECK-V8M-NEXT:    [[TMP3:%.*]] = shl nuw i32 [[TMP2]], 5
+; CHECK-V8M-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[SIZE:%.*]]
+; CHECK-V8M-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[TMP0]], i32 -32)
+; CHECK-V8M-NEXT:    [[TMP1:%.*]] = add i32 [[SIZE]], [[UMAX]]
+; CHECK-V8M-NEXT:    [[TMP2:%.*]] = add i32 [[TMP1]], 32
+; CHECK-V8M-NEXT:    [[TMP3:%.*]] = lshr i32 [[TMP2]], 5
+; CHECK-V8M-NEXT:    [[TMP4:%.*]] = shl nuw i32 [[TMP3]], 5
 ; CHECK-V8M-NEXT:    br label [[WHILE_COND:%.*]]
 ; CHECK-V8M:       while.cond:
 ; CHECK-V8M-NEXT:    br i1 false, label [[WHILE_COND]], label [[WHILE_END:%.*]]
 ; CHECK-V8M:       while.end:
-; CHECK-V8M-NEXT:    [[TMP4:%.*]] = sub i32 [[SIZE]], [[TMP3]]
-; CHECK-V8M-NEXT:    ret i32 [[TMP4]]
+; CHECK-V8M-NEXT:    [[TMP5:%.*]] = sub i32 [[SIZE]], [[TMP4]]
+; CHECK-V8M-NEXT:    ret i32 [[TMP5]]
 ;
 ; CHECK-V8A-LABEL: @remove_loop(
 ; CHECK-V8A-NEXT:  entry:
-; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[SIZE:%.*]], 31
-; CHECK-V8A-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[SIZE]], i32 31)
-; CHECK-V8A-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[UMIN]]
-; CHECK-V8A-NEXT:    [[TMP2:%.*]] = lshr i32 [[TMP1]], 5
-; CHECK-V8A-NEXT:    [[TMP3:%.*]] = shl nuw i32 [[TMP2]], 5
+; CHECK-V8A-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[SIZE:%.*]]
+; CHECK-V8A-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[TMP0]], i32 -32)
+; CHECK-V8A-NEXT:    [[TMP1:%.*]] = add i32 [[SIZE]], [[UMAX]]
+; CHECK-V8A-NEXT:    [[TMP2:%.*]] = add i32 [[TMP1]], 32
+; CHECK-V8A-NEXT:    [[TMP3:%.*]] = lshr i32 [[TMP2]], 5
+; CHECK-V8A-NEXT:    [[TMP4:%.*]] = shl nuw i32 [[TMP3]], 5
 ; CHECK-V8A-NEXT:    br label [[WHILE_COND:%.*]]
 ; CHECK-V8A:       while.cond:
 ; CHECK-V8A-NEXT:    br i1 false, label [[WHILE_COND]], label [[WHILE_END:%.*]]
 ; CHECK-V8A:       while.end:
-; CHECK-V8A-NEXT:    [[TMP4:%.*]] = sub i32 [[SIZE]], [[TMP3]]
-; CHECK-V8A-NEXT:    ret i32 [[TMP4]]
+; CHECK-V8A-NEXT:    [[TMP5:%.*]] = sub i32 [[SIZE]], [[TMP4]]
+; CHECK-V8A-NEXT:    ret i32 [[TMP5]]
 ;
 entry:
   br label %while.cond
 
 while.cond:                                       ; preds = %while.cond, %entry
   %size.addr.0 = phi i32 [ %size, %entry ], [ %sub, %while.cond ]
   %cmp = icmp ugt i32 %size.addr.0, 31
   %sub = add i32 %size.addr.0, -32

 ; CHECK-V8M-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8M-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0:![0-9]+]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]

 ; CHECK-V8A-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8A-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0:![0-9]+]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]

 ; CHECK-V8M-NEXT:  loop.preheader:
 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[N:%.*]], -1
 ; CHECK-V8M-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]
 ; CHECK-V8M-NEXT:    [[I_NEXT]] = add nuw i32 [[I]], 1
 ; CHECK-V8M-NEXT:    [[CONTINUE:%.*]] = icmp ne i32 [[I_NEXT]], [[N]]
 ; CHECK-V8M-NEXT:    br i1 [[CONTINUE]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-V8M:       exit:
 ; CHECK-V8M-NEXT:    [[RESULT:%.*]] = phi i32 [ [[LOOP_ACC_NEXT]], [[GUARDED]] ]
 ; CHECK-V8M-NEXT:    ret i32 [[RESULT]]
 ;
 ; CHECK-V8A-LABEL: @test2(
 ; CHECK-V8A-NEXT:  loop.preheader:
 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[N:%.*]], -1
 ; CHECK-V8A-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]

 ; CHECK-V8M-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8M-NEXT:    [[UMIN2:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN1]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[UMIN]], [[UMIN2]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]

 ; CHECK-V8A-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8A-NEXT:    [[UMIN2:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN1]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[UMIN]], [[UMIN2]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]

 ; CHECK-V8M-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8M-NEXT:    [[UMIN4:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN3]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[UMIN1]], [[UMIN4]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]

 ; CHECK-V8A-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8A-NEXT:    [[UMIN4:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN3]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[UMIN1]], [[UMIN4]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]

 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8M-NEXT:    [[UMIN1:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH_1]], [[UMIN1]]
 ; CHECK-V8M-NEXT:    [[TMP2:%.*]] = icmp ne i32 [[LENGTH_2]], [[UMIN1]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED1:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED1]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]
-; CHECK-V8M-NEXT:    br i1 [[TMP2]], label [[GUARDED1]], label [[DEOPT2:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP2]], label [[GUARDED1]], label [[DEOPT2:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt2:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded1:
 ; CHECK-V8M-NEXT:    [[ARRAY_3_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_3:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_3_I:%.*]] = load i32, i32* [[ARRAY_3_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC_1]], [[ARRAY_3_I]]
 ; CHECK-V8M-NEXT:    [[I_NEXT]] = add nuw i32 [[I]], 1

 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8A-NEXT:    [[UMIN1:%.*]] = call i32 @llvm.umin.i32(i32 [[UMIN]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH_1]], [[UMIN1]]
 ; CHECK-V8A-NEXT:    [[TMP2:%.*]] = icmp ne i32 [[LENGTH_2]], [[UMIN1]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED1:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED1]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]
-; CHECK-V8A-NEXT:    br i1 [[TMP2]], label [[GUARDED1]], label [[DEOPT2:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP2]], label [[GUARDED1]], label [[DEOPT2:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt2:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded1:
 ; CHECK-V8A-NEXT:    [[ARRAY_3_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_3:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_3_I:%.*]] = load i32, i32* [[ARRAY_3_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC_1]], [[ARRAY_3_I]]
 ; CHECK-V8A-NEXT:    [[I_NEXT]] = add nuw i32 [[I]], 1

 ; CHECK-V8M-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8M-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8M-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED1:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED1]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]
-; CHECK-V8M-NEXT:    br i1 true, label [[GUARDED1]], label [[DEOPT2:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 true, label [[GUARDED1]], label [[DEOPT2:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt2:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded1:
 ; CHECK-V8M-NEXT:    [[ARRAY_3_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_3:%.*]], i64 [[I_I64]]
 ; CHECK-V8M-NEXT:    [[ARRAY_3_I:%.*]] = load i32, i32* [[ARRAY_3_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC_1]], [[ARRAY_3_I]]
 ; CHECK-V8M-NEXT:    [[I_NEXT]] = add nuw i32 [[I]], 1
 ; CHECK-V8M-NEXT:    [[CONTINUE:%.*]] = icmp ult i32 [[I_NEXT]], [[N]]
 ; CHECK-V8M-NEXT:    br i1 [[CONTINUE]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-V8M:       exit:
 ; CHECK-V8M-NEXT:    [[RESULT:%.*]] = phi i32 [ [[LOOP_ACC_NEXT]], [[GUARDED1]] ]
 ; CHECK-V8M-NEXT:    ret i32 [[RESULT]]
 ;
 ; CHECK-V8A-LABEL: @duplicate_checks(
 ; CHECK-V8A-NEXT:  loop.preheader:
 ; CHECK-V8A-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[N:%.*]], i32 1)
 ; CHECK-V8A-NEXT:    [[TMP0:%.*]] = add i32 [[UMAX]], -1
 ; CHECK-V8A-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[LENGTH:%.*]], i32 [[TMP0]])
 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = icmp ne i32 [[LENGTH]], [[UMIN]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED1:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i32 [ [[I_NEXT:%.*]], [[GUARDED1]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP1]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[I_I64:%.*]] = zext i32 [[I]] to i64
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_1:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_1_I:%.*]] = load i32, i32* [[ARRAY_1_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_1:%.*]] = add i32 [[LOOP_ACC]], [[ARRAY_1_I]]
-; CHECK-V8A-NEXT:    br i1 true, label [[GUARDED1]], label [[DEOPT2:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 true, label [[GUARDED1]], label [[DEOPT2:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt2:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded1:
 ; CHECK-V8A-NEXT:    [[ARRAY_3_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY_3:%.*]], i64 [[I_I64]]
 ; CHECK-V8A-NEXT:    [[ARRAY_3_I:%.*]] = load i32, i32* [[ARRAY_3_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC_1]], [[ARRAY_3_I]]
 ; CHECK-V8A-NEXT:    [[I_NEXT]] = add nuw i32 [[I]], 1

 ; CHECK-V8M-NEXT:    [[TMP1:%.*]] = zext i32 [[LENGTH:%.*]] to i64
 ; CHECK-V8M-NEXT:    [[UMIN:%.*]] = call i64 @llvm.umin.i64(i64 [[TMP0]], i64 [[TMP1]])
 ; CHECK-V8M-NEXT:    [[TMP2:%.*]] = zext i32 [[LENGTH]] to i64
 ; CHECK-V8M-NEXT:    [[TMP3:%.*]] = icmp ne i64 [[TMP2]], [[UMIN]]
 ; CHECK-V8M-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8M:       loop:
 ; CHECK-V8M-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8M-NEXT:    [[I:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8M-NEXT:    br i1 [[TMP3]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8M-NEXT:    br i1 [[TMP3]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8M:       deopt:
 ; CHECK-V8M-NEXT:    call void @prevent_merging()
 ; CHECK-V8M-NEXT:    ret i32 -1
 ; CHECK-V8M:       guarded:
 ; CHECK-V8M-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I]]
 ; CHECK-V8M-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8M-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]
 ; CHECK-V8M-NEXT:    [[I_NEXT]] = add nuw nsw i64 [[I]], 1

 ; CHECK-V8A-NEXT:    [[TMP1:%.*]] = zext i32 [[LENGTH:%.*]] to i64
 ; CHECK-V8A-NEXT:    [[UMIN:%.*]] = call i64 @llvm.umin.i64(i64 [[TMP0]], i64 [[TMP1]])
 ; CHECK-V8A-NEXT:    [[TMP2:%.*]] = zext i32 [[LENGTH]] to i64
 ; CHECK-V8A-NEXT:    [[TMP3:%.*]] = icmp ne i64 [[TMP2]], [[UMIN]]
 ; CHECK-V8A-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-V8A:       loop:
 ; CHECK-V8A-NEXT:    [[LOOP_ACC:%.*]] = phi i32 [ [[LOOP_ACC_NEXT:%.*]], [[GUARDED:%.*]] ], [ 0, [[LOOP_PREHEADER:%.*]] ]
 ; CHECK-V8A-NEXT:    [[I:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[GUARDED]] ], [ 0, [[LOOP_PREHEADER]] ]
-; CHECK-V8A-NEXT:    br i1 [[TMP3]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0
+; CHECK-V8A-NEXT:    br i1 [[TMP3]], label [[GUARDED]], label [[DEOPT:%.*]], !prof [[PROF0]]
 ; CHECK-V8A:       deopt:
 ; CHECK-V8A-NEXT:    call void @prevent_merging()
 ; CHECK-V8A-NEXT:    ret i32 -1
 ; CHECK-V8A:       guarded:
 ; CHECK-V8A-NEXT:    [[ARRAY_I_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAY:%.*]], i64 [[I]]
 ; CHECK-V8A-NEXT:    [[ARRAY_I:%.*]] = load i32, i32* [[ARRAY_I_PTR]], align 4
 ; CHECK-V8A-NEXT:    [[LOOP_ACC_NEXT]] = add i32 [[LOOP_ACC]], [[ARRAY_I]]
 ; CHECK-V8A-NEXT:    [[I_NEXT]] = add nuw nsw i64 [[I]], 1

llvm/test/Transforms/IndVarSimplify/lftr-address-space-pointers.ll

 ; CHECK-NEXT:    br i1 [[CMP1]], label [[FOR_BODY_PREHEADER:%.*]], label [[FOR_END:%.*]]
 ; CHECK:       for.body.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[P_02:%.*]] = phi i8 addrspace(2)* [ [[INCDEC_PTR:%.*]], [[FOR_BODY]] ], [ [[BASE]], [[FOR_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[SUB_PTR_LHS_CAST:%.*]] = ptrtoint i8 addrspace(2)* [[P_02]] to i8
 ; CHECK-NEXT:    [[SUB_PTR_RHS_CAST:%.*]] = ptrtoint i8 addrspace(2)* [[BASE]] to i8
 ; CHECK-NEXT:    [[SUB_PTR_SUB:%.*]] = sub i8 [[SUB_PTR_LHS_CAST]], [[SUB_PTR_RHS_CAST]]
-; CHECK-NEXT:    store i8 [[SUB_PTR_SUB]], i8 addrspace(2)* [[P_02]]
+; CHECK-NEXT:    store i8 [[SUB_PTR_SUB]], i8 addrspace(2)* [[P_02]], align 1
 ; CHECK-NEXT:    [[INCDEC_PTR]] = getelementptr inbounds i8, i8 addrspace(2)* [[P_02]], i32 1
-; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i8 addrspace(2)* [[INCDEC_PTR]], [[ADD_PTR]]
-; CHECK-NEXT:    br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
+; CHECK-NEXT:    [[CMP:%.*]] = icmp ult i8 addrspace(2)* [[INCDEC_PTR]], [[ADD_PTR]]
+; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
 ; CHECK:       for.end.loopexit:
 ; CHECK-NEXT:    br label [[FOR_END]]
 ; CHECK:       for.end:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %idx.trunc = trunc i32 %n to i8
   %add.ptr = getelementptr inbounds i8, i8 addrspace(2)* %base, i8 %idx.trunc

 ; CHECK:       for.body.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[P_02:%.*]] = phi i8 addrspace(3)* [ [[INCDEC_PTR:%.*]], [[FOR_BODY]] ], [ [[BASE]], [[FOR_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[SUB_PTR_LHS_CAST:%.*]] = ptrtoint i8 addrspace(3)* [[P_02]] to i16
 ; CHECK-NEXT:    [[SUB_PTR_RHS_CAST:%.*]] = ptrtoint i8 addrspace(3)* [[BASE]] to i16
 ; CHECK-NEXT:    [[SUB_PTR_SUB:%.*]] = sub i16 [[SUB_PTR_LHS_CAST]], [[SUB_PTR_RHS_CAST]]
 ; CHECK-NEXT:    [[CONV:%.*]] = trunc i16 [[SUB_PTR_SUB]] to i8
-; CHECK-NEXT:    store i8 [[CONV]], i8 addrspace(3)* [[P_02]]
+; CHECK-NEXT:    store i8 [[CONV]], i8 addrspace(3)* [[P_02]], align 1
 ; CHECK-NEXT:    [[INCDEC_PTR]] = getelementptr inbounds i8, i8 addrspace(3)* [[P_02]], i32 1
-; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i8 addrspace(3)* [[INCDEC_PTR]], [[ADD_PTR]]
-; CHECK-NEXT:    br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
+; CHECK-NEXT:    [[CMP:%.*]] = icmp ult i8 addrspace(3)* [[INCDEC_PTR]], [[ADD_PTR]]
+; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
 ; CHECK:       for.end.loopexit:
 ; CHECK-NEXT:    br label [[FOR_END]]
 ; CHECK:       for.end:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %idx.trunc = trunc i32 %n to i16
   %add.ptr = getelementptr inbounds i8, i8 addrspace(3)* %base, i16 %idx.trunc

llvm/test/Transforms/IndVarSimplify/lftr-reuse.ll

 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[P_02:%.*]] = phi i8* [ [[INCDEC_PTR:%.*]], [[FOR_BODY]] ], [ [[BASE]], [[FOR_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[SUB_PTR_LHS_CAST:%.*]] = ptrtoint i8* [[P_02]] to i64
 ; CHECK-NEXT:    [[SUB_PTR_RHS_CAST:%.*]] = ptrtoint i8* [[BASE]] to i64
 ; CHECK-NEXT:    [[SUB_PTR_SUB:%.*]] = sub i64 [[SUB_PTR_LHS_CAST]], [[SUB_PTR_RHS_CAST]]
 ; CHECK-NEXT:    [[CONV:%.*]] = trunc i64 [[SUB_PTR_SUB]] to i8
 ; CHECK-NEXT:    store i8 [[CONV]], i8* [[P_02]], align 1
 ; CHECK-NEXT:    [[INCDEC_PTR]] = getelementptr inbounds i8, i8* [[P_02]], i32 1
-; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[INCDEC_PTR]], [[ADD_PTR]]
-; CHECK-NEXT:    br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
+; CHECK-NEXT:    [[CMP:%.*]] = icmp ult i8* [[INCDEC_PTR]], [[ADD_PTR]]
+; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
 ; CHECK:       for.end.loopexit:
 ; CHECK-NEXT:    br label [[FOR_END]]
 ; CHECK:       for.end:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %idx.ext = sext i32 %n to i64
   %add.ptr = getelementptr inbounds i8, i8* %base, i64 %idx.ext

 define void @aryptriv([256 x i8]* %base, i32 %n) nounwind {
 ; CHECK-LABEL: @aryptriv(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[IVSTART:%.*]] = getelementptr inbounds [256 x i8], [256 x i8]* [[BASE:%.*]], i32 0, i32 0
 ; CHECK-NEXT:    [[IVEND:%.*]] = getelementptr inbounds [256 x i8], [256 x i8]* [[BASE]], i32 0, i32 [[N:%.*]]
 ; CHECK-NEXT:    [[CMP_PH:%.*]] = icmp ult i8* [[IVSTART]], [[IVEND]]
 ; CHECK-NEXT:    br i1 [[CMP_PH]], label [[LOOP_PREHEADER:%.*]], label [[EXIT:%.*]]
 ; CHECK:       loop.preheader:
-; CHECK-NEXT:    [[TMP0:%.*]] = sext i32 [[N]] to i64
-; CHECK-NEXT:    [[SCEVGEP:%.*]] = getelementptr [256 x i8], [256 x i8]* [[BASE]], i64 0, i64 [[TMP0]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[APTR:%.*]] = phi i8* [ [[INCDEC_PTR:%.*]], [[LOOP]] ], [ [[IVSTART]], [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[INCDEC_PTR]] = getelementptr inbounds i8, i8* [[APTR]], i32 1
 ; CHECK-NEXT:    store i8 3, i8* [[APTR]], align 1
-; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i8* [[INCDEC_PTR]], [[SCEVGEP]]
-; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
+; CHECK-NEXT:    [[CMP:%.*]] = icmp ult i8* [[INCDEC_PTR]], [[IVEND]]
+; CHECK-NEXT:    br i1 [[CMP]], label [[LOOP]], label [[EXIT_LOOPEXIT:%.*]]
 ; CHECK:       exit.loopexit:
 ; CHECK-NEXT:    br label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %ivstart = getelementptr inbounds [256 x i8], [256 x i8]* %base, i32 0, i32 0
   %ivend = getelementptr inbounds [256 x i8], [256 x i8]* %base, i32 0, i32 %n

llvm/test/Transforms/IndVarSimplify/replace-loop-exit-folds.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -indvars -S -indvars-predicate-loops=0 < %s | FileCheck %s
 
 target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
 
 define i32 @remove_loop(i32 %size) {
 ; CHECK-LABEL: @remove_loop(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[TMP0:%.*]] = add i32 [[SIZE:%.*]], 31
-; CHECK-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[SIZE]], i32 31)
-; CHECK-NEXT:    [[TMP1:%.*]] = sub i32 [[TMP0]], [[UMIN]]
-; CHECK-NEXT:    [[TMP2:%.*]] = lshr i32 [[TMP1]], 5
-; CHECK-NEXT:    [[TMP3:%.*]] = shl nuw i32 [[TMP2]], 5
+; CHECK-NEXT:    [[TMP0:%.*]] = sub i32 -1, [[SIZE:%.*]]
+; CHECK-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[TMP0]], i32 -32)
+; CHECK-NEXT:    [[TMP1:%.*]] = add i32 [[SIZE]], [[UMAX]]
+; CHECK-NEXT:    [[TMP2:%.*]] = add i32 [[TMP1]], 32
+; CHECK-NEXT:    [[TMP3:%.*]] = lshr i32 [[TMP2]], 5
+; CHECK-NEXT:    [[TMP4:%.*]] = shl nuw i32 [[TMP3]], 5
 ; CHECK-NEXT:    br label [[WHILE_COND:%.*]]
 ; CHECK:       while.cond:
 ; CHECK-NEXT:    [[SIZE_ADDR_0:%.*]] = phi i32 [ [[SIZE]], [[ENTRY:%.*]] ], [ [[SUB:%.*]], [[WHILE_COND]] ]
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp ugt i32 [[SIZE_ADDR_0]], 31
 ; CHECK-NEXT:    [[SUB]] = add i32 [[SIZE_ADDR_0]], -32
 ; CHECK-NEXT:    br i1 [[CMP]], label [[WHILE_COND]], label [[WHILE_END:%.*]]
 ; CHECK:       while.end:
-; CHECK-NEXT:    [[TMP4:%.*]] = sub i32 [[SIZE]], [[TMP3]]
-; CHECK-NEXT:    ret i32 [[TMP4]]
+; CHECK-NEXT:    [[TMP5:%.*]] = sub i32 [[SIZE]], [[TMP4]]
+; CHECK-NEXT:    ret i32 [[TMP5]]
 ;
 entry:
   br label %while.cond
 
 while.cond:                                       ; preds = %while.cond, %entry
   %size.addr.0 = phi i32 [ %size, %entry ], [ %sub, %while.cond ]
   %cmp = icmp ugt i32 %size.addr.0, 31
   %sub = add i32 %size.addr.0, -32

llvm/test/Transforms/LoopVectorize/X86/pr35432.ll

 ; CHECK-NEXT:    [[CALL:%.*]] = call i32 (i32*, ...) bitcast (i32 (...)* @goo to i32 (i32*, ...)*)(i32* nonnull [[I]])
 ; CHECK-NEXT:    [[TMP2:%.*]] = load i32, i32* [[I]], align 4
 ; CHECK-NEXT:    [[STOREMERGE6:%.*]] = trunc i32 [[TMP2]] to i16
 ; CHECK-NEXT:    store i16 [[STOREMERGE6]], i16* [[S]], align 2
 ; CHECK-NEXT:    [[CONV17:%.*]] = and i32 [[TMP2]], 65472
 ; CHECK-NEXT:    [[CMP8:%.*]] = icmp eq i32 [[CONV17]], 0
 ; CHECK-NEXT:    br i1 [[CMP8]], label [[FOR_BODY_LR_PH:%.*]], label [[FOR_END12:%.*]]
 ; CHECK:       for.body.lr.ph:
+; CHECK-NEXT:    [[TMP3:%.*]] = sub i32 -1, [[TMP2]]
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[STOREMERGE_IN9:%.*]] = phi i32 [ [[TMP2]], [[FOR_BODY_LR_PH]] ], [ [[ADD:%.*]], [[FOR_INC9:%.*]] ]
 ; CHECK-NEXT:    [[CONV52:%.*]] = and i32 [[STOREMERGE_IN9]], 255
 ; CHECK-NEXT:    [[CMP63:%.*]] = icmp ult i32 [[TMP2]], [[CONV52]]
 ; CHECK-NEXT:    br i1 [[CMP63]], label [[FOR_BODY8_LR_PH:%.*]], label [[FOR_INC9]]
 ; CHECK:       for.body8.lr.ph:
 ; CHECK-NEXT:    [[CONV3:%.*]] = trunc i32 [[STOREMERGE_IN9]] to i8
 ; CHECK-NEXT:    [[DOTPROMOTED:%.*]] = load i32, i32* getelementptr inbounds ([192 x [192 x i32]], [192 x [192 x i32]]* @a, i64 0, i64 0, i64 0), align 16
-; CHECK-NEXT:    [[TMP3:%.*]] = add i8 [[CONV3]], -1
-; CHECK-NEXT:    [[TMP4:%.*]] = zext i8 [[TMP3]] to i32
-; CHECK-NEXT:    [[TMP5:%.*]] = add i32 [[TMP4]], 1
-; CHECK-NEXT:    [[UMIN1:%.*]] = call i32 @llvm.umin.i32(i32 [[TMP2]], i32 [[TMP4]])
-; CHECK-NEXT:    [[TMP6:%.*]] = sub i32 [[TMP5]], [[UMIN1]]
-; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[TMP6]], 8
+; CHECK-NEXT:    [[TMP4:%.*]] = add i8 [[CONV3]], -1
+; CHECK-NEXT:    [[TMP5:%.*]] = zext i8 [[TMP4]] to i32
+; CHECK-NEXT:    [[TMP6:%.*]] = sub i32 -1, [[TMP5]]
+; CHECK-NEXT:    [[UMAX1:%.*]] = call i32 @llvm.umax.i32(i32 [[TMP3]], i32 [[TMP6]])
+; CHECK-NEXT:    [[TMP7:%.*]] = add i32 [[UMAX1]], 2
+; CHECK-NEXT:    [[TMP8:%.*]] = add i32 [[TMP7]], [[TMP5]]
+; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[TMP8]], 8
 ; CHECK-NEXT:    br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_SCEVCHECK:%.*]]
 ; CHECK:       vector.scevcheck:
-; CHECK-NEXT:    [[TMP7:%.*]] = add i8 [[CONV3]], -1
-; CHECK-NEXT:    [[TMP8:%.*]] = zext i8 [[TMP7]] to i32
-; CHECK-NEXT:    [[UMIN:%.*]] = call i32 @llvm.umin.i32(i32 [[TMP2]], i32 [[TMP8]])
-; CHECK-NEXT:    [[TMP9:%.*]] = sub i32 [[TMP8]], [[UMIN]]
-; CHECK-NEXT:    [[TMP10:%.*]] = trunc i32 [[TMP9]] to i8
-; CHECK-NEXT:    [[MUL:%.*]] = call { i8, i1 } @llvm.umul.with.overflow.i8(i8 1, i8 [[TMP10]])
+; CHECK-NEXT:    [[TMP9:%.*]] = add i8 [[CONV3]], -1
+; CHECK-NEXT:    [[TMP10:%.*]] = zext i8 [[TMP9]] to i32
+; CHECK-NEXT:    [[TMP11:%.*]] = sub i32 -1, [[TMP10]]
+; CHECK-NEXT:    [[UMAX:%.*]] = call i32 @llvm.umax.i32(i32 [[TMP3]], i32 [[TMP11]])
+; CHECK-NEXT:    [[TMP12:%.*]] = add i32 [[UMAX]], 1
+; CHECK-NEXT:    [[TMP13:%.*]] = add i32 [[TMP12]], [[TMP10]]
+; CHECK-NEXT:    [[TMP14:%.*]] = trunc i32 [[TMP13]] to i8
+; CHECK-NEXT:    [[MUL:%.*]] = call { i8, i1 } @llvm.umul.with.overflow.i8(i8 1, i8 [[TMP14]])
 ; CHECK-NEXT:    [[MUL_RESULT:%.*]] = extractvalue { i8, i1 } [[MUL]], 0
 ; CHECK-NEXT:    [[MUL_OVERFLOW:%.*]] = extractvalue { i8, i1 } [[MUL]], 1
-; CHECK-NEXT:    [[TMP11:%.*]] = add i8 [[TMP7]], [[MUL_RESULT]]
-; CHECK-NEXT:    [[TMP12:%.*]] = sub i8 [[TMP7]], [[MUL_RESULT]]
-; CHECK-NEXT:    [[TMP13:%.*]] = icmp ugt i8 [[TMP12]], [[TMP7]]
-; CHECK-NEXT:    [[TMP14:%.*]] = icmp ult i8 [[TMP11]], [[TMP7]]
-; CHECK-NEXT:    [[TMP15:%.*]] = select i1 true, i1 [[TMP13]], i1 [[TMP14]]
-; CHECK-NEXT:    [[TMP16:%.*]] = icmp ugt i32 [[TMP9]], 255
-; CHECK-NEXT:    [[TMP17:%.*]] = or i1 [[TMP15]], [[TMP16]]
-; CHECK-NEXT:    [[TMP18:%.*]] = or i1 [[TMP17]], [[MUL_OVERFLOW]]
-; CHECK-NEXT:    [[TMP19:%.*]] = or i1 false, [[TMP18]]
-; CHECK-NEXT:    br i1 [[TMP19]], label [[SCALAR_PH]], label [[VECTOR_PH:%.*]]
+; CHECK-NEXT:    [[TMP15:%.*]] = add i8 [[TMP9]], [[MUL_RESULT]]
+; CHECK-NEXT:    [[TMP16:%.*]] = sub i8 [[TMP9]], [[MUL_RESULT]]
+; CHECK-NEXT:    [[TMP17:%.*]] = icmp ugt i8 [[TMP16]], [[TMP9]]
+; CHECK-NEXT:    [[TMP18:%.*]] = icmp ult i8 [[TMP15]], [[TMP9]]
+; CHECK-NEXT:    [[TMP19:%.*]] = select i1 true, i1 [[TMP17]], i1 [[TMP18]]
+; CHECK-NEXT:    [[TMP20:%.*]] = icmp ugt i32 [[TMP13]], 255
+; CHECK-NEXT:    [[TMP21:%.*]] = or i1 [[TMP19]], [[TMP20]]
+; CHECK-NEXT:    [[TMP22:%.*]] = or i1 [[TMP21]], [[MUL_OVERFLOW]]
+; CHECK-NEXT:    [[TMP23:%.*]] = or i1 false, [[TMP22]]
+; CHECK-NEXT:    br i1 [[TMP23]], label [[SCALAR_PH]], label [[VECTOR_PH:%.*]]
 ; CHECK:       vector.ph:
-; CHECK-NEXT:    [[N_MOD_VF:%.*]] = urem i32 [[TMP6]], 8
-; CHECK-NEXT:    [[N_VEC:%.*]] = sub i32 [[TMP6]], [[N_MOD_VF]]
+; CHECK-NEXT:    [[N_MOD_VF:%.*]] = urem i32 [[TMP8]], 8
+; CHECK-NEXT:    [[N_VEC:%.*]] = sub i32 [[TMP8]], [[N_MOD_VF]]
 ; CHECK-NEXT:    [[CAST_CRD:%.*]] = trunc i32 [[N_VEC]] to i8
 ; CHECK-NEXT:    [[IND_END:%.*]] = sub i8 [[CONV3]], [[CAST_CRD]]
-; CHECK-NEXT:    [[TMP20:%.*]] = insertelement <4 x i32> zeroinitializer, i32 [[DOTPROMOTED]], i32 0
+; CHECK-NEXT:    [[TMP24:%.*]] = insertelement <4 x i32> zeroinitializer, i32 [[DOTPROMOTED]], i32 0
 ; CHECK-NEXT:    br label [[VECTOR_BODY:%.*]]
 ; CHECK:       vector.body:
 ; CHECK-NEXT:    [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
-; CHECK-NEXT:    [[VEC_PHI:%.*]] = phi <4 x i32> [ [[TMP20]], [[VECTOR_PH]] ], [ [[TMP24:%.*]], [[VECTOR_BODY]] ]
-; CHECK-NEXT:    [[VEC_PHI2:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP25:%.*]], [[VECTOR_BODY]] ]
-; CHECK-NEXT:    [[TMP21:%.*]] = trunc i32 [[INDEX]] to i8
-; CHECK-NEXT:    [[OFFSET_IDX:%.*]] = sub i8 [[CONV3]], [[TMP21]]
-; CHECK-NEXT:    [[TMP22:%.*]] = add i8 [[OFFSET_IDX]], 0
-; CHECK-NEXT:    [[TMP23:%.*]] = add i8 [[OFFSET_IDX]], -4
-; CHECK-NEXT:    [[TMP24]] = add <4 x i32> [[VEC_PHI]], <i32 1, i32 1, i32 1, i32 1>
-; CHECK-NEXT:    [[TMP25]] = add <4 x i32> [[VEC_PHI2]], <i32 1, i32 1, i32 1, i32 1>
-; CHECK-NEXT:    [[TMP26:%.*]] = add i8 [[TMP22]], -1
-; CHECK-NEXT:    [[TMP27:%.*]] = add i8 [[TMP23]], -1
-; CHECK-NEXT:    [[TMP28:%.*]] = zext i8 [[TMP26]] to i32
-; CHECK-NEXT:    [[TMP29:%.*]] = zext i8 [[TMP27]] to i32
+; CHECK-NEXT:    [[VEC_PHI:%.*]] = phi <4 x i32> [ [[TMP24]], [[VECTOR_PH]] ], [ [[TMP28:%.*]], [[VECTOR_BODY]] ]
+; CHECK-NEXT:    [[VEC_PHI2:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP29:%.*]], [[VECTOR_BODY]] ]
+; CHECK-NEXT:    [[TMP25:%.*]] = trunc i32 [[INDEX]] to i8
+; CHECK-NEXT:    [[OFFSET_IDX:%.*]] = sub i8 [[CONV3]], [[TMP25]]
+; CHECK-NEXT:    [[TMP26:%.*]] = add i8 [[OFFSET_IDX]], 0
+; CHECK-NEXT:    [[TMP27:%.*]] = add i8 [[OFFSET_IDX]], -4
+; CHECK-NEXT:    [[TMP28]] = add <4 x i32> [[VEC_PHI]], <i32 1, i32 1, i32 1, i32 1>
+; CHECK-NEXT:    [[TMP29]] = add <4 x i32> [[VEC_PHI2]], <i32 1, i32 1, i32 1, i32 1>
+; CHECK-NEXT:    [[TMP30:%.*]] = add i8 [[TMP26]], -1
+; CHECK-NEXT:    [[TMP31:%.*]] = add i8 [[TMP27]], -1
+; CHECK-NEXT:    [[TMP32:%.*]] = zext i8 [[TMP30]] to i32
+; CHECK-NEXT:    [[TMP33:%.*]] = zext i8 [[TMP31]] to i32
 ; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 8
-; CHECK-NEXT:    [[TMP30:%.*]] = icmp eq i32 [[INDEX_NEXT]], [[N_VEC]]
-; CHECK-NEXT:    br i1 [[TMP30]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], [[LOOP0:!llvm.loop !.*]]
+; CHECK-NEXT:    [[TMP34:%.*]] = icmp eq i32 [[INDEX_NEXT]], [[N_VEC]]
+; CHECK-NEXT:    br i1 [[TMP34]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
 ; CHECK:       middle.block:
-; CHECK-NEXT:    [[BIN_RDX:%.*]] = add <4 x i32> [[TMP25]], [[TMP24]]
-; CHECK-NEXT:    [[TMP31:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX]])
-; CHECK-NEXT:    [[CMP_N:%.*]] = icmp eq i32 [[TMP6]], [[N_VEC]]
+; CHECK-NEXT:    [[BIN_RDX:%.*]] = add <4 x i32> [[TMP29]], [[TMP28]]
+; CHECK-NEXT:    [[TMP35:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX]])
+; CHECK-NEXT:    [[CMP_N:%.*]] = icmp eq i32 [[TMP8]], [[N_VEC]]
 ; CHECK-NEXT:    br i1 [[CMP_N]], label [[FOR_COND4_FOR_INC9_CRIT_EDGE:%.*]], label [[SCALAR_PH]]
 ; CHECK:       scalar.ph:
 ; CHECK-NEXT:    [[BC_RESUME_VAL:%.*]] = phi i8 [ [[IND_END]], [[MIDDLE_BLOCK]] ], [ [[CONV3]], [[FOR_BODY8_LR_PH]] ], [ [[CONV3]], [[VECTOR_SCEVCHECK]] ]
-; CHECK-NEXT:    [[BC_MERGE_RDX:%.*]] = phi i32 [ [[DOTPROMOTED]], [[FOR_BODY8_LR_PH]] ], [ [[DOTPROMOTED]], [[VECTOR_SCEVCHECK]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ]
+; CHECK-NEXT:    [[BC_MERGE_RDX:%.*]] = phi i32 [ [[DOTPROMOTED]], [[FOR_BODY8_LR_PH]] ], [ [[DOTPROMOTED]], [[VECTOR_SCEVCHECK]] ], [ [[TMP35]], [[MIDDLE_BLOCK]] ]
 ; CHECK-NEXT:    br label [[FOR_BODY8:%.*]]
 ; CHECK:       for.body8:
 ; CHECK-NEXT:    [[INC5:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[INC:%.*]], [[FOR_BODY8]] ]
 ; CHECK-NEXT:    [[C_04:%.*]] = phi i8 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[DEC:%.*]], [[FOR_BODY8]] ]
 ; CHECK-NEXT:    [[INC]] = add i32 [[INC5]], 1
 ; CHECK-NEXT:    [[DEC]] = add i8 [[C_04]], -1
 ; CHECK-NEXT:    [[CONV5:%.*]] = zext i8 [[DEC]] to i32
 ; CHECK-NEXT:    [[CMP6:%.*]] = icmp ult i32 [[TMP2]], [[CONV5]]
-; CHECK-NEXT:    br i1 [[CMP6]], label [[FOR_BODY8]], label [[FOR_COND4_FOR_INC9_CRIT_EDGE]], [[LOOP2:!llvm.loop !.*]]
+; CHECK-NEXT:    br i1 [[CMP6]], label [[FOR_BODY8]], label [[FOR_COND4_FOR_INC9_CRIT_EDGE]], !llvm.loop [[LOOP2:![0-9]+]]
 ; CHECK:       for.cond4.for.inc9_crit_edge:
-; CHECK-NEXT:    [[INC_LCSSA:%.*]] = phi i32 [ [[INC]], [[FOR_BODY8]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ]
+; CHECK-NEXT:    [[INC_LCSSA:%.*]] = phi i32 [ [[INC]], [[FOR_BODY8]] ], [ [[TMP35]], [[MIDDLE_BLOCK]] ]
 ; CHECK-NEXT:    store i32 [[INC_LCSSA]], i32* getelementptr inbounds ([192 x [192 x i32]], [192 x [192 x i32]]* @a, i64 0, i64 0, i64 0), align 16
 ; CHECK-NEXT:    br label [[FOR_INC9]]
 ; CHECK:       for.inc9:
 ; CHECK-NEXT:    [[CONV10:%.*]] = and i32 [[STOREMERGE_IN9]], 65535
 ; CHECK-NEXT:    [[ADD]] = add nuw nsw i32 [[CONV10]], 1
 ; CHECK-NEXT:    [[CONV1:%.*]] = and i32 [[ADD]], 65472
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i32 [[CONV1]], 0
 ; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_COND_FOR_END12_CRIT_EDGE:%.*]]

llvm/test/Transforms/LoopVectorize/first-order-recurrence.ll

+; XFAIL: *
 ; RUN: opt < %s -loop-vectorize -force-vector-width=4 -force-vector-interleave=1 -dce -instcombine -S | FileCheck %s
 ; RUN: opt < %s -loop-vectorize -force-vector-width=4 -force-vector-interleave=2 -dce -instcombine -S | FileCheck %s --check-prefix=UNROLL
 ; RUN: opt < %s -loop-vectorize -force-vector-width=4 -force-vector-interleave=2 -S | FileCheck %s --check-prefix=UNROLL-NO-IC
 ; RUN: opt < %s -loop-vectorize -force-vector-width=1 -force-vector-interleave=2 -S | FileCheck %s --check-prefix=UNROLL-NO-VF
 ; RUN: opt < %s -loop-vectorize -force-vector-width=4 -force-vector-interleave=1 -S | FileCheck %s --check-prefix=SINK-AFTER
 ; RUN: opt < %s -loop-vectorize -force-vector-width=4 -force-vector-interleave=1 -S | FileCheck %s --check-prefix=NO-SINK-AFTER
 
 target datalayout = "e-m:e-i64:64-i128:128-n32:64-S128"

llvm/test/Transforms/LoopVectorize/runtime-check-pointer-element-type.ll

 ; RUN: opt -loop-vectorize -force-vector-width=2 %s -S -debug 2>&1 | FileCheck %s
 ; RUN: opt -passes='loop-vectorize' -force-vector-width=2 %s -S -debug 2>&1 | FileCheck %s
 
 ; REQUIRES: asserts
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 
 ; FIXME
 ; Test case for PR47751. Make sure the runtime check includes a required
 ; addition of the size of the element type (a pointer) for the end bound.
 
 define void @test(i64 %arg, i32 %arg1, i8** %base) {
 ; CHECK:      LAA: Adding RT check for range:
-; CHECK-NEXT:  Start: ((8 * (zext i32 (-1 + %arg1)<nsw> to i64))<nuw><nsw> + (8 * (1 smin %arg)) + (-8 * %arg) + %base)
+; CHECK-NEXT:  Start: (-8 + (8 * (zext i32 (-1 + %arg1)<nsw> to i64))<nuw><nsw> + (-8 * ((-2 smax (-1 + (-1 * %arg))) + %arg)) + %base)
 ; CHECK-SAME:  End: (8 + (8 * (zext i32 (-1 + %arg1)<nsw> to i64))<nuw><nsw> + %base)
 ; CHECK-NEXT: LAA: Adding RT check for range:
-; CHECK-NEXT:  Start: ((8 * (1 smin %arg)) + %base)
+; CHECK-NEXT:  Start: (-8 + (-8 * (-2 smax (-1 + (-1 * %arg)))) + %base)
 ; CHECK-SAME:  End: (8 + (8 * %arg) + %base)
 
 ; CHECK: vector.body
 
 entry:
   br label %loop
 
 loop: