[WIP] Loop peeling opportunity for identity operators

[jamieschmeiser]

Summary

This is a work in progress and is to aid discussion in the loop opt working group.

An opportunity for peeling exists for loops like the following:

unsigned Sum = 0;
for (unsigned I = 0; I < 999; ++I)

Sum += g();

In this situation, the initial value results in an identity operation on the first iteration of the loop.
This loop could be peeled to avoid this operation:

unsigned Sum = g();
for (unsigned I = 0; I < 998; ++I)

Sum += g();

This idiom frequently occurs in the form of summing up the values in an array. It can also occur with *=, &&= and ||=.

The number of tests requiring changes here illustrates that this is a common idiom. There may be more tests that need updating
but since this is currently for discussion purposes, this task has not been completed. These tests were mostly altered by dis-allowing peeling or changing the initial value to 1 from 0, making it not an identity operation.

There are several questions about the role of peeling in the llvm opt pipeline but the main one raised by this sample follows: This will likely be a relatively minor small improvement and it may interfere with other optimizations so is it worth pursuing?

For example, there is an attempt to not interfere with vectorization in the code as it exists. This heuristic needs improvement as the above sample, for example, will not peel if the upper bound is 10,000 as it may fail the heuristic on some hardware.

[mkazantsev]

llvm/test/Transforms/LoopUnroll/AArch64/runtime-loop.ll

27

Looks like a bug, the number of iterations didn't change but sum did?

[jamieschmeiser]

llvm/test/Transforms/LoopUnroll/AArch64/runtime-loop.ll

27

This value is coming in from the entry block and represents the initial value. I changed it to 1 from 0 so that the identity operation would not apply and this test would not otherwise change. I did this to the tests here because these tests are for testing other behaviours, rather than testing the peeling of an identity operator.

[Meinersbur]

The approach looks good to me. Got some smaller remarks, nothing critical since it doesn't affect correctness.

llvm/lib/Transforms/Utils/LoopPeel.cpp

471–487

Consider using doxygen comments (starting with ///). Doxygen code samples are indented by 4 spaces.

Could you explicitly list all the cases that should be handled. E.g.

• Integer, initial value 0, add
• Integer, initial value 1, mul
• Float, initial value 0, fadd
• Float, initial value 0, fmadd
• Float, initial value 1, fmul
• ...

506

[style] Functions start with a verb, e.g. getConstIn

516

ConstantFP::isZeroValue() returns true for negative and positive zero; should we do this only with -ffast-math (or the subflag that controls this)?

529

Would both (or and add; respectively mul and and with isAllOnes()) be valid independent of bit with. E.g. The PHI value might be a flags enum so we'd do a bitwise operation.

flags_t Flags = 0;
for (I = 0; I < 100000; ++I) {
   Total |= enable_flag(I);
}

544

[style] Functions start with a verb

558

128 is is really big; x86 is 2, 4, 8 or 16 elements. Is TTI to get native vector size?

llvm/test/Transforms/LoopUnroll/peel-loop-identity-op.ll

4

Consider adding a description to the test

7

[nikic]

Strong -1 on this change as implemented. There's a lot of red flags here, for example that you are trying to artificially limit this transform so as not to completely break LoopVectorize (indicating that you're breaking a canonical form), and the number of times you have to suppress the transform in tests, where it looks like the transform would be clearly non-profitable if it were actually applied.

There might be something to the general idea here, but this would need entirely different cost modelling from the current peeling transform. Loop peeling is a tradeoff between simplifying the loop body vs increasing code size. In the cases where current peeling transforms are performed, we expect some significant simplification of the loop, like the elimination of branches that are constant after some iterations. For the transform proposed here the benefit is very marginal, you're basically saving a single operation. This might make sense if that operation is pretty much the only thing the loop does, but not if you need to duplicate a large loop body that also does many other things.

[jamieschmeiser]

@nikic, thank you for your comments. I too have concerns about this proposal. I agree that the benefits are marginal but I would like to address some of your comments.

In D142255#4082028, @nikic wrote:

Strong -1 on this change as implemented. There's a lot of red flags here, for example that you are trying to artificially limit this transform so as not to completely break LoopVectorize (indicating that you're breaking a canonical form), and the number of times you have to suppress the transform in tests, where it looks like the transform would be clearly non-profitable if it were actually applied.

How this affects other transforms, LoopVectorize in particular, is one of my biggest concerns, but for different reasons. I don't think that it is breaking a canonical form, but rather that it may cause different remainders after vectorizing. This could be beneficial in some cases, harmful in others, depending on how it happens to line up. This was one of the things we discussed in the loop-opt group meeting and there was some question as to why peeling was being done in the full looop unroll pass before vectorizing as well as after in the loop unroll pass. I investigated this and found that peeling was specifically turned on in loop full unroll to aid vectorizing (commit 35b3989a30eefa66cd6edca4c6e1ec061c05ad96) because, as you indicated, the existing peeling strategies tend to eliminate phis, which can help the vectorizer. A proposed solution from this discussion was to not do this new peeling strategy in full loop unrolling but only after vectorization. This also removes the need for that bit of code for limiting the peel.
I disagree that the number of times I suppressed it shows it is not profitable. In fact, the frequency that it occurs shows its usefulness, but I suppressed it because it isn't germane to the tests in question and obfuscates their meaning and purpose.

There might be something to the general idea here, but this would need entirely different cost modelling from the current peeling transform. Loop peeling is a tradeoff between simplifying the loop body vs increasing code size. In the cases where current peeling transforms are performed, we expect some significant simplification of the loop, like the elimination of branches that are constant after some iterations. For the transform proposed here the benefit is very marginal, you're basically saving a single operation. This might make sense if that operation is pretty much the only thing the loop does, but not if you need to duplicate a large loop body that also does many other things.

The peeling code already considers the size of code growth and establishes limits to the amount of peeling that can occur. The proposed code does not change that and only suggests the peel when it is within the established limits.

I am preparing a revision of this code that adds a new peeling flag to the peeling control struct called allow-aggressive-peeling. This would control opportunities that do not necessarily simplify phi structure of control flow but exist for other opportunities such as this one. It would be set to false for full loop unroll (before vectorizing) but on for the loop unroll pass. Like other flags in this struct, it will have an option for controlling it and it will be suppressed using the option in the loop unrolling tests for the reasons given above but not specified for other ones. Does this approach address your concerns? I recognize that the change would need to be examined before any decision is made...

[nikic]

In D142255#4086518, @jamieschmeiser wrote:

I disagree that the number of times I suppressed it shows it is not profitable. In fact, the frequency that it occurs shows its usefulness, but I suppressed it because it isn't germane to the tests in question and obfuscates their meaning and purpose.

I think my phrasing here was ambiguous: It's not the number of times it is suppressed, but if you take a look at the specific cases where it is suppressed, many of them look non-profitable to me. E.g. the second test file llvm/test/Transforms/LoopUnroll/ARM/multi-blocks.ll already has a non-trivial loop body with internal control flow, where I would expect the marginal cost of more code size to already outweigh the marginal benefit of this optimization.

The peeling code already considers the size of code growth and establishes limits to the amount of peeling that can occur. The proposed code does not change that and only suggests the peel when it is within the established limits.

My point here was that the existing cost model is inappropriate for this transform. This transform basically saves you from executing a single instruction over the entire loop (not per iteration!) It's a very small improvement, that will be outweighed by code size increases for anything but the smallest loops. The existing cost model will perform this optimization in too many cases.

I am preparing a revision of this code that adds a new peeling flag to the peeling control struct called allow-aggressive-peeling. This would control opportunities that do not necessarily simplify phi structure of control flow but exist for other opportunities such as this one. It would be set to false for full loop unroll (before vectorizing) but on for the loop unroll pass. Like other flags in this struct, it will have an option for controlling it and it will be suppressed using the option in the loop unrolling tests for the reasons given above but not specified for other ones. Does this approach address your concerns? I recognize that the change would need to be examined before any decision is made...

Delaying this to runtime unrolling is certainly a good start, but I don't think it will be sufficient, and I'm not really convinced that this optimization is worth investing into in the first place. Do you have any performance data that suggests that this is really a worthwhile thing to do?

[jamieschmeiser]

I agree that the potential gain is limited and probably not worth the potential increase in code size. This patch captures the changes/discussion if someone else is interested in the future. I am abandoning this revision.

llvm/lib/Transforms/Utils/LoopPeel.cpp

 //===- LoopPeel.cpp -------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Loop Peeling Utilities.
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/Utils/LoopPeel.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/Statistic.h"
+#include "llvm/Analysis/IVDescriptors.h"
 #include "llvm/Analysis/Loads.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/LoopIterator.h"
 #include "llvm/Analysis/ScalarEvolution.h"
 #include "llvm/Analysis/ScalarEvolutionExpressions.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Dominators.h"
static bool violatesLegacyMultiExitLoopCheck(Loop *L) {
 
   SmallVector<BasicBlock *, 4> ExitBlocks;
   L->getUniqueNonLatchExitBlocks(ExitBlocks);
   return any_of(ExitBlocks, [](const BasicBlock *EB) {
       return !EB->getTerminatingDeoptimizeCall();
     });
 }
 
+// When a loop contains an accumulator, an initial identity operation
+// may be avoided by peeling a single time.  For example, consider this loop
+// int Total = 0;
+// int Product = 1;
+// for (I = 0; I < 100000; ++I) {
+//   Total += foo(I);
+//   Product *= bar(I);
+// }
+// Peeling a single time will avoid the initial addtion and multiplication:
+// int Total = foo(I);
+// int Product = bar(I);
+// for (I = 1; I < 100000; ++I) {
+//   Total += foo(I);
+//   Product *= bar(I);
+// }
+// This may work for +=, *=, &&= and ||= depending on initial value.
+//
+class IdentityOpPeeler {
+public:
+  IdentityOpPeeler(Loop &L, ScalarEvolution &SE) : L(L), SE(SE) {
+    assert(canPeel(&L) && L.getHeader() && "expected peelable loop");
+  }
+  bool shouldPeelOnce() {
+    return any_of(L.getHeader()->phis(),
+                  [&](PHINode &Phi) { return isAccumulatorPhi(Phi); });
+  }
+
+protected:
+  // Return true if \p Phi in not an induction variable and is an accumulator
+  // that starts with an identity operation
+  bool isAccumulatorPhi(PHINode &Phi) const {
+    if (&Phi == L.getInductionVariable(SE))
+      return false;
+
+    // Return the incoming constant value from \p BB or nullptr
+    auto constIn = [&](BasicBlock *BB, bool) -> Constant * {
+      if (L.contains(BB))
+        // not a value coming from outside the loop
+        return nullptr;
+      return dyn_cast<Constant>(Phi.getIncomingValueForBlock(BB));
+    };
+
+    // find a single constant value coming from outside the loop
+    if (Constant *C = find_singleton<Constant>(Phi.blocks(), constIn, true)) {
+      if (ConstantFP *CFP = dyn_cast<ConstantFP>(C))
+        return CFP->isZeroValue() && any_of(Phi.users(), [&](User *U) -> bool {
+                 if (beneficial(*U, Phi, Instruction::FAdd))
+                   return true;
+                 // M += <exp> may be expressed using the FMulAdd
+                 // intrinsic with arguments (M, M, 0)
+                 Instruction *I = dyn_cast<Instruction>(U);
+                 return I && RecurrenceDescriptor::isFMulAddIntrinsic(I) &&
+                        I->getOperand(2) == &Phi;
+               });
+
+      if (ConstantInt *CI = dyn_cast<ConstantInt>(C)) {
+        BinaryOperator::BinaryOps Op;
+        if (CI->isZeroValue())
+          Op = (CI->getBitWidth() == 1) ? Instruction::Or : Instruction::Add;
+        else if (CI->isOneValue())
+          Op = (CI->getBitWidth() == 1) ? Instruction::And : Instruction::Mul;
+        else
+          return false;
+
+        return any_of(Phi.users(), [&](const User *U) -> bool {
+          return beneficial(*U, Phi, Op);
+        });
+      }
+    }
+    return false;
+  }
+
+  // Peeling may be beneficial for a use in the loop with operator \p Op.
+  bool beneficial(const User &U, const PHINode &Phi,
+                  const BinaryOperator::BinaryOps Op) const {
+    if (const Instruction *I = dyn_cast<const Instruction>(&U))
+      if (L.contains(I->getParent()))
+        if (const BinaryOperator *BO = dyn_cast<const BinaryOperator>(I))
+          return (BO->getOpcode() == Op) &&
+                 // SCEV might not recognize an induction variable so assume it
+                 // is only beneficial if the other operand is not a constant
+                 (!isa<Constant>(
+                     BO->getOperand(BO->getOperand(0) == &Phi ? 1 : 0))) &&
+                 // try not to get in the way of the vectorizer by seeing if the
+                 // count of the loop multiplied by the size is divisible by 128
+                 (SE.getSmallConstantMaxTripCount(&L) == 0 ||
+                  SE.getSmallConstantMaxTripCount(&L) *
+                          Phi.getType()->getScalarSizeInBits() % 128 !=
+                      0);
+    return false;
+  }
+
+  const Loop &L;
+  ScalarEvolution &SE;
+};
 
 // Return the number of iterations we want to peel off.
 void llvm::computePeelCount(Loop *L, unsigned LoopSize,
                             TargetTransformInfo::PeelingPreferences &PP,
                             unsigned TripCount, DominatorTree &DT,
                             ScalarEvolution &SE, AssumptionCache *AC,
                             unsigned Threshold) {
   assert(LoopSize > 0 && "Zero loop size is not allowed!");
void llvm::computePeelCount(Loop *L, unsigned LoopSize,
   }
 
   DesiredPeelCount = std::max(DesiredPeelCount,
                               countToEliminateCompares(*L, MaxPeelCount, SE));
 
   if (DesiredPeelCount == 0)
     DesiredPeelCount = peelToTurnInvariantLoadsDerefencebale(*L, DT, AC);
 
+  if (DesiredPeelCount == 0 && MaxPeelCount)
+    DesiredPeelCount = IdentityOpPeeler(*L, SE).shouldPeelOnce() ? 1 : 0;
+
   if (DesiredPeelCount > 0) {
     DesiredPeelCount = std::min(DesiredPeelCount, MaxPeelCount);
     // Consider max peel count limitation.
     assert(DesiredPeelCount > 0 && "Wrong loop size estimation?");
     if (DesiredPeelCount + AlreadyPeeled <= UnrollPeelMaxCount) {
       LLVM_DEBUG(dbgs() << "Peel " << DesiredPeelCount
                         << " iteration(s) to turn"
                         << " some Phis into invariants.\n");

llvm/test/Transforms/LoopUnroll/AArch64/runtime-loop.ll

 
 define i32 @test(ptr nocapture %a, i32 %n) nounwind uwtable readonly {
 entry:
   %cmp1 = icmp eq i32 %n, 0
   br i1 %cmp1, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %n
   br i1 %exitcond, label %for.end, label %for.body
 
 for.end:                                          ; preds = %for.body, %entry
   %sum.0.lcssa = phi i32 [ 0, %entry ], [ %add, %for.body ]
   ret i32 %sum.0.lcssa
 }
 
 

llvm/test/Transforms/LoopUnroll/ARM/multi-blocks.ll

 ; CHECK:       for.cond.cleanup.loopexit.unr-lcssa.loopexit:
 ; CHECK-NEXT:    [[TEMP_1_LCSSA_PH_PH:%.*]] = phi i32 [ [[TEMP_1_3:%.*]], [[FOR_INC_3:%.*]] ]
 ; CHECK-NEXT:    [[J_010_UNR_PH:%.*]] = phi i32 [ [[INC_3:%.*]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[TEMP_09_UNR_PH:%.*]] = phi i32 [ [[TEMP_1_3]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    br label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]]
 ; CHECK:       for.cond.cleanup.loopexit.unr-lcssa:
 ; CHECK-NEXT:    [[TEMP_1_LCSSA_PH:%.*]] = phi i32 [ undef, [[FOR_BODY_PREHEADER]] ], [ [[TEMP_1_LCSSA_PH_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT:%.*]] ]
 ; CHECK-NEXT:    [[J_010_UNR:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER]] ], [ [[J_010_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
-; CHECK-NEXT:    [[TEMP_09_UNR:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER]] ], [ [[TEMP_09_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
+; CHECK-NEXT:    [[TEMP_09_UNR:%.*]] = phi i32 [ 1, [[FOR_BODY_PREHEADER]] ], [ [[TEMP_09_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[LCMP_MOD:%.*]] = icmp ne i32 [[XTRAITER]], 0
 ; CHECK-NEXT:    br i1 [[LCMP_MOD]], label [[FOR_BODY_EPIL_PREHEADER:%.*]], label [[FOR_COND_CLEANUP_LOOPEXIT:%.*]]
 ; CHECK:       for.body.epil.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY_EPIL:%.*]]
 ; CHECK:       for.body.epil:
 ; CHECK-NEXT:    [[ARRAYIDX_EPIL:%.*]] = getelementptr inbounds i32, ptr [[CONDITION:%.*]], i32 [[J_010_UNR]]
 ; CHECK-NEXT:    [[I_EPIL:%.*]] = load i32, ptr [[ARRAYIDX_EPIL]], align 4
 ; CHECK-NEXT:    [[TOBOOL_EPIL:%.*]] = icmp eq i32 [[I_EPIL]], 0

 ; CHECK-NEXT:    [[TEMP_1_LCSSA:%.*]] = phi i32 [ [[TEMP_1_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ], [ [[TEMP_1_LCSSA_PH1]], [[FOR_COND_CLEANUP_LOOPEXIT_EPILOG_LCSSA]] ]
 ; CHECK-NEXT:    br label [[FOR_COND_CLEANUP]]
 ; CHECK:       for.cond.cleanup:
 ; CHECK-NEXT:    [[TEMP_0_LCSSA:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[TEMP_1_LCSSA]], [[FOR_COND_CLEANUP_LOOPEXIT]] ]
 ; CHECK-NEXT:    store i32 [[TEMP_0_LCSSA]], ptr [[OUTPUT:%.*]], align 4
 ; CHECK-NEXT:    ret void
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[J_010:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER_NEW]] ], [ [[INC_3]], [[FOR_INC_3]] ]
-; CHECK-NEXT:    [[TEMP_09:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER_NEW]] ], [ [[TEMP_1_3]], [[FOR_INC_3]] ]
+; CHECK-NEXT:    [[TEMP_09:%.*]] = phi i32 [ 1, [[FOR_BODY_PREHEADER_NEW]] ], [ [[TEMP_1_3]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[NITER:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER_NEW]] ], [ [[NITER_NEXT_3:%.*]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[CONDITION]], i32 [[J_010]]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[TOBOOL:%.*]] = icmp eq i32 [[I]], 0
 ; CHECK-NEXT:    br i1 [[TOBOOL]], label [[FOR_INC:%.*]], label [[IF_THEN:%.*]]
 ; CHECK:       if.then:
 ; CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds i32, ptr [[INPUT]], i32 [[J_010]]
 ; CHECK-NEXT:    [[I1:%.*]] = load i32, ptr [[ARRAYIDX1]], align 4

 
 for.cond.cleanup:                                 ; preds = %for.inc, %entry
   %temp.0.lcssa = phi i32 [ 0, %entry ], [ %temp.1, %for.inc ]
   store i32 %temp.0.lcssa, ptr %Output, align 4
   ret void
 
 for.body:                                         ; preds = %for.inc, %for.body.preheader
   %j.010 = phi i32 [ %inc, %for.inc ], [ 0, %for.body.preheader ]
-  %temp.09 = phi i32 [ %temp.1, %for.inc ], [ 0, %for.body.preheader ]
+  %temp.09 = phi i32 [ %temp.1, %for.inc ], [ 1, %for.body.preheader ]
   %arrayidx = getelementptr inbounds i32, ptr %Condition, i32 %j.010
   %i = load i32, ptr %arrayidx, align 4
   %tobool = icmp eq i32 %i, 0
   br i1 %tobool, label %for.inc, label %if.then
 
 if.then:                                          ; preds = %for.body
   %arrayidx1 = getelementptr inbounds i32, ptr %Input, i32 %j.010
   %i1 = load i32, ptr %arrayidx1, align 4

 ; CHECK-LABEL: @test_two_exits(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP14:%.*]] = icmp eq i32 [[MAXJ:%.*]], 0
 ; CHECK-NEXT:    br i1 [[CMP14]], label [[CLEANUP:%.*]], label [[FOR_BODY_PREHEADER:%.*]]
 ; CHECK:       for.body.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[J_016:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER]] ], [ [[INC_3:%.*]], [[IF_END_3:%.*]] ]
-; CHECK-NEXT:    [[TEMP_015:%.*]] = phi i32 [ 0, [[FOR_BODY_PREHEADER]] ], [ [[TEMP_0_ADD_3:%.*]], [[IF_END_3]] ]
+; CHECK-NEXT:    [[TEMP_015:%.*]] = phi i32 [ 1, [[FOR_BODY_PREHEADER]] ], [ [[TEMP_0_ADD_3:%.*]], [[IF_END_3]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[INPUT:%.*]], i32 [[J_016]]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[CMP1:%.*]] = icmp ugt i32 [[I]], 65535
 ; CHECK-NEXT:    br i1 [[CMP1]], label [[CLEANUP_LOOPEXIT:%.*]], label [[IF_END:%.*]]
 ; CHECK:       if.end:
 ; CHECK-NEXT:    [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[CONDITION:%.*]], i32 [[J_016]]
 ; CHECK-NEXT:    [[I1:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4
 ; CHECK-NEXT:    [[TOBOOL:%.*]] = icmp eq i32 [[I1]], 0
entry:
   %cmp14 = icmp eq i32 %MaxJ, 0
   br i1 %cmp14, label %cleanup, label %for.body.preheader
 
 for.body.preheader:                               ; preds = %entry
   br label %for.body
 
 for.body:                                         ; preds = %if.end, %for.body.preheader
   %j.016 = phi i32 [ %inc, %if.end ], [ 0, %for.body.preheader ]
-  %temp.015 = phi i32 [ %temp.0.add, %if.end ], [ 0, %for.body.preheader ]
+  %temp.015 = phi i32 [ %temp.0.add, %if.end ], [ 1, %for.body.preheader ]
   %arrayidx = getelementptr inbounds i32, ptr %Input, i32 %j.016
   %i = load i32, ptr %arrayidx, align 4
   %cmp1 = icmp ugt i32 %i, 65535
   br i1 %cmp1, label %cleanup, label %if.end
 
 if.end:                                           ; preds = %for.body
   %arrayidx2 = getelementptr inbounds i32, ptr %Condition, i32 %j.016
   %i1 = load i32, ptr %arrayidx2, align 4

 ; CHECK-LABEL: @test_three_exits(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP20:%.*]] = icmp eq i32 [[MAXJ:%.*]], 0
 ; CHECK-NEXT:    br i1 [[CMP20]], label [[CLEANUP:%.*]], label [[FOR_BODY_PREHEADER:%.*]]
 ; CHECK:       for.body.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[J_022:%.*]] = phi i32 [ [[INC:%.*]], [[IF_END5:%.*]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
-; CHECK-NEXT:    [[TEMP_021:%.*]] = phi i32 [ [[TEMP_0_ADD:%.*]], [[IF_END5]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
+; CHECK-NEXT:    [[TEMP_021:%.*]] = phi i32 [ [[TEMP_0_ADD:%.*]], [[IF_END5]] ], [ 1, [[FOR_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[CONDITION:%.*]], i32 [[J_022]]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[CMP1:%.*]] = icmp ugt i32 [[I]], 65535
 ; CHECK-NEXT:    br i1 [[CMP1]], label [[CLEANUP_LOOPEXIT:%.*]], label [[IF_END:%.*]]
 ; CHECK:       if.end:
 ; CHECK-NEXT:    [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[INPUT:%.*]], i32 [[J_022]]
 ; CHECK-NEXT:    [[I1:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4
 ; CHECK-NEXT:    [[CMP3:%.*]] = icmp ugt i32 [[I1]], 65535
entry:
   %cmp20 = icmp eq i32 %MaxJ, 0
   br i1 %cmp20, label %cleanup, label %for.body.preheader
 
 for.body.preheader:                               ; preds = %entry
   br label %for.body
 
 for.body:                                         ; preds = %if.end5, %for.body.preheader
   %j.022 = phi i32 [ %inc, %if.end5 ], [ 0, %for.body.preheader ]
-  %temp.021 = phi i32 [ %temp.0.add, %if.end5 ], [ 0, %for.body.preheader ]
+  %temp.021 = phi i32 [ %temp.0.add, %if.end5 ], [ 1, %for.body.preheader ]
   %arrayidx = getelementptr inbounds i32, ptr %Condition, i32 %j.022
   %i = load i32, ptr %arrayidx, align 4
   %cmp1 = icmp ugt i32 %i, 65535
   br i1 %cmp1, label %cleanup, label %if.end
 
 if.end:                                           ; preds = %for.body
   %arrayidx2 = getelementptr inbounds i32, ptr %Input, i32 %j.022
   %i1 = load i32, ptr %arrayidx2, align 4

 ; CHECK-NEXT:    [[I_UNR_PH:%.*]] = phi i32 [ [[I2_3:%.*]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[J_027_UNR_PH:%.*]] = phi i32 [ [[INC_3:%.*]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[TEMP_026_UNR_PH:%.*]] = phi i32 [ [[TEMP_1_3]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    br label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]]
 ; CHECK:       for.cond.cleanup.loopexit.unr-lcssa:
 ; CHECK-NEXT:    [[TEMP_1_LCSSA_PH:%.*]] = phi i32 [ undef, [[FOR_BODY_LR_PH]] ], [ [[TEMP_1_LCSSA_PH_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT:%.*]] ]
 ; CHECK-NEXT:    [[I_UNR:%.*]] = phi i32 [ [[DOTPRE]], [[FOR_BODY_LR_PH]] ], [ [[I_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[J_027_UNR:%.*]] = phi i32 [ 1, [[FOR_BODY_LR_PH]] ], [ [[J_027_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
-; CHECK-NEXT:    [[TEMP_026_UNR:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH]] ], [ [[TEMP_026_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
+; CHECK-NEXT:    [[TEMP_026_UNR:%.*]] = phi i32 [ 1, [[FOR_BODY_LR_PH]] ], [ [[TEMP_026_UNR_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[LCMP_MOD:%.*]] = icmp ne i32 [[XTRAITER]], 0
 ; CHECK-NEXT:    br i1 [[LCMP_MOD]], label [[FOR_BODY_EPIL_PREHEADER:%.*]], label [[FOR_COND_CLEANUP_LOOPEXIT:%.*]]
 ; CHECK:       for.body.epil.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY_EPIL:%.*]]
 ; CHECK:       for.body.epil:
 ; CHECK-NEXT:    [[ARRAYIDX_EPIL:%.*]] = getelementptr inbounds i32, ptr [[CONDITION:%.*]], i32 [[J_027_UNR]]
 ; CHECK-NEXT:    [[I1_EPIL:%.*]] = load i32, ptr [[ARRAYIDX_EPIL]], align 4
 ; CHECK-NEXT:    [[CMP1_EPIL:%.*]] = icmp ugt i32 [[I1_EPIL]], 65535

 ; CHECK-NEXT:    br label [[FOR_COND_CLEANUP]]
 ; CHECK:       for.cond.cleanup:
 ; CHECK-NEXT:    [[TEMP_0_LCSSA:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[TEMP_1_LCSSA]], [[FOR_COND_CLEANUP_LOOPEXIT]] ]
 ; CHECK-NEXT:    store i32 [[TEMP_0_LCSSA]], ptr [[OUTPUT:%.*]], align 4
 ; CHECK-NEXT:    ret void
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[I:%.*]] = phi i32 [ [[DOTPRE]], [[FOR_BODY_LR_PH_NEW]] ], [ [[I2_3]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[J_027:%.*]] = phi i32 [ 1, [[FOR_BODY_LR_PH_NEW]] ], [ [[INC_3]], [[FOR_INC_3]] ]
-; CHECK-NEXT:    [[TEMP_026:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH_NEW]] ], [ [[TEMP_1_3]], [[FOR_INC_3]] ]
+; CHECK-NEXT:    [[TEMP_026:%.*]] = phi i32 [ 1, [[FOR_BODY_LR_PH_NEW]] ], [ [[TEMP_1_3]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[NITER:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH_NEW]] ], [ [[NITER_NEXT_3:%.*]], [[FOR_INC_3]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[CONDITION]], i32 [[J_027]]
 ; CHECK-NEXT:    [[I1:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[CMP1:%.*]] = icmp ugt i32 [[I1]], 65535
 ; CHECK-NEXT:    [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[INPUT]], i32 [[J_027]]
 ; CHECK-NEXT:    [[I2:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4
 ; CHECK-NEXT:    [[CMP4:%.*]] = icmp ugt i32 [[I2]], [[I]]
 ; CHECK-NEXT:    br i1 [[CMP1]], label [[IF_THEN:%.*]], label [[IF_ELSE:%.*]]

 for.cond.cleanup:                                 ; preds = %for.inc, %entry
   %temp.0.lcssa = phi i32 [ 0, %entry ], [ %temp.1, %for.inc ]
   store i32 %temp.0.lcssa, ptr %Output, align 4
   ret void
 
 for.body:                                         ; preds = %for.inc, %for.body.lr.ph
   %i = phi i32 [ %.pre, %for.body.lr.ph ], [ %i2, %for.inc ]
   %j.027 = phi i32 [ 1, %for.body.lr.ph ], [ %inc, %for.inc ]
-  %temp.026 = phi i32 [ 0, %for.body.lr.ph ], [ %temp.1, %for.inc ]
+  %temp.026 = phi i32 [ 1, %for.body.lr.ph ], [ %temp.1, %for.inc ]
   %arrayidx = getelementptr inbounds i32, ptr %Condition, i32 %j.027
   %i1 = load i32, ptr %arrayidx, align 4
   %cmp1 = icmp ugt i32 %i1, 65535
   %arrayidx2 = getelementptr inbounds i32, ptr %Input, i32 %j.027
   %i2 = load i32, ptr %arrayidx2, align 4
   %cmp4 = icmp ugt i32 %i2, %i
   br i1 %cmp1, label %if.then, label %if.else
 

 ; CHECK-LABEL: @test_five_blocks(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP24:%.*]] = icmp ugt i32 [[MAXJ:%.*]], 1
 ; CHECK-NEXT:    br i1 [[CMP24]], label [[FOR_BODY_PREHEADER:%.*]], label [[CLEANUP:%.*]]
 ; CHECK:       for.body.preheader:
 ; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
 ; CHECK:       for.body:
 ; CHECK-NEXT:    [[J_026:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_INC:%.*]] ], [ 1, [[FOR_BODY_PREHEADER]] ]
-; CHECK-NEXT:    [[TEMP_025:%.*]] = phi i32 [ [[TEMP_1:%.*]], [[FOR_INC]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
+; CHECK-NEXT:    [[TEMP_025:%.*]] = phi i32 [ [[TEMP_1:%.*]], [[FOR_INC]] ], [ 1, [[FOR_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[INPUT:%.*]], i32 [[J_026]]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[ADD:%.*]] = add i32 [[I]], [[TEMP_025]]
 ; CHECK-NEXT:    [[CMP1:%.*]] = icmp ugt i32 [[ADD]], 16777215
 ; CHECK-NEXT:    br i1 [[CMP1]], label [[CLEANUP_LOOPEXIT:%.*]], label [[IF_END:%.*]]
 ; CHECK:       if.end:
 ; CHECK-NEXT:    [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[CONDITION:%.*]], i32 [[J_026]]
 ; CHECK-NEXT:    [[I1:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4
entry:
   %cmp24 = icmp ugt i32 %MaxJ, 1
   br i1 %cmp24, label %for.body.preheader, label %cleanup
 
 for.body.preheader:                               ; preds = %entry
   br label %for.body
 
 for.body:                                         ; preds = %for.inc, %for.body.preheader
   %j.026 = phi i32 [ %inc, %for.inc ], [ 1, %for.body.preheader ]
-  %temp.025 = phi i32 [ %temp.1, %for.inc ], [ 0, %for.body.preheader ]
+  %temp.025 = phi i32 [ %temp.1, %for.inc ], [ 1, %for.body.preheader ]
   %arrayidx = getelementptr inbounds i32, ptr %Input, i32 %j.026
   %i = load i32, ptr %arrayidx, align 4
   %add = add i32 %i, %temp.025
   %cmp1 = icmp ugt i32 %add, 16777215
   br i1 %cmp1, label %cleanup, label %if.end
 
 if.end:                                           ; preds = %for.body
   %arrayidx2 = getelementptr inbounds i32, ptr %Condition, i32 %j.026

llvm/test/Transforms/LoopUnroll/ARM/v6munroll.ll

 
 for.body.preheader:                               ; preds = %entry
   br label %for.body
 
 for.body:                                         ; preds = %for.body.preheader, %for.inc
   %x.addr.062 = phi ptr [ %incdec.ptr, %for.inc ], [ %x, %for.body.preheader ]
   %y.addr.061 = phi ptr [ %incdec.ptr19, %for.inc ], [ %y, %for.body.preheader ]
   %rhs_cols_idx.060 = phi i32 [ %dec, %for.inc ], [ %n, %for.body.preheader ]
-  %res11.059 = phi i32 [ %add18, %for.inc ], [ 0, %for.body.preheader ]
-  %res10.058 = phi i32 [ %add16, %for.inc ], [ 0, %for.body.preheader ]
-  %res01.057 = phi i32 [ %add8, %for.inc ], [ 0, %for.body.preheader ]
-  %res00.056 = phi i32 [ %add, %for.inc ], [ 0, %for.body.preheader ]
+  %res11.059 = phi i32 [ %add18, %for.inc ], [ 1, %for.body.preheader ]
+  %res10.058 = phi i32 [ %add16, %for.inc ], [ 1, %for.body.preheader ]
+  %res01.057 = phi i32 [ %add8, %for.inc ], [ 1, %for.body.preheader ]
+  %res00.056 = phi i32 [ %add, %for.inc ], [ 1, %for.body.preheader ]
   %0 = load i32, ptr %x.addr.062, align 4
   %1 = load i32, ptr %y.addr.061, align 4
   %conv5 = and i32 %1, 255
   %conv6 = and i32 %0, 255
   %mul = mul nuw nsw i32 %conv5, %conv6
   %add = add nuw nsw i32 %mul, %res00.056
   %add8 = add nuw nsw i32 %conv5, %res01.057
   %cmp9 = icmp ugt i32 %add8, 100

llvm/test/Transforms/LoopUnroll/PowerPC/a2-unrolling.ll

 ; RUN: opt < %s -S -mtriple=powerpc64-unknown-linux-gnu -mcpu=a2 -passes=loop-unroll -unroll-runtime-epilog=true  | FileCheck %s -check-prefix=EPILOG
 ; RUN: opt < %s -S -mtriple=powerpc64-unknown-linux-gnu -mcpu=a2 -passes=loop-unroll -unroll-runtime-epilog=false | FileCheck %s -check-prefix=PROLOG
 
 define i32 @test(ptr nocapture %a, i32 %n) nounwind uwtable readonly {
 entry:
   %cmp1 = icmp eq i32 %n, 0
   br i1 %cmp1, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %n
   br i1 %exitcond, label %for.end, label %for.body
 

llvm/test/Transforms/LoopUnroll/X86/partial.ll

 
 define zeroext i16 @test1(ptr nocapture readonly %arr, i32 %n) #0 {
 entry:
   %cmp25 = icmp eq i32 %n, 0
   br i1 %cmp25, label %for.end, label %for.body
 
 for.body:                                         ; preds = %entry, %for.body
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %reduction.026 = phi i16 [ %add14, %for.body ], [ 0, %entry ]
+  %reduction.026 = phi i16 [ %add14, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i16, ptr %arr, i64 %indvars.iv
   %0 = load i16, ptr %arrayidx, align 2
   %mul = shl i16 %0, 1
   %add = add i16 %mul, %reduction.026
   %sext = mul i64 %indvars.iv, 12884901888
   %idxprom3 = ashr exact i64 %sext, 32
   %arrayidx4 = getelementptr inbounds i16, ptr %arr, i64 %idxprom3
   %1 = load i16, ptr %arrayidx4, align 2

llvm/test/Transforms/LoopUnroll/followup.ll

 ; RUN: opt < %s -S -passes=loop-unroll -unroll-count=2 | FileCheck %s -check-prefixes=COUNT,COMMON
 ; RUN: opt < %s -S -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=true  | FileCheck %s -check-prefixes=EPILOG,COMMON
 ; RUN: opt < %s -S -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=false | FileCheck %s -check-prefixes=PROLOG,COMMON
 ;
 ; Check that followup-attributes are applied after LoopUnroll.
 ;
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 
 define i32 @test(ptr nocapture %a, i32 %n) nounwind uwtable readonly {
 entry:
   %cmp1 = icmp eq i32 %n, 0
   br i1 %cmp1, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %n
   br i1 %exitcond, label %for.end, label %for.body, !llvm.loop !4
 

llvm/test/Transforms/LoopUnroll/partially-unroll-unconditional-latch.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes=loop-unroll -unroll-allow-partial -S %s -verify-loop-info -verify-dom-info -verify-loop-lcssa | FileCheck %s
+; RUN: opt -passes=loop-unroll -unroll-allow-partial -S %s -verify-loop-info -verify-dom-info -verify-loop-lcssa -unroll-allow-peeling=false | FileCheck %s
 
 @table = internal unnamed_addr global [344 x i32] zeroinitializer, align 16
 
 define i32 @test_partial_unroll_with_breakout_at_iter0() {
 ;
 ;
 ; CHECK-LABEL: @test_partial_unroll_with_breakout_at_iter0(
 ; CHECK-NEXT:  entry:

llvm/test/Transforms/LoopUnroll/peel-loop-identity-op.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -S -passes=loop-unroll -unroll-allow-peeling | FileCheck %s
+
+declare i32 @g()
+
+define i32 @f() {
+; Preheader:
+; CHECK-LABEL: @f(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    br label [[FOR_BODY_PEEL_BEGIN:%.*]]
+; CHECK:       for.body.peel.begin:
+; CHECK-NEXT:    br label [[FOR_BODY_PEEL:%.*]]
+; CHECK:       for.body.peel:
+; CHECK-NEXT:    [[CALL_PEEL:%.*]] = tail call signext i32 @g(i32 noundef signext 0)
+; CHECK-NEXT:    [[ADD_PEEL:%.*]] = add nsw i32 [[CALL_PEEL]], 0
+; CHECK-NEXT:    [[INC_PEEL:%.*]] = add nuw nsw i32 0, 1
+; CHECK-NEXT:    [[EXITCOND_PEEL:%.*]] = icmp ne i32 [[INC_PEEL]], 999
+; CHECK-NEXT:    br i1 [[EXITCOND_PEEL]], label [[FOR_BODY_PEEL_NEXT:%.*]], label [[FOR_COND_CLEANUP:%.*]]
+; CHECK:       for.body.peel.next:
+; CHECK-NEXT:    br label [[FOR_BODY_PEEL_NEXT1:%.*]]
+; CHECK:       for.body.peel.next1:
+; CHECK-NEXT:    br label [[ENTRY_PEEL_NEWPH:%.*]]
+; CHECK:       entry.peel.newph:
+; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
+; CHECK:       for.body:
+; CHECK-NEXT:    [[J_05:%.*]] = phi i32 [ [[INC_PEEL]], [[ENTRY_PEEL_NEWPH]] ], [ [[INC:%.*]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[TOTAL_04:%.*]] = phi i32 [ [[ADD_PEEL]], [[ENTRY_PEEL_NEWPH]] ], [ [[ADD:%.*]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[CALL:%.*]] = tail call signext i32 @g(i32 noundef signext [[J_05]])
+; CHECK-NEXT:    [[ADD]] = add nsw i32 [[CALL]], [[TOTAL_04]]
+; CHECK-NEXT:    [[INC]] = add nuw nsw i32 [[J_05]], 1
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[INC]], 999
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_COND_CLEANUP_LOOPEXIT:%.*]], !llvm.loop [[LOOP0:![0-9]+]]
+; CHECK:       for.cond.cleanup.loopexit:
+; CHECK-NEXT:    [[ADD_LCSSA_PH:%.*]] = phi i32 [ [[ADD]], [[FOR_BODY]] ]
+; CHECK-NEXT:    br label [[FOR_COND_CLEANUP]]
+; CHECK:       for.cond.cleanup:
+; CHECK-NEXT:    [[ADD_LCSSA:%.*]] = phi i32 [ [[ADD_PEEL]], [[FOR_BODY_PEEL]] ], [ [[ADD_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT]] ]
+; CHECK-NEXT:    ret i32 [[ADD_LCSSA]]
+;
+entry:
+  br label %for.body
+
+; Loop:
+for.body:                                         ; preds = %entry, %for.body
+  %j.05 = phi i32 [ 0, %entry ], [ %inc, %for.body ]
+  %total.04 = phi i32 [ 0, %entry ], [ %add, %for.body ]
+  %call = tail call signext i32 @g(i32 noundef signext %j.05)
+  %add = add nsw i32 %call, %total.04
+  %inc = add nuw nsw i32 %j.05, 1
+  %exitcond = icmp ne i32 %inc, 999
+  br i1 %exitcond, label %for.body, label %for.cond.cleanup
+
+; Exit blocks
+for.cond.cleanup:                                 ; preds = %for.body
+  %add.lcssa = phi i32 [ %add, %for.body ]
+  ret i32 %add.lcssa
+}

llvm/test/Transforms/LoopUnroll/peel-loop-not-forced.ll

 ; CHECK:         %sum = phi
 entry:
   br label %loop
 
 loop:
   %i = phi i32 [ 0, %entry ], [ %inc, %loop ]
   %phi.a = phi i32 [ %a, %entry ], [ %phi.b, %loop ]
   %phi.b = phi i32 [ %b, %entry ], [ %phi.a, %loop ]
-  %sum = phi i32 [ 0, %entry], [ %incsum, %loop ]
+  %sum = phi i32 [ 2, %entry], [ %incsum, %loop ]
   %incsum = add i32 %sum, %phi.a
   %inc = add i32 %i, 1
   %cmp = icmp slt i32 %i, 1000
 
   br i1 %cmp, label %loop, label %exit
 
 exit:
   ret i32 %sum
 }

llvm/test/Transforms/LoopUnroll/peel-to-turn-invariant-accesses-dereferenceable.ll

 }
 
 define i32 @peel_readonly_to_make_loads_derefenceable_exits_lead_to_unreachable(ptr %ptr, i32 %N, ptr %inv, i1 %c.1) {
 ; CHECK-LABEL: @peel_readonly_to_make_loads_derefenceable_exits_lead_to_unreachable(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[THEN:%.*]], label [[EXIT_2:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[THEN_2:%.*]], label [[EXIT_2]]
 ; CHECK:       then.2:
 ; CHECK-NEXT:    [[C_4:%.*]] = icmp ult i32 [[I]], 4
 ; CHECK-NEXT:    br i1 [[C_4]], label [[LOOP_LATCH]], label [[EXIT_3:%.*]]

 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   br i1 %c.1, label %then, label %exit.2
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %then.2, label %exit.2
 
 then.2:

 }
 
 define i32 @do_not_peel_readonly_load_in_header(ptr %ptr, i32 %N, ptr %inv, i1 %c.1) {
 ; CHECK-LABEL: @do_not_peel_readonly_load_in_header(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[THEN:%.*]], label [[UNREACHABLE_EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[LOOP_LATCH]], label [[UNREACHABLE_EXIT]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4
 ; CHECK-NEXT:    [[SUM_NEXT]] = add i32 [[SUM]], [[LV]]
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    [[C_3:%.*]] = icmp ult i32 [[IV]], 1000
 ; CHECK-NEXT:    br i1 [[C_3]], label [[LOOP_HEADER]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[SUM_NEXT_LCSSA:%.*]] = phi i32 [ [[SUM_NEXT]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    ret i32 [[SUM_NEXT_LCSSA]]
 ; CHECK:       unreachable.exit:
 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %then, label %unreachable.exit
 
 then:
   br i1 %c.1, label %loop.latch, label %unreachable.exit
 
 loop.latch:

 }
 
 define i32 @do_not_peel_readonly_but_wont_turn_dereferenceable(ptr %ptr, i32 %N, i32 %x, ptr %inv) {
 ; CHECK-LABEL: @do_not_peel_readonly_but_wont_turn_dereferenceable(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    [[C_1:%.*]] = icmp eq i32 [[IV]], [[X:%.*]]
 ; CHECK-NEXT:    br i1 [[C_1]], label [[THEN:%.*]], label [[ELSE:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp eq i32 [[I]], 20
 ; CHECK-NEXT:    br i1 [[C_2]], label [[UNREACHABLE_EXIT:%.*]], label [[LOOP_LATCH]]
 ; CHECK:       else:
 ; CHECK-NEXT:    br label [[LOOP_LATCH]]

 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   %c.1 = icmp eq i32 %iv, %x
   br i1 %c.1, label %then, label %else
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp eq i32 %i, 20
   br i1 %c.2, label %unreachable.exit, label %loop.latch
 

 }
 
 define i32 @do_not_peel_write1(ptr %ptr, i32 %N, i32 %x, ptr %inv, ptr %dst, i1 %c.1) {
 ; CHECK-LABEL: @do_not_peel_write1(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[THEN:%.*]], label [[UNREACHABLE_EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP_LATCH]], label [[UNREACHABLE_EXIT]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4

 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   br i1 %c.1, label %then, label %unreachable.exit
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %loop.latch, label %unreachable.exit
 
 loop.latch:

 }
 
 define i32 @do_not_peel_write2(ptr %ptr, i32 %N, ptr %inv, ptr %dst) {
 ; CHECK-LABEL: @do_not_peel_write2(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_1:%.*]] = icmp eq i32 [[I]], 20
 ; CHECK-NEXT:    br i1 [[C_1]], label [[THEN:%.*]], label [[ELSE:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    store i32 [[I]], ptr [[DST:%.*]], align 4
 ; CHECK-NEXT:    br label [[LOOP_LATCH]]
 ; CHECK:       else:
 ; CHECK-NEXT:    br label [[UNREACHABLE_EXIT:%.*]]

 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   %i = load i32, ptr %inv
   %c.1 = icmp eq i32 %i, 20
   br i1 %c.1, label %then, label %else
 
 then:
   store i32 %i, ptr %dst
   br label %loop.latch
 

 declare i32 @llvm.experimental.deoptimize.i32(...)
 
 define i32 @peel_with_deopt_exit(ptr %ptr, i32 %N, ptr %inv, i1 %c.1) {
 ; CHECK-LABEL: @peel_with_deopt_exit(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[THEN:%.*]], label [[DEOPT_EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP_LATCH]], label [[DEOPT_EXIT]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4

 ; CHECK-NEXT:    [[RVAL:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"(i32 [[SUM_LCSSA]]) ]
 ; CHECK-NEXT:    ret i32 [[RVAL]]
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   br i1 %c.1, label %then, label %deopt.exit
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %loop.latch, label %deopt.exit
 
 loop.latch:

 }
 
 define i32 @do_not_peel_when_header_exiting(ptr %ptr, i32 %N, ptr %inv) {
 ; CHECK-LABEL: @do_not_peel_when_header_exiting(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    [[C_1:%.*]] = icmp ult i32 [[IV]], 1000
 ; CHECK-NEXT:    br i1 [[C_1]], label [[THEN:%.*]], label [[EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP_LATCH]], label [[UNREACHABLE_EXIT:%.*]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4
 ; CHECK-NEXT:    [[SUM_NEXT]] = add i32 [[SUM]], [[LV]]
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br label [[LOOP_HEADER]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[SUM_LCSSA:%.*]] = phi i32 [ [[SUM]], [[LOOP_HEADER]] ]
 ; CHECK-NEXT:    ret i32 [[SUM_LCSSA]]
 ; CHECK:       unreachable.exit:
 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   %c.1 = icmp ult i32 %iv, 1000
   br i1 %c.1, label %then, label %exit
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %loop.latch, label %unreachable.exit
 

 }
 
 define i32 @do_not_peel_readonly_to_make_loads_derefenceable_but_does_not_control_exit(ptr %ptr, i32 %N, ptr %inv, i1 %c.1, i32 %N.2) {
 ; CHECK-LABEL: @do_not_peel_readonly_to_make_loads_derefenceable_but_does_not_control_exit(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[THEN:%.*]], label [[UNREACHABLE_EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[INV:%.*]], align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[IV]], [[N_2:%.*]]
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP_LATCH]], label [[UNREACHABLE_EXIT]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4
 ; CHECK-NEXT:    [[SUM_NEXT]] = add i32 [[SUM]], [[LV]]
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    [[C_3:%.*]] = icmp ult i32 [[IV]], 1000
 ; CHECK-NEXT:    br i1 [[C_3]], label [[LOOP_HEADER]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[SUM_NEXT_LCSSA:%.*]] = phi i32 [ [[SUM_NEXT]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    ret i32 [[SUM_NEXT_LCSSA]]
 ; CHECK:       unreachable.exit:
 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   br i1 %c.1, label %then, label %unreachable.exit
 
 then:
   %i = load i32, ptr %inv
   %c.2 = icmp ult i32 %iv, %N.2
   br i1 %c.2, label %loop.latch, label %unreachable.exit
 
 loop.latch:

 @glob = global i32 10
 
 define i32 @do_not_peel_readonly_but_already_deref_glob(ptr %ptr, i32 %N, i1 %c.1) {
 ; CHECK-LABEL: @do_not_peel_readonly_but_already_deref_glob(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_HEADER:%.*]]
 ; CHECK:       loop.header:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP_LATCH:%.*]] ]
-; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
+; CHECK-NEXT:    [[SUM:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_NEXT:%.*]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[THEN:%.*]], label [[UNREACHABLE_EXIT:%.*]]
 ; CHECK:       then:
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr @glob, align 4
 ; CHECK-NEXT:    [[C_2:%.*]] = icmp ult i32 [[I]], 2
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP_LATCH]], label [[UNREACHABLE_EXIT]]
 ; CHECK:       loop.latch:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[PTR:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[LV:%.*]] = load i32, ptr [[GEP]], align 4
 ; CHECK-NEXT:    [[SUM_NEXT]] = add i32 [[SUM]], [[LV]]
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    [[C_3:%.*]] = icmp ult i32 [[IV]], 1000
 ; CHECK-NEXT:    br i1 [[C_3]], label [[LOOP_HEADER]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[SUM_NEXT_LCSSA:%.*]] = phi i32 [ [[SUM_NEXT]], [[LOOP_LATCH]] ]
 ; CHECK-NEXT:    ret i32 [[SUM_NEXT_LCSSA]]
 ; CHECK:       unreachable.exit:
 ; CHECK-NEXT:    call void @foo()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br label %loop.header
 
 loop.header:
   %iv = phi i32 [ 1, %entry ], [ %iv.next, %loop.latch ]
-  %sum = phi i32 [ 0, %entry ], [ %sum.next, %loop.latch ]
+  %sum = phi i32 [ 1, %entry ], [ %sum.next, %loop.latch ]
   br i1 %c.1, label %then, label %unreachable.exit
 
 then:
   %i = load i32, ptr @glob
   %c.2 = icmp ult i32 %i, 2
   br i1 %c.2, label %loop.latch, label %unreachable.exit
 
 loop.latch:

llvm/test/Transforms/LoopUnroll/runtime-epilog-debuginfo.ll

-; RUN: opt -passes=loop-unroll -unroll-runtime -unroll-runtime-epilog -S %s | FileCheck %s
+; RUN: opt -passes=loop-unroll -unroll-runtime -unroll-runtime-epilog -unroll-allow-peeling=false -S %s | FileCheck %s
 
 ; Test that epilogue is tagged with the same debug information as original loop body rather than original loop exit.
 
 ; CHECK: for.body.i:
 ; CHECK:   br i1 {{.*}}, label %lee1.exit.loopexit.unr-lcssa.loopexit, label %for.body.i, !dbg ![[LOOP_LOC:[0-9]+]]
 ; CHECK: lee1.exit.loopexit.unr-lcssa.loopexit:
 ; CHECK:   br label %lee1.exit.loopexit.unr-lcssa, !dbg ![[LOOP_LOC]]
 ; CHECK: lee1.exit.loopexit.unr-lcssa:

llvm/test/Transforms/LoopUnroll/runtime-loop-at-most-two-exits.ll

 ; ENABLED-NEXT:    [[XTRAITER:%.*]] = and i64 [[TMP0]], 7
 ; ENABLED-NEXT:    [[TMP2:%.*]] = icmp ult i64 [[TMP1]], 7
 ; ENABLED-NEXT:    br i1 [[TMP2]], label [[FOR_END_UNR_LCSSA:%.*]], label [[ENTRY_NEW:%.*]]
 ; ENABLED:       entry.new:
 ; ENABLED-NEXT:    [[UNROLL_ITER:%.*]] = sub i64 [[TMP0]], [[XTRAITER]]
 ; ENABLED-NEXT:    br label [[HEADER:%.*]]
 ; ENABLED:       header:
 ; ENABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[FOR_BODY_7:%.*]] ]
-; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 0, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[FOR_BODY_7]] ]
+; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 1, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[FOR_BODY_7]] ]
 ; ENABLED-NEXT:    [[NITER:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[NITER_NEXT_7:%.*]], [[FOR_BODY_7]] ]
 ; ENABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N]], 42
 ; ENABLED-NEXT:    br i1 [[CMP]], label [[FOR_EXIT2_LOOPEXIT:%.*]], label [[FOR_BODY:%.*]]
 ; ENABLED:       for.body:
 ; ENABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; ENABLED-NEXT:    [[TMP3:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; ENABLED-NEXT:    [[ADD:%.*]] = add nsw i32 [[TMP3]], [[SUM_02]]
 ; ENABLED-NEXT:    [[INDVARS_IV_NEXT:%.*]] = add nuw nsw i64 [[INDVARS_IV]], 1

 ; ENABLED:       for.end.unr-lcssa.loopexit:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH_PH:%.*]] = phi i32 [ [[ADD_7]], [[FOR_BODY_7]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR_PH:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_7]], [[FOR_BODY_7]] ]
 ; ENABLED-NEXT:    [[SUM_02_UNR_PH:%.*]] = phi i32 [ [[ADD_7]], [[FOR_BODY_7]] ]
 ; ENABLED-NEXT:    br label [[FOR_END_UNR_LCSSA]]
 ; ENABLED:       for.end.unr-lcssa:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH:%.*]] = phi i32 [ undef, [[ENTRY:%.*]] ], [ [[SUM_0_LCSSA_PH_PH]], [[FOR_END_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_UNR_PH]], [[FOR_END_UNR_LCSSA_LOOPEXIT]] ]
-; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[FOR_END_UNR_LCSSA_LOOPEXIT]] ]
+; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[FOR_END_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[LCMP_MOD:%.*]] = icmp ne i64 [[XTRAITER]], 0
 ; ENABLED-NEXT:    br i1 [[LCMP_MOD]], label [[HEADER_EPIL_PREHEADER:%.*]], label [[FOR_END:%.*]]
 ; ENABLED:       header.epil.preheader:
 ; ENABLED-NEXT:    br label [[HEADER_EPIL:%.*]]
 ; ENABLED:       header.epil:
 ; ENABLED-NEXT:    [[INDVARS_IV_EPIL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_EPIL:%.*]], [[FOR_BODY_EPIL:%.*]] ], [ [[INDVARS_IV_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[SUM_02_EPIL:%.*]] = phi i32 [ [[ADD_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[SUM_02_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ 0, [[HEADER_EPIL_PREHEADER]] ], [ [[EPIL_ITER_NEXT:%.*]], [[FOR_BODY_EPIL]] ]

 ; ENABLED-NEXT:    [[RETVAL:%.*]] = phi i32 [ [[RETVAL_PH]], [[FOR_EXIT2_LOOPEXIT]] ], [ [[RETVAL_PH3]], [[FOR_EXIT2_LOOPEXIT2]] ]
 ; ENABLED-NEXT:    ret i32 [[RETVAL]]
 ;
 ; DISABLED-LABEL: @test(
 ; DISABLED-NEXT:  entry:
 ; DISABLED-NEXT:    br label [[HEADER:%.*]]
 ; DISABLED:       header:
 ; DISABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; DISABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[FOR_BODY]] ], [ 0, [[ENTRY]] ]
+; DISABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[FOR_BODY]] ], [ 1, [[ENTRY]] ]
 ; DISABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; DISABLED-NEXT:    br i1 [[CMP]], label [[FOR_EXIT2:%.*]], label [[FOR_BODY]]
 ; DISABLED:       for.body:
 ; DISABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; DISABLED-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; DISABLED-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; DISABLED-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; DISABLED-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N]]
 ; DISABLED-NEXT:    br i1 [[EXITCOND]], label [[FOR_END:%.*]], label [[HEADER]]
 ; DISABLED:       for.end:
 ; DISABLED-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ [[ADD]], [[FOR_BODY]] ]
 ; DISABLED-NEXT:    ret i32 [[SUM_0_LCSSA]]
 ; DISABLED:       for.exit2:
 ; DISABLED-NEXT:    [[RETVAL:%.*]] = phi i32 [ [[SUM_02]], [[HEADER]] ]
 ; DISABLED-NEXT:    ret i32 [[RETVAL]]
 ;
 entry:
   br label %header
 
 header:
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %cmp = icmp eq i64 %n, 42
   br i1 %cmp, label %for.exit2, label %for.body
 
 for.body:
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1

llvm/test/Transforms/LoopUnroll/runtime-loop-multiple-exits.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: -p
-; RUN: opt < %s -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=true -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S | FileCheck %s -check-prefixes=CHECK,EPILOG
-; RUN: opt < %s -passes=loop-unroll -unroll-runtime -unroll-count=2 -unroll-runtime-epilog=true -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S | FileCheck %s -check-prefixes=CHECK,EPILOG-BLOCK
-; RUN: opt < %s -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=false -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S | FileCheck %s -check-prefixes=CHECK,PROLOG
-; RUN: opt < %s -passes=loop-unroll -unroll-runtime -unroll-runtime-epilog=false -unroll-count=2 -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S | FileCheck %s -check-prefixes=CHECK,PROLOG-BLOCK
+; RUN: opt < %s -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=true -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S -unroll-allow-peeling=false | FileCheck %s -check-prefixes=CHECK,EPILOG
+; RUN: opt < %s -passes=loop-unroll -unroll-runtime -unroll-count=2 -unroll-runtime-epilog=true -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S -unroll-allow-peeling=false | FileCheck %s -check-prefixes=CHECK,EPILOG-BLOCK
+; RUN: opt < %s -passes=loop-unroll -unroll-runtime=true -unroll-runtime-epilog=false -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S -unroll-allow-peeling=false | FileCheck %s -check-prefixes=CHECK,PROLOG
+; RUN: opt < %s -passes=loop-unroll -unroll-runtime -unroll-runtime-epilog=false -unroll-count=2 -unroll-runtime-multi-exit=true -verify-loop-lcssa -verify-dom-info -verify-loop-info -S -unroll-allow-peeling=false | FileCheck %s -check-prefixes=CHECK,PROLOG-BLOCK
 
 ; REQUIRES: asserts
 
 ; test with three exiting and three exit blocks.
 ; none of the exit blocks have successors
 define void @test1(i64 %trip, i1 %cond) {
 ; EPILOG-LABEL: @test1(
 ; EPILOG-NEXT:  entry:

llvm/test/Transforms/LoopUnroll/runtime-loop.ll

 
 define i32 @test(ptr nocapture %a, i32 %n) nounwind uwtable readonly {
 entry:
   %cmp1 = icmp eq i32 %n, 0
   br i1 %cmp1, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %n
   br i1 %exitcond, label %for.end, label %for.body
 
 for.end:                                          ; preds = %for.body, %entry
-  %sum.0.lcssa = phi i32 [ 0, %entry ], [ %add, %for.body ]
+  %sum.0.lcssa = phi i32 [ 1, %entry ], [ %add, %for.body ]
   ret i32 %sum.0.lcssa
 }
 
 
 ; Still try to completely unroll loops with compile-time trip counts
 ; even if the -unroll-runtime is specified
 
 ; COMMON-LABEL: @test1(
 ; COMMON: for.body:
 ; COMMON-NOT: for.body.epil:
 ; COMMON-NOT: for.body.prol:
 
 define i32 @test1(ptr nocapture %a) nounwind uwtable readonly {
 entry:
   br label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
-  %sum.01 = phi i32 [ 0, %entry ], [ %add, %for.body ]
+  %sum.01 = phi i32 [ 1, %entry ], [ %add, %for.body ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.01
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, 5
   br i1 %exitcond, label %for.end, label %for.body
 

 define zeroext i16 @down(ptr nocapture %p, i32 %len) nounwind uwtable readonly {
 entry:
   %cmp2 = icmp eq i32 %len, 0
   br i1 %cmp2, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %p.addr.05 = phi ptr [ %incdec.ptr, %for.body ], [ %p, %entry ]
   %len.addr.04 = phi i32 [ %sub, %for.body ], [ %len, %entry ]
-  %res.03 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %res.03 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %incdec.ptr = getelementptr inbounds i16, ptr %p.addr.05, i64 1
   %0 = load i16, ptr %p.addr.05, align 2
   %conv = zext i16 %0 to i32
   %add = add i32 %conv, %res.03
   %sub = add nsw i32 %len.addr.04, -2
   %cmp = icmp eq i32 %sub, 0
   br i1 %cmp, label %for.cond.for.end_crit_edge, label %for.body
 

 define zeroext i16 @test2(ptr nocapture %p, i32 %len) nounwind uwtable readonly {
 entry:
   %cmp2 = icmp eq i32 %len, 0
   br i1 %cmp2, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %p.addr.05 = phi ptr [ %incdec.ptr, %for.body ], [ %p, %entry ]
   %len.addr.04 = phi i32 [ %sub, %for.body ], [ %len, %entry ]
-  %res.03 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %res.03 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %incdec.ptr = getelementptr inbounds i16, ptr %p.addr.05, i64 1
   %0 = load i16, ptr %p.addr.05, align 2
   %conv = zext i16 %0 to i32
   %add = add i32 %conv, %res.03
   %sub = add nsw i32 %len.addr.04, -2
   %cmp = icmp eq i32 %sub, 0
   br i1 %cmp, label %for.cond.for.end_crit_edge, label %for.body, !llvm.loop !0
 

llvm/test/Transforms/LoopUnroll/runtime-loop2.ll

 
 define i32 @test(ptr nocapture %a, i32 %n) nounwind uwtable readonly {
 entry:
   %cmp1 = icmp eq i32 %n, 0
   br i1 %cmp1, label %for.end, label %for.body
 
 for.body:                                         ; preds = %for.body, %entry
   %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %for.body ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %for.body ], [ 1, %entry ]
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv
   %0 = load i32, ptr %arrayidx, align 4
   %add = add nsw i32 %0, %sum.02
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %n
   br i1 %exitcond, label %for.end, label %for.body
 
 for.end:                                          ; preds = %for.body, %entry
   %sum.0.lcssa = phi i32 [ 0, %entry ], [ %add, %for.body ]
   ret i32 %sum.0.lcssa
 }

llvm/test/Transforms/LoopUnroll/runtime-loop3.ll

for.cond1.preheader.lr.ph:                        ; preds = %entry
   br label %for.cond1.preheader
 
 for.cond1.preheader:                              ; preds = %for.inc5, %for.cond1.preheader.lr.ph
   %indvars.iv16 = phi i64 [ 0, %for.cond1.preheader.lr.ph ], [ %indvars.iv.next17, %for.inc5 ]
   %sum.012 = phi i32 [ 0, %for.cond1.preheader.lr.ph ], [ %sum.1.lcssa, %for.inc5 ]
   br i1 %cmp28, label %for.body3, label %for.inc5
 
 for.body3:                                        ; preds = %for.cond1.preheader, %for.body3
-  %indvars.iv = phi i64 [ %indvars.iv.next, %for.body3 ], [ 0, %for.cond1.preheader ]
+  %indvars.iv = phi i64 [ %indvars.iv.next, %for.body3 ], [ 1, %for.cond1.preheader ]
   %sum.19 = phi i32 [ %add4, %for.body3 ], [ %sum.012, %for.cond1.preheader ]
   %0 = add nsw i64 %indvars.iv, %indvars.iv16
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %0
   %1 = load i32, ptr %arrayidx, align 4
   %add4 = add nsw i32 %1, %sum.19
   %indvars.iv.next = add i64 %indvars.iv, 1
   %lftr.wideiv = trunc i64 %indvars.iv.next to i32
   %exitcond = icmp eq i32 %lftr.wideiv, %m

llvm/test/Transforms/LoopUnroll/runtime-multiexit-heuristic.ll

 ; CHECK-NEXT:    [[XTRAITER:%.*]] = and i64 [[TMP0]], 7
 ; CHECK-NEXT:    [[TMP2:%.*]] = icmp ult i64 [[TMP1]], 7
 ; CHECK-NEXT:    br i1 [[TMP2]], label [[LATCHEXIT_UNR_LCSSA:%.*]], label [[ENTRY_NEW:%.*]]
 ; CHECK:       entry.new:
 ; CHECK-NEXT:    [[UNROLL_ITER:%.*]] = and i64 [[TMP0]], -8
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[LATCH_7:%.*]] ]
-; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ 0, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
+; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ 1, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
 ; CHECK-NEXT:    [[NITER:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[NITER_NEXT_7:%.*]], [[LATCH_7]] ]
 ; CHECK-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; CHECK:       for.exiting_block:
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i64 [[TMP0]], 42
 ; CHECK-NEXT:    br i1 [[CMP]], label [[OTHEREXIT_LOOPEXIT:%.*]], label [[LATCH:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[TMP3:%.*]] = load i32, ptr [[ARRAYIDX]], align 4

 ; CHECK-NEXT:    [[NITER_NEXT_7]] = add i64 [[NITER]], 8
 ; CHECK-NEXT:    [[NITER_NCMP_7:%.*]] = icmp eq i64 [[NITER_NEXT_7]], [[UNROLL_ITER]]
 ; CHECK-NEXT:    br i1 [[NITER_NCMP_7]], label [[LATCHEXIT_UNR_LCSSA_LOOPEXIT:%.*]], label [[HEADER]]
 ; CHECK:       latchexit.unr-lcssa.loopexit:
 ; CHECK-NEXT:    br label [[LATCHEXIT_UNR_LCSSA]]
 ; CHECK:       latchexit.unr-lcssa:
 ; CHECK-NEXT:    [[SUM_0_LCSSA_PH:%.*]] = phi i32 [ undef, [[ENTRY:%.*]] ], [ [[ADD_7]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[INDVARS_IV_UNR:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT_7]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
-; CHECK-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD_7]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
+; CHECK-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[ADD_7]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[LCMP_MOD_NOT:%.*]] = icmp eq i64 [[XTRAITER]], 0
 ; CHECK-NEXT:    br i1 [[LCMP_MOD_NOT]], label [[LATCHEXIT:%.*]], label [[HEADER_EPIL_PREHEADER:%.*]]
 ; CHECK:       header.epil.preheader:
 ; CHECK-NEXT:    br label [[HEADER_EPIL:%.*]]
 ; CHECK:       header.epil:
 ; CHECK-NEXT:    [[INDVARS_IV_EPIL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_EPIL:%.*]], [[LATCH_EPIL:%.*]] ], [ [[INDVARS_IV_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; CHECK-NEXT:    [[SUM_02_EPIL:%.*]] = phi i32 [ [[ADD_EPIL:%.*]], [[LATCH_EPIL]] ], [ [[SUM_02_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; CHECK-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_NEXT:%.*]], [[LATCH_EPIL]] ], [ 0, [[HEADER_EPIL_PREHEADER]] ]

 ; CHECK-NEXT:    [[RVAL:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"(i32 [[SUM_02_LCSSA]]) ]
 ; CHECK-NEXT:    ret i32 [[RVAL]]
 ;
 ; NOUNROLL-LABEL: @test1(
 ; NOUNROLL-NEXT:  entry:
 ; NOUNROLL-NEXT:    br label [[HEADER:%.*]]
 ; NOUNROLL:       header:
 ; NOUNROLL-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; NOUNROLL-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; NOUNROLL:       for.exiting_block:
 ; NOUNROLL-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; NOUNROLL-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; NOUNROLL:       latch:
 ; NOUNROLL-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; NOUNROLL-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; NOUNROLL-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]

 ; ENABLED-NEXT:    [[XTRAITER:%.*]] = and i64 [[TMP0]], 7
 ; ENABLED-NEXT:    [[TMP2:%.*]] = icmp ult i64 [[TMP1]], 7
 ; ENABLED-NEXT:    br i1 [[TMP2]], label [[LATCHEXIT_UNR_LCSSA:%.*]], label [[ENTRY_NEW:%.*]]
 ; ENABLED:       entry.new:
 ; ENABLED-NEXT:    [[UNROLL_ITER:%.*]] = sub i64 [[TMP0]], [[XTRAITER]]
 ; ENABLED-NEXT:    br label [[HEADER:%.*]]
 ; ENABLED:       header:
 ; ENABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[LATCH_7:%.*]] ]
-; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 0, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
+; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 1, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[NITER:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[NITER_NEXT_7:%.*]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; ENABLED:       for.exiting_block:
 ; ENABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N]], 42
 ; ENABLED-NEXT:    br i1 [[CMP]], label [[OTHEREXIT_LOOPEXIT:%.*]], label [[LATCH:%.*]]
 ; ENABLED:       latch:
 ; ENABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; ENABLED-NEXT:    [[TMP3:%.*]] = load i32, ptr [[ARRAYIDX]], align 4

 ; ENABLED:       latchexit.unr-lcssa.loopexit:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH_PH:%.*]] = phi i32 [ [[ADD_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR_PH:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[SUM_02_UNR_PH:%.*]] = phi i32 [ [[ADD_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    br label [[LATCHEXIT_UNR_LCSSA]]
 ; ENABLED:       latchexit.unr-lcssa:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH:%.*]] = phi i32 [ undef, [[ENTRY:%.*]] ], [ [[SUM_0_LCSSA_PH_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
-; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
+; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[LCMP_MOD:%.*]] = icmp ne i64 [[XTRAITER]], 0
 ; ENABLED-NEXT:    br i1 [[LCMP_MOD]], label [[HEADER_EPIL_PREHEADER:%.*]], label [[LATCHEXIT:%.*]]
 ; ENABLED:       header.epil.preheader:
 ; ENABLED-NEXT:    br label [[HEADER_EPIL:%.*]]
 ; ENABLED:       header.epil:
 ; ENABLED-NEXT:    [[INDVARS_IV_EPIL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_EPIL:%.*]], [[LATCH_EPIL:%.*]] ], [ [[INDVARS_IV_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[SUM_02_EPIL:%.*]] = phi i32 [ [[ADD_EPIL:%.*]], [[LATCH_EPIL]] ], [ [[SUM_02_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ 0, [[HEADER_EPIL_PREHEADER]] ], [ [[EPIL_ITER_NEXT:%.*]], [[LATCH_EPIL]] ]

 ; ENABLED-NEXT:    [[RVAL:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"(i32 [[SUM_02_LCSSA]]) ]
 ; ENABLED-NEXT:    ret i32 [[RVAL]]
 ;
 entry:
   br label %header
 
 header:
   %indvars.iv = phi i64 [ %indvars.iv.next, %latch ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %latch ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %latch ], [ 1, %entry ]
   br label %for.exiting_block
 
 for.exiting_block:
   %cmp = icmp eq i64 %n, 42
   br i1 %cmp, label %otherexit, label %latch
 
 latch:
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv

 ; the exit block is not a deopt block.
 define i32 @test2(ptr nocapture %a, i64 %n) {
 ;
 ; CHECK-LABEL: @test2(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; CHECK-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; CHECK:       for.exiting_block:
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; CHECK-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]]
 ; CHECK:       latchexit:
 ; CHECK-NEXT:    ret i32 [[ADD]]
 ; CHECK:       otherexit:
 ; CHECK-NEXT:    ret i32 [[SUM_02]]
 ;
 ; NOUNROLL-LABEL: @test2(
 ; NOUNROLL-NEXT:  entry:
 ; NOUNROLL-NEXT:    br label [[HEADER:%.*]]
 ; NOUNROLL:       header:
 ; NOUNROLL-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; NOUNROLL-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; NOUNROLL:       for.exiting_block:
 ; NOUNROLL-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; NOUNROLL-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; NOUNROLL:       latch:
 ; NOUNROLL-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; NOUNROLL-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; NOUNROLL-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]

 ; ENABLED-NEXT:    [[XTRAITER:%.*]] = and i64 [[TMP0]], 7
 ; ENABLED-NEXT:    [[TMP2:%.*]] = icmp ult i64 [[TMP1]], 7
 ; ENABLED-NEXT:    br i1 [[TMP2]], label [[LATCHEXIT_UNR_LCSSA:%.*]], label [[ENTRY_NEW:%.*]]
 ; ENABLED:       entry.new:
 ; ENABLED-NEXT:    [[UNROLL_ITER:%.*]] = sub i64 [[TMP0]], [[XTRAITER]]
 ; ENABLED-NEXT:    br label [[HEADER:%.*]]
 ; ENABLED:       header:
 ; ENABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[LATCH_7:%.*]] ]
-; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 0, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
+; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ 1, [[ENTRY_NEW]] ], [ [[ADD_7:%.*]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[NITER:%.*]] = phi i64 [ 0, [[ENTRY_NEW]] ], [ [[NITER_NEXT_7:%.*]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; ENABLED:       for.exiting_block:
 ; ENABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N]], 42
 ; ENABLED-NEXT:    br i1 [[CMP]], label [[OTHEREXIT_LOOPEXIT:%.*]], label [[LATCH:%.*]]
 ; ENABLED:       latch:
 ; ENABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; ENABLED-NEXT:    [[TMP3:%.*]] = load i32, ptr [[ARRAYIDX]], align 4

 ; ENABLED:       latchexit.unr-lcssa.loopexit:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH_PH:%.*]] = phi i32 [ [[ADD_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR_PH:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    [[SUM_02_UNR_PH:%.*]] = phi i32 [ [[ADD_7]], [[LATCH_7]] ]
 ; ENABLED-NEXT:    br label [[LATCHEXIT_UNR_LCSSA]]
 ; ENABLED:       latchexit.unr-lcssa:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA_PH:%.*]] = phi i32 [ undef, [[ENTRY:%.*]] ], [ [[SUM_0_LCSSA_PH_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[INDVARS_IV_UNR:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
-; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
+; ENABLED-NEXT:    [[SUM_02_UNR:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[SUM_02_UNR_PH]], [[LATCHEXIT_UNR_LCSSA_LOOPEXIT]] ]
 ; ENABLED-NEXT:    [[LCMP_MOD:%.*]] = icmp ne i64 [[XTRAITER]], 0
 ; ENABLED-NEXT:    br i1 [[LCMP_MOD]], label [[HEADER_EPIL_PREHEADER:%.*]], label [[LATCHEXIT:%.*]]
 ; ENABLED:       header.epil.preheader:
 ; ENABLED-NEXT:    br label [[HEADER_EPIL:%.*]]
 ; ENABLED:       header.epil:
 ; ENABLED-NEXT:    [[INDVARS_IV_EPIL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_EPIL:%.*]], [[LATCH_EPIL:%.*]] ], [ [[INDVARS_IV_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[SUM_02_EPIL:%.*]] = phi i32 [ [[ADD_EPIL:%.*]], [[LATCH_EPIL]] ], [ [[SUM_02_UNR]], [[HEADER_EPIL_PREHEADER]] ]
 ; ENABLED-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ 0, [[HEADER_EPIL_PREHEADER]] ], [ [[EPIL_ITER_NEXT:%.*]], [[LATCH_EPIL]] ]

 ; ENABLED-NEXT:    [[RVAL:%.*]] = phi i32 [ [[RVAL_PH]], [[OTHEREXIT_LOOPEXIT]] ], [ [[RVAL_PH3]], [[OTHEREXIT_LOOPEXIT2]] ]
 ; ENABLED-NEXT:    ret i32 [[RVAL]]
 ;
 entry:
   br label %header
 
 header:
   %indvars.iv = phi i64 [ %indvars.iv.next, %latch ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %latch ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %latch ], [ 1, %entry ]
   br label %for.exiting_block
 
 for.exiting_block:
   %cmp = icmp eq i64 %n, 42
   br i1 %cmp, label %otherexit, label %latch
 
 latch:
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv

 ; the loop is not worth unrolling.  We probably should peel said loop, but
 ; currently don't.
 define i32 @test3(ptr nocapture %a, i64 %n) !prof !{!"function_entry_count", i64 2048} {
 ; CHECK-LABEL: @test3(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; CHECK-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; CHECK:       for.exiting_block:
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; CHECK-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]], !prof [[PROF3:![0-9]+]]
 ; CHECK:       latchexit:
 ; CHECK-NEXT:    ret i32 [[ADD]]
 ; CHECK:       otherexit:
 ; CHECK-NEXT:    ret i32 57
 ;
 ; NOUNROLL-LABEL: @test3(
 ; NOUNROLL-NEXT:  entry:
 ; NOUNROLL-NEXT:    br label [[HEADER:%.*]]
 ; NOUNROLL:       header:
 ; NOUNROLL-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; NOUNROLL-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; NOUNROLL:       for.exiting_block:
 ; NOUNROLL-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; NOUNROLL-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; NOUNROLL:       latch:
 ; NOUNROLL-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; NOUNROLL-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; NOUNROLL-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; NOUNROLL-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; NOUNROLL-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N]]
 ; NOUNROLL-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]], !prof [[PROF1:![0-9]+]]
 ; NOUNROLL:       latchexit:
 ; NOUNROLL-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ [[ADD]], [[LATCH]] ]
 ; NOUNROLL-NEXT:    ret i32 [[SUM_0_LCSSA]]
 ; NOUNROLL:       otherexit:
 ; NOUNROLL-NEXT:    ret i32 57
 ;
 ; ENABLED-LABEL: @test3(
 ; ENABLED-NEXT:  entry:
 ; ENABLED-NEXT:    br label [[HEADER:%.*]]
 ; ENABLED:       header:
 ; ENABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; ENABLED-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; ENABLED:       for.exiting_block:
 ; ENABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[N:%.*]], 42
 ; ENABLED-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; ENABLED:       latch:
 ; ENABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; ENABLED-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; ENABLED-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; ENABLED-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; ENABLED-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N]]
 ; ENABLED-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]], !prof [[PROF4:![0-9]+]]
 ; ENABLED:       latchexit:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ [[ADD]], [[LATCH]] ]
 ; ENABLED-NEXT:    ret i32 [[SUM_0_LCSSA]]
 ; ENABLED:       otherexit:
 ; ENABLED-NEXT:    ret i32 57
 ;
 entry:
   br label %header
 
 header:
   %indvars.iv = phi i64 [ %indvars.iv.next, %latch ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %latch ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %latch ], [ 1, %entry ]
   br label %for.exiting_block
 
 for.exiting_block:
   %cmp = icmp eq i64 %n, 42
   br i1 %cmp, label %otherexit, label %latch
 
 latch:
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv

 ; seems to inhibit unrolling for some unclear reason.
 define i32 @test4(ptr nocapture %a, i64 %n) !prof !{!"function_entry_count", i64 2048} {
 ;
 ; CHECK-LABEL: @test4(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[HEADER:%.*]]
 ; CHECK:       header:
 ; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; CHECK-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; CHECK-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; CHECK:       for.exiting_block:
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp eq i64 [[INDVARS_IV]], 4096
 ; CHECK-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]]
 ; CHECK:       latchexit:
 ; CHECK-NEXT:    ret i32 [[ADD]]
 ; CHECK:       otherexit:
 ; CHECK-NEXT:    ret i32 57
 ;
 ; NOUNROLL-LABEL: @test4(
 ; NOUNROLL-NEXT:  entry:
 ; NOUNROLL-NEXT:    br label [[HEADER:%.*]]
 ; NOUNROLL:       header:
 ; NOUNROLL-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; NOUNROLL-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; NOUNROLL-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; NOUNROLL:       for.exiting_block:
 ; NOUNROLL-NEXT:    [[CMP:%.*]] = icmp eq i64 [[INDVARS_IV]], 4096
 ; NOUNROLL-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; NOUNROLL:       latch:
 ; NOUNROLL-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; NOUNROLL-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; NOUNROLL-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; NOUNROLL-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; NOUNROLL-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; NOUNROLL-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]]
 ; NOUNROLL:       latchexit:
 ; NOUNROLL-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ [[ADD]], [[LATCH]] ]
 ; NOUNROLL-NEXT:    ret i32 [[SUM_0_LCSSA]]
 ; NOUNROLL:       otherexit:
 ; NOUNROLL-NEXT:    ret i32 57
 ;
 ; ENABLED-LABEL: @test4(
 ; ENABLED-NEXT:  entry:
 ; ENABLED-NEXT:    br label [[HEADER:%.*]]
 ; ENABLED:       header:
 ; ENABLED-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[LATCH:%.*]] ], [ 0, [[ENTRY:%.*]] ]
-; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 0, [[ENTRY]] ]
+; ENABLED-NEXT:    [[SUM_02:%.*]] = phi i32 [ [[ADD:%.*]], [[LATCH]] ], [ 1, [[ENTRY]] ]
 ; ENABLED-NEXT:    br label [[FOR_EXITING_BLOCK:%.*]]
 ; ENABLED:       for.exiting_block:
 ; ENABLED-NEXT:    [[CMP:%.*]] = icmp eq i64 [[INDVARS_IV]], 4096
 ; ENABLED-NEXT:    br i1 [[CMP]], label [[OTHEREXIT:%.*]], label [[LATCH]]
 ; ENABLED:       latch:
 ; ENABLED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[INDVARS_IV]]
 ; ENABLED-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; ENABLED-NEXT:    [[ADD]] = add nsw i32 [[TMP0]], [[SUM_02]]
 ; ENABLED-NEXT:    [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1
 ; ENABLED-NEXT:    [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; ENABLED-NEXT:    br i1 [[EXITCOND]], label [[LATCHEXIT:%.*]], label [[HEADER]]
 ; ENABLED:       latchexit:
 ; ENABLED-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ [[ADD]], [[LATCH]] ]
 ; ENABLED-NEXT:    ret i32 [[SUM_0_LCSSA]]
 ; ENABLED:       otherexit:
 ; ENABLED-NEXT:    ret i32 57
 ;
 entry:
   br label %header
 
 header:
   %indvars.iv = phi i64 [ %indvars.iv.next, %latch ], [ 0, %entry ]
-  %sum.02 = phi i32 [ %add, %latch ], [ 0, %entry ]
+  %sum.02 = phi i32 [ %add, %latch ], [ 1, %entry ]
   br label %for.exiting_block
 
 for.exiting_block:
   %cmp = icmp eq i64 %indvars.iv, 4096
   br i1 %cmp, label %otherexit, label %latch
 
 latch:
   %arrayidx = getelementptr inbounds i32, ptr %a, i64 %indvars.iv

llvm/test/Transforms/LoopUnroll/unroll-cost-symbolic-execute.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes=loop-unroll -S -unroll-threshold=120 -unroll-max-iteration-count-to-analyze=100 < %s | FileCheck %s
+; RUN: opt -passes=loop-unroll -S -unroll-threshold=120 -unroll-max-iteration-count-to-analyze=100 -unroll-allow-peeling=false < %s | FileCheck %s
 
 ; TODO: None of the if.false blocks are reachable, it would be nice if
 ; the output of unrolling made this obvious and didn't rely on other
 ; passes to cleanup code the cost model already knew was dead.
 
 @G = global i32 0
 
 ; Symbolic simplification can prove the value of %zext on first

llvm/test/Transforms/LoopUnroll/unroll-heuristics-pgo.ll

 ; CHECK: %mul.3 = mul
 ; CHECK: loop.epil:
 define i32 @bar_prof(ptr noalias nocapture readonly %src, i64 %c) !prof !1 {
 entry:
   br label %loop
 
 loop:
   %iv = phi i64 [ 0, %entry ], [ %inc, %loop ]
-  %r  = phi i32 [ 0, %entry ], [ %add, %loop ]
+  %r  = phi i32 [ 1, %entry ], [ %add, %loop ]
   %arrayidx = getelementptr inbounds i32, ptr %src, i64 %iv
   %src_element = load i32, ptr %arrayidx, align 4
   %array_const_idx = getelementptr inbounds [9 x i32], ptr @known_constant, i64 0, i64 %iv
   %const_array_element = load i32, ptr %array_const_idx, align 4
   %mul = mul nsw i32 %src_element, %const_array_element
   %add = add nsw i32 %mul, %r
   %inc = add nuw nsw i64 %iv, 1
   %exitcond86.i = icmp eq i64 %inc, %c
   br i1 %exitcond86.i, label %loop.end, label %loop, !prof !2
 
 loop.end:
   %r.lcssa = phi i32 [ %r, %loop ]
   ret i32 %r.lcssa
 }
 
 ; CHECK-LABEL: @bar_prof_flat
 ; CHECK-NOT: loop.epil
 define i32 @bar_prof_flat(ptr noalias nocapture readonly %src, i64 %c) !prof !1 {
 entry:
   br label %loop
 
 loop:
   %iv = phi i64 [ 0, %entry ], [ %inc, %loop ]
-  %r  = phi i32 [ 0, %entry ], [ %add, %loop ]
+  %r  = phi i32 [ 1, %entry ], [ %add, %loop ]
   %arrayidx = getelementptr inbounds i32, ptr %src, i64 %iv
   %src_element = load i32, ptr %arrayidx, align 4
   %array_const_idx = getelementptr inbounds [9 x i32], ptr @known_constant, i64 0, i64 %iv
   %const_array_element = load i32, ptr %array_const_idx, align 4
   %mul = mul nsw i32 %src_element, %const_array_element
   %add = add nsw i32 %mul, %r
   %inc = add nuw nsw i64 %iv, 1
   %exitcond86.i = icmp eq i64 %inc, %c

llvm/test/Transforms/PhaseOrdering/ARM/arm_mean_q7.ll

 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP_NOT10:%.*]] = icmp ult i32 [[BLOCKSIZE:%.*]], 16
 ; CHECK-NEXT:    br i1 [[CMP_NOT10]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK:       while.body.preheader:
 ; CHECK-NEXT:    [[SHR:%.*]] = lshr i32 [[BLOCKSIZE]], 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = and i32 [[BLOCKSIZE]], -16
 ; CHECK-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK:       while.body:
-; CHECK-NEXT:    [[SUM_013:%.*]] = phi i32 [ [[TMP3:%.*]], [[WHILE_BODY]] ], [ 0, [[WHILE_BODY_PREHEADER]] ]
+; CHECK-NEXT:    [[SUM_013:%.*]] = phi i32 [ [[TMP3:%.*]], [[WHILE_BODY]] ], [ 1, [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[PSRC_ADDR_012:%.*]] = phi ptr [ [[ADD_PTR:%.*]], [[WHILE_BODY]] ], [ [[PSRC:%.*]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[BLKCNT_011:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY]] ], [ [[SHR]], [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[TMP1:%.*]] = load <16 x i8>, ptr [[PSRC_ADDR_012]], align 1
 ; CHECK-NEXT:    [[TMP2:%.*]] = tail call i32 @llvm.arm.mve.addv.v16i8(<16 x i8> [[TMP1]], i32 0)
 ; CHECK-NEXT:    [[TMP3]] = add i32 [[TMP2]], [[SUM_013]]
 ; CHECK-NEXT:    [[DEC]] = add nsw i32 [[BLKCNT_011]], -1
 ; CHECK-NEXT:    [[ADD_PTR]] = getelementptr inbounds i8, ptr [[PSRC_ADDR_012]], i32 16
 ; CHECK-NEXT:    [[CMP_NOT:%.*]] = icmp eq i32 [[DEC]], 0
 ; CHECK-NEXT:    br i1 [[CMP_NOT]], label [[WHILE_END_LOOPEXIT:%.*]], label [[WHILE_BODY]]
 ; CHECK:       while.end.loopexit:
 ; CHECK-NEXT:    [[UGLYGEP:%.*]] = getelementptr i8, ptr [[PSRC]], i32 [[TMP0]]
 ; CHECK-NEXT:    br label [[WHILE_END]]
 ; CHECK:       while.end:
 ; CHECK-NEXT:    [[PSRC_ADDR_0_LCSSA:%.*]] = phi ptr [ [[PSRC]], [[ENTRY:%.*]] ], [ [[UGLYGEP]], [[WHILE_END_LOOPEXIT]] ]
-; CHECK-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP3]], [[WHILE_END_LOOPEXIT]] ]
+; CHECK-NEXT:    [[SUM_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[TMP3]], [[WHILE_END_LOOPEXIT]] ]
 ; CHECK-NEXT:    [[AND:%.*]] = and i32 [[BLOCKSIZE]], 15
 ; CHECK-NEXT:    [[CMP2_NOT15:%.*]] = icmp eq i32 [[AND]], 0
 ; CHECK-NEXT:    br i1 [[CMP2_NOT15]], label [[WHILE_END5:%.*]], label [[VECTOR_BODY:%.*]]
 ; CHECK:       vector.body:
 ; CHECK-NEXT:    [[ACTIVE_LANE_MASK:%.*]] = tail call <16 x i1> @llvm.get.active.lane.mask.v16i1.i32(i32 0, i32 [[AND]])
 ; CHECK-NEXT:    [[WIDE_MASKED_LOAD:%.*]] = tail call <16 x i8> @llvm.masked.load.v16i8.p0(ptr [[PSRC_ADDR_0_LCSSA]], i32 1, <16 x i1> [[ACTIVE_LANE_MASK]], <16 x i8> poison)
 ; CHECK-NEXT:    [[TMP4:%.*]] = sext <16 x i8> [[WIDE_MASKED_LOAD]] to <16 x i32>
 ; CHECK-NEXT:    [[TMP5:%.*]] = select <16 x i1> [[ACTIVE_LANE_MASK]], <16 x i32> [[TMP4]], <16 x i32> zeroinitializer
entry:
   %vecSrc = alloca <16 x i8>, align 1
   %sum = alloca i32, align 4
   store ptr %pSrc, ptr %pSrc.addr, align 4
   store i32 %blockSize, ptr %blockSize.addr, align 4
   store ptr %pResult, ptr %pResult.addr, align 4
   call void @llvm.lifetime.start.p0(i64 4, ptr %blkCnt) #3
   call void @llvm.lifetime.start.p0(i64 16, ptr %vecSrc) #3
   call void @llvm.lifetime.start.p0(i64 4, ptr %sum) #3
-  store i32 0, ptr %sum, align 4
+  store i32 1, ptr %sum, align 4
   %0 = load i32, ptr %blockSize.addr, align 4
   %shr = lshr i32 %0, 4
   store i32 %shr, ptr %blkCnt, align 4
   br label %while.cond
 
 while.cond:                                       ; preds = %while.body, %entry
   %1 = load i32, ptr %blkCnt, align 4
   %cmp = icmp ugt i32 %1, 0

llvm/test/Transforms/PhaseOrdering/SystemZ/sub-xor.ll

 
 define dso_local zeroext i32 @foo(ptr noundef %a) #0 {
 ; CHECK-LABEL: @foo(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    tail call void @populate(ptr noundef nonnull @ARR) #[[ATTR2:[0-9]+]]
 ; CHECK-NEXT:    br label [[FOR_BODY4:%.*]]
 ; CHECK:       for.body4:
 ; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[FOR_BODY4]] ]
-; CHECK-NEXT:    [[SUM_11:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD_7:%.*]], [[FOR_BODY4]] ]
+; CHECK-NEXT:    [[SUM_11:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[ADD_7:%.*]], [[FOR_BODY4]] ]
 ; CHECK-NEXT:    [[IDX_NEG:%.*]] = sub nsw i64 0, [[INDVARS_IV]]
 ; CHECK-NEXT:    [[ADD_PTR:%.*]] = getelementptr inbounds i32, ptr getelementptr inbounds ([100 x i32], ptr @ARR, i64 0, i64 99), i64 [[IDX_NEG]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ADD_PTR]], align 4, !tbaa [[TBAA3:![0-9]+]]
 ; CHECK-NEXT:    [[ADD:%.*]] = add i32 [[TMP0]], [[SUM_11]]
 ; CHECK-NEXT:    [[IDX_NEG_19:%.*]] = xor i64 [[INDVARS_IV]], -1
 ; CHECK-NEXT:    [[ADD_PTR_110:%.*]] = getelementptr inbounds i32, ptr getelementptr inbounds ([100 x i32], ptr @ARR, i64 0, i64 99), i64 [[IDX_NEG_19]]
 ; CHECK-NEXT:    [[TMP1:%.*]] = load i32, ptr [[ADD_PTR_110]], align 4, !tbaa [[TBAA3]]
 ; CHECK-NEXT:    [[ADD_111:%.*]] = add i32 [[TMP1]], [[ADD]]

 ; CHECK-NEXT:    ret i32 [[ADD_2_7]]
 ;
 entry:
   call void @populate(ptr noundef @ARR)
   br label %for.cond
 
 for.cond:                                         ; preds = %for.inc5, %entry
   %j.0 = phi i32 [ 1, %entry ], [ %inc6, %for.inc5 ]
-  %sum.0 = phi i32 [ 0, %entry ], [ %sum.1, %for.inc5 ]
+  %sum.0 = phi i32 [ 1, %entry ], [ %sum.1, %for.inc5 ]
   %cmp = icmp slt i32 %j.0, 4
   br i1 %cmp, label %for.body, label %for.cond.cleanup
 
 for.cond.cleanup:                                 ; preds = %for.cond
   br label %for.end7
 
 for.body:                                         ; preds = %for.cond
   br label %for.cond1

llvm/test/Transforms/PhaseOrdering/X86/ctlz-loop.ll

 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TOBOOL_NOT1:%.*]] = icmp eq i32 [[N:%.*]], 0
 ; CHECK-NEXT:    br i1 [[TOBOOL_NOT1]], label [[WHILE_END:%.*]], label [[WHILE_BODY_PREHEADER:%.*]]
 ; CHECK:       while.body.preheader:
 ; CHECK-NEXT:    [[TMP0:%.*]] = tail call i32 @llvm.abs.i32(i32 [[N]], i1 true)
 ; CHECK-NEXT:    br label [[WHILE_BODY:%.*]]
 ; CHECK:       while.body:
 ; CHECK-NEXT:    [[N_ADDR_03:%.*]] = phi i32 [ [[TMP1:%.*]], [[WHILE_BODY]] ], [ [[TMP0]], [[WHILE_BODY_PREHEADER]] ]
-; CHECK-NEXT:    [[I_02:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_BODY]] ], [ 0, [[WHILE_BODY_PREHEADER]] ]
+; CHECK-NEXT:    [[I_02:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_BODY]] ], [ 1, [[WHILE_BODY_PREHEADER]] ]
 ; CHECK-NEXT:    [[TMP1]] = lshr i32 [[N_ADDR_03]], 1
 ; CHECK-NEXT:    [[INC]] = add nuw nsw i32 [[I_02]], 1
 ; CHECK-NEXT:    [[TOBOOL_NOT:%.*]] = icmp ult i32 [[N_ADDR_03]], 2
 ; CHECK-NEXT:    br i1 [[TOBOOL_NOT]], label [[WHILE_END]], label [[WHILE_BODY]]
 ; CHECK:       while.end:
-; CHECK-NEXT:    [[I_0_LCSSA:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[INC]], [[WHILE_BODY]] ]
+; CHECK-NEXT:    [[I_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ], [ [[INC]], [[WHILE_BODY]] ]
 ; CHECK-NEXT:    ret i32 [[I_0_LCSSA]]
 ;
 entry:
   %cmp = icmp sge i32 %n, 0
   br i1 %cmp, label %cond.true, label %cond.false
 
 cond.true:                                        ; preds = %entry
   br label %cond.end
 
 cond.false:                                       ; preds = %entry
   %sub = sub nsw i32 0, %n
   br label %cond.end
 
 cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi i32 [ %n, %cond.true ], [ %sub, %cond.false ]
   br label %while.cond
 
 while.cond:                                       ; preds = %while.body, %cond.end
-  %i.0 = phi i32 [ 0, %cond.end ], [ %inc, %while.body ]
+  %i.0 = phi i32 [ 1, %cond.end ], [ %inc, %while.body ]
   %n.addr.0 = phi i32 [ %cond, %cond.end ], [ %shr, %while.body ]
   %tobool = icmp ne i32 %n.addr.0, 0
   br i1 %tobool, label %while.body, label %while.end
 
 while.body:                                       ; preds = %while.cond
   %shr = ashr i32 %n.addr.0, 1
   %inc = add nsw i32 %i.0, 1
   br label %while.cond
 
 while.end:                                        ; preds = %while.cond
   ret i32 %i.0
 }

llvm/test/Transforms/PhaseOrdering/pr44461-br-to-switch-rotate.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S -O2 < %s | FileCheck %s
 ; RUN: opt -S -passes='default<O2>' < %s | FileCheck %s
 
 ; FIXME: The br -> switch conversion blocks loop rotation.
 
 define i64 @test() {
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT:  start:
 ; CHECK-NEXT:    br label [[BB10:%.*]]
 ; CHECK:       bb10:
 ; CHECK-NEXT:    [[ITER1_SROA_5_0:%.*]] = phi i64 [ 100000, [[START:%.*]] ], [ [[SPEC_SELECT:%.*]], [[BB3_I_I:%.*]] ]
 ; CHECK-NEXT:    [[ITER1_SROA_9_0:%.*]] = phi i8 [ 2, [[START]] ], [ [[TMP3:%.*]], [[BB3_I_I]] ]
-; CHECK-NEXT:    [[COUNT_1:%.*]] = phi i64 [ 0, [[START]] ], [ [[TMP4:%.*]], [[BB3_I_I]] ]
+; CHECK-NEXT:    [[COUNT_1:%.*]] = phi i64 [ 1, [[START]] ], [ [[TMP4:%.*]], [[BB3_I_I]] ]
 ; CHECK-NEXT:    switch i8 [[ITER1_SROA_9_0]], label [[BB12:%.*]] [
 ; CHECK-NEXT:    i8 2, label [[BB3_I_I]]
 ; CHECK-NEXT:    i8 0, label [[BB3_I_I]]
 ; CHECK-NEXT:    ]
 ; CHECK:       bb3.i.i:
 ; CHECK-NEXT:    [[TMP2:%.*]] = icmp eq i64 [[ITER1_SROA_5_0]], 0
 ; CHECK-NEXT:    [[TMP3]] = zext i1 [[TMP2]] to i8
 ; CHECK-NEXT:    [[SPEC_SELECT]] = tail call i64 @llvm.usub.sat.i64(i64 [[ITER1_SROA_5_0]], i64 1)
 ; CHECK-NEXT:    [[TMP4]] = add i64 [[COUNT_1]], [[ITER1_SROA_5_0]]
 ; CHECK-NEXT:    br label [[BB10]]
 ; CHECK:       bb12:
 ; CHECK-NEXT:    ret i64 [[COUNT_1]]
 ;
 start:
   br label %bb10
 
 bb10:                                             ; preds = %bb3.i.i, %bb6
   %iter1.sroa.5.0 = phi i64 [ 100000, %start ], [ %spec.select, %bb3.i.i ]
   %iter1.sroa.9.0 = phi i8 [ 2, %start ], [ %tmp3, %bb3.i.i ]
-  %count.1 = phi i64 [ 0, %start ], [ %tmp4, %bb3.i.i ]
+  %count.1 = phi i64 [ 1, %start ], [ %tmp4, %bb3.i.i ]
   %c1 = icmp eq i8 %iter1.sroa.9.0, 2
   %c2 = icmp eq i8 %iter1.sroa.9.0, 0
   %c3 = or i1 %c1, %c2
   br i1 %c3, label %bb3.i.i, label %bb12
 
 bb3.i.i:                                          ; preds = %bb10, %bb10
   %tmp2 = icmp eq i64 %iter1.sroa.5.0, 0
   %tmp3 = zext i1 %tmp2 to i8