LFTR for multiple exit loops

[reames]

Summary

Teach IndVarSimply's LinearFunctionTestReplace transform to handle multiple exit loops. LFTR does two key things 1) it rewrites (all) exit tests in terms of a common IV potentially eliminating one in the process and 2) it moves any offset/indexing/f(i) style logic out of the loop.

This turns out to actually be pretty easy to implement. SCEV already has all the information we need to know what the backedge taken count is for each individual exit. (We use that when computing the BE taken count for the loop as a whole.) We basically just need to iterate through the exiting blocks and apply the existing logic with the exit specific BE taken count. (The previously landed NFC makes this super obvious.)

I chose to go ahead and apply this to all loop exits instead of only latch exits as originally proposed. After reviewing other passes, the only case I could find where LFTR form was harmful was LoopPredication. I've fixed the latch case, and guards aren't LFTRed anyways. We'll have some more work to do on the way towards widenable_conditions, but that's easily deferred.

[reames]

Clean up towards a practical patch. Need to land a bunch of tests, and then rebase. Also included the LoopPred fix needed to make the increase in scope non-pessimizing in practice. (There may be more pieces like that needed.)

[reames]

Rebase on tests, and minor generalization. Still have a couple of other test failures to track down, so not yet ready for review.

[reames]

The NFC portion of this turned out to be a bit more involved than expected. It's been separated into D62880. Once that lands, I'll rebase for the small remaining functional bit. I'm debating just going to all loop exits instead of just the latch exit. On further reflection, I'm not sure it's really going to matter impact wise, and fleshing out problems faster might be worthwhile.

Also, this is still blocked on one last correctness issue in the existing LFTR implementation. Specifically, we're not clearing the inbounds parameter on a pointer IV when moving to a dynamically dead IV. That needs fixed before we generalize the code this much and increase the number of opportunities for miscompiles sharply.

[reames]

Rebase on landed prep patches. Ready for actual review, though won't land until after https://reviews.llvm.org/D62939.

[nikic]

This looks good, but I'd like to see some tests that involve an IV change. In particular I would expect that this transformation can go from exit conditions on two different IVs, to exit conditions on a single IV and it would be nice to see that.

[reames]

rebase on requested tests

Just to note, I'm deliberately not adding any tests for possibly poison IVs. I don't think there are any unique issues for multiple exit IVs, and giving that I want to leave the coverage concentrated on things which are unique to multiple exits.

[nikic]

LGTM, thanks for the extra tests.

lib/Transforms/Scalar/IndVarSimplify.cpp

2634 ↗

(On Diff #204387)

Rename BETakenCount to ExitCount here and also in linearFunctionTestReplace, possibly as NFC precommit? With multiple exits, this is not the backedge count anymore...

[reames]

lib/Transforms/Scalar/IndVarSimplify.cpp

2634 ↗

(On Diff #204387)

Done in 363293.

[reames]

I just submitted rL363964 which fixes a bug introduced in this change. I can't find an example which actually triggers it so it may be entirely latent, but if anyone is chasing a miscompile tied to this change, try applying that patch.

[SjoerdMeijer]

Hello,
I am going to be a bit unhelpful here. We see some regressions caused by this change, but I didn't have a chance to look into it today and so I don't have a reproducer. And here's my unhelpful question: are you aware or have you seen regressions, or are you still working on this? I am not really familiar with this pass, but from reading the description, it should do good things. Taking a wild guess, I see changes in the tests and they change comparisons from ult to ne, could it for example be that this confuses other passes/analysis?
Anyway, I will try to look into this tomorrow.
Cheers.

[reames]

In D62625#1552433, @SjoerdMeijer wrote:

Hello,
I am going to be a bit unhelpful here. We see some regressions caused by this change, but I didn't have a chance to look into it today and so I don't have a reproducer. And here's my unhelpful question: are you aware or have you seen regressions, or are you still working on this? I am not really familiar with this pass, but from reading the description, it should do good things. Taking a wild guess, I see changes in the tests and they change comparisons from ult to ne, could it for example be that this confuses other passes/analysis?
Anyway, I will try to look into this tomorrow.
Cheers.

Just to be clear, you mean performance regressions correct? Not correctness ones?

I won't be surprised if this exposes weaknesses in other passes. If you can find me examples, I'm happy to go about trying to fix them. If there's too many, or the regressions are too large, we can also disable this behind a flag while we work through issues.

One case I know of is that you have a hot loop which runs very few iterations, this could end up being a regression. I have ideas on how to address that, but since they're involved, I'd really like to confirm a test case first.

Look forward to more details when you have them.

[mstorsjo]

I just ran into a performance regression due to this commit. With https://martin.st/temp/glew-preproc-i686.c, compiling with clang -target i686-w64-mingw32 -c -O3 glew-preproc-i686.c went from 73 seconds before this commit, to 133 seconds afterwards.

[dendibakh]

There should be also positive impact on LSR pass from this change, since LSR likes equality comparisons.

[reames]

In D62625#1552681, @mstorsjo wrote:

I just ran into a performance regression due to this commit. With https://martin.st/temp/glew-preproc-i686.c, compiling with clang -target i686-w64-mingw32 -c -O3 glew-preproc-i686.c went from 73 seconds before this commit, to 133 seconds afterwards.

Clarification here. Do you mean this was a compile time regression? Or a runtime regression? (i.e. is your clang here build with a different compiler? Or is this a self host?)

[mstorsjo]

In D62625#1552791, @reames wrote:

In D62625#1552681, @mstorsjo wrote:

I just ran into a performance regression due to this commit. With https://martin.st/temp/glew-preproc-i686.c, compiling with clang -target i686-w64-mingw32 -c -O3 glew-preproc-i686.c went from 73 seconds before this commit, to 133 seconds afterwards.

Clarification here. Do you mean this was a compile time regression? Or a runtime regression? (i.e. is your clang here build with a different compiler? Or is this a self host?)

Compile time regression. (I build the project it's part of with a 120 second timeout per translation unit, enforced with ulimit -t 120, that's why I noticed.)

[SjoerdMeijer]

Just to confirm that in my case it's a performance regression.
I am going to take a look now.

[SjoerdMeijer]

I think I am mainly looking at micro-architecture sensitivities and some unfortunate knock on effects of this patch.
I don't think I can blame anything of that on this patch. If I change my mind about that, I will let you know. :-) But then I'd of course have to come up with a reproducer (or a fix).
Thanks for your help, sorry for the noise!

[reames]

In D62625#1553095, @mstorsjo wrote:

Compile time regression. (I build the project it's part of with a 120 second timeout per translation unit, enforced with ulimit -t 120, that's why I noticed.)

If you could provide an IR reproducer, and ideally a hint where to look, I can see what I find. Please file a bug with reproduction instructions and an IR attachment.

In D62625#1553525, @SjoerdMeijer wrote:

I think I am mainly looking at micro-architecture sensitivities and some unfortunate knock on effects of this patch.
I don't think I can blame anything of that on this patch. If I change my mind about that, I will let you know. :-) But then I'd of course have to come up with a reproducer (or a fix).
Thanks for your help, sorry for the noise!

Np. I'm happy to help debate fixes for issues you might find. Feel free to add me as a reviewer (if appropriate), or start an email conversation with preliminary findings if it's helpful.

[mstorsjo]

In D62625#1553942, @reames wrote:

In D62625#1553095, @mstorsjo wrote:

Compile time regression. (I build the project it's part of with a 120 second timeout per translation unit, enforced with ulimit -t 120, that's why I noticed.)

If you could provide an IR reproducer, and ideally a hint where to look, I can see what I find. Please file a bug with reproduction instructions and an IR attachment.

A C level reproducer is already linked (https://martin.st/temp/glew-preproc-i686.c, compiled with clang -target i686-w64-mingw32 -c -O3 glew-preproc-i686.c) I filed https://bugs.llvm.org/show_bug.cgi?id=42357 now as well, with that, and matching IR for reproducing on that level. The issue is not only that compilation takes longer, the output also almost doubled in size.

llvm/trunk/lib/Transforms/Scalar/IndVarSimplify.cpp

 /// block is also the latch.
 static ICmpInst *getLoopTest(Loop *L, BasicBlock *ExitingBB) {
 
   BasicBlock *LatchBlock = L->getLoopLatch();
   // Don't bother with LFTR if the loop is not properly simplified.
   if (!LatchBlock)
     return nullptr;
 
-  BranchInst *BI = dyn_cast<BranchInst>(ExitingBB->getTerminator());
-  assert(BI && "expected exit branch");
-
+  BranchInst *BI = cast<BranchInst>(ExitingBB->getTerminator());
   return dyn_cast<ICmpInst>(BI->getCondition());
 }
 
 /// linearFunctionTestReplace policy. Return true unless we can show that the
 /// current exit test is already sufficiently canonical.
 static bool needsLFTR(Loop *L, BasicBlock *ExitingBB) {
+  // Avoid converting a constant or loop invariant test back to a runtime
+  // test.  This is critical for when SCEV's cached ExitCount is less precise
+  // than the current IR (such as after we've proven a particular exit is
+  // actually dead and thus the BE count never reaches our ExitCount.)
+  BranchInst *BI = cast<BranchInst>(ExitingBB->getTerminator());
+  if (L->isLoopInvariant(BI->getCondition()))
+    return false;
+  
   // Do LFTR to simplify the exit condition to an ICMP.
   ICmpInst *Cond = getLoopTest(L, ExitingBB);
   if (!Cond)
     return true;
 
   // Do LFTR to simplify the exit ICMP to EQ/NE
   ICmpInst::Predicate Pred = Cond->getPredicate();
   if (Pred != ICmpInst::ICMP_NE && Pred != ICmpInst::ICMP_EQ)
for (BasicBlock::iterator I = L->getHeader()->begin(); isa<PHINode>(I); ++I) {
     uint64_t PhiWidth = SE->getTypeSizeInBits(AR->getType());
     if (PhiWidth < BCWidth || !DL.isLegalInteger(PhiWidth))
       continue;
 
     // Avoid reusing a potentially undef value to compute other values that may
     // have originally had a concrete definition.
     if (!hasConcreteDef(Phi)) {
       // We explicitly allow unknown phis as long as they are already used by
-      // the loop test. In this case we assume that performing LFTR could not
-      // increase the number of undef users.
-      // TODO: Generalize this to allow *any* loop exit which is known to
-      // execute on each iteration
-      if (L->getExitingBlock())
-        if (ICmpInst *Cond = getLoopTest(L, ExitingBB))
-          if (Phi != getLoopPhiForCounter(Cond->getOperand(0), L) &&
-              Phi != getLoopPhiForCounter(Cond->getOperand(1), L))
-            continue;
+      // the loop exit test.  This is legal since performing LFTR could not
+      // increase the number of undef users. 
     }
 
     // Avoid introducing undefined behavior due to poison which didn't exist in
     // the original program.  (Annoyingly, the rules for poison and undef
     // propagation are distinct, so this does NOT cover the undef case above.)
     // We have to ensure that we don't introduce UB by introducing a use on an
     // iteration where said IV produces poison.  Our strategy here differs for
     // pointers and integer IVs.  For integers, we strip and reinfer as needed,
if (ReplaceExitValue != NeverRepl &&
     Changed |= rewriteLoopExitValues(L, Rewriter);
 
   // Eliminate redundant IV cycles.
   NumElimIV += Rewriter.replaceCongruentIVs(L, DT, DeadInsts);
 
   // If we have a trip count expression, rewrite the loop's exit condition
   // using it.  
   if (!DisableLFTR) {
-    // For the moment, we only do LFTR for single exit loops.  The code is
-    // structured as it is in the expectation of generalization to multi-exit
-    // loops in the near future.  See D62625 for context.
     SmallVector<BasicBlock*, 16> ExitingBlocks;
-    if (auto *ExitingBB = L->getExitingBlock())
-      ExitingBlocks.push_back(ExitingBB);
+    L->getExitingBlocks(ExitingBlocks);
     for (BasicBlock *ExitingBB : ExitingBlocks) {
       // Can't rewrite non-branch yet.
       if (!isa<BranchInst>(ExitingBB->getTerminator()))
         continue;
       
       if (!needsLFTR(L, ExitingBB))
         continue;
 

llvm/trunk/test/Transforms/IndVarSimplify/eliminate-comparison.ll

 
 
 define i32 @func_11() nounwind uwtable {
 ; CHECK-LABEL: @func_11(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[FORCOND:%.*]]
 ; CHECK:       forcond:
 ; CHECK-NEXT:    [[__KEY6_0:%.*]] = phi i32 [ 2, [[ENTRY:%.*]] ], [ [[TMP37:%.*]], [[NOASSERT:%.*]] ]
-; CHECK-NEXT:    [[TMP5:%.*]] = icmp ult i32 [[__KEY6_0]], 10
-; CHECK-NEXT:    br i1 [[TMP5]], label [[NOASSERT]], label [[FORCOND38_PREHEADER:%.*]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[__KEY6_0]], 10
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[NOASSERT]], label [[FORCOND38_PREHEADER:%.*]]
 ; CHECK:       forcond38.preheader:
 ; CHECK-NEXT:    br label [[FORCOND38:%.*]]
 ; CHECK:       noassert:
 ; CHECK-NEXT:    [[TMP13:%.*]] = sdiv i32 -32768, [[__KEY6_0]]
 ; CHECK-NEXT:    [[TMP2936:%.*]] = shl i32 [[TMP13]], 24
 ; CHECK-NEXT:    [[SEXT23:%.*]] = shl i32 [[TMP13]], 24
 ; CHECK-NEXT:    [[TMP32:%.*]] = icmp eq i32 [[TMP2936]], [[SEXT23]]
 ; CHECK-NEXT:    [[TMP37]] = add nuw nsw i32 [[__KEY6_0]], 1
 ; CHECK-NEXT:    br i1 [[TMP32]], label [[FORCOND]], label [[ASSERT33:%.*]]
 ; CHECK:       assert33:
 ; CHECK-NEXT:    tail call void @llvm.trap()
 ; CHECK-NEXT:    unreachable
 ; CHECK:       forcond38:
 ; CHECK-NEXT:    [[__KEY8_0:%.*]] = phi i32 [ [[TMP81:%.*]], [[NOASSERT68:%.*]] ], [ 2, [[FORCOND38_PREHEADER]] ]
-; CHECK-NEXT:    [[TMP46:%.*]] = icmp ult i32 [[__KEY8_0]], 10
-; CHECK-NEXT:    br i1 [[TMP46]], label [[NOASSERT68]], label [[UNROLLEDEND:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[__KEY8_0]], 10
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[NOASSERT68]], label [[UNROLLEDEND:%.*]]
 ; CHECK:       noassert68:
 ; CHECK-NEXT:    [[TMP57:%.*]] = sdiv i32 -32768, [[__KEY8_0]]
 ; CHECK-NEXT:    [[SEXT34:%.*]] = shl i32 [[TMP57]], 16
 ; CHECK-NEXT:    [[SEXT21:%.*]] = shl i32 [[TMP57]], 16
 ; CHECK-NEXT:    [[TMP76:%.*]] = icmp eq i32 [[SEXT34]], [[SEXT21]]
 ; CHECK-NEXT:    [[TMP81]] = add nuw nsw i32 [[__KEY8_0]], 1
 ; CHECK-NEXT:    br i1 [[TMP76]], label [[FORCOND38]], label [[ASSERT77:%.*]]
 ; CHECK:       assert77:

 
 ; In this case the second loop only has a single iteration, fold the header away
 define i32 @func_12() nounwind uwtable {
 ; CHECK-LABEL: @func_12(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[FORCOND:%.*]]
 ; CHECK:       forcond:
 ; CHECK-NEXT:    [[__KEY6_0:%.*]] = phi i32 [ 2, [[ENTRY:%.*]] ], [ [[TMP37:%.*]], [[NOASSERT:%.*]] ]
-; CHECK-NEXT:    [[TMP5:%.*]] = icmp ult i32 [[__KEY6_0]], 10
-; CHECK-NEXT:    br i1 [[TMP5]], label [[NOASSERT]], label [[FORCOND38_PREHEADER:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[__KEY6_0]], 10
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[NOASSERT]], label [[FORCOND38_PREHEADER:%.*]]
 ; CHECK:       forcond38.preheader:
 ; CHECK-NEXT:    br label [[FORCOND38:%.*]]
 ; CHECK:       noassert:
 ; CHECK-NEXT:    [[TMP13:%.*]] = sdiv i32 -32768, [[__KEY6_0]]
 ; CHECK-NEXT:    [[TMP2936:%.*]] = shl i32 [[TMP13]], 24
 ; CHECK-NEXT:    [[SEXT23:%.*]] = shl i32 [[TMP13]], 24
 ; CHECK-NEXT:    [[TMP32:%.*]] = icmp eq i32 [[TMP2936]], [[SEXT23]]
 ; CHECK-NEXT:    [[TMP37]] = add nuw nsw i32 [[__KEY6_0]], 1

 }
 
 declare void @side_effect()
 
 define void @func_13(i32* %len.ptr) {
 ; CHECK-LABEL: @func_13(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, i32* [[LEN_PTR:%.*]], !range !0
-; CHECK-NEXT:    [[LEN_SUB_1:%.*]] = add i32 [[LEN]], -1
 ; CHECK-NEXT:    [[LEN_IS_ZERO:%.*]] = icmp eq i32 [[LEN]], 0
 ; CHECK-NEXT:    br i1 [[LEN_IS_ZERO]], label [[LEAVE:%.*]], label [[LOOP_PREHEADER:%.*]]
 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    call void @side_effect()
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp ult i32 [[IV]], [[LEN_SUB_1]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LEN]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %len = load i32, i32* %len.ptr, !range !0
   %len.sub.1 = add i32 %len, -1

 leave:
   ret void
 }
 
 define void @func_14(i32* %len.ptr) {
 ; CHECK-LABEL: @func_14(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, i32* [[LEN_PTR:%.*]], !range !0
-; CHECK-NEXT:    [[LEN_SUB_1:%.*]] = add i32 [[LEN]], -1
 ; CHECK-NEXT:    [[LEN_IS_ZERO:%.*]] = icmp eq i32 [[LEN]], 0
 ; CHECK-NEXT:    [[LEN_IS_INT_MIN:%.*]] = icmp eq i32 [[LEN]], -2147483648
 ; CHECK-NEXT:    [[NO_ENTRY:%.*]] = or i1 [[LEN_IS_ZERO]], [[LEN_IS_INT_MIN]]
 ; CHECK-NEXT:    br i1 [[NO_ENTRY]], label [[LEAVE:%.*]], label [[LOOP_PREHEADER:%.*]]
 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    call void @side_effect()
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp slt i32 [[IV]], [[LEN_SUB_1]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LEN]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %len = load i32, i32* %len.ptr, !range !0
   %len.sub.1 = add i32 %len, -1

 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    call void @side_effect()
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp ult i32 [[IV]], [[LEN]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LEN_ADD_1]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %len = load i32, i32* %len.ptr, !range !0
   %len.add.1 = add i32 %len, 1

 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, i32* [[LEN_PTR:%.*]], !range !0
 ; CHECK-NEXT:    [[LEN_ADD_5:%.*]] = add i32 [[LEN]], 5
 ; CHECK-NEXT:    [[ENTRY_COND_0:%.*]] = icmp slt i32 [[LEN]], 2147483643
 ; CHECK-NEXT:    [[ENTRY_COND_1:%.*]] = icmp slt i32 4, [[LEN_ADD_5]]
 ; CHECK-NEXT:    [[ENTRY_COND:%.*]] = and i1 [[ENTRY_COND_0]], [[ENTRY_COND_1]]
 ; CHECK-NEXT:    br i1 [[ENTRY_COND]], label [[LOOP_PREHEADER:%.*]], label [[LEAVE:%.*]]
 ; CHECK:       loop.preheader:
+; CHECK-NEXT:    [[TMP0:%.*]] = add nuw nsw i32 [[LEN]], 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    call void @side_effect()
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp ult i32 [[IV]], [[LEN]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[TMP0]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %len = load i32, i32* %len.ptr, !range !0
   %len.add.5 = add i32 %len, 5

 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp ult i32 [[IV_INC]], [[LENGTH]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LENGTH]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %length = load i32, i32* %length.ptr, !range !0
   %length.is.nonzero = icmp ne i32 %length, 0

 ; CHECK-NEXT:    [[LENGTH:%.*]] = load i32, i32* [[LENGTH_PTR:%.*]]
 ; CHECK-NEXT:    [[LENGTH_IS_NONZERO:%.*]] = icmp ne i32 [[LENGTH]], 0
 ; CHECK-NEXT:    br i1 [[LENGTH_IS_NONZERO]], label [[LOOP_PREHEADER:%.*]], label [[LEAVE:%.*]]
 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
-; CHECK-NEXT:    [[RANGE_CHECK:%.*]] = icmp ult i32 [[IV]], [[LENGTH]]
-; CHECK-NEXT:    br i1 [[RANGE_CHECK]], label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[LENGTH]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
 ; CHECK-NEXT:    [[BE_COND:%.*]] = icmp slt i32 [[IV_INC]], [[LENGTH]]
 ; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void

 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp slt i32 [[IV_INC]], [[LIM]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LIM]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %length = load i32, i32* %length.ptr, !range !0
   %lim = sub i32 %length, 1

 }
 
 ; This checks that the backedge condition, (I + 1) < Length - 1 implies
 ; (I + 1) < Length
 define void @func_22(i32* %length.ptr) {
 ; CHECK-LABEL: @func_22(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LENGTH:%.*]] = load i32, i32* [[LENGTH_PTR:%.*]], !range !0
-; CHECK-NEXT:    [[LIM:%.*]] = sub i32 [[LENGTH]], 1
 ; CHECK-NEXT:    [[ENTRY_COND:%.*]] = icmp sgt i32 [[LENGTH]], 1
 ; CHECK-NEXT:    br i1 [[ENTRY_COND]], label [[LOOP_PREHEADER:%.*]], label [[LEAVE:%.*]]
 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 0, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp sle i32 [[IV_INC]], [[LIM]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LENGTH]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %length = load i32, i32* %length.ptr, !range !0
   %lim = sub i32 %length, 1

 ; CHECK:       loop.preheader:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ [[IV_INC:%.*]], [[BE:%.*]] ], [ 4, [[LOOP_PREHEADER]] ]
 ; CHECK-NEXT:    [[IV_INC]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    br i1 true, label [[BE]], label [[LEAVE_LOOPEXIT:%.*]]
 ; CHECK:       be:
 ; CHECK-NEXT:    call void @side_effect()
-; CHECK-NEXT:    [[BE_COND:%.*]] = icmp ult i32 [[IV_INC]], [[LENGTH]]
-; CHECK-NEXT:    br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_INC]], [[LENGTH]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT]]
 ; CHECK:       leave.loopexit:
 ; CHECK-NEXT:    br label [[LEAVE]]
 ; CHECK:       leave:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %length = load i32, i32* %length.ptr, !range !0
   %entry.cond = icmp ult i32 4, %length

llvm/trunk/test/Transforms/IndVarSimplify/lftr-multi-exit.ll

 @A = external global i32
 
 define void @analyzeable_early_exit(i32 %n) {
 ; CHECK-LABEL: @analyzeable_early_exit(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    store i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[IV_NEXT]], 1000
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV_NEXT]], 1000
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
 ; CHECK-NEXT:    [[VOL:%.*]] = load volatile i32, i32* @A
 ; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ne i32 [[VOL]], 0
 ; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    store i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[IV_NEXT]], 1000
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_NEXT]], 1000
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 
 
 define void @multiple_early_exits(i32 %n, i32 %m) {
 ; CHECK-LABEL: @multiple_early_exits(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[CONTINUE:%.*]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[CONTINUE:%.*]], label [[EXIT:%.*]]
 ; CHECK:       continue:
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[EARLYCND2:%.*]] = icmp ult i32 [[IV]], [[M:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND2]], label [[LATCH]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV]], [[M:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LATCH]], label [[EXIT]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[IV_NEXT]], 1000
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND2:%.*]] = icmp ne i32 [[IV_NEXT]], 1000
+; CHECK-NEXT:    br i1 [[EXITCOND2]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
 ; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
 ; CHECK-NEXT:    [[EARLYCND2:%.*]] = icmp ult i32 [[IV]], [[M:%.*]]
 ; CHECK-NEXT:    [[AND:%.*]] = and i1 [[EARLYCND]], [[EARLYCND2]]
 ; CHECK-NEXT:    br i1 [[AND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[IV_NEXT]], 1000
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV_NEXT]], 1000
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 
 
 define void @unanalyzeable_latch(i32 %n) {
 ; CHECK-LABEL: @unanalyzeable_latch(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    store i32 [[IV]], i32* @A
 ; CHECK-NEXT:    [[VOL:%.*]] = load volatile i32, i32* @A
 ; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[VOL]], 1000
 ; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void

 ; Multiple exits which could be LFTRed, but the latch itself is not an
 ; exiting block.
 define void @no_latch_exit(i32 %n, i32 %m) {
 ; CHECK-LABEL: @no_latch_exit(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[CONTINUE:%.*]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[CONTINUE:%.*]], label [[EXIT:%.*]]
 ; CHECK:       continue:
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[EARLYCND2:%.*]] = icmp ult i32 [[IV]], [[M:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND2]], label [[LATCH]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV]], [[M:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LATCH]], label [[EXIT]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    br label [[LOOP]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:

 ;; Show the value of multiple exit LFTR (being able to eliminate all but
 ;; one IV when exit tests involve multiple IVs).
 define void @combine_ivs(i32 %n) {
 ; CHECK-LABEL: @combine_ivs(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[IV2:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[IV2_NEXT:%.*]], [[LATCH]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
-; CHECK-NEXT:    [[IV2_NEXT]] = add nuw nsw i32 [[IV2]], 1
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[IV2_NEXT]], 1000
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV_NEXT]], 999
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 
 ; We can remove the decrementing IV entirely
 define void @combine_ivs2(i32 %n) {
 ; CHECK-LABEL: @combine_ivs2(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[IV2:%.*]] = phi i32 [ 1000, [[ENTRY]] ], [ [[IV2_NEXT:%.*]], [[LATCH]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
-; CHECK-NEXT:    [[IV2_NEXT]] = sub nuw nsw i32 [[IV2]], 1
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ugt i32 [[IV2_NEXT]], 0
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV_NEXT]], 1000
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]

 ; An example where we can eliminate an f(i) computation entirely
 ; from a multiple exit loop with LFTR.
 define void @simplify_exit_test(i32 %n) {
 ; CHECK-LABEL: @simplify_exit_test(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
-; CHECK-NEXT:    [[EARLYCND:%.*]] = icmp ult i32 [[IV]], [[N:%.*]]
-; CHECK-NEXT:    br i1 [[EARLYCND]], label [[LATCH]], label [[EXIT:%.*]]
+; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i32 [[IV]], [[N:%.*]]
+; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LATCH]], label [[EXIT:%.*]]
 ; CHECK:       latch:
 ; CHECK-NEXT:    [[IV_NEXT]] = add nuw nsw i32 [[IV]], 1
-; CHECK-NEXT:    [[FX:%.*]] = shl i32 [[IV]], 4
 ; CHECK-NEXT:    store volatile i32 [[IV]], i32* @A
-; CHECK-NEXT:    [[C:%.*]] = icmp ult i32 [[FX]], 1024
-; CHECK-NEXT:    br i1 [[C]], label [[LOOP]], label [[EXIT]]
+; CHECK-NEXT:    [[EXITCOND1:%.*]] = icmp ne i32 [[IV_NEXT]], 65
+; CHECK-NEXT:    br i1 [[EXITCOND1]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %loop
 
 loop:
   %iv = phi i32 [ 0, %entry], [ %iv.next, %latch]