[SpeculateAroundPHIs] Avoid speculation on loop back edges

[bjope]

Summary

In https://bugs.llvm.org/show_bug.cgi?id=48821 there are some
discussions around problems with SpeculateAroundPHIs and back-edges
in loops.

One problem observed is that when splitting a critical edge that
is a loop latch, the pass isn't transferring loop metadata to the
new latch (nor removing the old loop metadata). So if we want to
allow speculation for such scenarios we need to fix that.

There have also been observations that (downstream) backends have
had problems with identifying hwloops when allowing speculation
involving back-edges. That might be a target specific problem
that potentially could be resolved by adding a TTI hook, or possibly
by keeping the loop form by also rotating loops after the pass (need
to decide if opt should canonicalize the loop in such manner or if
a backend need to rotate loops in the codegen pipeline).

This also avoid some differences between LegacyPM and NewPM when
it comes to how we optimize loops. So this could make the transition
to the new pass manager more clean, avoiding to change both pass
manager and potentially introducing new phase ordering issues at the
same time.

[bjope]

I think the problem with loop metadata not being handled properly is a bug (that also could impact performance negatively) , so this is a quick fix (but I guess it could be possible to handle transition of metadata to the new latch with some effort if the transform is beneficial).

Then there is the question about loop form and what can be expected at the end of the opt pipeline. Should the loop be in simplified/rotated form (I think legacy PM ends with SimplifyCFG, but new PM is running this pass that might unrotate(?) the loop). At least our downstream benchmarks suffered significantly since it failed to detect hwloops when having speculated around a phi involving a latch. If there is no canonical form, then maybe we need to improve codegen to either undo the transform (rotate loops) or improve the hwloop detection somehow.

Maybe I should add a TTI hook to simplify for specific targets to use this workaround (although, the loop metadata problem could possible be messing up for any target)?

[Meinersbur]

Some passes passes might not handle critical edges, in addition to SpeculateAroundPHIs . Wouldn't it be simpler to fix llvm::SplitCriticalEdge to make it put the loop metadata to the correct branch?

[bjope]

In D95789#2575065, @Meinersbur wrote:

Some passes passes might not handle critical edges, in addition to SpeculateAroundPHIs . Wouldn't it be simpler to fix llvm::SplitCriticalEdge to make it put the loop metadata to the correct branch?

That could perhaps be a long-term solution, although I'm not familiar with all uses of SplitCriticalEdge to know if it is feasible (I figure that sometimes the metadata should be updated as well, not only being moved).
Other passes, e.g. LoopRotate, is handling loop metadata "manually". But maybe there are other passes using SplitCriticalEdge that also suffer from not updating loop metadata.

In this patch I put focused on getting rid of the "bug" (having loop metadata on the wrong branch should ideally be caught be the verifier but as I recall it isn't that easy to verify).

[Meinersbur]

IIUC, this patch inhibits optimizations opportunities because of a bug elsewhere? Don't we want to fix lost loop metadata anywhere else too?

[bjope]

In D95789#2604835, @Meinersbur wrote:

IIUC, this patch inhibits optimizations opportunities because of a bug elsewhere?

I haven't understood if the rewrite really is an optimization or not when the critical edge split occur for a backedge. All I see for my target are performance penalties (either by no longer getting an "hwloop", or due to that the rewrite introduces a conditional branch inside the loop body).
This patch both restores performance loss introduced by changing from legacy PM to new PM, and avoids the problem with incorrect loop metadata after such non-profitable rewrites. If someone could motivate the transform (why it should be considered as beneficial), then I agree that the loop metadata problem should be solved instead. And then maybe we also need some TTI hook or something to allow targets to skip the transform if it isn't wanted. Those rewrites at least seem to be devastating for our OOT benchmark suite when switching from legacy PM to new PM.

[lebedev.ri]

I will need to take another look, but i also have seen significant perf regressions due to SpeculateAroundPHIs.
I think a better/less controversial fix would be to move SpeculateAroundPHIs to the end of IR portion of codegen pipeline,
after last extension point.

[bjope]

In D95789#2609897, @lebedev.ri wrote:

I will need to take another look, but i also have seen significant perf regressions due to SpeculateAroundPHIs.
I think a better/less controversial fix would be to move SpeculateAroundPHIs to the end of IR portion of codegen pipeline,
after last extension point.

Moving the pass still doesn't solve the problem that the loop metadata can be messed up (I think that having stale loop metadata on the wrong branch is bad, a couple of transformations later it might end up being on a latch belonging to a totally different loop).
That could ofcourse also be solved by another patch.

I think it was pretty annoying that the switch of pass manager introduce these problems as a side effect. And the idea here is simply to avoid the bugs/regressions seen.
Then someone could try to optimize these patterns again (hopefully in a correct way), or move things around in the pipeline etc.

[lebedev.ri]

I find it worrying that the authors of the pass (i.e. google) still haven't found time to reply to the post-commit feedback here / in the original review.

[nikic]

It seems like the addition of this pass has been somewhat botched by adding it only to the NewPM pipeline -- for no clear reason I can discern. This was at a time where barely anyone (outside Google) was using the NewPM, which means that this received very little evaluation from other parties. Now people are hitting this as part of the LegacyPM -> NewPM migration instead, which really shouldn't be the case.

It might make sense to remove this pass from the NewPM default pipeline again (or put it behind an option) and then restart the process of enabling it by default, this time with due diligence.

[lebedev.ri]

In D95789#2812692, @nikic wrote:

It seems like the addition of this pass has been somewhat botched by adding it only to the NewPM pipeline -- for no clear reason I can discern. This was at a time where barely anyone (outside Google) was using the NewPM, which means that this received very little evaluation from other parties. Now people are hitting this as part of the LegacyPM -> NewPM migration instead, which really shouldn't be the case.

It might make sense to remove this pass from the NewPM default pipeline again (or put it behind an option) and then restart the process of enabling it by default, this time with due diligence.

Aha, so i'm not the only one with that sentiment. D104099 it is then.

[lebedev.ri]

The pass has now been reverted by rGe52364532afb2748c324f360bc1cc12605d314f3.
Abandon this?

[bjope]

The pass has been removed from the code base, so this can be abandonded.

(Hopefully someone will remember about https://bugs.llvm.org/show_bug.cgi?id=48821 if trying to revive the pass in the future.)

llvm/lib/Passes/PassBuilder.cpp

 //===- Parsing, selection, and construction of pass pipelines -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 /// \file
PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
 
   // LoopSink (and other loop passes since the last simplifyCFG) might have
   // resulted in single-entry-single-exit or empty blocks. Clean up the CFG.
   OptimizePM.addPass(SimplifyCFGPass());
 
   // Optimize PHIs by speculating around them when profitable. Note that this
   // pass needs to be run after any PRE or similar pass as it is essentially
   // inserting redundancies into the program. This even includes SimplifyCFG.
+  // FIXME: Lots of backends run SimplifyCFG (and possibly similar passes) in
+  // the backend pipiline. So it would be nice to elaborate a bit more about
+  // "needs to be run after". Or maybe the SpeculateAroundPHIsPass should be
+  // moved to the backend pipeline to ensure that it is run after IR passes in
+  // the backend pipeline as well?
   OptimizePM.addPass(SpeculateAroundPHIsPass());
 
   if (PTO.Coroutines)
     OptimizePM.addPass(CoroCleanupPass());
 
   // Add the core optimizing pipeline.
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(OptimizePM)));
 

llvm/lib/Transforms/Scalar/SpeculateAroundPHIs.cpp

 //===- SpeculateAroundPHIs.cpp --------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
findProfitablePHIs(ArrayRef<PHINode *> PNs,
   return SpecPNs;
 }
 
 /// Speculate users around a set of PHI nodes.
 ///
 /// This routine does the actual speculation around a set of PHI nodes where we
 /// have determined this to be both safe and profitable.
 ///
-/// This routine handles any spliting of critical edges necessary to create
+/// This routine handles any splitting of critical edges necessary to create
 /// a safe block to speculate into as well as cloning the instructions and
 /// rewriting all uses.
 static void speculatePHIs(ArrayRef<PHINode *> SpecPNs,
                           SmallPtrSetImpl<Instruction *> &PotentialSpecSet,
                           SmallSetVector<BasicBlock *, 16> &PredSet,
                           DominatorTree &DT) {
   LLVM_DEBUG(dbgs() << "  Speculating around " << SpecPNs.size() << " PHIs!\n");
   NumPHIsSpeculated += SpecPNs.size();
static bool tryToSpeculatePHIs(SmallVectorImpl<PHINode *> &PNs,
   // speculate into the incoming edges of this basic block. This avoids
   // recomputing them for each PHI node we check. This set is specific to this
   // block though as things are pruned out of it based on what is available
   // along incoming edges.
   SmallPtrSet<Instruction *, 16> UnsafeSet;
 
   // For each PHI node in this block, check whether there are immediate folding
   // opportunities from speculation, and whether that speculation will be
-  // valid. This determise the set of safe PHIs to speculate.
+  // valid. This determine the set of safe PHIs to speculate.
   llvm::erase_if(PNs, [&](PHINode *PN) {
     return !isSafeAndProfitableToSpeculateAroundPHI(
         *PN, CostSavingsMap, PotentialSpecSet, UnsafeSet, DT, TTI);
   });
   // If no PHIs were profitable, skip.
   if (PNs.empty()) {
     LLVM_DEBUG(dbgs() << "  No safe and profitable PHIs found!\n");
     return false;
if (isa<IndirectBrInst>(TermInst) ||
       return false;
     }
   }
   if (PredSet.size() < 2) {
     LLVM_DEBUG(dbgs() << "  Unimportant: phi with only one predecessor\n");
     return false;
   }
 
+  // Avoid speculation around back-edges. The speculation would result in a
+  // critical edge split, in effect moving the latch to a new BB. In such
+  // situations we would need to transfer any loop metadata associated with the
+  // loop to the new latch, but support for that has not been implemented in
+  // this pass yet.
+  //
+  // This kind of speculation has also been identified as being bad for certain
+  // targets with HW-loops, as it changes the loop-form at the end of the
+  // optimization pipeline (maybe it would help if we also rotate the loop
+  // afterwards).
+  for (auto &Pred : PredSet)
+    if (DT.dominates(PNs[0]->getParent(), Pred)) {
+      LLVM_DEBUG(dbgs() << "  Involves a back-edge\n");
+      return false;
+    }
+
   SmallVector<PHINode *, 16> SpecPNs = findProfitablePHIs(
       PNs, CostSavingsMap, PotentialSpecSet, PredSet.size(), DT, TTI);
   if (SpecPNs.empty())
     // Nothing to do.
     return false;
 
   speculatePHIs(SpecPNs, PotentialSpecSet, PredSet, DT);
   return true;

llvm/test/Transforms/PhaseOrdering/loop-rotation-vs-common-code-hoisting.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -O3 -rotation-max-header-size=0 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=HOIST
 ; RUN: opt -passes='default<O3>' -rotation-max-header-size=0 -S < %s  | FileCheck %s --check-prefix=HOIST
 
 ; RUN: opt -O3 -rotation-max-header-size=1 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=HOIST
 ; RUN: opt -passes='default<O3>' -rotation-max-header-size=1 -S < %s  | FileCheck %s --check-prefix=HOIST
 
-; RUN: opt -O3 -rotation-max-header-size=2 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=ROTATED_LATER_OLDPM
-; RUN: opt -passes='default<O3>' -rotation-max-header-size=2 -S < %s  | FileCheck %s --check-prefix=ROTATED_LATER_NEWPM
+; RUN: opt -O3 -rotation-max-header-size=2 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=ROTATED_LATER
+; RUN: opt -passes='default<O3>' -rotation-max-header-size=2 -S < %s  | FileCheck %s --check-prefix=ROTATED_LATER
 
-; RUN: opt -O3 -rotation-max-header-size=3 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=ROTATE_OLDPM
-; RUN: opt -passes='default<O3>' -rotation-max-header-size=3 -S < %s  | FileCheck %s --check-prefix=ROTATE_NEWPM
+; RUN: opt -O3 -rotation-max-header-size=3 -S -enable-new-pm=0 < %s   | FileCheck %s --check-prefix=ROTATE
+; RUN: opt -passes='default<O3>' -rotation-max-header-size=3 -S < %s  | FileCheck %s --check-prefix=ROTATE
 
 ; This example is produced from a very basic C code:
 ;
 ;   void f0();
 ;   void f1();
 ;   void f2();
 ;
 ;   void loop(int width) {

 ; HOIST-NEXT:    br label [[RETURN]]
 ; HOIST:       for.body:
 ; HOIST-NEXT:    tail call void @f1()
 ; HOIST-NEXT:    [[INC]] = add nuw i32 [[I_0]], 1
 ; HOIST-NEXT:    br label [[FOR_COND]]
 ; HOIST:       return:
 ; HOIST-NEXT:    ret void
 ;
-; ROTATED_LATER_OLDPM-LABEL: @_Z4loopi(
-; ROTATED_LATER_OLDPM-NEXT:  entry:
-; ROTATED_LATER_OLDPM-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
-; ROTATED_LATER_OLDPM-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
-; ROTATED_LATER_OLDPM:       for.cond.preheader:
-; ROTATED_LATER_OLDPM-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
-; ROTATED_LATER_OLDPM-NEXT:    [[EXITCOND_NOT3:%.*]] = icmp eq i32 [[TMP0]], 0
-; ROTATED_LATER_OLDPM-NEXT:    br i1 [[EXITCOND_NOT3]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY:%.*]]
-; ROTATED_LATER_OLDPM:       for.cond.cleanup:
-; ROTATED_LATER_OLDPM-NEXT:    tail call void @f0()
-; ROTATED_LATER_OLDPM-NEXT:    tail call void @f2()
-; ROTATED_LATER_OLDPM-NEXT:    br label [[RETURN]]
-; ROTATED_LATER_OLDPM:       for.body:
-; ROTATED_LATER_OLDPM-NEXT:    [[I_04:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_COND_PREHEADER]] ]
-; ROTATED_LATER_OLDPM-NEXT:    tail call void @f0()
-; ROTATED_LATER_OLDPM-NEXT:    tail call void @f1()
-; ROTATED_LATER_OLDPM-NEXT:    [[INC]] = add nuw i32 [[I_04]], 1
-; ROTATED_LATER_OLDPM-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC]], [[TMP0]]
-; ROTATED_LATER_OLDPM-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]]
-; ROTATED_LATER_OLDPM:       return:
-; ROTATED_LATER_OLDPM-NEXT:    ret void
-;
-; ROTATED_LATER_NEWPM-LABEL: @_Z4loopi(
-; ROTATED_LATER_NEWPM-NEXT:  entry:
-; ROTATED_LATER_NEWPM-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
-; ROTATED_LATER_NEWPM-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
-; ROTATED_LATER_NEWPM:       for.cond.preheader:
-; ROTATED_LATER_NEWPM-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
-; ROTATED_LATER_NEWPM-NEXT:    [[EXITCOND_NOT3:%.*]] = icmp eq i32 [[TMP0]], 0
-; ROTATED_LATER_NEWPM-NEXT:    br i1 [[EXITCOND_NOT3]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_COND_PREHEADER_FOR_BODY_CRIT_EDGE:%.*]]
-; ROTATED_LATER_NEWPM:       for.cond.preheader.for.body_crit_edge:
-; ROTATED_LATER_NEWPM-NEXT:    [[INC_1:%.*]] = add nuw i32 0, 1
-; ROTATED_LATER_NEWPM-NEXT:    br label [[FOR_BODY:%.*]]
-; ROTATED_LATER_NEWPM:       for.cond.cleanup:
-; ROTATED_LATER_NEWPM-NEXT:    tail call void @f0()
-; ROTATED_LATER_NEWPM-NEXT:    tail call void @f2()
-; ROTATED_LATER_NEWPM-NEXT:    br label [[RETURN]]
-; ROTATED_LATER_NEWPM:       for.body:
-; ROTATED_LATER_NEWPM-NEXT:    [[INC_PHI:%.*]] = phi i32 [ [[INC_0:%.*]], [[FOR_BODY_FOR_BODY_CRIT_EDGE:%.*]] ], [ [[INC_1]], [[FOR_COND_PREHEADER_FOR_BODY_CRIT_EDGE]] ]
-; ROTATED_LATER_NEWPM-NEXT:    tail call void @f0()
-; ROTATED_LATER_NEWPM-NEXT:    tail call void @f1()
-; ROTATED_LATER_NEWPM-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC_PHI]], [[TMP0]]
-; ROTATED_LATER_NEWPM-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_FOR_BODY_CRIT_EDGE]]
-; ROTATED_LATER_NEWPM:       for.body.for.body_crit_edge:
-; ROTATED_LATER_NEWPM-NEXT:    [[INC_0]] = add nuw i32 [[INC_PHI]], 1
-; ROTATED_LATER_NEWPM-NEXT:    br label [[FOR_BODY]]
-; ROTATED_LATER_NEWPM:       return:
-; ROTATED_LATER_NEWPM-NEXT:    ret void
-;
-; ROTATE_OLDPM-LABEL: @_Z4loopi(
-; ROTATE_OLDPM-NEXT:  entry:
-; ROTATE_OLDPM-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
-; ROTATE_OLDPM-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
-; ROTATE_OLDPM:       for.cond.preheader:
-; ROTATE_OLDPM-NEXT:    [[CMP13_NOT:%.*]] = icmp eq i32 [[WIDTH]], 1
-; ROTATE_OLDPM-NEXT:    br i1 [[CMP13_NOT]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_PREHEADER:%.*]]
-; ROTATE_OLDPM:       for.body.preheader:
-; ROTATE_OLDPM-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
-; ROTATE_OLDPM-NEXT:    br label [[FOR_BODY:%.*]]
-; ROTATE_OLDPM:       for.cond.cleanup:
-; ROTATE_OLDPM-NEXT:    tail call void @f0()
-; ROTATE_OLDPM-NEXT:    tail call void @f2()
-; ROTATE_OLDPM-NEXT:    br label [[RETURN]]
-; ROTATE_OLDPM:       for.body:
-; ROTATE_OLDPM-NEXT:    [[I_04:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
-; ROTATE_OLDPM-NEXT:    tail call void @f0()
-; ROTATE_OLDPM-NEXT:    tail call void @f1()
-; ROTATE_OLDPM-NEXT:    [[INC]] = add nuw nsw i32 [[I_04]], 1
-; ROTATE_OLDPM-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC]], [[TMP0]]
-; ROTATE_OLDPM-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]]
-; ROTATE_OLDPM:       return:
-; ROTATE_OLDPM-NEXT:    ret void
-;
-; ROTATE_NEWPM-LABEL: @_Z4loopi(
-; ROTATE_NEWPM-NEXT:  entry:
-; ROTATE_NEWPM-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
-; ROTATE_NEWPM-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
-; ROTATE_NEWPM:       for.cond.preheader:
-; ROTATE_NEWPM-NEXT:    [[CMP13_NOT:%.*]] = icmp eq i32 [[WIDTH]], 1
-; ROTATE_NEWPM-NEXT:    br i1 [[CMP13_NOT]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_PREHEADER:%.*]]
-; ROTATE_NEWPM:       for.body.preheader:
-; ROTATE_NEWPM-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
-; ROTATE_NEWPM-NEXT:    [[INC_1:%.*]] = add nuw nsw i32 0, 1
-; ROTATE_NEWPM-NEXT:    br label [[FOR_BODY:%.*]]
-; ROTATE_NEWPM:       for.cond.cleanup:
-; ROTATE_NEWPM-NEXT:    tail call void @f0()
-; ROTATE_NEWPM-NEXT:    tail call void @f2()
-; ROTATE_NEWPM-NEXT:    br label [[RETURN]]
-; ROTATE_NEWPM:       for.body:
-; ROTATE_NEWPM-NEXT:    [[INC_PHI:%.*]] = phi i32 [ [[INC_0:%.*]], [[FOR_BODY_FOR_BODY_CRIT_EDGE:%.*]] ], [ [[INC_1]], [[FOR_BODY_PREHEADER]] ]
-; ROTATE_NEWPM-NEXT:    tail call void @f0()
-; ROTATE_NEWPM-NEXT:    tail call void @f1()
-; ROTATE_NEWPM-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC_PHI]], [[TMP0]]
-; ROTATE_NEWPM-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_FOR_BODY_CRIT_EDGE]]
-; ROTATE_NEWPM:       for.body.for.body_crit_edge:
-; ROTATE_NEWPM-NEXT:    [[INC_0]] = add nuw nsw i32 [[INC_PHI]], 1
-; ROTATE_NEWPM-NEXT:    br label [[FOR_BODY]]
-; ROTATE_NEWPM:       return:
-; ROTATE_NEWPM-NEXT:    ret void
+; ROTATED_LATER-LABEL: @_Z4loopi(
+; ROTATED_LATER-NEXT:  entry:
+; ROTATED_LATER-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
+; ROTATED_LATER-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
+; ROTATED_LATER:       for.cond.preheader:
+; ROTATED_LATER-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
+; ROTATED_LATER-NEXT:    [[EXITCOND_NOT3:%.*]] = icmp eq i32 [[TMP0]], 0
+; ROTATED_LATER-NEXT:    br i1 [[EXITCOND_NOT3]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY:%.*]]
+; ROTATED_LATER:       for.cond.cleanup:
+; ROTATED_LATER-NEXT:    tail call void @f0()
+; ROTATED_LATER-NEXT:    tail call void @f2()
+; ROTATED_LATER-NEXT:    br label [[RETURN]]
+; ROTATED_LATER:       for.body:
+; ROTATED_LATER-NEXT:    [[I_04:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_COND_PREHEADER]] ]
+; ROTATED_LATER-NEXT:    tail call void @f0()
+; ROTATED_LATER-NEXT:    tail call void @f1()
+; ROTATED_LATER-NEXT:    [[INC]] = add nuw i32 [[I_04]], 1
+; ROTATED_LATER-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC]], [[TMP0]]
+; ROTATED_LATER-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]]
+; ROTATED_LATER:       return:
+; ROTATED_LATER-NEXT:    ret void
+;
+; ROTATE-LABEL: @_Z4loopi(
+; ROTATE-NEXT:  entry:
+; ROTATE-NEXT:    [[CMP:%.*]] = icmp slt i32 [[WIDTH:%.*]], 1
+; ROTATE-NEXT:    br i1 [[CMP]], label [[RETURN:%.*]], label [[FOR_COND_PREHEADER:%.*]]
+; ROTATE:       for.cond.preheader:
+; ROTATE-NEXT:    [[CMP13_NOT:%.*]] = icmp eq i32 [[WIDTH]], 1
+; ROTATE-NEXT:    br i1 [[CMP13_NOT]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_PREHEADER:%.*]]
+; ROTATE:       for.body.preheader:
+; ROTATE-NEXT:    [[TMP0:%.*]] = add nsw i32 [[WIDTH]], -1
+; ROTATE-NEXT:    br label [[FOR_BODY:%.*]]
+; ROTATE:       for.cond.cleanup:
+; ROTATE-NEXT:    tail call void @f0()
+; ROTATE-NEXT:    tail call void @f2()
+; ROTATE-NEXT:    br label [[RETURN]]
+; ROTATE:       for.body:
+; ROTATE-NEXT:    [[I_04:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
+; ROTATE-NEXT:    tail call void @f0()
+; ROTATE-NEXT:    tail call void @f1()
+; ROTATE-NEXT:    [[INC]] = add nuw nsw i32 [[I_04]], 1
+; ROTATE-NEXT:    [[EXITCOND_NOT:%.*]] = icmp eq i32 [[INC]], [[TMP0]]
+; ROTATE-NEXT:    br i1 [[EXITCOND_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]]
+; ROTATE:       return:
+; ROTATE-NEXT:    ret void
 ;
 entry:
   %width.addr = alloca i32, align 4
   %i = alloca i32, align 4
   store i32 %width, i32* %width.addr, align 4
   %i1 = load i32, i32* %width.addr, align 4
   %cmp = icmp slt i32 %i1, 1
   br i1 %cmp, label %if.then, label %if.end

llvm/test/Transforms/PhaseOrdering/reassociate-after-unroll.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; REQUIRES: powerpc-registered-target
-; RUN: opt -O2 -enable-new-pm=0 -S < %s | FileCheck %s --check-prefix=OLDPM
-; RUN: opt -passes='default<O2>' -S < %s | FileCheck %s --check-prefix=NEWPM
+; RUN: opt -O2 -enable-new-pm=0 -S < %s | FileCheck %s --check-prefix=CHECK
+; RUN: opt -passes='default<O2>' -S < %s | FileCheck %s --check-prefix=CHECK
 
 target datalayout = "e-m:e-i64:64-n32:64"
 target triple = "powerpc64le-unknown-linux-gnu"
 
 define dso_local i64 @func(i64 %blah, i64 %limit) #0 {
-; OLDPM-LABEL: @func(
-; OLDPM-NEXT:  entry:
-; OLDPM-NEXT:    [[CMP4:%.*]] = icmp eq i64 [[LIMIT:%.*]], 0
-; OLDPM-NEXT:    br i1 [[CMP4]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_LR_PH:%.*]]
-; OLDPM:       for.body.lr.ph:
-; OLDPM-NEXT:    [[CONV:%.*]] = and i64 [[BLAH:%.*]], 4294967295
-; OLDPM-NEXT:    [[TMP0:%.*]] = add i64 [[LIMIT]], -1
-; OLDPM-NEXT:    [[XTRAITER:%.*]] = and i64 [[LIMIT]], 7
-; OLDPM-NEXT:    [[TMP1:%.*]] = icmp ult i64 [[TMP0]], 7
-; OLDPM-NEXT:    br i1 [[TMP1]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY_LR_PH_NEW:%.*]]
-; OLDPM:       for.body.lr.ph.new:
-; OLDPM-NEXT:    [[UNROLL_ITER:%.*]] = and i64 [[LIMIT]], -8
-; OLDPM-NEXT:    br label [[FOR_BODY:%.*]]
-; OLDPM:       for.cond.cleanup.loopexit.unr-lcssa:
-; OLDPM-NEXT:    [[ADD_LCSSA_PH:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH]] ], [ [[ADD_7:%.*]], [[FOR_BODY]] ]
-; OLDPM-NEXT:    [[K_05_UNR:%.*]] = phi i64 [ 1, [[FOR_BODY_LR_PH]] ], [ [[AND:%.*]], [[FOR_BODY]] ]
-; OLDPM-NEXT:    [[LCMP_MOD:%.*]] = icmp eq i64 [[XTRAITER]], 0
-; OLDPM-NEXT:    br i1 [[LCMP_MOD]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL:%.*]]
-; OLDPM:       for.body.epil:
-; OLDPM-NEXT:    [[G_06_EPIL:%.*]] = phi i64 [ [[ADD_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[ADD_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; OLDPM-NEXT:    [[K_05_EPIL:%.*]] = phi i64 [ [[AND_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[K_05_UNR]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; OLDPM-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; OLDPM-NEXT:    [[AND_EPIL]] = and i64 [[CONV]], [[K_05_EPIL]]
-; OLDPM-NEXT:    [[ADD_EPIL]] = add i64 [[AND_EPIL]], [[G_06_EPIL]]
-; OLDPM-NEXT:    [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
-; OLDPM-NEXT:    [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
-; OLDPM-NEXT:    br i1 [[EPIL_ITER_CMP]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL]], !llvm.loop !0
-; OLDPM:       for.cond.cleanup:
-; OLDPM-NEXT:    [[G_0_LCSSA:%.*]] = phi i64 [ undef, [[ENTRY:%.*]] ], [ [[ADD_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ], [ [[ADD_EPIL]], [[FOR_BODY_EPIL]] ]
-; OLDPM-NEXT:    ret i64 [[G_0_LCSSA]]
-; OLDPM:       for.body:
-; OLDPM-NEXT:    [[G_06:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH_NEW]] ], [ [[ADD_7]], [[FOR_BODY]] ]
-; OLDPM-NEXT:    [[K_05:%.*]] = phi i64 [ 1, [[FOR_BODY_LR_PH_NEW]] ], [ [[AND]], [[FOR_BODY]] ]
-; OLDPM-NEXT:    [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[FOR_BODY_LR_PH_NEW]] ], [ [[NITER_NSUB_7:%.*]], [[FOR_BODY]] ]
-; OLDPM-NEXT:    [[AND]] = and i64 [[CONV]], [[K_05]]
-; OLDPM-NEXT:    [[REASS_ADD:%.*]] = shl nuw nsw i64 [[AND]], 1
-; OLDPM-NEXT:    [[ADD_1:%.*]] = add i64 [[G_06]], [[REASS_ADD]]
-; OLDPM-NEXT:    [[REASS_ADD9:%.*]] = shl nuw nsw i64 [[AND]], 1
-; OLDPM-NEXT:    [[ADD_3:%.*]] = add i64 [[ADD_1]], [[REASS_ADD9]]
-; OLDPM-NEXT:    [[REASS_ADD10:%.*]] = shl nuw nsw i64 [[AND]], 1
-; OLDPM-NEXT:    [[ADD_5:%.*]] = add i64 [[ADD_3]], [[REASS_ADD10]]
-; OLDPM-NEXT:    [[REASS_ADD11:%.*]] = shl nuw nsw i64 [[AND]], 1
-; OLDPM-NEXT:    [[ADD_7]] = add i64 [[ADD_5]], [[REASS_ADD11]]
-; OLDPM-NEXT:    [[NITER_NSUB_7]] = add i64 [[NITER]], -8
-; OLDPM-NEXT:    [[NITER_NCMP_7:%.*]] = icmp eq i64 [[NITER_NSUB_7]], 0
-; OLDPM-NEXT:    br i1 [[NITER_NCMP_7]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]], label [[FOR_BODY]]
-;
-; NEWPM-LABEL: @func(
-; NEWPM-NEXT:  entry:
-; NEWPM-NEXT:    [[CMP4:%.*]] = icmp eq i64 [[LIMIT:%.*]], 0
-; NEWPM-NEXT:    br i1 [[CMP4]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_LR_PH:%.*]]
-; NEWPM:       for.body.lr.ph:
-; NEWPM-NEXT:    [[CONV:%.*]] = and i64 [[BLAH:%.*]], 4294967295
-; NEWPM-NEXT:    [[TMP0:%.*]] = add i64 [[LIMIT]], -1
-; NEWPM-NEXT:    [[XTRAITER:%.*]] = and i64 [[LIMIT]], 7
-; NEWPM-NEXT:    [[TMP1:%.*]] = icmp ult i64 [[TMP0]], 7
-; NEWPM-NEXT:    br i1 [[TMP1]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY_LR_PH_NEW:%.*]]
-; NEWPM:       for.body.lr.ph.new:
-; NEWPM-NEXT:    [[UNROLL_ITER:%.*]] = and i64 [[LIMIT]], -8
-; NEWPM-NEXT:    [[AND_0:%.*]] = and i64 [[CONV]], 1
-; NEWPM-NEXT:    br label [[FOR_BODY:%.*]]
-; NEWPM:       for.cond.cleanup.loopexit.unr-lcssa:
-; NEWPM-NEXT:    [[ADD_LCSSA_PH:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH]] ], [ [[ADD_7:%.*]], [[FOR_BODY]] ]
-; NEWPM-NEXT:    [[K_05_UNR:%.*]] = phi i64 [ 1, [[FOR_BODY_LR_PH]] ], [ [[AND_PHI:%.*]], [[FOR_BODY]] ]
-; NEWPM-NEXT:    [[LCMP_MOD:%.*]] = icmp eq i64 [[XTRAITER]], 0
-; NEWPM-NEXT:    br i1 [[LCMP_MOD]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL:%.*]]
-; NEWPM:       for.body.epil:
-; NEWPM-NEXT:    [[G_06_EPIL:%.*]] = phi i64 [ [[ADD_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[ADD_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; NEWPM-NEXT:    [[K_05_EPIL:%.*]] = phi i64 [ [[AND_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[K_05_UNR]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; NEWPM-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
-; NEWPM-NEXT:    [[AND_EPIL]] = and i64 [[CONV]], [[K_05_EPIL]]
-; NEWPM-NEXT:    [[ADD_EPIL]] = add i64 [[AND_EPIL]], [[G_06_EPIL]]
-; NEWPM-NEXT:    [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
-; NEWPM-NEXT:    [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
-; NEWPM-NEXT:    br i1 [[EPIL_ITER_CMP]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL]], !llvm.loop !0
-; NEWPM:       for.cond.cleanup:
-; NEWPM-NEXT:    [[G_0_LCSSA:%.*]] = phi i64 [ undef, [[ENTRY:%.*]] ], [ [[ADD_LCSSA_PH]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ], [ [[ADD_EPIL]], [[FOR_BODY_EPIL]] ]
-; NEWPM-NEXT:    ret i64 [[G_0_LCSSA]]
-; NEWPM:       for.body:
-; NEWPM-NEXT:    [[G_06:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH_NEW]] ], [ [[ADD_7]], [[FOR_BODY_FOR_BODY_CRIT_EDGE:%.*]] ]
-; NEWPM-NEXT:    [[AND_PHI]] = phi i64 [ [[AND_0]], [[FOR_BODY_LR_PH_NEW]] ], [ [[AND_1:%.*]], [[FOR_BODY_FOR_BODY_CRIT_EDGE]] ]
-; NEWPM-NEXT:    [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[FOR_BODY_LR_PH_NEW]] ], [ [[NITER_NSUB_7:%.*]], [[FOR_BODY_FOR_BODY_CRIT_EDGE]] ]
-; NEWPM-NEXT:    [[REASS_ADD:%.*]] = shl nuw nsw i64 [[AND_PHI]], 1
-; NEWPM-NEXT:    [[ADD_1:%.*]] = add i64 [[G_06]], [[REASS_ADD]]
-; NEWPM-NEXT:    [[REASS_ADD9:%.*]] = shl nuw nsw i64 [[AND_PHI]], 1
-; NEWPM-NEXT:    [[ADD_3:%.*]] = add i64 [[ADD_1]], [[REASS_ADD9]]
-; NEWPM-NEXT:    [[REASS_ADD10:%.*]] = shl nuw nsw i64 [[AND_PHI]], 1
-; NEWPM-NEXT:    [[ADD_5:%.*]] = add i64 [[ADD_3]], [[REASS_ADD10]]
-; NEWPM-NEXT:    [[REASS_ADD11:%.*]] = shl nuw nsw i64 [[AND_PHI]], 1
-; NEWPM-NEXT:    [[ADD_7]] = add i64 [[ADD_5]], [[REASS_ADD11]]
-; NEWPM-NEXT:    [[NITER_NSUB_7]] = add i64 [[NITER]], -8
-; NEWPM-NEXT:    [[NITER_NCMP_7:%.*]] = icmp eq i64 [[NITER_NSUB_7]], 0
-; NEWPM-NEXT:    br i1 [[NITER_NCMP_7]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]], label [[FOR_BODY_FOR_BODY_CRIT_EDGE]]
-; NEWPM:       for.body.for.body_crit_edge:
-; NEWPM-NEXT:    [[AND_1]] = and i64 [[CONV]], [[AND_PHI]]
-; NEWPM-NEXT:    br label [[FOR_BODY]]
+; CHECK-LABEL: @func(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP4_NOT:%.*]] = icmp eq i64 [[LIMIT:%.*]], 0
+; CHECK-NEXT:    br i1 [[CMP4_NOT]], label [[FOR_COND_CLEANUP:%.*]], label [[FOR_BODY_LR_PH:%.*]]
+; CHECK:       for.body.lr.ph:
+; CHECK-NEXT:    [[CONV:%.*]] = and i64 [[BLAH:%.*]], 4294967295
+; CHECK-NEXT:    [[TMP0:%.*]] = add i64 [[LIMIT]], -1
+; CHECK-NEXT:    [[XTRAITER:%.*]] = and i64 [[LIMIT]], 7
+; CHECK-NEXT:    [[TMP1:%.*]] = icmp ult i64 [[TMP0]], 7
+; CHECK-NEXT:    br i1 [[TMP1]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY_LR_PH_NEW:%.*]]
+; CHECK:       for.body.lr.ph.new:
+; CHECK-NEXT:    [[UNROLL_ITER:%.*]] = and i64 [[LIMIT]], -8
+; CHECK-NEXT:    br label [[FOR_BODY:%.*]]
+; CHECK:       for.cond.cleanup.loopexit.unr-lcssa:
+; CHECK-NEXT:    [[G_06_UNR:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH]] ], [ [[ADD_7:%.*]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[K_05_UNR:%.*]] = phi i64 [ 1, [[FOR_BODY_LR_PH]] ], [ [[AND:%.*]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[LCMP_MOD_NOT:%.*]] = icmp eq i64 [[XTRAITER]], 0
+; CHECK-NEXT:    br i1 [[LCMP_MOD_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL:%.*]]
+; CHECK:       for.body.epil:
+; CHECK-NEXT:    [[G_06_EPIL:%.*]] = phi i64 [ [[ADD_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[G_06_UNR]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
+; CHECK-NEXT:    [[K_05_EPIL:%.*]] = phi i64 [ [[AND_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[K_05_UNR]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
+; CHECK-NEXT:    [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ]
+; CHECK-NEXT:    [[AND_EPIL]] = and i64 [[CONV]], [[K_05_EPIL]]
+; CHECK-NEXT:    [[ADD_EPIL]] = add i64 [[AND_EPIL]], [[G_06_EPIL]]
+; CHECK-NEXT:    [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
+; CHECK-NEXT:    [[EPIL_ITER_CMP_NOT:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
+; CHECK-NEXT:    br i1 [[EPIL_ITER_CMP_NOT]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY_EPIL]], [[LOOP0:!llvm.loop !.*]]
+; CHECK:       for.cond.cleanup:
+; CHECK-NEXT:    [[G_0_LCSSA:%.*]] = phi i64 [ undef, [[ENTRY:%.*]] ], [ [[G_06_UNR]], [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]] ], [ [[ADD_EPIL]], [[FOR_BODY_EPIL]] ]
+; CHECK-NEXT:    ret i64 [[G_0_LCSSA]]
+; CHECK:       for.body:
+; CHECK-NEXT:    [[G_06:%.*]] = phi i64 [ undef, [[FOR_BODY_LR_PH_NEW]] ], [ [[ADD_7]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[K_05:%.*]] = phi i64 [ 1, [[FOR_BODY_LR_PH_NEW]] ], [ [[AND]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[FOR_BODY_LR_PH_NEW]] ], [ [[NITER_NSUB_7:%.*]], [[FOR_BODY]] ]
+; CHECK-NEXT:    [[AND]] = and i64 [[CONV]], [[K_05]]
+; CHECK-NEXT:    [[REASS_ADD:%.*]] = shl nuw nsw i64 [[AND]], 1
+; CHECK-NEXT:    [[ADD_1:%.*]] = add i64 [[G_06]], [[REASS_ADD]]
+; CHECK-NEXT:    [[REASS_ADD9:%.*]] = shl nuw nsw i64 [[AND]], 1
+; CHECK-NEXT:    [[ADD_3:%.*]] = add i64 [[ADD_1]], [[REASS_ADD9]]
+; CHECK-NEXT:    [[REASS_ADD10:%.*]] = shl nuw nsw i64 [[AND]], 1
+; CHECK-NEXT:    [[ADD_5:%.*]] = add i64 [[ADD_3]], [[REASS_ADD10]]
+; CHECK-NEXT:    [[REASS_ADD11:%.*]] = shl nuw nsw i64 [[AND]], 1
+; CHECK-NEXT:    [[ADD_7]] = add i64 [[ADD_5]], [[REASS_ADD11]]
+; CHECK-NEXT:    [[NITER_NSUB_7]] = add i64 [[NITER]], -8
+; CHECK-NEXT:    [[NITER_NCMP_7:%.*]] = icmp eq i64 [[NITER_NSUB_7]], 0
+; CHECK-NEXT:    br i1 [[NITER_NCMP_7]], label [[FOR_COND_CLEANUP_LOOPEXIT_UNR_LCSSA]], label [[FOR_BODY]]
 ;
 entry:
   %blah.addr = alloca i64, align 8
   %limit.addr = alloca i64, align 8
   %k = alloca i32, align 4
   %g = alloca i64, align 8
   %i = alloca i64, align 8
   store i64 %blah, i64* %blah.addr, align 8

llvm/test/Transforms/SpeculateAroundPHIs/loop-latch.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S -passes="spec-phis,print<loops>" %s  2>&1 | FileCheck %s
+
+; This check aims at verifying that loop metadata isn't discarded and that it
+; is placed on the latch (detected by the print<loops> pass)..
+; CHECK: Parallel Loop at depth 1 containing:
+
+; Also verify that we do not split a critical back-edge. If we do, we'd need
+; to transfer the loop metadata to the new latch (or discard it). We also need
+; to evaluate if such speculation is beneficial (at least downstream targets
+; have seen problems with missing HW-loops when letting the pass speculate
+; around back-edges).
+define void @hello(i16 %n) {
+; CHECK-LABEL: @hello(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[CMP1:%.*]] = icmp slt i16 [[N:%.*]], 8
+; CHECK-NEXT:    br i1 [[CMP1]], label [[FOR_BODY:%.*]], label [[FOR_END:%.*]]
+; CHECK:       for.body:
+; CHECK-NEXT:    [[I_02:%.*]] = phi i16 [ [[ADD:%.*]], [[FOR_BODY]] ], [ 0, [[ENTRY:%.*]] ]
+; CHECK-NEXT:    [[ADD]] = add nuw nsw i16 [[I_02]], 1
+; CHECK-NEXT:    [[CMP:%.*]] = icmp slt i16 [[ADD]], [[N]]
+; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END]], [[LOOP0:!llvm.loop !.*]]
+; CHECK:       for.end:
+; CHECK-NEXT:    ret void
+;
+entry:
+  %cmp1 = icmp slt i16 %n, 8
+  br i1 %cmp1, label %for.body, label %for.end
+
+for.body:
+  %i.02 = phi i16 [ %add, %for.body ], [ 0, %entry ]
+  %add = add nuw nsw i16 %i.02, 1
+  %cmp = icmp slt i16 %add, %n
+  br i1 %cmp, label %for.body, label %for.end, !llvm.loop !0
+
+for.end:
+  ret void
+}
+
+!0 = !{!0, !1}
+!1 = !{!"llvm.loop.vectorize.width", i32 1}