[SimplifyCFG/JumpThreading] Do not simplify empty blocks with unconditional branches if this causes loop metadata confusion.

[Meinersbur]

Summary

When simplifying an empty block with an unconditional branch by skipping over it, the skipped block might have been a loop's latch. In that case transfer the loop's metadata attached to the latch to the new latch. If it already is also a latch with different metadata, do not skip the block.

Part of fix for llvm.org/PR50060.

[lebedev.ri]

I'm not really in favor of using LoopHeaders for this.
It is not updated *anywhere*, which means that most of the time it's data is both stale and incomplete.

[Meinersbur]

LoopHeaders is already used by both SimplifyCFG and JumpThreading. If it is stale and incomplete, it is already a problem now.

Do you have an alternative suggestion?

[jeroen.dobbelaere]

Did you verify the effect of this on the performance ? In a similar (but different) trial where I tried to block merging of blocks when that would result in a conflict with loop annotations, I noticed unacceptable regressions.

[lebedev.ri]

Patch's subject is wrong. This doesn't "Transfer loop metadata to new latch.", it prevents folding.
Or is this a wrong diff?

In D101142#2711280, @Meinersbur wrote:

LoopHeaders is already used by both SimplifyCFG and JumpThreading. If it is stale and incomplete, it is already a problem now.

Do you have an alternative suggestion?

I don't know, but not introducing more problems by using problematic interface (LoopHeaders) seems like a good suggestion.
if you already have metadata, why do you need LoopHeaders ?
But basically, i'm unconvinced that we want to restrict simplifycfg in this way.

[Meinersbur]

In D101142#2711283, @jeroen.dobbelaere wrote:

Did you verify the effect of this on the performance ? In a similar (but different) trial where I tried to block merging of blocks when that would result in a conflict with loop annotations, I noticed unacceptable regressions.

I did not. This is a correctness fix. Arguing for performance over correctness feels weird.

In D101142#2711289, @lebedev.ri wrote:

Patch's subject is wrong. This doesn't "Transfer loop metadata to new latch.", it prevents folding.

Fixed title.

if you already have metadata, why do you need LoopHeaders ?

To identify which blocks are latches. If moving metadata from another loop there may cause correctness problems, for instance if a mustprogress annotation is associated with a loop that does not need to progress.

Unfortunately the metadata itself is not sufficient. Let A and B be to blocks which are latches of two different loops. A has metadata, but B has not. Combining the blocks and using the metadata of A for it will be wrong, as it would apply the same metadata (e.g. mustprogress) to both loops, as happened in the PR50060. The alternative would be to not combine any blocks if any of the two has loop metadata. Or dropping both metadata.

[lebedev.ri]

In D101142#2711394, @Meinersbur wrote:

In D101142#2711283, @jeroen.dobbelaere wrote:

Did you verify the effect of this on the performance ? In a similar (but different) trial where I tried to block merging of blocks when that would result in a conflict with loop annotations, I noticed unacceptable regressions.

I did not. This is a correctness fix. Arguing for performance over correctness feels weird.

In D101142#2711289, @lebedev.ri wrote:

Patch's subject is wrong. This doesn't "Transfer loop metadata to new latch.", it prevents folding.

Fixed title.

if you already have metadata, why do you need LoopHeaders ?

To identify which blocks are latches. If moving metadata from another loop there may cause correctness problems, for instance if a mustprogress annotation is associated with a loop that does not need to progress.

Unfortunately the metadata itself is not sufficient. Let A and B be to blocks which are latches of two different loops. A has metadata, but B has not. Combining the blocks and using the metadata of A for it will be wrong, as it would apply the same metadata (e.g. mustprogress) to both loops, as happened in the PR50060. The alternative would be to not combine any blocks if any of the two has loop metadata. Or dropping both metadata.

What i meant is, doesn't that metadata belong to the loop latches?
So if we really want to treat loop latches specially, isn't merely the presence of said metadata enough?

[jeroen.dobbelaere]

In D101142#2711453, @lebedev.ri wrote:

In D101142#2711394, @Meinersbur wrote:

In D101142#2711283, @jeroen.dobbelaere wrote:

Did you verify the effect of this on the performance ? In a similar (but different) trial where I tried to block merging of blocks when that would result in a conflict with loop annotations, I noticed unacceptable regressions.

I did not. This is a correctness fix. Arguing for performance over correctness feels weird.

It indeed feels weird and maybe your testing shows that it is not that big of a problem. My feeling was that maybe a different approach in describing loop information might be more appropriate, resulting in accurate information, without the observed performance impact. That given, I have no idea yet how that could look like :(.

[Meinersbur]

In D101142#2711453, @lebedev.ri wrote:

What i meant is, doesn't that metadata belong to the loop latches?
So if we really want to treat loop latches specially, isn't merely the presence of said metadata enough?

This is what my example was trying to show. A latch that previously had no metadata must not be assigned metadata from a different loop.

In D101142#2711513, @jeroen.dobbelaere wrote:

It indeed feels weird and maybe your testing shows that it is not that big of a problem. My feeling was that maybe a different approach in describing loop information might be more appropriate, resulting in accurate information, without the observed performance impact. That given, I have no idea yet how that could look like :(.

This is not about preserving loop metadata, but ensuring that the loop does not get assigned the wrong metadata. Like a mustprogress annotation or a #pragma clang loop vectorize(assume_safety) from a different loop.

I agree that a different approach might be better, such as Johannes' suggestion to allow Metadata on (Header) BasicBlocks (which may come with other problems), or introducing an intrinsic that holds that metadata (which comes with a performance penalty of a different kind: every other pass has to iterate over ir). But until this happens, this fixes a potential miscompilation.

[jeroen.dobbelaere]

llvm/lib/Transforms/Utils/Local.cpp

1090

Is (LoopMD != PredMD) a sufficient rule ?

Assume you have two nested loops with both a 'loop unroll 4' annotation. Can we get in a situation where both latches would be merged ? If so, is it valid then to still merge the latches ?

[Meinersbur]

llvm/lib/Transforms/Utils/Local.cpp

1090

The idea was that if the loops have the same properties, then they will still have the same properties after merging. That itself might be problematic since changing the LoopMD (e.g. LoopVectorize sets a isvectorized property when unable to vectorize) would apply to both loops.

However, also conside LoopMD are distinct and therefore different loop should have different MDNodes. This is not strictly enforced (e.g. Inliner or LoopUnroll duplicate entire loops), so it can happen for different loops to have the same LoopMD.

I can make the condition stricter by comparing the header blocks they branch to.

Is there still interest in this patch?

[lebedev.ri]

This review seems to be stuck/dead, consider abandoning if no longer relevant.

llvm/include/llvm/Transforms/Scalar/JumpThreading.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TRANSFORMS_SCALAR_JUMPTHREADING_H
 #define LLVM_TRANSFORMS_SCALAR_JUMPTHREADING_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/SmallPtrSet.h"
-#include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Analysis/BlockFrequencyInfo.h"
 #include "llvm/Analysis/BranchProbabilityInfo.h"
 #include "llvm/Analysis/DomTreeUpdater.h"
 #include "llvm/IR/ValueHandle.h"
 #include <memory>
 #include <utility>
 
class JumpThreadingPass : public PassInfoMixin<JumpThreadingPass> {
   TargetLibraryInfo *TLI;
   LazyValueInfo *LVI;
   AAResults *AA;
   DomTreeUpdater *DTU;
   std::unique_ptr<BlockFrequencyInfo> BFI;
   std::unique_ptr<BranchProbabilityInfo> BPI;
   bool HasProfileData = false;
   bool HasGuards = false;
-#ifdef NDEBUG
   SmallPtrSet<const BasicBlock *, 16> LoopHeaders;
-#else
-  SmallSet<AssertingVH<const BasicBlock>, 16> LoopHeaders;
-#endif
 
   unsigned BBDupThreshold;
   unsigned DefaultBBDupThreshold;
   bool InsertFreezeWhenUnfoldingSelect;
 
 public:
   JumpThreadingPass(bool InsertFreezeWhenUnfoldingSelect = false, int T = -1);
 

llvm/include/llvm/Transforms/Utils/Local.h

 /// successor (BB!). Eliminate the edge between them, moving the instructions in
 /// the predecessor into BB. This deletes the predecessor block.
 void MergeBasicBlockIntoOnlyPred(BasicBlock *BB, DomTreeUpdater *DTU = nullptr);
 
 /// BB is known to contain an unconditional branch, and contains no instructions
 /// other than PHI nodes, potential debug intrinsics and the branch. If
 /// possible, eliminate BB by rewriting all the predecessors to branch to the
 /// successor block and return true. If we can't transform, return false.
-bool TryToSimplifyUncondBranchFromEmptyBlock(BasicBlock *BB,
-                                             DomTreeUpdater *DTU = nullptr);
+///
+/// The set of loop headers is needed to avoid that loop metadata is associated
+/// with the wrong loop.
+bool TryToSimplifyUncondBranchFromEmptyBlock(
+    BasicBlock *BB, const SmallPtrSetImpl<const BasicBlock *> &LoopHeaders,
 
 /// Check for and eliminate duplicate PHI nodes in this block. This doesn't try
 /// to be clever about PHI nodes which differ only in the order of the incoming
 /// values, but instcombine orders them so it usually won't matter.
 bool EliminateDuplicatePHINodes(BasicBlock *BB);
 
 /// This function is used to do simplification of a CFG.  For example, it
 /// adjusts branches to branches to eliminate the extra hop, it eliminates

llvm/lib/Transforms/Scalar/JumpThreading.cpp

for (auto &BB : F) {
       if (BI && BI->isUnconditional()) {
         BasicBlock *Succ = BI->getSuccessor(0);
         if (
             // The terminator must be the only non-phi instruction in BB.
             BB.getFirstNonPHIOrDbg(true)->isTerminator() &&
             // Don't alter Loop headers and latches to ensure another pass can
             // detect and transform nested loops later.
             !LoopHeaders.count(&BB) && !LoopHeaders.count(Succ) &&
-            TryToSimplifyUncondBranchFromEmptyBlock(&BB, DTU)) {
+            TryToSimplifyUncondBranchFromEmptyBlock(&BB, LoopHeaders, DTU)) {
           RemoveRedundantDbgInstrs(Succ, true);
           // BB is valid for cleanup here because we passed in DTU. F remains
           // BB's parent until a DTU->getDomTree() event.
           LVI->eraseBlock(&BB);
           Changed = true;
         }
       }
     }

llvm/lib/Transforms/Utils/Local.cpp

for (unsigned i = 0, e = BBPreds.size(); i != e; ++i) {
       // newly retargeted branch.
       PN->addIncoming(Selected, PredBB);
     }
   }
 
   replaceUndefValuesInPhi(PN, IncomingValues);
 }
 
-bool llvm::TryToSimplifyUncondBranchFromEmptyBlock(BasicBlock *BB,
-                                                   DomTreeUpdater *DTU) {
+bool llvm::TryToSimplifyUncondBranchFromEmptyBlock(
+    BasicBlock *BB, const SmallPtrSetImpl<const BasicBlock *> &LoopHeaders,
   assert(BB != &BB->getParent()->getEntryBlock() &&
          "TryToSimplifyUncondBranchFromEmptyBlock called on entry block!");
 
   // We can't eliminate infinite loops.
   BasicBlock *Succ = cast<BranchInst>(BB->getTerminator())->getSuccessor(0);
   if (BB == Succ) return false;
 
   // Check to see if merging these blocks would cause conflicts for any of the
while (isa<PHINode>(*BBI)) {
         } else {
           return false;
         }
       }
       ++BBI;
     }
   }
 
+  // Return whether the BB is a latch of some loop. If yes, it's llvm.loop
+  // metadata has to be preserved.
+  auto IsLoopLatch = [&LoopHeaders](BasicBlock *BB) {
+    for (BasicBlock *Succ : successors(BB))
+      if (is_contained(LoopHeaders, Succ))
+        return true;
+    return false;
+  };
+
+  unsigned LoopMDKind = BB->getContext().getMDKindID("llvm.loop");
+  Instruction *TI = BB->getTerminator();
+  MDNode *LoopMD = TI ? TI->getMetadata(LoopMDKind) : nullptr;
+  bool BBIsLatch = IsLoopLatch(BB);
+
   // We cannot fold the block if it's a branch to an already present callbr
   // successor because that creates duplicate successors.
   for (BasicBlock *PredBB : predecessors(BB)) {
     if (auto *CBI = dyn_cast<CallBrInst>(PredBB->getTerminator())) {
       if (Succ == CBI->getDefaultDest())
         return false;
       for (unsigned i = 0, e = CBI->getNumIndirectDests(); i != e; ++i)
         if (Succ == CBI->getIndirectDest(i))
           return false;
     }
+
+    // If both BBs are latches, then do not merge them to keep their metadata
+    // distinct.
+    Instruction *PredTI = PredBB->getTerminator();
+    MDNode *PredMD = PredTI ? PredTI->getMetadata(LoopMDKind) : nullptr;
+    if (LoopMD != PredMD && BBIsLatch && IsLoopLatch(PredBB))
+      return false;
   }
 
   LLVM_DEBUG(dbgs() << "Killing Trivial BB: \n" << *BB);
 
   SmallVector<DominatorTree::UpdateType, 32> Updates;
   if (DTU) {
     // All predecessors of BB will be moved to Succ.
     SmallPtrSet<BasicBlock *, 8> PredsOfBB(pred_begin(BB), pred_end(BB));
bool llvm::TryToSimplifyUncondBranchFromEmptyBlock(
   } else {
     while (PHINode *PN = dyn_cast<PHINode>(&BB->front())) {
       // We explicitly check for such uses in CanPropagatePredecessorsForPHIs.
       assert(PN->use_empty() && "There shouldn't be any uses here!");
       PN->eraseFromParent();
     }
   }
 
-  // If the unconditional branch we replaced contains llvm.loop metadata, we
-  // add the metadata to the branch instructions in the predecessors.
-  unsigned LoopMDKind = BB->getContext().getMDKindID("llvm.loop");
-  Instruction *TI = BB->getTerminator();
-  if (TI)
-    if (MDNode *LoopMD = TI->getMetadata(LoopMDKind))
-      for (BasicBlock *Pred : predecessors(BB))
-        Pred->getTerminator()->setMetadata(LoopMDKind, LoopMD);
+  // We previously ensured that either BB is not a loop latch, or none of its
+  // predecessor is (or they have the same metadata already). If BB is a latch,
+  // then its predecessors will become latches of the same loop; therefore
+  // transfer the loop's metadata to the new latches. If a predecessor was a
+  // latch, then nothing changes.
+  if (BBIsLatch)
 
   // For AutoFDO, since BB is going to be removed, we won't be able to sample
   // it. To avoid assigning a zero weight for BB, move all its pseudo probes
   // into Succ and mark them dangling. This should allow the counts inference a
   // chance to get a more reasonable weight for BB.
   moveAndDanglePseudoProbes(BB, &*Succ->getFirstInsertionPt());
 
   // Everything that jumped to BB now goes to Succ.

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

return Branch->isUnconditional() ? simplifyUncondBranch(Branch, Builder)
                                    : simplifyCondBranch(Branch, Builder);
 }
 
 bool SimplifyCFGOpt::simplifyUncondBranch(BranchInst *BI,
                                           IRBuilder<> &Builder) {
   BasicBlock *BB = BI->getParent();
   BasicBlock *Succ = BI->getSuccessor(0);
 
+  SmallPtrSet<const BasicBlock *, 16> LoopHeaderSet;
+  for (WeakVH L : LoopHeaders)
+    if (L)
+      LoopHeaderSet.insert(cast<BasicBlock>(L));
+
   // If the Terminator is the only non-phi instruction, simplify the block.
   // If LoopHeader is provided, check if the block or its successor is a loop
   // header. (This is for early invocations before loop simplify and
   // vectorization to keep canonical loop forms for nested loops. These blocks
   // can be eliminated when the pass is invoked later in the back-end.)
   // Note that if BB has only one predecessor then we do not introduce new
   // backedge, so we can eliminate BB.
   bool NeedCanonicalLoop =
       Options.NeedCanonicalLoop &&
       (!LoopHeaders.empty() && BB->hasNPredecessorsOrMore(2) &&
        (is_contained(LoopHeaders, BB) || is_contained(LoopHeaders, Succ)));
   BasicBlock::iterator I = BB->getFirstNonPHIOrDbg(true)->getIterator();
   if (I->isTerminator() && BB != &BB->getParent()->getEntryBlock() &&
-      !NeedCanonicalLoop && TryToSimplifyUncondBranchFromEmptyBlock(BB, DTU))
+      !NeedCanonicalLoop &&
+      TryToSimplifyUncondBranchFromEmptyBlock(BB, LoopHeaderSet, DTU))
     return true;
 
   // If the only instruction in the block is a seteq/setne comparison against a
   // constant, try to simplify the block.
   if (ICmpInst *ICI = dyn_cast<ICmpInst>(I))
     if (ICI->isEquality() && isa<ConstantInt>(ICI->getOperand(1))) {
       for (++I; isa<DbgInfoIntrinsic>(I); ++I)
         ;

llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-accesses-masked-group.ll

 ; ENABLED_MASKED_STRIDED-NEXT:    [[WIDE_MASKED_VEC:%.*]] = call <16 x i8> @llvm.masked.load.v16i8.p0v16i8(<16 x i8>* [[TMP3]], i32 1, <16 x i1> [[INTERLEAVED_MASK]], <16 x i8> poison)
 ; ENABLED_MASKED_STRIDED-NEXT:    [[STRIDED_VEC:%.*]] = shufflevector <16 x i8> [[WIDE_MASKED_VEC]], <16 x i8> poison, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
 ; ENABLED_MASKED_STRIDED-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[Q:%.*]], i32 [[INDEX]]
 ; ENABLED_MASKED_STRIDED-NEXT:    [[TMP5:%.*]] = bitcast i8* [[TMP4]] to <8 x i8>*
 ; ENABLED_MASKED_STRIDED-NEXT:    call void @llvm.masked.store.v8i8.p0v8i8(<8 x i8> [[STRIDED_VEC]], <8 x i8>* [[TMP5]], i32 1, <8 x i1> [[TMP0]])
 ; ENABLED_MASKED_STRIDED-NEXT:    [[INDEX_NEXT]] = add i32 [[INDEX]], 8
 ; ENABLED_MASKED_STRIDED-NEXT:    [[VEC_IND_NEXT]] = add <8 x i32> [[VEC_IND]], <i32 8, i32 8, i32 8, i32 8, i32 8, i32 8, i32 8, i32 8>
 ; ENABLED_MASKED_STRIDED-NEXT:    [[TMP6:%.*]] = icmp eq i32 [[INDEX_NEXT]], 1016
-; ENABLED_MASKED_STRIDED-NEXT:    br i1 [[TMP6]], label [[FOR_BODY:%.*]], label [[VECTOR_BODY]], [[LOOP0:!llvm.loop !.*]]
+; ENABLED_MASKED_STRIDED-NEXT:    br i1 [[TMP6]], label [[SCALAR_PH:%.*]], label [[VECTOR_BODY]], [[LOOP0:!llvm.loop !.*]]
+; ENABLED_MASKED_STRIDED:       scalar.ph:
+; ENABLED_MASKED_STRIDED-NEXT:    br label [[FOR_BODY:%.*]]
 ; ENABLED_MASKED_STRIDED:       for.body:
-; ENABLED_MASKED_STRIDED-NEXT:    [[IX_09:%.*]] = phi i32 [ [[INC:%.*]], [[FOR_INC:%.*]] ], [ 1016, [[VECTOR_BODY]] ]
+; ENABLED_MASKED_STRIDED-NEXT:    [[IX_09:%.*]] = phi i32 [ 1016, [[SCALAR_PH]] ], [ [[INC:%.*]], [[FOR_INC:%.*]] ]
 ; ENABLED_MASKED_STRIDED-NEXT:    [[CMP1:%.*]] = icmp ugt i32 [[IX_09]], [[CONV]]
 ; ENABLED_MASKED_STRIDED-NEXT:    br i1 [[CMP1]], label [[IF_THEN:%.*]], label [[FOR_INC]]
 ; ENABLED_MASKED_STRIDED:       if.then:
 ; ENABLED_MASKED_STRIDED-NEXT:    [[MUL:%.*]] = shl nuw nsw i32 [[IX_09]], 1
 ; ENABLED_MASKED_STRIDED-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i8, i8* [[P]], i32 [[MUL]]
 ; ENABLED_MASKED_STRIDED-NEXT:    [[TMP7:%.*]] = load i8, i8* [[ARRAYIDX]], align 1
 ; ENABLED_MASKED_STRIDED-NEXT:    [[ARRAYIDX3:%.*]] = getelementptr inbounds i8, i8* [[Q]], i32 [[IX_09]]
 ; ENABLED_MASKED_STRIDED-NEXT:    store i8 [[TMP7]], i8* [[ARRAYIDX3]], align 1

llvm/test/Transforms/SimplifyCFG/pr50060-jumpthread.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S -simplifycfg < %s | FileCheck %s
+;
+; When simplifying the unconditional branch do.body.loopexit
+; (latch of%do.body), ensure that the loop metadata is transferred to
+; the new latch (for.cond.cleanup3).
+;
+; llvm.org/PR50060
+
+@n = dso_local global i32 0, align 4
+@C = dso_local global i32 0, align 4
+
+; Function Attrs: nounwind
+define dso_local void @_Z6test01v() addrspace(1) #0 {
+; CHECK-LABEL: @_Z6test01v(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TMP0:%.*]] = load i32, i32* @n, align 4, !tbaa [[TBAA2:![0-9]+]]
+; CHECK-NEXT:    [[CMP26:%.*]] = icmp slt i32 0, [[TMP0]]
+; CHECK-NEXT:    br label [[DO_BODY:%.*]]
+; CHECK:       do.body.loopexit:
+; CHECK-NEXT:    br label [[DO_BODY]], !llvm.loop [[LOOP6:![0-9]+]]
+; CHECK:       do.body:
+; CHECK-NEXT:    br label [[FOR_COND1_PREHEADER:%.*]]
+; CHECK:       for.cond1.preheader:
+; CHECK-NEXT:    [[J_08:%.*]] = phi i32 [ 0, [[DO_BODY]] ], [ [[INC7:%.*]], [[FOR_COND_CLEANUP3:%.*]] ]
+; CHECK-NEXT:    br i1 [[CMP26]], label [[FOR_BODY4:%.*]], label [[FOR_COND_CLEANUP3]]
+; CHECK:       for.cond.cleanup3:
+; CHECK-NEXT:    [[INC7]] = add nuw nsw i32 [[J_08]], 1
+; CHECK-NEXT:    [[CMP:%.*]] = icmp ult i32 [[INC7]], 3
+; CHECK-NEXT:    br i1 [[CMP]], label [[FOR_COND1_PREHEADER]], label [[DO_BODY_LOOPEXIT:%.*]], !llvm.loop [[LOOP8:![0-9]+]]
+; CHECK:       for.body4:
+; CHECK-NEXT:    [[I_07:%.*]] = phi i32 [ [[INC5:%.*]], [[FOR_BODY4]] ], [ 0, [[FOR_COND1_PREHEADER]] ]
+; CHECK-NEXT:    store volatile i32 [[I_07]], i32* @C, align 4, !tbaa [[TBAA2]]
+; CHECK-NEXT:    [[INC5]] = add nuw nsw i32 [[I_07]], 1
+; CHECK-NEXT:    [[CMP2:%.*]] = icmp slt i32 [[INC5]], [[TMP0]]
+; CHECK-NEXT:    br i1 [[CMP2]], label [[FOR_BODY4]], label [[FOR_COND_CLEANUP3]], !llvm.loop [[LOOP11:![0-9]+]]
+;
+entry:
+  %0 = load i32, i32* @n, align 4, !tbaa !2
+  %cmp26 = icmp slt i32 0, %0
+  br label %do.body
+
+do.body.loopexit:                                 ; preds = %for.cond.cleanup3
+  br label %do.body, !llvm.loop !6
+
+do.body:                                          ; preds = %do.body.loopexit, %entry
+  br label %for.cond1.preheader
+
+for.cond1.preheader:                              ; preds = %do.body, %for.cond.cleanup3
+  %j.08 = phi i32 [ 0, %do.body ], [ %inc7, %for.cond.cleanup3 ]
+  br i1 %cmp26, label %for.body4.lr.ph, label %for.cond.cleanup3
+
+for.body4.lr.ph:                                  ; preds = %for.cond1.preheader
+  br label %for.body4
+
+for.cond1.for.cond.cleanup3_crit_edge:            ; preds = %for.body4
+  br label %for.cond.cleanup3
+
+for.cond.cleanup3:                                ; preds = %for.cond1.for.cond.cleanup3_crit_edge, %for.cond1.preheader
+  %inc7 = add nuw nsw i32 %j.08, 1
+  %cmp = icmp ult i32 %inc7, 3
+  br i1 %cmp, label %for.cond1.preheader, label %do.body.loopexit, !llvm.loop !8
+
+for.body4:                                        ; preds = %for.body4.lr.ph, %for.body4
+  %i.07 = phi i32 [ 0, %for.body4.lr.ph ], [ %inc5, %for.body4 ]
+  store volatile i32 %i.07, i32* @C, align 4, !tbaa !2
+  %inc5 = add nuw nsw i32 %i.07, 1
+  %cmp2 = icmp slt i32 %inc5, %0
+  br i1 %cmp2, label %for.body4, label %for.cond1.for.cond.cleanup3_crit_edge, !llvm.loop !11
+}
+
+; Function Attrs: argmemonly nofree nosync nounwind willreturn
+declare void @llvm.lifetime.start.p0i8(i64 immarg, i8* nocapture) addrspace(1) #1
+
+; Function Attrs: argmemonly nofree nosync nounwind willreturn
+declare void @llvm.lifetime.end.p0i8(i64 immarg, i8* nocapture) addrspace(1) #1
+
+attributes #0 = { nounwind "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { argmemonly nofree nosync nounwind willreturn }
+
+!llvm.module.flags = !{!0}
+!llvm.ident = !{!1}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{!"clang)"}
+!2 = !{!3, !3, i64 0, i64 4}
+!3 = !{!4, i64 4, !"int"}
+!4 = !{!5, i64 1, !"omnipotent char"}
+!5 = !{!"Simple C++ TBAA"}
+!6 = distinct !{!6, !7}
+!7 = !{!"llvm.loop.unroll.count", i32 2}
+!8 = distinct !{!8, !9, !10}
+!9 = !{!"llvm.loop.mustprogress"}
+!10 = !{!"llvm.loop.unroll.disable"}
+!11 = distinct !{!11, !9}