[LoopUnroll] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed

[jeroen.dobbelaere]

Summary

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=39282. Compared to D90104, this version is based on part of the full restrict patched (D68484) and uses the @llvm.experimental.noalias.scope.decl intrinsic to track the location where !noalias and !alias.scope scopes have been introduced. This allows us to only duplicate the scopes that are really needed.

Notes:

• it also includes changes and tests from D90104

[nikic]

I'd suggest to split this up into (at least) three parts:

• LangRef patch (not present yet here) and the addition of the intrinsic.
• Teaching everything relevant to look past the intrinsic and emitting it in inliner.
• Making use of it in loop unrolling.

This will make it easier to identify regressions caused by the extra intrinsic blocking some transform, because of missing special handling or similar.

[jeroen.dobbelaere]

In D92887#2442047, @nikic wrote:

I'd suggest to split this up into (at least) three parts:

• LangRef patch (not present yet here) and the addition of the intrinsic.
• Teaching everything relevant to look past the intrinsic and emitting it in inliner.
• Making use of it in loop unrolling.

This will make it easier to identify regressions caused by the extra intrinsic blocking some transform, because of missing special handling or similar.

Yes, that makes sense. I hope to have an update tomorrow.

[jeroen.dobbelaere]

Rebased and adapted to new naming

[nikic]

The logic here looks sounds to me.

llvm/include/llvm/Transforms/Utils/Cloning.h

272–274

277

I don't think we use this kind of out_ style anywhere.

llvm/lib/Transforms/Utils/CloneFunction.cpp

933

nit: No braces here.

942

Can operands be null?

954

nit: No braces here.

978

nit: No braces here.

985

nit: auto * or BasicBlock *.

llvm/lib/Transforms/Utils/InlineFunction.cpp

86 ↗

(On Diff #315134)

These changes should have probably ended up in D93040.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

591

llvm:: is presumably unnecessary here.

690

With the implementation factored out the "phase 2" and "phase 3" comments look a bit out of place.

693

Possible the ext parameter should be Twine rather than StringRef. But I'm not really familiar with our string API conventions.

[jeroen.dobbelaere]

llvm/include/llvm/Transforms/Utils/Cloning.h

277

I personally like the the out_/inout_/in_ style for making the purpose clear. But I understand that in the LLVM context it is better to have a common style.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

693

I would rather keep it. When using a Twine, the Twine will be evaluated every time a string is needed.
By using a StringRef, at least that part will only be done once.

[jeroen.dobbelaere]

• Update according to comments

[jeroen.dobbelaere]

• removed more unnecessary braces

[jeroen.dobbelaere]

Rebased.
Changed comment to indicate that, while scopes are duplicated, the domain is kept.

[nikic]

LGTM

llvm/include/llvm/Transforms/Utils/Cloning.h

271

nit: "back" is too much here.

301

nit: Missing period at end of sentence.

llvm/lib/Transforms/Utils/CloneFunction.cpp

901

nit: Presumably unnecessary llvm:: prefix.

927

I believe you should be able to write this as for (Use &U : I->operands()) and then use U.set() to replace the operand.

930

Should be possible to use something like if (MetadataAsValue *NewMV = ClonedMVScopes.lookup(MV)) here.

937

nit: Should be NeedsReplacement.

942

Here again, something like MDNode *NewMD = ClonedScopes.lookup(MD)) should be possible.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

588

nit: The "Phase1" here no longer makes sense (was dropped below).

llvm/test/Transforms/LoopUnroll/noalias.ll

63

It might be worthwhile to also add a test with noalias.scope.decl outside the loop, that shows that no scopes are cloned in that case.

[nikic]

llvm/test/Transforms/PhaseOrdering/pr39282.ll

18

This TODO should be dropped :)

[jeroen.dobbelaere]

Adapted to comments.

[asbirlea]

My concern with this patch is in the iteration on all instructions in the blocks in cloneAndAdaptNoAliasScopes and identifyNoAliasScopesToClone.
Perhaps for LoopUnroll is not an issue, if unroll cost model is taken into account before this (i.e. unroll doesn't happen for large BBs). But when doing loop rotate, this may explode for IRs with large BBs, so I'd say some measurements would be helpful here.
For loop rotate, which instructions are cloned from the header to the preheader is known at the callsite, so as to not iterate over the entire original preheader, but I'm not sure this is sufficient.

llvm/test/Transforms/PhaseOrdering/pr39282.ll

57

Nit: CHECKs for MDNodes?

[jeroen.dobbelaere]

Make use of NoAliasScopeDeclInst

llvm/include/llvm/IR/Metadata.h

public:
   const MDNode *getNode() const { return Node; }
 
   /// Get the MDNode for this AliasScopeNode's domain.
   const MDNode *getDomain() const {
     if (Node->getNumOperands() < 2)
       return nullptr;
     return dyn_cast_or_null<MDNode>(Node->getOperand(1));
   }
+  StringRef getName() const {
+    if (Node->getNumOperands() > 2)
+      if (MDString *N = dyn_cast_or_null<MDString>(Node->getOperand(2)))
+        return N->getString();
+    return StringRef();
+  }
 };
 
 /// Typed iterator through MDNode operands.
 ///
 /// An iterator that transforms an \a MDNode::iterator into an iterator over a
 /// particular Metadata subclass.
 template <class T>
 class TypedMDOperandIterator

llvm/include/llvm/Transforms/Utils/Cloning.h

 
 /// Updates profile information by adjusting the entry count by adding
 /// entryDelta then scaling callsite information by the new count divided by the
 /// old count. VMap is used during inlinng to also update the new clone
 void updateProfileCallee(
     Function *Callee, int64_t entryDelta,
     const ValueMap<const Value *, WeakTrackingVH> *VMap = nullptr);
 
+/// Find back the 'llvm.experimental.noalias.scope.decl' intrinsics in the
+/// specified basic blocks and extract their scope. These are candidates for
+/// duplication when cloning.
+void identifyNoAliasScopesToClone(
+    ArrayRef<BasicBlock *> BBs,
+    SmallVectorImpl<MetadataAsValue *> &NoAliasDeclScopes);
+
+/// Duplicate the specified list of noalias decl scopes.
+/// The 'Ext' string is added as an extension to the name.
+/// Afterwards, the ClonedMVScopes contains a mapping of the original MV onto
+/// the cloned version.
+/// The ClonedScopes contains the mapping of the original scope MDNode onto the
+/// cloned scope.
+void cloneNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    StringRef Ext, LLVMContext &Context);
+
+/// Adapt the metadata for the specified instruction according to the
+/// provided mapping. This is normally used after cloning an instruction, when
+/// some noalias scopes needed to be cloned.
+void adaptNoAliasScopes(
+    llvm::Instruction *I, const DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    const DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    LLVMContext &Context);
+
+/// Clone the specified noalias decl scopes. Then adapt all instructions in the
+/// NewBlocks basicblocks to the cloned versions.
+/// 'Ext' will be added to the duplicate scope names
+void cloneAndAdaptNoAliasScopes(ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+                                ArrayRef<BasicBlock *> NewBlocks,
+                                LLVMContext &Context, StringRef Ext);
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_UTILS_CLONING_H

llvm/lib/Transforms/Utils/CloneFunction.cpp

 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DebugInfo.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/MDBuilder.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
 #include "llvm/Transforms/Utils/Cloning.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/ValueMapper.h"
 #include <map>
 using namespace llvm;
 
+#define DEBUG_TYPE "clone-function"
+
 /// See comments in Cloning.h.
 BasicBlock *llvm::CloneBasicBlock(const BasicBlock *BB, ValueToValueMapTy &VMap,
                                   const Twine &NameSuffix, Function *F,
                                   ClonedCodeInfo *CodeInfo,
                                   DebugInfoFinder *DIFinder) {
   DenseMap<const MDNode *, MDNode *> Cache;
   BasicBlock *NewBB = BasicBlock::Create(BB->getContext(), "", F);
   if (BB->hasName())
for (unsigned i = 0, e = New->getNumOperands(); i != e; ++i)
         auto I = ValueMapping.find(Inst);
         if (I != ValueMapping.end())
           New->setOperand(i, I->second);
       }
   }
 
   return NewBB;
 }
+void llvm::cloneNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    StringRef Ext, LLVMContext &Context) {
+  MDBuilder MDB(Context);
+
+  for (auto *MV : NoAliasDeclScopes) {
+    SmallVector<Metadata *, 4> ScopeList;
+    for (auto &MDOperand : cast<MDNode>(MV->getMetadata())->operands()) {
+      if (MDNode *MD = dyn_cast<MDNode>(MDOperand)) {
+        llvm::AliasScopeNode SNANode(MD);
+
+        std::string Name;
+        auto ScopeName = SNANode.getName();
+        if (!ScopeName.empty()) {
+          Name = (Twine(ScopeName) + ":" + Ext).str();
+        } else {
+          Name = std::string(Ext);
+        }
+
+        MDNode *NewScope = MDB.createAnonymousAliasScope(
+            const_cast<MDNode *>(SNANode.getDomain()), Name);
+        ClonedScopes.insert(std::make_pair(MD, NewScope));
+        ScopeList.push_back(NewScope);
+      }
+    }
+    MDNode *NewScopeList = MDNode::get(Context, ScopeList);
+    ClonedMVScopes.insert(
+        std::make_pair(MV, MetadataAsValue::get(Context, NewScopeList)));
+  }
+}
+
+void llvm::adaptNoAliasScopes(
+    Instruction *I, const DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    const DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    LLVMContext &Context) {
+  // MetadataAsValue will always be replaced !
+  for (int opI = 0, opIEnd = I->getNumOperands(); opI < opIEnd; ++opI) {
+    if (MetadataAsValue *MV = dyn_cast<MetadataAsValue>(I->getOperand(opI))) {
+      auto MvIt = ClonedMVScopes.find(MV);
+      if (MvIt != ClonedMVScopes.end())
+        I->setOperand(opI, MvIt->second);
+    }
+  }
+
+  auto replaceWhenNeeded = [&](unsigned MD_ID) {
+    if (const MDNode *CSNoAlias = I->getMetadata(MD_ID)) {
+      bool needsReplacement = false;
+      SmallVector<Metadata *, 8> NewScopeList;
+      for (auto &MDOp : CSNoAlias->operands()) {
+        if (MDNode *MD = dyn_cast<MDNode>(MDOp)) {
+          auto MdIt = ClonedScopes.find(MD);
+          if (MdIt != ClonedScopes.end()) {
+            NewScopeList.push_back(MdIt->second);
+            needsReplacement = true;
+            continue;
+          }
+          NewScopeList.push_back(MD);
+        }
+      }
+      if (needsReplacement)
+        I->setMetadata(MD_ID, MDNode::get(Context, NewScopeList));
+    }
+  };
+  replaceWhenNeeded(LLVMContext::MD_noalias);
+  replaceWhenNeeded(LLVMContext::MD_alias_scope);
+}
+
+void llvm::cloneAndAdaptNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    ArrayRef<BasicBlock *> NewBlocks, LLVMContext &Context, StringRef Ext) {
+  if (NoAliasDeclScopes.empty())
+    return;
+
+  DenseMap<MDNode *, MDNode *> ClonedScopes;
+  DenseMap<MetadataAsValue *, MetadataAsValue *> ClonedMVScopes;
+  LLVM_DEBUG(llvm::dbgs() << "cloneAndAdaptNoAliasScopes: cloning "
+                          << NoAliasDeclScopes.size() << " node(s)\n");
+
+  cloneNoAliasScopes(NoAliasDeclScopes, ClonedScopes, ClonedMVScopes, Ext,
+                     Context);
+  // Identify instructions using metadata that needs adaptation
+  for (BasicBlock *NewBlock : NewBlocks) {
+    for (Instruction &I : *NewBlock) {
+      adaptNoAliasScopes(&I, ClonedScopes, ClonedMVScopes, Context);
+    }
+  }
+}
+
+void llvm::identifyNoAliasScopesToClone(
+    ArrayRef<BasicBlock *> BBs,
+    SmallVectorImpl<MetadataAsValue *> &NoAliasDeclScopes) {
+  for (BasicBlock *BB : BBs) {
+    for (Instruction &I : *BB) {
+      if (auto II = dyn_cast<IntrinsicInst>(&I)) {
+        if (II->getIntrinsicID() ==
+            Intrinsic::experimental_noalias_scope_decl) {
+          NoAliasDeclScopes.push_back(cast<MetadataAsValue>(
+              II->getOperand(Intrinsic::NoAliasScopeDeclScopeArg)));
+        }
+      }
+    }
+  }
+}

llvm/lib/Transforms/Utils/LoopUnroll.cpp

for (BasicBlock *BB : L->getBlocks())
             if (NewDIL)
               I.setDebugLoc(NewDIL.getValue());
             else
               LLVM_DEBUG(dbgs()
                          << "Failed to create new discriminator: "
                          << DIL->getFilename() << " Line: " << DIL->getLine());
           }
 
+  // Phase1: Identify what noalias metadata is inside the loop: if it is inside
+  // the loop, the associated metadata must be cloned for each iteration.
+  SmallVector<MetadataAsValue *, 6> LoopLocalNoAliasDeclScopes;
+  identifyNoAliasScopesToClone(L->getBlocks(), LoopLocalNoAliasDeclScopes);
+
   for (unsigned It = 1; It != ULO.Count; ++It) {
     SmallVector<BasicBlock *, 8> NewBlocks;
     SmallDenseMap<const Loop *, Loop *, 4> NewLoops;
     NewLoops[L] = L;
 
     for (LoopBlocksDFS::RPOIterator BB = BlockBegin; BB != BlockEnd; ++BB) {
       ValueToValueMapTy VMap;
       BasicBlock *New = CloneBasicBlock(*BB, VMap, "." + Twine(It));
for (unsigned It = 1; It != ULO.Count; ++It) {
     remapInstructionsInBlocks(NewBlocks, LastValueMap);
     for (BasicBlock *NewBlock : NewBlocks) {
       for (Instruction &I : *NewBlock) {
         if (auto *II = dyn_cast<IntrinsicInst>(&I))
           if (II->getIntrinsicID() == Intrinsic::assume)
             AC->registerAssumption(II);
       }
     }
+
+    {
+      // Identify what other metadata depends on the cloned version. After
+      // cloning, replace the metadata with the corrected version for both
+      // memory instructions and noalias intrinsics.
+      std::string ext = (Twine("It") + Twine(It)).str();
+      cloneAndAdaptNoAliasScopes(LoopLocalNoAliasDeclScopes, NewBlocks,
+                                 Header->getContext(), ext);
+    }
   }
 
   // Loop over the PHI nodes in the original block, setting incoming values.
   for (PHINode *PN : OrigPHINode) {
     if (CompletelyUnroll) {
       PN->replaceAllUsesWith(PN->getIncomingValueForBlock(Preheader));
       Header->getInstList().erase(PN);
     } else if (ULO.Count > 1) {

llvm/test/Transforms/LoopUnroll/noalias.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S -loop-unroll -unroll-count=4 < %s | FileCheck %s
+
+define void @pr39282(i32* %addr1, i32* %addr2) {
+; CHECK-LABEL: @pr39282(
+; CHECK-NEXT:  start:
+; CHECK-NEXT:    br label [[BODY:%.*]]
+; CHECK:       body:
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !0)
+; CHECK-NEXT:    [[X:%.*]] = load i32, i32* [[ADDR1:%.*]], align 4, !alias.scope !0
+; CHECK-NEXT:    store i32 [[X]], i32* [[ADDR2:%.*]], align 4, !noalias !0
+; CHECK-NEXT:    [[ADDR1I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i32 1
+; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i32 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !3)
+; CHECK-NEXT:    [[X_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !3
+; CHECK-NEXT:    store i32 [[X_1]], i32* [[ADDR2I_1]], align 4, !noalias !3
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !5)
+; CHECK-NEXT:    [[X_2:%.*]] = load i32, i32* [[ADDR1]], align 4, !alias.scope !5
+; CHECK-NEXT:    store i32 [[X_2]], i32* [[ADDR2]], align 4, !noalias !5
+; CHECK-NEXT:    [[ADDR1I_3:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i32 1
+; CHECK-NEXT:    [[ADDR2I_3:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i32 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !7)
+; CHECK-NEXT:    [[X_3:%.*]] = load i32, i32* [[ADDR1I_3]], align 4, !alias.scope !7
+; CHECK-NEXT:    store i32 [[X_3]], i32* [[ADDR2I_3]], align 4, !noalias !7
+; CHECK-NEXT:    ret void
+;
+start:
+  br label %body
+
+body:
+  %i = phi i32 [ 0, %start ], [ %i2, %body ]
+  %j = and i32 %i, 1
+  %addr1i = getelementptr inbounds i32, i32* %addr1, i32 %j
+  %addr2i = getelementptr inbounds i32, i32* %addr2, i32 %j
+
+  call void @llvm.experimental.noalias.scope.decl(metadata !2)
+  %x = load i32, i32* %addr1i, !alias.scope !2
+  store i32 %x, i32* %addr2i, !noalias !2
+
+  %i2 = add i32 %i, 1
+  %cmp = icmp slt i32 %i2, 4
+  br i1 %cmp, label %body, label %end
+
+end:
+  ret void
+}
+
+declare void @llvm.experimental.noalias.scope.decl(metadata)
+
+!0 = distinct !{!0}
+!1 = distinct !{!1, !0}
+!2 = !{!1}
+
+; CHECK: !0 = !{!1}
+; CHECK: !1 = distinct !{!1, !2}
+; CHECK: !2 = distinct !{!2}
+; CHECK: !3 = !{!4}
+; CHECK: !4 = distinct !{!4, !2, !"It1"}
+; CHECK: !5 = !{!6}
+; CHECK: !6 = distinct !{!6, !2, !"It2"}
+; CHECK: !7 = !{!8}
+; CHECK: !8 = distinct !{!8, !2, !"It3"}

llvm/test/Transforms/PhaseOrdering/pr39282.ll

 ;
   %x = load i32, i32* %from
   store i32 %x, i32* %to
   ret void
 }
 
 ; Consider that %addr1 = %addr2 + 1, in which case %addr2i and %addr1i are
 ; noalias within one iteration, but may alias across iterations.
 ; TODO: This is a micompile.
 define void @pr39282(i32* %addr1, i32* %addr2) {
 ; CHECK-LABEL: @pr39282(
 ; CHECK-NEXT:  start:
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0:metadata !.*]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3:metadata !.*]])
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
 ; CHECK-NEXT:    [[X_I:%.*]] = load i32, i32* [[ADDR1:%.*]], align 4, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    store i32 [[X_I]], i32* [[ADDR2:%.*]], align 4, !alias.scope !0, !noalias !3
 ; CHECK-NEXT:    [[ADDR1I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i64 1
-; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2:%.*]], i64 1
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    [[X_I_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !3, !noalias !0
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    store i32 [[X_I]], i32* [[ADDR2]], align 4, !alias.scope !0, !noalias !3
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    store i32 [[X_I_1]], i32* [[ADDR2I_1]], align 4, !alias.scope !0, !noalias !3
+; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i64 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !7, !noalias !5
+; CHECK-NEXT:    store i32 [[X_I_1]], i32* [[ADDR2I_1]], align 4, !alias.scope !5, !noalias !7
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_2:%.*]] = load i32, i32* [[ADDR1]], align 4, !alias.scope !11, !noalias !9
+; CHECK-NEXT:    store i32 [[X_I_2]], i32* [[ADDR2]], align 4, !alias.scope !9, !noalias !11
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_3:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !15, !noalias !13
+; CHECK-NEXT:    store i32 [[X_I_3]], i32* [[ADDR2I_1]], align 4, !alias.scope !13, !noalias !15
 ; CHECK-NEXT:    ret void
 ;
 start:
   br label %body
 
 body:
   %i = phi i32 [ 0, %start ], [ %i.next, %body ]
   %j = and i32 %i, 1
   %addr1i = getelementptr inbounds i32, i32* %addr1, i32 %j
   %addr2i = getelementptr inbounds i32, i32* %addr2, i32 %j
   call void @copy(i32* %addr2i, i32* %addr1i)
   %i.next = add i32 %i, 1
   %cmp = icmp slt i32 %i.next, 4
   br i1 %cmp, label %body, label %end
 
 end:
   ret void
 }