[LoopUnroll] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed

[jeroen.dobbelaere]

Summary

This is a fix for https://bugs.llvm.org/show_bug.cgi?id=39282. Compared to D90104, this version is based on part of the full restrict patched (D68484) and uses the @llvm.experimental.noalias.scope.decl intrinsic to track the location where !noalias and !alias.scope scopes have been introduced. This allows us to only duplicate the scopes that are really needed.

Notes:

• it also includes changes and tests from D90104

[nikic]

I'd suggest to split this up into (at least) three parts:

• LangRef patch (not present yet here) and the addition of the intrinsic.
• Teaching everything relevant to look past the intrinsic and emitting it in inliner.
• Making use of it in loop unrolling.

This will make it easier to identify regressions caused by the extra intrinsic blocking some transform, because of missing special handling or similar.

[jeroen.dobbelaere]

In D92887#2442047, @nikic wrote:

I'd suggest to split this up into (at least) three parts:

• LangRef patch (not present yet here) and the addition of the intrinsic.
• Teaching everything relevant to look past the intrinsic and emitting it in inliner.
• Making use of it in loop unrolling.

This will make it easier to identify regressions caused by the extra intrinsic blocking some transform, because of missing special handling or similar.

Yes, that makes sense. I hope to have an update tomorrow.

[jeroen.dobbelaere]

Rebased and adapted to new naming

[nikic]

The logic here looks sounds to me.

llvm/include/llvm/Transforms/Utils/Cloning.h

295–297

300

I don't think we use this kind of out_ style anywhere.

llvm/lib/Transforms/Utils/CloneFunction.cpp

1031

nit: No braces here.

1040

Can operands be null?

1052

nit: No braces here.

1076

nit: No braces here.

1083

nit: auto * or BasicBlock *.

llvm/lib/Transforms/Utils/InlineFunction.cpp

86

These changes should have probably ended up in D93040.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

591

llvm:: is presumably unnecessary here.

691

With the implementation factored out the "phase 2" and "phase 3" comments look a bit out of place.

694

Possible the ext parameter should be Twine rather than StringRef. But I'm not really familiar with our string API conventions.

[jeroen.dobbelaere]

llvm/include/llvm/Transforms/Utils/Cloning.h

300

I personally like the the out_/inout_/in_ style for making the purpose clear. But I understand that in the LLVM context it is better to have a common style.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

694

I would rather keep it. When using a Twine, the Twine will be evaluated every time a string is needed.
By using a StringRef, at least that part will only be done once.

[jeroen.dobbelaere]

• Update according to comments

[jeroen.dobbelaere]

• removed more unnecessary braces

[jeroen.dobbelaere]

Rebased.
Changed comment to indicate that, while scopes are duplicated, the domain is kept.

[nikic]

LGTM

llvm/include/llvm/Transforms/Utils/Cloning.h

272

nit: "back" is too much here.

302

nit: Missing period at end of sentence.

llvm/lib/Transforms/Utils/CloneFunction.cpp

901

nit: Presumably unnecessary llvm:: prefix.

927

I believe you should be able to write this as for (Use &U : I->operands()) and then use U.set() to replace the operand.

930

Should be possible to use something like if (MetadataAsValue *NewMV = ClonedMVScopes.lookup(MV)) here.

937

nit: Should be NeedsReplacement.

942

Here again, something like MDNode *NewMD = ClonedScopes.lookup(MD)) should be possible.

llvm/lib/Transforms/Utils/LoopUnroll.cpp

588

nit: The "Phase1" here no longer makes sense (was dropped below).

llvm/test/Transforms/LoopUnroll/noalias.ll

63

It might be worthwhile to also add a test with noalias.scope.decl outside the loop, that shows that no scopes are cloned in that case.

[nikic]

llvm/test/Transforms/PhaseOrdering/pr39282.ll

18

This TODO should be dropped :)

[jeroen.dobbelaere]

Adapted to comments.

[asbirlea]

My concern with this patch is in the iteration on all instructions in the blocks in cloneAndAdaptNoAliasScopes and identifyNoAliasScopesToClone.
Perhaps for LoopUnroll is not an issue, if unroll cost model is taken into account before this (i.e. unroll doesn't happen for large BBs). But when doing loop rotate, this may explode for IRs with large BBs, so I'd say some measurements would be helpful here.
For loop rotate, which instructions are cloned from the header to the preheader is known at the callsite, so as to not iterate over the entire original preheader, but I'm not sure this is sufficient.

llvm/test/Transforms/PhaseOrdering/pr39282.ll

57

Nit: CHECKs for MDNodes?

[jeroen.dobbelaere]

Make use of NoAliasScopeDeclInst

llvm/include/llvm/Transforms/Utils/Cloning.h

 
 /// Updates profile information by adjusting the entry count by adding
 /// entryDelta then scaling callsite information by the new count divided by the
 /// old count. VMap is used during inlinng to also update the new clone
 void updateProfileCallee(
     Function *Callee, int64_t entryDelta,
     const ValueMap<const Value *, WeakTrackingVH> *VMap = nullptr);
 
 /// Utility for cloning !noalias and !alias.scope metadata. When a code region
 /// using scoped alias metadata is cloned, the aliasing relationships may not
 /// hold between the two clones, in which case it is necessary to clone the
 /// metadata using this utility. This comes up with inlining and unrolling.
 class ScopedAliasMetadataCloner {
   using MetadataMap = DenseMap<const MDNode *, TrackingMDNodeRef>;
   SetVector<const MDNode *> MD;
   MetadataMap Map;
   void addRecursiveMetadataUses();
 
 public:
   ScopedAliasMetadataCloner(ArrayRef<BasicBlock *> Blocks);
   ScopedAliasMetadataCloner(const Function *F);
 
   /// Create a new clone of the scoped alias metadata, which will be used by
   /// subsequent remap() calls.
   void clone();
 
   /// Remap instructions in the given VMap from the original to the cloned
   /// metadata.
   void remap(ValueToValueMapTy &VMap);
 };
+
+/// Find back the 'llvm.experimental.noalias.scope.decl' intrinsics in the
+/// specified basic blocks and extract their scope. This are candidates for
+/// duplication when cloning.
+void identifyNoAliasScopesToClone(
+    ArrayRef<BasicBlock *> BBs,
+    SmallVectorImpl<MetadataAsValue *> &out_NoAliasDeclScopes);
+
+/// Duplicate the specified list of noalias decl scopes.
+/// The 'Ext' string is added as an extension to the name.
+/// Afterwards, the out_ClonedMVScopes contains a mapping of the original MV
+/// onto the cloned version.
+/// The out_ClonedScopes contains the mapping of the original scope MDNode
+/// onto the cloned scope.
+void cloneNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    DenseMap<MDNode *, MDNode *> &out_ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &out_ClonedMVScopes,
+    StringRef Ext, LLVMContext &Context);
+
+/// Adapt the metadata for the specified instruction according to the
+/// provided mapping. This is normally used after cloning an instruction, when
+/// some noalias scopes needed to be cloned.
+void adaptNoAliasScopes(
+    Instruction *I, DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    LLVMContext &Context);
+
+/// Clone the specified noalias decl scopes. Then adapt all instructions in the
+/// NewBlocks basicblocks to the cloned versions.
+/// 'Ext' will be added to the duplicate scope names
+void cloneAndAdaptNoAliasScopes(ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+                                ArrayRef<BasicBlock *> NewBlocks,
+                                LLVMContext &Context, StringRef Ext);
 } // end namespace llvm
+
 #endif // LLVM_TRANSFORMS_UTILS_CLONING_H

llvm/lib/Transforms/Utils/CloneFunction.cpp

for (BasicBlock *BB : Blocks) {
     for (const Instruction &I : *BB) {
       if (const MDNode *M = I.getMetadata(LLVMContext::MD_alias_scope))
         MD.insert(M);
       if (const MDNode *M = I.getMetadata(LLVMContext::MD_noalias))
         MD.insert(M);
     }
   }
   addRecursiveMetadataUses();
 }
 
 ScopedAliasMetadataCloner::ScopedAliasMetadataCloner(const Function *F) {
   for (const BasicBlock &BB : *F) {
     for (const Instruction &I : BB) {
       if (const MDNode *M = I.getMetadata(LLVMContext::MD_alias_scope))
         MD.insert(M);
       if (const MDNode *M = I.getMetadata(LLVMContext::MD_noalias))
         MD.insert(M);
for (const BasicBlock &BB : *F) {
     }
   }
   addRecursiveMetadataUses();
 }
 
 void ScopedAliasMetadataCloner::addRecursiveMetadataUses() {
   SmallVector<const Metadata *, 16> Queue(MD.begin(), MD.end());
   while (!Queue.empty()) {
     const MDNode *M = cast<MDNode>(Queue.pop_back_val());
     for (const Metadata *Op : M->operands())
       if (const MDNode *OpMD = dyn_cast<MDNode>(Op))
         if (MD.insert(OpMD))
           Queue.push_back(OpMD);
   }
 }
 
 void ScopedAliasMetadataCloner::clone() {
   // Discard a previous clone that may exist.
   Map.clear();
 
   SmallVector<TempMDTuple, 16> DummyNodes;
   for (const MDNode *I : MD) {
     DummyNodes.push_back(MDTuple::getTemporary(I->getContext(), None));
     Map[I].reset(DummyNodes.back().get());
   }
 
   // Create new metadata nodes to replace the dummy nodes, replacing old
   // metadata references with either a dummy node or an already-created new
   // node.
   SmallVector<Metadata *, 4> NewOps;
   for (const MDNode *I : MD) {
     for (const Metadata *Op : I->operands()) {
for (auto Entry : VMap) {
 
     if (MDNode *M = I->getMetadata(LLVMContext::MD_alias_scope))
       I->setMetadata(LLVMContext::MD_alias_scope, Map[M]);
 
     if (MDNode *M = I->getMetadata(LLVMContext::MD_noalias))
       I->setMetadata(LLVMContext::MD_noalias, Map[M]);
   }
 }
+
+void llvm::cloneNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    DenseMap<MDNode *, MDNode *> &out_ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &out_ClonedMVScopes,
+    StringRef Ext, LLVMContext &Context) {
+  MDBuilder MDB(Context);
+
+  for (auto *MV : NoAliasDeclScopes) {
+    SmallVector<Metadata *, 4> ScopeList;
+    for (auto &MDOperand : cast<MDNode>(MV->getMetadata())->operands()) {
+      if (MDNode *MD = dyn_cast<MDNode>(MDOperand)) {
+        llvm::AliasScopeNode SNANode(MD);
+
+        std::string Name;
+        auto ScopeName = SNANode.getName();
+        if (!ScopeName.empty()) {
+          Name = (Twine(ScopeName) + ":" + Ext).str();
+        } else {
+          Name = std::string(Ext);
+        }
+
+        MDNode *NewScope = MDB.createAnonymousAliasScope(
+            const_cast<MDNode *>(SNANode.getDomain()), Name);
+        out_ClonedScopes.insert(std::make_pair(MD, NewScope));
+        ScopeList.push_back(NewScope);
+      }
+    }
+    MDNode *NewScopeList = MDNode::get(Context, ScopeList);
+    out_ClonedMVScopes.insert(
+        std::make_pair(MV, MetadataAsValue::get(Context, NewScopeList)));
+  }
+}
+
+void llvm::adaptNoAliasScopes(
+    Instruction *I, DenseMap<MDNode *, MDNode *> &ClonedScopes,
+    DenseMap<MetadataAsValue *, MetadataAsValue *> &ClonedMVScopes,
+    LLVMContext &Context) {
+  // MetadataAsValue will always be replaced !
+  for (int opI = 0, opIEnd = I->getNumOperands(); opI < opIEnd; ++opI) {
+    if (MetadataAsValue *MV = dyn_cast<MetadataAsValue>(I->getOperand(opI))) {
+      auto MvIt = ClonedMVScopes.find(MV);
+      if (MvIt != ClonedMVScopes.end()) {
+        I->setOperand(opI, MvIt->second);
+      }
+    }
+  }
+
+  auto replaceWhenNeeded = [&](unsigned MD_ID) {
+    if (const MDNode *CSNoAlias = I->getMetadata(MD_ID)) {
+      bool needsReplacement = false;
+      SmallVector<Metadata *, 8> NewScopeList;
+      for (auto &MDOp : CSNoAlias->operands()) {
+        if (MDNode *MD = dyn_cast_or_null<MDNode>(MDOp)) {
+          auto MdIt = ClonedScopes.find(MD);
+          if (MdIt != ClonedScopes.end()) {
+            NewScopeList.push_back(MdIt->second);
+            needsReplacement = true;
+            continue;
+          }
+          NewScopeList.push_back(MD);
+        }
+      }
+      if (needsReplacement) {
+        I->setMetadata(MD_ID, MDNode::get(Context, NewScopeList));
+      }
+    }
+  };
+  replaceWhenNeeded(LLVMContext::MD_noalias);
+  replaceWhenNeeded(LLVMContext::MD_alias_scope);
+}
+
+void llvm::cloneAndAdaptNoAliasScopes(
+    ArrayRef<MetadataAsValue *> NoAliasDeclScopes,
+    ArrayRef<BasicBlock *> NewBlocks, LLVMContext &Context, StringRef Ext) {
+  if (NoAliasDeclScopes.empty())
+    return;
+
+  DenseMap<MDNode *, MDNode *> ClonedScopes;
+  DenseMap<MetadataAsValue *, MetadataAsValue *> ClonedMVScopes;
+  LLVM_DEBUG(llvm::dbgs() << "cloneAndAdaptNoAliasScopes: cloning "
+                          << NoAliasDeclScopes.size() << " node(s)\n");
+
+  cloneNoAliasScopes(NoAliasDeclScopes, ClonedScopes, ClonedMVScopes, Ext,
+                     Context);
+  // Identify instructions using metadata that needs adaptation
+  for (BasicBlock *NewBlock : NewBlocks) {
+    for (Instruction &I : *NewBlock) {
+      adaptNoAliasScopes(&I, ClonedScopes, ClonedMVScopes, Context);
+    }
+  }
+}
+
+void llvm::identifyNoAliasScopesToClone(
+    ArrayRef<BasicBlock *> BBs,
+    SmallVectorImpl<MetadataAsValue *> &out_NoAliasDeclScopes) {
+  for (auto BB : BBs) {
+    for (Instruction &I : *BB) {
+      if (auto II = dyn_cast<IntrinsicInst>(&I)) {
+        if (II->getIntrinsicID() ==
+            Intrinsic::experimental_noalias_scope_decl) {
+          out_NoAliasDeclScopes.push_back(cast<MetadataAsValue>(
+              II->getOperand(Intrinsic::NoAliasScopeDeclScopeArg)));
+        }
+      }
+    }
+  }
+}

llvm/lib/Transforms/Utils/InlineFunction.cpp

 using namespace llvm;
 using ProfileCount = Function::ProfileCount;
 
 static cl::opt<bool>
 EnableNoAliasConversion("enable-noalias-to-md-conversion", cl::init(true),
   cl::Hidden,
   cl::desc("Convert noalias attributes to metadata during inlining."));
 
-static cl::opt<bool> UseNoAliasIntrinsic(
-    "use-noalias-intrinsic-during-inlining", cl::Hidden, cl::ZeroOrMore,
-    cl::init(true),
-    cl::desc("Use the llvm.noalias.decl intrinsic during inlining."));
+static cl::opt<bool>
+    UseNoAliasIntrinsic("use-noalias-intrinsic-during-inlining", cl::Hidden,
+                        cl::ZeroOrMore, cl::init(true),
+                        cl::desc("Use the llvm.experimental.noalias.scope.decl "
+                                 "intrinsic during inlining."));
 
 // Disabled by default, because the added alignment assumptions may increase
 // compile-time and block optimizations. This option is not suitable for use
 // with frontends that emit comprehensive parameter alignment annotations.
 static cl::opt<bool>
 PreserveAlignmentAssumptions("preserve-alignment-assumptions-during-inlining",
   cl::init(false), cl::Hidden,
   cl::desc("Convert align attributes to assumptions during inlining."));
for (unsigned i = 0, e = NoAliasArgs.size(); i != e; ++i) {
 
     // Note: We always create a new anonymous root here. This is true regardless
     // of the linkage of the callee because the aliasing "scope" is not just a
     // property of the callee, but also all control dependencies in the caller.
     MDNode *NewScope = MDB.createAnonymousAliasScope(NewDomain, Name);
     NewScopes.insert(std::make_pair(A, NewScope));
 
     if (UseNoAliasIntrinsic) {
-      // Introduce a llvm.noalias.decl for the noalias argument.
+      // Introduce a llvm.experimental.noalias.scope.decl for the noalias
+      // argument.
       MDNode *AScopeList = MDNode::get(CalledFunc->getContext(), NewScope);
       auto *NoAliasDecl =
           IRBuilder<>(&CB).CreateNoAliasScopeDeclaration(AScopeList);
       // Ignore the result for now. The result will be used when the
       // llvm.noalias intrinsic is introduced.
       (void)NoAliasDecl;
     }
   }

llvm/lib/Transforms/Utils/LoopUnroll.cpp

for (BasicBlock *BB : L->getBlocks())
             if (NewDIL)
               I.setDebugLoc(NewDIL.getValue());
             else
               LLVM_DEBUG(dbgs()
                          << "Failed to create new discriminator: "
                          << DIL->getFilename() << " Line: " << DIL->getLine());
           }
 
+  // Phase1: Identify what noalias metadata is inside the loop: if it is inside
+  // the loop, the associated metadata must be cloned for each iteration.
+  SmallVector<MetadataAsValue *, 6> LoopLocalNoAliasDeclScopes;
+  llvm::identifyNoAliasScopesToClone(L->getBlocks(),
+                                     LoopLocalNoAliasDeclScopes);
+
   for (unsigned It = 1; It != ULO.Count; ++It) {
     SmallVector<BasicBlock *, 8> NewBlocks;
     SmallDenseMap<const Loop *, Loop *, 4> NewLoops;
     NewLoops[L] = L;
 
     for (LoopBlocksDFS::RPOIterator BB = BlockBegin; BB != BlockEnd; ++BB) {
       ValueToValueMapTy VMap;
       BasicBlock *New = CloneBasicBlock(*BB, VMap, "." + Twine(It));
for (unsigned It = 1; It != ULO.Count; ++It) {
     remapInstructionsInBlocks(NewBlocks, LastValueMap);
     for (BasicBlock *NewBlock : NewBlocks) {
       for (Instruction &I : *NewBlock) {
         if (auto *II = dyn_cast<IntrinsicInst>(&I))
           if (II->getIntrinsicID() == Intrinsic::assume)
             AC->registerAssumption(II);
       }
     }
+
+    {
+      // Phase2: identify what other metadata depends on the cloned version
+      // Phase3: after cloning, replace the metadata with the corrected version
+      // for both memory instructions and noalias intrinsics
+      std::string ext = (Twine("It") + Twine(It)).str();
+      cloneAndAdaptNoAliasScopes(LoopLocalNoAliasDeclScopes, NewBlocks,
+                                 Header->getContext(), ext);
+    }
   }
 
   // Loop over the PHI nodes in the original block, setting incoming values.
   for (PHINode *PN : OrigPHINode) {
     if (CompletelyUnroll) {
       PN->replaceAllUsesWith(PN->getIncomingValueForBlock(Preheader));
       Header->getInstList().erase(PN);
     } else if (ULO.Count > 1) {

llvm/test/Transforms/LoopUnroll/noalias.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -S -loop-unroll -unroll-count=4 < %s | FileCheck %s
+
+define void @pr39282(i32* %addr1, i32* %addr2) {
+; CHECK-LABEL: @pr39282(
+; CHECK-NEXT:  start:
+; CHECK-NEXT:    br label [[BODY:%.*]]
+; CHECK:       body:
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !0)
+; CHECK-NEXT:    [[X:%.*]] = load i32, i32* [[ADDR1:%.*]], align 4, !alias.scope !0
+; CHECK-NEXT:    store i32 [[X]], i32* [[ADDR2:%.*]], align 4, !noalias !0
+; CHECK-NEXT:    [[ADDR1I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i32 1
+; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i32 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !3)
+; CHECK-NEXT:    [[X_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !3
+; CHECK-NEXT:    store i32 [[X_1]], i32* [[ADDR2I_1]], align 4, !noalias !3
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !5)
+; CHECK-NEXT:    [[X_2:%.*]] = load i32, i32* [[ADDR1]], align 4, !alias.scope !5
+; CHECK-NEXT:    store i32 [[X_2]], i32* [[ADDR2]], align 4, !noalias !5
+; CHECK-NEXT:    [[ADDR1I_3:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i32 1
+; CHECK-NEXT:    [[ADDR2I_3:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i32 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl(metadata !7)
+; CHECK-NEXT:    [[X_3:%.*]] = load i32, i32* [[ADDR1I_3]], align 4, !alias.scope !7
+; CHECK-NEXT:    store i32 [[X_3]], i32* [[ADDR2I_3]], align 4, !noalias !7
+; CHECK-NEXT:    ret void
+;
+start:
+  br label %body
+
+body:
+  %i = phi i32 [ 0, %start ], [ %i2, %body ]
+  %j = and i32 %i, 1
+  %addr1i = getelementptr inbounds i32, i32* %addr1, i32 %j
+  %addr2i = getelementptr inbounds i32, i32* %addr2, i32 %j
+
+  call void @llvm.experimental.noalias.scope.decl(metadata !2)
+  %x = load i32, i32* %addr1i, !alias.scope !2
+  store i32 %x, i32* %addr2i, !noalias !2
+
+  %i2 = add i32 %i, 1
+  %cmp = icmp slt i32 %i2, 4
+  br i1 %cmp, label %body, label %end
+
+end:
+  ret void
+}
+
+declare void @llvm.experimental.noalias.scope.decl(metadata)
+
+!0 = distinct !{!0}
+!1 = distinct !{!1, !0}
+!2 = !{!1}
+
+; CHECK: !0 = !{!1}
+; CHECK: !1 = distinct !{!1, !2}
+; CHECK: !2 = distinct !{!2}
+; CHECK: !3 = !{!4}
+; CHECK: !4 = distinct !{!4, !2, !"It1"}
+; CHECK: !5 = !{!6}
+; CHECK: !6 = distinct !{!6, !2, !"It2"}
+; CHECK: !7 = !{!8}
+; CHECK: !8 = distinct !{!8, !2, !"It3"}

llvm/test/Transforms/PhaseOrdering/pr39282.ll

 ;
   %x = load i32, i32* %from
   store i32 %x, i32* %to
   ret void
 }
 
 ; Consider that %addr1 = %addr2 + 1, in which case %addr2i and %addr1i are
 ; noalias within one iteration, but may alias across iterations.
 ; TODO: This is a micompile.
 define void @pr39282(i32* %addr1, i32* %addr2) {
 ; CHECK-LABEL: @pr39282(
 ; CHECK-NEXT:  start:
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0:metadata !.*]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3:metadata !.*]])
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
 ; CHECK-NEXT:    [[X_I:%.*]] = load i32, i32* [[ADDR1:%.*]], align 4, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    store i32 [[X_I]], i32* [[ADDR2:%.*]], align 4, !alias.scope !0, !noalias !3
 ; CHECK-NEXT:    [[ADDR1I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR1]], i64 1
-; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2:%.*]], i64 1
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    [[X_I_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !3, !noalias !0
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    store i32 [[X_I]], i32* [[ADDR2]], align 4, !alias.scope !0, !noalias !3
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META0]])
-; CHECK-NEXT:    tail call void @llvm.experimental.noalias.scope.decl([[META3]])
-; CHECK-NEXT:    store i32 [[X_I_1]], i32* [[ADDR2I_1]], align 4, !alias.scope !0, !noalias !3
+; CHECK-NEXT:    [[ADDR2I_1:%.*]] = getelementptr inbounds i32, i32* [[ADDR2]], i64 1
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_1:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !7, !noalias !5
+; CHECK-NEXT:    store i32 [[X_I_1]], i32* [[ADDR2I_1]], align 4, !alias.scope !5, !noalias !7
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_2:%.*]] = load i32, i32* [[ADDR1]], align 4, !alias.scope !11, !noalias !9
+; CHECK-NEXT:    store i32 [[X_I_2]], i32* [[ADDR2]], align 4, !alias.scope !9, !noalias !11
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    call void @llvm.experimental.noalias.scope.decl
+; CHECK-NEXT:    [[X_I_3:%.*]] = load i32, i32* [[ADDR1I_1]], align 4, !alias.scope !15, !noalias !13
+; CHECK-NEXT:    store i32 [[X_I_3]], i32* [[ADDR2I_1]], align 4, !alias.scope !13, !noalias !15
 ; CHECK-NEXT:    ret void
 ;
 start:
   br label %body
 
 body:
   %i = phi i32 [ 0, %start ], [ %i.next, %body ]
   %j = and i32 %i, 1
   %addr1i = getelementptr inbounds i32, i32* %addr1, i32 %j
   %addr2i = getelementptr inbounds i32, i32* %addr2, i32 %j
   call void @copy(i32* %addr2i, i32* %addr1i)
   %i.next = add i32 %i, 1
   %cmp = icmp slt i32 %i.next, 4
   br i1 %cmp, label %body, label %end
 
 end:
   ret void
 }