Add support for annotated minimum dependence distance

[ArchDRobison]

Summary

This patch enables producers of LLVM IR to tell the LoopVectorizer (and other tools) that there is a minimum dependence distance, such as with the OpenMP 4.0 "safelen" clause. See email chain http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-August/075701.html for discussion. The updates to LangRef explain the final design.

[ArchDRobison]

Can I enlist one of you to review this patch? I've updated to adjust for Renato's recent updates to hint processing..

[hfinkel]

Generally speaking, I think this is fine. You definitely need to run this through clang-format (which you can do directly on the patch file, IIRC), or manually fix the formatting to comply with the LLVM coding conventions.

However, if the main use case for this is dealing with OpenMP's safelen (and possibly other similar user-provided safety assertions), then I would like to get a resolution on the lexical dependency question (i.e. whether we really also need some ordering construct here).

Also, for future reference, please upload full-context diffs (see http://llvm.org/docs/Phabricator.html for instructions).

[ArchDRobison]

Update adjusts patch for this morning's LLVM trunk, and was cleaned up with clang-format-diff.py.

[rengolin]

Any progress in this? If not, can we close this review?

[hfinkel]

I believe that we had discussed this in a different thread, and decided that we needed to record a partial ordering in this metadata. It looks like a new patch has not yet been developed.

[ArchDRobison]

I believe that support for forward lexical dependences was the item that required partial ordering support, but was orthogonal to this patch, which addresses minimum dependence distance.

• Arch

[hfinkel]

In D5123#137651, @ArchDRobison wrote:

I believe that support for forward lexical dependences was the item that required partial ordering support, but was orthogonal to this patch, which addresses minimum dependence distance.

I was not under the impression that it is orthogonal (or it is a prerequisite). Right now, we only have a way to specify that a loop is completely data parallel (no loop carried dependencies at all), so it does not really matter. It does matter, however, if we have a minimum dependency distance, because that distance can have a validity dependence on lexical dependencies (forward and backward) that are sensitive to potential reordering.

Also, as a separate matter, the unroller, when it does partial unrolling, needs to update this metadata (to divide the distances by the unrolling factor).

[ArchDRobison]

Thanks for the reminder about the reordering issue. I'll abandon this patch.

docs/LangRef.rst

 This metadata either suggests that the loop should be unrolled fully. The
 metadata has a single operand which is the string ``llvm.loop.unroll.disable``.
 For example:
 
 .. code-block:: llvm
 
    !0 = metadata !{ metadata !"llvm.loop.unroll.full" }
 
+'``llvm.loop.minimum_dependence_distance``' Metadata
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This metadata specifies a minimum dependence distance for accesses marked
+with ``llvm.mem.parallel_loop_access``. The first operand is the string
+``llvm.loop.minimum_dependence_distance`` and the second operand is a
+positive integer specifying the minimum distance.
+For example:
+
+.. code-block:: llvm
+
+   !0 = metadata !{metadata !"llvm.loop.minimum_dependence_distance", i32 4}
+
+specifies that parallel accesses have a minimum dependence distance of 4.
+
 '``llvm.mem``'
 ^^^^^^^^^^^^^^^
 
 Metadata types used to annotate memory accesses with information helpful
 for optimizations are prefixed with ``llvm.mem``.
 
 '``llvm.mem.parallel_loop_access``' Metadata
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The ``llvm.mem.parallel_loop_access`` metadata refers to a loop identifier, 
 or metadata containing a list of loop identifiers for nested loops. 
 The metadata is attached to memory accessing instructions and denotes that 
-no loop carried memory dependence exist between it and other instructions denoted 
-with the same loop identifier.
+any loop carried memory dependence between it and other instructions denoted 
+with the same loop identifier have a minimum dependence distance of ``k``,
+where ``k`` is specified by ``llvm.mem.minimum_dependence_distance`` metadata
+on the loop.  If ``k`` is not specified by such metadata, it is positive 
+infinity.
 
 Precisely, given two instructions ``m1`` and ``m2`` that both have the 
 ``llvm.mem.parallel_loop_access`` metadata, with ``L1`` and ``L2`` being the 
 set of loops associated with that metadata, respectively, then there is no loop 
-carried dependence between ``m1`` and ``m2`` for loops in both ``L1`` and 
-``L2``.
+carried dependence with distance less than ``k1`` and ``k2`` between 
+``m1`` and ``m2`` for loops in both ``L1`` and ``L2``, where ``k1`` and ``k2``
+are the minimum distancesd specified by ``llvm.loop.minimum_dependence_distance``
+on ``L1`` and ``L2`` respectively (or the default of infinity if not specified).  
 
 As a special case, if all memory accessing instructions in a loop have 
-``llvm.mem.parallel_loop_access`` metadata that refers to that loop, then the 
-loop has no loop carried memory dependences and is considered to be a parallel 
-loop.  
+``llvm.mem.parallel_loop_access`` metadata that refers to that loop and
+``llvm.loop.minimum_dependence_distance`` is not specified for the loop,
+then the loop has no loop carried memory dependences and is considered to 
+be a parallel loop.  
 
 Note that if not all memory access instructions have such metadata referring to 
 the loop, then the loop is considered not being trivially parallel. Additional 
 memory dependence analysis is required to make that determination.  As a fail 
 safe mechanism, this causes loops that were originally parallel to be considered 
 sequential (if optimization passes that are unaware of the parallel semantics 
 insert new memory instructions into the loop body).
 

include/llvm/Analysis/LoopInfo.h

public:
   /// isLoopSimplifyForm - Return true if the Loop is in the form that
   /// the LoopSimplify form transforms loops to, which is sometimes called
   /// normal form.
   bool isLoopSimplifyForm() const;
 
   /// isSafeToClone - Return true if the loop body is safe to clone in practice.
   bool isSafeToClone() const;
 
-  /// Returns true if the loop is annotated parallel.
+  /// Returns true if the loop is annotated parallel *without* distance
+  /// restrictions.
   ///
   /// A parallel loop can be assumed to not contain any dependencies between
   /// iterations by the compiler. That is, any loop-carried dependency checking
   /// can be skipped completely when parallelizing the loop on the target
   /// machine. Thus, if the parallel loop information originates from the
   /// programmer, e.g. via the OpenMP parallel for pragma, it is the
   /// programmer's responsibility to ensure there are no loop-carried
   /// dependencies. The final execution order of the instructions across
   /// iterations is not guaranteed, thus, the end result might or might not
   /// implement actual concurrent execution of instructions across multiple
   /// iterations.
-  bool isAnnotatedParallel() const;
+  bool isAnnotatedParallel() const {
+    return getAnnotatedRestrictedParallel().second == 0;
+  };
+
+  /// Determine if the loop is annotated parallel, possibly with distance
+  /// restrictions.
+  ///
+  /// Let (p,k) denote the returned pair.
+  /// If p==false, no assumptions can be made.  k is always 1 in this case.
+  /// If p==true && k==0, then the loop can be assumed to have no
+  /// dependencies between iterations, i.e. isAnnotatedParallel()==true.
+  /// If p==true && k>0, the loop can be assumed to have no dependencies
+  /// with distance less than k between iterations.
+  std::pair<bool, unsigned> getAnnotatedRestrictedParallel() const;
 
   /// Return the llvm.loop loop id metadata node for this loop if it is present.
   ///
   /// If this loop contains the same llvm.loop metadata on each branch to the
   /// header then the node is returned. If any latch instruction does not
   /// contain llvm.loop or or if multiple latches contain different nodes then
   /// 0 is returned.
   MDNode *getLoopID() const;

lib/Analysis/LoopInfo.cpp

for (block_iterator I = block_begin(), IE = block_end(); I != IE; ++I) {
     TerminatorInst *TI = (*I)->getTerminator();
     for (unsigned i = 0, ie = TI->getNumSuccessors(); i != ie; ++i) {
       if (TI->getSuccessor(i) == H)
         TI->setMetadata(LoopMDName, LoopID);
     }
   }
 }
 
-bool Loop::isAnnotatedParallel() const {
+std::pair<bool, unsigned> Loop::getAnnotatedRestrictedParallel() const {
   MDNode *desiredLoopIdMetadata = getLoopID();
 
   if (!desiredLoopIdMetadata)
-      return false;
+    return std::make_pair(false, 1u);
 
   // The loop branch contains the parallel loop metadata. In order to ensure
   // that any parallel-loop-unaware optimization pass hasn't added loop-carried
   // dependencies (thus converted the loop back to a sequential loop), check
   // that all the memory instructions in the loop contain parallelism metadata
   // that point to the same unique "loop id metadata" the loop branch does.
   for (block_iterator BB = block_begin(), BE = block_end(); BB != BE; ++BB) {
     for (BasicBlock::iterator II = (*BB)->begin(), EE = (*BB)->end();
          II != EE; II++) {
 
       if (!II->mayReadOrWriteMemory())
         continue;
 
       // The memory instruction can refer to the loop identifier metadata
       // directly or indirectly through another list metadata (in case of
       // nested parallel loops). The loop identifier metadata refers to
       // itself so we can check both cases with the same routine.
       MDNode *loopIdMD = II->getMetadata("llvm.mem.parallel_loop_access");
 
       if (!loopIdMD)
-        return false;
+        return std::make_pair(false, 1u);
 
       bool loopIdMDFound = false;
       for (unsigned i = 0, e = loopIdMD->getNumOperands(); i < e; ++i) {
         if (loopIdMD->getOperand(i) == desiredLoopIdMetadata) {
           loopIdMDFound = true;
           break;
         }
       }
 
       if (!loopIdMDFound)
-        return false;
+        return std::make_pair(false, 1);
     }
   }
-  return true;
-}
 
+  // Check for restriction to distance d.  If present, it appears as a non-first
+  // operand of the loopId with the form
+  //   'metadata !"llvm.loop.minimum_dependence_distance", i32 d'
+  for (unsigned i = 1, ie = desiredLoopIdMetadata->getNumOperands(); i < ie;
+       ++i)
+    if (const MDNode *MD =
+            dyn_cast<MDNode>(desiredLoopIdMetadata->getOperand(i)))
+      if (MD->getNumOperands() == 2)
+        if (MDString *s = dyn_cast<MDString>(MD->getOperand(0)))
+          if (s->getString() == "llvm.loop.minimum_dependence_distance")
+            return std::make_pair(
+                true, dyn_cast<ConstantInt>(MD->getOperand(1))->getZExtValue());
+
+  // Zero is convention for unrestricted.
+  return std::make_pair(true, 0);
+}
 
 /// hasDedicatedExits - Return true if no exit block for the loop
 /// has a predecessor that is outside the loop.
 bool Loop::hasDedicatedExits() const {
   // Each predecessor of each exit block of a normal loop is contained
   // within the loop.
   SmallVector<BasicBlock *, 4> ExitBlocks;
   getExitBlocks(ExitBlocks);

lib/Transforms/Vectorize/LoopVectorize.cpp

public:
   unsigned NumStores;
   unsigned NumPredStores;
 
   LoopVectorizationLegality(Loop *L, ScalarEvolution *SE, const DataLayout *DL,
                             DominatorTree *DT, TargetLibraryInfo *TLI,
                             AliasAnalysis *AA, Function *F)
       : NumLoads(0), NumStores(0), NumPredStores(0), TheLoop(L), SE(SE), DL(DL),
         DT(DT), TLI(TLI), AA(AA), TheFunction(F), Induction(nullptr),
-        WidestIndTy(nullptr), HasFunNoNaNAttr(false), MaxSafeDepDistBytes(-1U) {
-  }
+        WidestIndTy(nullptr), HasFunNoNaNAttr(false), MaxSafeDepDistBytes(-1U),
+        AnnotatedMinDepDist(1) {}
 
   /// This enum represents the kinds of reductions that we support.
   enum ReductionKind {
     RK_NoReduction, ///< Not a reduction.
     RK_IntegerAdd,  ///< Sum of integers.
     RK_IntegerMult, ///< Product of integers.
     RK_IntegerOr,   ///< Bitwise or logical OR of numbers.
     RK_IntegerAnd,  ///< Bitwise or logical AND of numbers.
public:
   RuntimePointerCheck *getRuntimePointerCheck() { return &PtrRtCheck; }
 
   /// This function returns the identity element (or neutral element) for
   /// the operation K.
   static Constant *getReductionIdentity(ReductionKind K, Type *Tp);
 
   unsigned getMaxSafeDepDistBytes() { return MaxSafeDepDistBytes; }
 
+  void adjustForAnnotatedMinDepDist(unsigned WidestTypeBytes) {
+    if (AnnotatedMinDepDist != 1)
+      MaxSafeDepDistBytes = AnnotatedMinDepDist < -1u / WidestTypeBytes
+                                ? AnnotatedMinDepDist * WidestTypeBytes
+                                : -1u;
+  }
+
   bool hasStride(Value *V) { return StrideSet.count(V); }
   bool mustCheckStrides() { return !StrideSet.empty(); }
   SmallPtrSet<Value *, 8>::iterator strides_begin() {
     return StrideSet.begin();
   }
   SmallPtrSet<Value *, 8>::iterator strides_end() { return StrideSet.end(); }
 
 private:
private:
   SmallPtrSet<Instruction*, 4> Uniforms;
   /// We need to check that all of the pointers in this list are disjoint
   /// at runtime.
   RuntimePointerCheck PtrRtCheck;
   /// Can we assume the absence of NaNs.
   bool HasFunNoNaNAttr;
 
   unsigned MaxSafeDepDistBytes;
+  unsigned AnnotatedMinDepDist;
 
   ValueToValueMap Strides;
   SmallPtrSet<Value *, 8> StrideSet;
 };
 
 /// LoopVectorizationCostModel - estimates the expected speedups due to
 /// vectorization.
 /// In many cases vectorization is not profitable. This can happen because of

 /// values based on information in the loop.
 /// We cannot write all values to metadata, as the mere presence of some info,
 /// for example 'force', means a decision has been made. So, we need to be
 /// careful NOT to add them if the user hasn't specifically asked so.
 class LoopVectorizeHints {
   enum HintKind {
     HK_WIDTH,
     HK_UNROLL,
-    HK_FORCE
+    HK_FORCE,
+    HK_MINIMUM_DEPEENDENCE_DISTANCE
   };
 
   /// Hint - associates name and validation with the hint value.
   struct Hint {
     const char * Name;
     unsigned Value; // This may have to change for non-numeric values.
     HintKind Kind;
 
     Hint(const char * Name, unsigned Value, HintKind Kind)
       : Name(Name), Value(Value), Kind(Kind) { }
 
     bool validate(unsigned Val) {
       switch (Kind) {
       case HK_WIDTH:
         return isPowerOf2_32(Val) && Val <= MaxVectorWidth;
       case HK_UNROLL:
         return isPowerOf2_32(Val) && Val <= MaxInterleaveFactor;
       case HK_FORCE:
         return (Val <= 1);
+      case HK_MINIMUM_DEPEENDENCE_DISTANCE:
+        return Val >= 1;
       }
       return false;
     }
+
+    /// True if hint should not be written out after vectorization.
+    bool squash() const {
+      return Value == 0 && Kind == HK_MINIMUM_DEPEENDENCE_DISTANCE;
+    }
   };
 
   /// Vectorization width.
   Hint Width;
   /// Vectorization interleave factor.
   Hint Interleave;
   /// Vectorization forced
   Hint Force;
+  /// Minimum dependence distance. Technically, is not a hint, but it's handled
+  /// syntactically the same way as a hint.
+  Hint MinimumDependenceDistance;
   /// Array to help iterating through all hints.
-  Hint *Hints[3]; // avoiding initialisation due to MSVC2012
+  Hint *Hints[4]; // avoiding initialisation due to MSVC2012
 
   /// Return the loop metadata prefix.
   static StringRef Prefix() { return "llvm.loop."; }
 
 public:
   enum ForceKind {
     FK_Undefined = -1, ///< Not selected.
     FK_Disabled = 0,   ///< Forcing disabled.
     FK_Enabled = 1,    ///< Forcing enabled.
   };
 
   LoopVectorizeHints(const Loop *L, bool DisableInterleaving)
       : Width("vectorize.width", VectorizationFactor, HK_WIDTH),
         Interleave("interleave.count", DisableInterleaving, HK_UNROLL),
         Force("vectorize.enable", FK_Undefined, HK_FORCE),
+        MinimumDependenceDistance("minimum_dependence_distance", 0,
+                                  HK_MINIMUM_DEPEENDENCE_DISTANCE),
         TheLoop(L) {
     // FIXME: Move this up initialisation when MSVC requirement is 2013+
     Hints[0] = &Width;
     Hints[1] = &Interleave;
     Hints[2] = &Force;
+    Hints[3] = &MinimumDependenceDistance;
 
     // Populate values with existing loop metadata.
     getHintsFromMetadata();
 
     // force-vector-interleave overrides DisableInterleaving.
     if (VectorizationInterleave.getNumOccurrences() > 0)
       Interleave.Value = VectorizationInterleave;
 
     DEBUG(if (DisableInterleaving && Interleave.Value == 1) dbgs()
           << "LV: Interleaving disabled by the pass manager\n");
   }
 
   /// Mark the loop L as already vectorized by setting the width to 1.
   void setAlreadyVectorized() {
     Width.Value = Interleave.Value = 1;
+    // If distance was finite, conservatively assume it shrunk to 1
+    if (MinimumDependenceDistance.Value > 0)
+      MinimumDependenceDistance.Value = 1;
+
     // FIXME: Change all lines below for this when we can use MSVC 2013+
-    //writeHintsToMetadata({ Width, Unroll });
+    // writeHintsToMetadata({ Width, Unroll, MinimumDependenceDistance });
     std::vector<Hint> hints;
-    hints.reserve(2);
+    hints.reserve(3);
     hints.emplace_back(Width);
     hints.emplace_back(Interleave);
+    hints.emplace_back(MinimumDependenceDistance);
     writeHintsToMetadata(std::move(hints));
   }
 
   /// Dumps all the hint information.
   std::string emitRemark() const {
     Report R;
     if (Force.Value == LoopVectorizeHints::FK_Disabled)
       R << "vectorization is explicitly disabled";
if (LoopID) {
         // If node in update list, ignore old value.
         if (!matchesHintMetadataName(Node, HintTypes))
           Vals.push_back(Node);
       }
     }
 
     // Now, add the missing hints.
     for (auto H : HintTypes)
+      // Elide hint with value of zero.
+      if (!H.squash())
         Vals.push_back(
             createHintMetadata(Twine(Prefix(), H.Name).str(), H.Value));
 
     // Replace current metadata node with new one.
     LLVMContext &Context = TheLoop->getHeader()->getContext();
     MDNode *NewLoopID = MDNode::get(Context, Vals);
     // Set operand 0 to refer to the loop id itself.
     NewLoopID->replaceOperandWith(0, NewLoopID);
 
     TheLoop->setLoopID(NewLoopID);
bool LoopVectorizationLegality::canVectorizeMemory() {
 
   // Holds all the different accesses in the loop.
   unsigned NumReads = 0;
   unsigned NumReadWrites = 0;
 
   PtrRtCheck.Pointers.clear();
   PtrRtCheck.Need = false;
 
-  const bool IsAnnotatedParallel = TheLoop->isAnnotatedParallel();
+  const auto Annotations = TheLoop->getAnnotatedRestrictedParallel();
+  // Logic below handles fully parallel loops by treating them
+  // as having a very large dependence distance.
+  AnnotatedMinDepDist = Annotations.second ? Annotations.second : -1u;
+
+  const bool NeedDepCheck = AnnotatedMinDepDist == 1;
   MemoryDepChecker DepChecker(SE, DL, TheLoop);
 
   // For each block.
   for (Loop::block_iterator bb = TheLoop->block_begin(),
        be = TheLoop->block_end(); bb != be; ++bb) {
 
     // Scan the BB and collect legal loads and stores.
     for (BasicBlock::iterator it = (*bb)->begin(), e = (*bb)->end(); it != e;
          ++it) {
 
       // If this is a load, save it. If this instruction can read from memory
       // but is not a load, then we quit. Notice that we don't handle function
       // calls that read or write.
       if (it->mayReadFromMemory()) {
         // Many math library functions read the rounding mode. We will only
         // vectorize a loop if it contains known function calls that don't set
         // the flag. Therefore, it is safe to ignore this read from memory.
         CallInst *Call = dyn_cast<CallInst>(it);
         if (Call && getIntrinsicIDForCall(Call, TLI))
           continue;
 
         LoadInst *Ld = dyn_cast<LoadInst>(it);
-        if (!Ld || (!Ld->isSimple() && !IsAnnotatedParallel)) {
+        if (!Ld || (!Ld->isSimple() && NeedDepCheck)) {
           emitAnalysis(Report(Ld)
                        << "read with atomic ordering or volatile read");
           DEBUG(dbgs() << "LV: Found a non-simple load.\n");
           return false;
         }
         NumLoads++;
         Loads.push_back(Ld);
         DepChecker.addAccess(Ld);
         continue;
       }
 
       // Save 'store' instructions. Abort if other instructions write to memory.
       if (it->mayWriteToMemory()) {
         StoreInst *St = dyn_cast<StoreInst>(it);
         if (!St) {
           emitAnalysis(Report(it) << "instruction cannot be vectorized");
           return false;
         }
-        if (!St->isSimple() && !IsAnnotatedParallel) {
+        if (!St->isSimple() && NeedDepCheck) {
           emitAnalysis(Report(St)
                        << "write with atomic ordering or volatile write");
           DEBUG(dbgs() << "LV: Found a non-simple store.\n");
           return false;
         }
         NumStores++;
         Stores.push_back(St);
         DepChecker.addAccess(St);
if (Seen.insert(Ptr)) {
       // need runtime pointer checks.
       if (blockNeedsPredication(ST->getParent()))
         Loc.AATags.TBAA = nullptr;
 
       Accesses.addStore(Loc);
     }
   }
 
-  if (IsAnnotatedParallel) {
-    DEBUG(dbgs()
-          << "LV: A loop annotated parallel, ignore memory dependency "
-          << "checks.\n");
+  if (!NeedDepCheck) {
+    DEBUG(dbgs() << "LV: A loop annotated with min dependence distance "
+                 << AnnotatedMinDepDist
+                 << ", ignore memory dependency checks.\n");
     return true;
   }
 
   for (I = Loads.begin(), IE = Loads.end(); I != IE; ++I) {
     LoadInst *LD = cast<LoadInst>(*I);
     Value* Ptr = LD->getPointerOperand();
     // If we did *not* see this pointer before, insert it to the
     // read list. If we *did* see it before, then it is already in
LoopVectorizationCostModel::selectVectorizationFactor(bool OptForSize) {
 
   // Find the trip count.
   unsigned TC = SE->getSmallConstantTripCount(TheLoop, TheLoop->getLoopLatch());
   DEBUG(dbgs() << "LV: Found trip count: " << TC << '\n');
 
   unsigned WidestType = getWidestType();
   unsigned WidestRegister = TTI.getRegisterBitWidth(true);
   unsigned MaxSafeDepDist = -1U;
+  Legal->adjustForAnnotatedMinDepDist(WidestType / 8);
   if (Legal->getMaxSafeDepDistBytes() != -1U)
     MaxSafeDepDist = Legal->getMaxSafeDepDistBytes() * 8;
-  WidestRegister = ((WidestRegister < MaxSafeDepDist) ?
-                    WidestRegister : MaxSafeDepDist);
+  WidestRegister =
+      ((WidestRegister < MaxSafeDepDist) ? WidestRegister : MaxSafeDepDist);
   unsigned MaxVectorSize = WidestRegister / WidestType;
   DEBUG(dbgs() << "LV: The Widest type: " << WidestType << " bits.\n");
   DEBUG(dbgs() << "LV: The Widest register is: "
           << WidestRegister << " bits.\n");
 
   if (MaxVectorSize == 0) {
     DEBUG(dbgs() << "LV: The target has no vector registers.\n");
     MaxVectorSize = 1;

test/Transforms/LoopVectorize/X86/min-dep-dist.ll

+; RUN: opt < %s  -loop-vectorize -mcpu=corei7-avx -S | FileCheck %s
+; CHECK-NOT: memcheck
+; CHECK-NOT: <8 x float
+; CHECK-NOT: <4 x float
+; CHECK: <2 x float
+; CHECK-NOT: minimum_dependence_distance", i32 2}
+ 
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; Simple routine that would be vectorized wider than 2
+; if llvm.loop.minimum_dependence_distance were not present.
+define void @foo(float* %a, float* %b) {
+entry:
+  br label %for.body
+
+for.body:
+  %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
+  %i = getelementptr inbounds float* %a, i64 %indvars.iv
+  %t = load float* %i, align 4, !llvm.mem.parallel_loop_access !0
+  %mul = fmul float %t, %t
+  %j = getelementptr inbounds float* %b, i64 %indvars.iv
+  store float %mul, float* %j, align 4, !llvm.mem.parallel_loop_access !0
+  %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
+  %exitcond = icmp eq i64 %indvars.iv.next, 1024
+  br i1 %exitcond, label %for.end, label %for.body, !llvm.loop !0
+
+for.end:
+  ret void
+}
+
+!0 = metadata !{metadata !0, metadata !1}
+!1 = metadata !{metadata !"llvm.loop.minimum_dependence_distance", i32 2}