[LAA] Function 'isStridedPtr' returns additional result “Loop *Lp” via function argument and add appropriate checks out of the 'isStridedPtr'.

[roman.shirokiy]

Summary

The patch fixes PR26314. Before rL257134 there was in fact no restriction for pointer access function to be strided over innermost loop. This fix mostly keeps rL257134 logic, but relaxes "stride-over-innermost loop" condition in "AccessAnalysis::canCheckPtrAtRT" function. It allows vectorization in the case described in PR26314.

[Ayal]

Looks ok to me; leaving for Hal or Adam to formally LGTMize.

include/llvm/Analysis/LoopAccessAnalysis.h

655–661

Preds? (Admittedly not part of your patch)

Suggest to start by describing that the function "Returns the stride of \p Ptr or zero if \p Ptr is not strided". Then add that "\p Lp is an output parameter, expected to be null on entry; if \Ptr is strided across loop L, \Lp is set to L."

If \Ptr is strided across multiple loops (as in PR26314), \Lp is set to the innermost of them, right?

Alternatively isStridedPtr() can return Lp and set the Stride as an output parameter. That however would require two checks for consecutivity.

It might have been clearer to call this getStrideOfPtr().

lib/Analysis/LoopAccessAnalysis.cpp

586

"&& Lp != nullptr &&" >> "&& Lp &&"

861–871

func. >> function

test/Analysis/LoopAccessAnalysis/stride-vectorization.ll

3 ↗

(On Diff #47987)

better check for a vector type?

[anemet]

lib/Transforms/Vectorize/LoopVectorize.cpp

5003–5004

Rather than complicating the interface of isStridedPtr, we should just check ScalarEvoluiton::isLoopInvariant on the SCEV of Ptr here.

test/Analysis/LoopAccessAnalysis/stride-vectorization.ll

1 ↗

(On Diff #47987)

Please don't put vectorization-only tests under LAA. Those should go under Transforms/LoopVectorize.

You *should* have a test under LAA but it should then check the result of the analysis. See some examples in the directory.

There are ways to combine a single test to get tested both for vectorization and LAA. There should be examples in the LAA directory.

3 ↗

(On Diff #47987)

+1

[Ayal]

lib/Analysis/LoopAccessAnalysis.cpp

586

Can you indeed check if ScalarEvolution::isLoopInvariant on the SCEV of Ptr here wrt TheLoop, as Adam suggested below, instead of checking if Lp->contains(TheLoop)?

[anemet]

lib/Analysis/LoopAccessAnalysis.cpp

586

I don't think this is needed after rL264243.

[roman.shirokiy]

Ayal, Adam, thanks for the review!

Please, excuse me for messing up with the function signature, but for this time I've again considered it to be the good idea, since the restriction on loop being the innermost one seem to be redundant for "isStridedPtr".

If I understood it right, "SCEVAddRecExpr::getLoop", which is called for the SCEV of the pointer with access striding over some loop, should return this particular loop, and for the "isStridedPtr" function it is correct to compute pointer stride over this loop.

I've removed "const Loop *Lp" argument from "isStridedPtr", so the "loop-strided-over" is obtained inside the function. Now there is "ScalarEvolution::isLoopInvariant" check, where it is necessary to get stride value regarding specific (innermost) loop: stride is considered to be zero for invariant pointer access.

Updated tests, now there is one for the analysis results and one for the vectorization.

Also, change of "isStridedPtr" signature made me stumble upon rL263058. With this patch it is unnecessary to pass "Loop* L" through functions affected by rL263058, but there is also suspicious condition "isStridedPtr(PSE, LoadPtr, L) != 1 || isStridedPtr(PSE, LoadPtr, L) != 1", which looks like typo (or relying on side-effects?).

LLVM build with this patch passes LIT-testing.

[hfinkel]

include/llvm/Analysis/LoopAccessAnalysis.h

655–661

It might have been clearer to call this getStrideOfPtr().

I agree, and given that this change touches all calls anyway, let's rename it to make it more clear. How about, getPtrStride()?

lib/Analysis/LoopAccessAnalysis.cpp

1623

This use of isStridedPtr also needs a loop check. If the pointer is strided w.r.t. an outer loop only, then it is not consecutive.

lib/Transforms/Scalar/LoopLoadElimination.cpp

80–81

Loop check here. If a pointer is strided only w.r.t. an outer loop, it is not unit stride here.

lib/Transforms/Vectorize/LoopVectorize.cpp

5003–5004

Maybe we should have an extra overload of the function that takes a loop to check? I think that's cleaner than forcing the caller to repeat PSE.getSE()->isLoopInvariant in each case.

[roman.shirokiy]

Thanks Hal for the review!

Updated diff according to Hal's corrections. However, extra overload for this function turns up as a too much of repetitive code for the sake of few different lines, but approach with Lp being nullptr seems to be clean and allows the needed change for PR26314.

[roman.shirokiy]

Marked comments as done.

[anemet]

In D17268#400137, @roman.shirokiy wrote:

Also, change of "isStridedPtr" signature made me stumble upon rL263058. With this patch it is unnecessary to pass "Loop* L" through functions affected by rL263058, but there is also suspicious condition "isStridedPtr(PSE, LoadPtr, L) != 1 || isStridedPtr(PSE, LoadPtr, L) != 1", which looks like typo (or relying on side-effects?).

Thanks for noticing. One of them was supposed to check StorePtr. Fixed in r268251.

[hfinkel]

LGTM

[anemet]

Sorry for not commenting on this earlier but I didn't find the time.

I am not sure what the patch does anymore. Looks like there is a single call site were we pass in nullptr for the loop and that is the place where we should already accept loop-invariant pointers after http://reviews.llvm.org/rL264243.

Can you please summarize?

I also think that if you want to rename the function you should probably do that in a separate patch. BTW, I agree with the renaming.

Thanks,
Adam

[roman.shirokiy]

Thanks for the review!

I'll try to describe proposed change using the following example (PR26314). Up-to-date Clang (cfe/trunk 268486) is unable to vectorize this case.

#define Z 32
typedef struct s {
	int v1[Z];
	int v2[Z];
	int v3[Z][Z];
} s;

void slow_function (s* const obj) {
    for (int j=0; j<Z; j++) {
        for (int k=0; k<Z; k++) {
            int x = obj->v1[k] + obj->v2[j];
            obj->v3[j][k] += x;
        }
    }
}

Vectorization in this case is denied due to false result "AccessAnalysis::canCheckPtrAtRT" function.

Two calls of "AccessAnalysis::canCheckPtrAtRT" are possible during loop analysis, and check of pointers strides is performed only with the second call, which occurs only in particular case of failed memory dependency check of pointers with non-constant distance between them.

"AccessAnalysis::canCheckPtrAtRT" returns false on the example because if-condition below is false due to "isStridedPtr" zero result on inner-loop-invariant pointer check w.r.t. inner loop "TheLoop".

if (hasComputableBounds(PSE, StridesMap, Ptr, TheLoop) &&
    // When we run after a failing dependency check we have to make sure
    // we don't have wrapping pointers.
    (!ShouldCheckStride ||
     isStridedPtr(PSE, Ptr, TheLoop, StridesMap) == 1)) {...}
else {
  DEBUG(dbgs() << "LAA: Can't find bounds for ptr:" << *Ptr << '\n');
  CanDoRT = false;
}

After rL264243 "hasComputableBounds" returns true in case of loop-invariant pointer (like "v2[j]" in the example), but, if I understood it correctly, current "isStridedPtr" will always return zero in this case because there is a check for equality of "TheLoop" (inner) and the loop which is obtained with SCEV of "Ptr" (outer loop for "v2[j]").

When it comes to stride check in "AccessAnalysis::canCheckPtrAtRT" we have as given that there was successful previous run (without the stride check), so either no RT check is needed at all or the pointers are already found suitable for RT check. If we only have to make sure that none of the pointers are wrapping then unit stride of the pointer w.r.t. any loop might be enough?

In D17268#420503, @anemet wrote:

I also think that if you want to rename the function you should probably do that in a separate patch. BTW, I agree with the renaming.

Should I create separate diff for review?

[anemet]

In D17268#421474, @roman.shirokiy wrote:

Thanks for the review!

I'll try to describe proposed change using the following example (PR26314). Up-to-date Clang (cfe/trunk 268486) is unable to vectorize this case.

#define Z 32
typedef struct s {
	int v1[Z];
	int v2[Z];
	int v3[Z][Z];
} s;

void slow_function (s* const obj) {
    for (int j=0; j<Z; j++) {
        for (int k=0; k<Z; k++) {
            int x = obj->v1[k] + obj->v2[j];
            obj->v3[j][k] += x;
        }
    }
}

Vectorization in this case is denied due to false result "AccessAnalysis::canCheckPtrAtRT" function.

Two calls of "AccessAnalysis::canCheckPtrAtRT" are possible during loop analysis, and check of pointers strides is performed only with the second call, which occurs only in particular case of failed memory dependency check of pointers with non-constant distance between them.

"AccessAnalysis::canCheckPtrAtRT" returns false on the example because if-condition below is false due to "isStridedPtr" zero result on inner-loop-invariant pointer check w.r.t. inner loop "TheLoop".

if (hasComputableBounds(PSE, StridesMap, Ptr, TheLoop) &&
    // When we run after a failing dependency check we have to make sure
    // we don't have wrapping pointers.
    (!ShouldCheckStride ||
     isStridedPtr(PSE, Ptr, TheLoop, StridesMap) == 1)) {...}
else {
  DEBUG(dbgs() << "LAA: Can't find bounds for ptr:" << *Ptr << '\n');
  CanDoRT = false;
}

After rL264243 "hasComputableBounds" returns true in case of loop-invariant pointer (like "v2[j]" in the example), but, if I understood it correctly, current "isStridedPtr" will always return zero in this case because there is a check for equality of "TheLoop" (inner) and the loop which is obtained with SCEV of "Ptr" (outer loop for "v2[j]").

When it comes to stride check in "AccessAnalysis::canCheckPtrAtRT" we have as given that there was successful previous run (without the stride check), so either no RT check is needed at all or the pointers are already found suitable for RT check. If we only have to make sure that none of the pointers are wrapping then unit stride of the pointer w.r.t. any loop might be enough?

Ah, thanks for writing this up. For some reason, I thought that we were failing somewhere else during the second phase (when we prove independence with RT checks only).

I still don't think that tweaking isStridedPtr to prove non-wrapping for more cases is the right approach. If the address is not a recurrence (e.g. just a global address) that is also trivially non-wrapping but would fail early in getPtrStride.

How about leaving isStridedPtr/getPtrStride unchanged and creating a new helper isNoWrap (similar to isNoWrapAddRec). isNoWrap would essentially be isLoopInvariant(...) || getPtrStride(...) == 1.

That would also satisfy Hal's request to hide this behind a call.

What do you think?

In D17268#420503, @anemet wrote:

I also think that if you want to rename the function you should probably do that in a separate patch. BTW, I agree with the renaming.

Should I create separate diff for review?

No need to. You can just commit that part right away and then rebase this patch on top of that.

[roman.shirokiy]

In D17268#421675, @anemet wrote:

Ah, thanks for writing this up. For some reason, I thought that we were failing somewhere else during the second phase (when we prove independence with RT checks only).

I still don't think that tweaking isStridedPtr to prove non-wrapping for more cases is the right approach. If the address is not a recurrence (e.g. just a global address) that is also trivially non-wrapping but would fail early in getPtrStride.

How about leaving isStridedPtr/getPtrStride unchanged and creating a new helper isNoWrap (similar to isNoWrapAddRec). isNoWrap would essentially be isLoopInvariant(...) || getPtrStride(...) == 1.

That would also satisfy Hal's request to hide this behind a call.

What do you think?

Hi Adam!
Sorry for late response, I've been distracted due to Russian holidays.
Thanks for the suggestion! I really try grasping LAA-stuff, but my understanding of it is quite superficial. I've almost prepared updated diff, but I wonder why originally there was no check for negative unit stride?

In D17268#421675, @anemet wrote:

No need to. You can just commit that part right away and then rebase this patch on top of that.

This was commited in r269020.

[roman.shirokiy]

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

for (int j=0; j<Z; j++) {

for (int k=Z-1; k>=0; k--) {
    int x = obj->v1[k] + obj->v2[j];
    obj->v3[j][k] += x;
}

}

[anemet]

In D17268#426175, @roman.shirokiy wrote:

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

Can you please not change this for now. I have been thinking about this code and I am not sure I understand the original version either.

Particularly unclear is why we only check non-wrapping when we retry proving vectorization safety with memchecks only. It seems to me that we'd have to do that for any pointer participating in memchecks.

@Hal, do you remember this? I asked Arnold and he didn't remember anymore.

This check feels parallel to the SCEV tests in isDependent because the dep distance is only correct if the pointers don't wrap. However in this case I think we're only interested if due to wrapping we need to consider the "inverse" interval (end->begin) rather than the normal (begin->end).

[anemet]

@hfinkel: ^^

[hfinkel]

In D17268#432173, @anemet wrote:

In D17268#426175, @roman.shirokiy wrote:

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

Can you please not change this for now. I have been thinking about this code and I am not sure I understand the original version either.

Particularly unclear is why we only check non-wrapping when we retry proving vectorization safety with memchecks only. It seems to me that we'd have to do that for any pointer participating in memchecks.

@Hal, do you remember this? I asked Arnold and he didn't remember anymore.

I don't in detail, but would the SCEV simplify in the first place if the expression might wrap? When we have memchecks we force the non-wrapping, and so we need to check.

This check feels parallel to the SCEV tests in isDependent because the dep distance is only correct if the pointers don't wrap. However in this case I think we're only interested if due to wrapping we need to consider the "inverse" interval (end->begin) rather than the normal (begin->end).

[anemet]

In D17268#433960, @hfinkel wrote:

In D17268#432173, @anemet wrote:

In D17268#426175, @roman.shirokiy wrote:

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

Can you please not change this for now. I have been thinking about this code and I am not sure I understand the original version either.

Particularly unclear is why we only check non-wrapping when we retry proving vectorization safety with memchecks only. It seems to me that we'd have to do that for any pointer participating in memchecks.

@Hal, do you remember this? I asked Arnold and he didn't remember anymore.

I don't in detail, but would the SCEV simplify in the first place if the expression might wrap? When we have memchecks we force the non-wrapping, and so we need to check.

Right but non-wrapping should be true for any memcheck not just those that we use instead of the failed SCEV-based testing. E.g. in

for (i) {

A[i] = B[i];

}

we will insert memchecks right away (i.e. ShouldCheckStride=false) but we should first make sure that the pointers don't wrap.

This check feels parallel to the SCEV tests in isDependent because the dep distance is only correct if the pointers don't wrap. However in this case I think we're only interested if due to wrapping we need to consider the "inverse" interval (end->begin) rather than the normal (begin->end).

[roman.shirokiy]

In D17268#433986, @anemet wrote:

In D17268#433960, @hfinkel wrote:

In D17268#432173, @anemet wrote:

In D17268#426175, @roman.shirokiy wrote:

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

Can you please not change this for now. I have been thinking about this code and I am not sure I understand the original version either.

Particularly unclear is why we only check non-wrapping when we retry proving vectorization safety with memchecks only. It seems to me that we'd have to do that for any pointer participating in memchecks.

@Hal, do you remember this? I asked Arnold and he didn't remember anymore.

I don't in detail, but would the SCEV simplify in the first place if the expression might wrap? When we have memchecks we force the non-wrapping, and so we need to check.

Right but non-wrapping should be true for any memcheck not just those that we use instead of the failed SCEV-based testing. E.g. in

for (i) {

A[i] = B[i];

}

we will insert memchecks right away (i.e. ShouldCheckStride=false) but we should first make sure that the pointers don't wrap.

Hi Adam!

Can we check for non-wrapping only in case if we already have memchecks, but Accesses.isDependencyCheckNeeded() is somehow false? If it is true then non-wrapping is checked in isDependent, as far as I understand. If it is false then we can traverse through participating pointers and check them for non-wrapping (with isNoWrap maybe), and if any one of them wraps then report that there is unsafe dependent memory operations i.e. CanVecMem = false.

[hfinkel]

In D17268#437021, @roman.shirokiy wrote:

In D17268#433986, @anemet wrote:

In D17268#433960, @hfinkel wrote:

In D17268#432173, @anemet wrote:

In D17268#426175, @roman.shirokiy wrote:

Updated diff according to Adam suggestions. Also added negative unit stride check since it seems to not break anything and allows to vectorize

Can you please not change this for now. I have been thinking about this code and I am not sure I understand the original version either.

Particularly unclear is why we only check non-wrapping when we retry proving vectorization safety with memchecks only. It seems to me that we'd have to do that for any pointer participating in memchecks.

@Hal, do you remember this? I asked Arnold and he didn't remember anymore.

I don't in detail, but would the SCEV simplify in the first place if the expression might wrap? When we have memchecks we force the non-wrapping, and so we need to check.

Right but non-wrapping should be true for any memcheck not just those that we use instead of the failed SCEV-based testing. E.g. in

for (i) {

A[i] = B[i];

}

we will insert memchecks right away (i.e. ShouldCheckStride=false) but we should first make sure that the pointers don't wrap.

Hi Adam!

Can we check for non-wrapping only in case if we already have memchecks, but Accesses.isDependencyCheckNeeded() is somehow false? If it is true then non-wrapping is checked in isDependent, as far as I understand. If it is false then we can traverse through participating pointers and check them for non-wrapping (with isNoWrap maybe), and if any one of them wraps then report that there is unsafe dependent memory operations i.e. CanVecMem = false.

Hopefully, in almost all such cases in practice, SCEV itself will be able to prove nowrap on the pointer-expression AddRec. Also, FWIW, if we need to do this, we might want to do it as part of support for alignment-based multiversioning (because in the case where the pointers are aligned, even if we wrap, the vectorized loads/stores don't cross the final page boundary, and so it is not actually a problem).

[roman.shirokiy]

Removed negative unit stride condition.

[anemet]

LGTM with the improved comments in the testcases. Thanks!

test/Analysis/LoopAccessAnalysis/multiple-strides-rt-memory-checks.ll

2–3

Can you please add a bit more explanation here and in the other testcase.

Please include the C version and a few words like:

When we were retrying dependence checking with memchecks only, the loop-invariant access in the inner loop was incorrectly determined to be wrapping because it was not strided in the inner loop.

include/llvm/Analysis/LoopAccessAnalysis.h

 ///
 /// If \p OrigPtr is not null, use it to look up the stride value instead of \p
 /// Ptr.  \p PtrToStride provides the mapping between the pointer value and its
 /// stride as collected by LoopVectorizationLegality::collectStridedAccess.
 const SCEV *replaceSymbolicStrideSCEV(PredicatedScalarEvolution &PSE,
                                       const ValueToValueMap &PtrToStride,
                                       Value *Ptr, Value *OrigPtr = nullptr);
 
 /// \brief If the pointer has a constant stride return it in units of its
-/// element size.  Otherwise return zero.
+/// element size. Otherwise return zero. If loop \p Lp is nullptr
+/// then strided loop is obtained within the function,
+/// else stride is computed with regard to the specified loop \p Lp.
 ///
 /// Ensure that it does not wrap in the address space, assuming the predicate
 /// associated with \p PSE is true.
 ///
 /// If necessary this method will version the stride of the pointer according
 /// to \p PtrToStride and therefore add further predicates to \p PSE.
 /// The \p Assume parameter indicates if we are allowed to make additional
 /// run-time assumptions.
-int isStridedPtr(PredicatedScalarEvolution &PSE, Value *Ptr, const Loop *Lp,
+int getPtrStride(PredicatedScalarEvolution &PSE, Value *Ptr, const Loop *Lp,
                  const ValueToValueMap &StridesMap = ValueToValueMap(),
                  bool Assume = false);
 
 /// \brief Returns true if the memory operations \p A and \p B are consecutive.
 /// This is a simple API that does not depend on the analysis pass. 
 bool isConsecutiveAccess(Value *A, Value *B, const DataLayout &DL,
                          ScalarEvolution &SE, bool CheckType = true);
 

lib/Analysis/LoopAccessAnalysis.cpp

for (auto A : AS) {
         ++NumWritePtrChecks;
       else
         ++NumReadPtrChecks;
 
       if (hasComputableBounds(PSE, StridesMap, Ptr, TheLoop) &&
           // When we run after a failing dependency check we have to make sure
           // we don't have wrapping pointers.
           (!ShouldCheckStride ||
-           isStridedPtr(PSE, Ptr, TheLoop, StridesMap) == 1)) {
+           getPtrStride(PSE, Ptr, /*Lp = */nullptr, StridesMap) == 1)) {
         // The id of the dependence set.
         unsigned DepId;
 
         if (IsDepCheckNeeded) {
           Value *Leader = DepCands.getLeaderValue(Access).getPointer();
           unsigned &LeaderId = DepSetId[Leader];
           if (!LeaderId)
             LeaderId = RunningDepId++;
if (OBO->hasNoSignedWrap() &&
       if (auto *OpAR = dyn_cast<SCEVAddRecExpr>(OpScev))
         return OpAR->getLoop() == L && OpAR->getNoWrapFlags(SCEV::FlagNSW);
     }
 
   return false;
 }
 
 /// \brief Check whether the access through \p Ptr has a constant stride.
-int llvm::isStridedPtr(PredicatedScalarEvolution &PSE, Value *Ptr,
+int llvm::getPtrStride(PredicatedScalarEvolution &PSE, Value *Ptr,
                        const Loop *Lp, const ValueToValueMap &StridesMap,
                        bool Assume) {
   Type *Ty = Ptr->getType();
   assert(Ty->isPointerTy() && "Unexpected non-ptr");
 
   // Make sure that the pointer does not point to aggregate types.
   auto *PtrTy = cast<PointerType>(Ty);
   if (PtrTy->getElementType()->isAggregateType()) {
if (Assume && !AR)
     AR = PSE.getAsAddRec(Ptr);
 
   if (!AR) {
     DEBUG(dbgs() << "LAA: Bad stride - Not an AddRecExpr pointer " << *Ptr
                  << " SCEV: " << *PtrScev << "\n");
     return 0;
   }
 
-  // The accesss function must stride over the innermost loop.
-  if (Lp != AR->getLoop()) {
-    DEBUG(dbgs() << "LAA: Bad stride - Not striding over innermost loop " <<
+  // Check whether we should get stride with regard to specified loop.
+  if (Lp) {
+    // Make sure Ptr is strided over Lp.
+    if (PSE.getSE()->isLoopInvariant(PtrScev, Lp)) {
+      DEBUG(dbgs() << "LAA: Bad stride - Not striding over provided loop " <<
             *Ptr << " SCEV: " << *AR << "\n");
       return 0;
     }
+  } else
+    // Obtain the loop over which access function is strided. 
+    Lp = AR->getLoop();
 
   // The address calculation must not wrap. Otherwise, a dependence could be
   // inverted.
   // An inbounds getelementptr that is a AddRec with a unit stride
   // cannot wrap per definition. The unit stride requirement is checked later.
   // An getelementptr without an inbounds attribute and unit stride would have
   // to access the pointer value "0" which is undefined behavior in address
   // space 0, therefore we can also vectorize this case.
MemoryDepChecker::isDependent(const MemAccessInfo &A, unsigned AIdx,
   // We cannot check pointers in different address spaces.
   if (APtr->getType()->getPointerAddressSpace() !=
       BPtr->getType()->getPointerAddressSpace())
     return Dependence::Unknown;
 
   const SCEV *AScev = replaceSymbolicStrideSCEV(PSE, Strides, APtr);
   const SCEV *BScev = replaceSymbolicStrideSCEV(PSE, Strides, BPtr);
 
-  int StrideAPtr = isStridedPtr(PSE, APtr, InnermostLoop, Strides, true);
-  int StrideBPtr = isStridedPtr(PSE, BPtr, InnermostLoop, Strides, true);
+  int StrideAPtr = getPtrStride(PSE, APtr, InnermostLoop, Strides, true);
+  int StrideBPtr = getPtrStride(PSE, BPtr, InnermostLoop, Strides, true);
 
   const SCEV *Src = AScev;
   const SCEV *Sink = BScev;
 
   // If the induction step is negative we have to invert source and sink of the
   // dependence.
   if (StrideAPtr < 0) {
     //Src = BScev;
for (I = Loads.begin(), IE = Loads.end(); I != IE; ++I) {
     // read list. If we *did* see it before, then it is already in
     // the read-write list. This allows us to vectorize expressions
     // such as A[i] += x;  Because the address of A[i] is a read-write
     // pointer. This only works if the index of A[i] is consecutive.
     // If the address of i is unknown (for example A[B[i]]) then we may
     // read a few words, modify, and write a few words, and some of the
     // words may be written to the same address.
     bool IsReadOnlyPtr = false;
-    if (Seen.insert(Ptr).second || !isStridedPtr(PSE, Ptr, TheLoop, Strides)) {
+    if (Seen.insert(Ptr).second || !getPtrStride(PSE, Ptr, TheLoop, Strides)) {
       ++NumReads;
       IsReadOnlyPtr = true;
     }
 
     MemoryLocation Loc = MemoryLocation::get(LD);
     // The TBAA metadata could have a control dependency on the predication
     // condition, so we cannot rely on it when determining whether or not we
     // need runtime pointer checks.

lib/Transforms/Scalar/LoopLoadElimination.cpp

bool isDependenceDistanceOfOne(PredicatedScalarEvolution &PSE,
     assert(LoadPtrType->getPointerAddressSpace() ==
                StorePtr->getType()->getPointerAddressSpace() &&
            LoadType == StorePtr->getType()->getPointerElementType() &&
            "Should be a known dependence");
 
     // Currently we only support accesses with unit stride.  FIXME: we should be
     // able to handle non unit stirde as well as long as the stride is equal to
     // the dependence distance.
-    if (isStridedPtr(PSE, LoadPtr, L) != 1 ||
-        isStridedPtr(PSE, LoadPtr, L) != 1)
+    if (getPtrStride(PSE, LoadPtr, L) != 1 ||
+        getPtrStride(PSE, LoadPtr, L) != 1)
       return false;
 
     auto &DL = Load->getParent()->getModule()->getDataLayout();
     unsigned TypeByteSize = DL.getTypeAllocSize(const_cast<Type *>(LoadType));
 
     auto *LoadPtrSCEV = cast<SCEVAddRecExpr>(PSE.getSCEV(LoadPtr));
     auto *StorePtrSCEV = cast<SCEVAddRecExpr>(PSE.getSCEV(StorePtr));
 

lib/Transforms/Vectorize/LoopVectorize.cpp

if (AccessList.empty())
     return;
 
   auto &DL = TheLoop->getHeader()->getModule()->getDataLayout();
   for (auto I : AccessList) {
     LoadInst *LI = dyn_cast<LoadInst>(I);
     StoreInst *SI = dyn_cast<StoreInst>(I);
 
     Value *Ptr = LI ? LI->getPointerOperand() : SI->getPointerOperand();
-    int Stride = isStridedPtr(PSE, Ptr, TheLoop, Strides);
+    int Stride = getPtrStride(PSE, Ptr, TheLoop, Strides);
 
     // The factor of the corresponding interleave group.
     unsigned Factor = std::abs(Stride);
 
     // Ignore the access if the factor is too small or too large.
     if (Factor < 2 || Factor > MaxInterleaveGroupFactor)
       continue;
 
     const SCEV *Scev = replaceSymbolicStrideSCEV(PSE, Strides, Ptr);
     PointerType *PtrTy = dyn_cast<PointerType>(Ptr->getType());

test/Analysis/LoopAccessAnalysis/multiple-strides-rt-memory-checks.ll

+; This is the test case from PR26314.
+; RUN: opt -loop-accesses -analyze -S < %s | FileCheck %s
+; CHECK: function 'Test':
+; CHECK:   .inner:
+; CHECK-NEXT:     Memory dependences are safe
+; CHECK-NEXT:     Dependences:
+; CHECK-NEXT:     Run-time memory checks:
+; CHECK:          Check 0:
+; CHECK:          Check 1:
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%struct.s = type { [32 x i32], [32 x i32], [32 x [32 x i32]] }
+
+define void @Test(%struct.s* nocapture %obj) #0 {
+  br label %.outer.preheader
+
+
+.outer.preheader:
+  %i = phi i64 [ 0, %0 ], [ %i.next, %.outer ]
+  %1 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 1, i64 %i
+  br label %.inner
+
+.exit:
+  ret void
+ 
+.outer:
+  %i.next = add nuw nsw i64 %i, 1
+  %exitcond.outer = icmp eq i64 %i.next, 32
+  br i1 %exitcond.outer, label %.exit, label %.outer.preheader
+
+.inner:
+  %j = phi i64 [ 0, %.outer.preheader ], [ %j.next, %.inner ]
+  %2 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 0, i64 %j
+  %3 = load i32, i32* %2
+  %4 = load i32, i32* %1
+  %5 = add nsw i32 %4, %3
+  %6 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 2, i64 %i, i64 %j
+  %7 = load i32, i32* %6
+  %8 = add nsw i32 %5, %7
+  store i32 %8, i32* %6  
+  %j.next = add nuw nsw i64 %j, 1
+  %exitcond.inner = icmp eq i64 %j.next, 32
+  br i1 %exitcond.inner, label %.outer, label %.inner
+}

test/Transforms/LoopVectorize/multiple-strides-vectorization.ll

+; This is the test case from PR26314.
+; RUN: opt -loop-vectorize -S < %s | FileCheck %s
+; CHECK-LABEL: Test
+; CHECK: <4 x i64>
+; CHECK: <4 x i32>, <4 x i32>
+; CHECK: llvm.loop.vectorize.width
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%struct.s = type { [32 x i32], [32 x i32], [32 x [32 x i32]] }
+
+define void @Test(%struct.s* nocapture %obj) #0 {
+  br label %.outer.preheader
+
+
+.outer.preheader:
+  %i = phi i64 [ 0, %0 ], [ %i.next, %.outer ]
+  %1 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 1, i64 %i
+  br label %.inner
+
+.exit:
+  ret void
+ 
+.outer:
+  %i.next = add nuw nsw i64 %i, 1
+  %exitcond.outer = icmp eq i64 %i.next, 32
+  br i1 %exitcond.outer, label %.exit, label %.outer.preheader
+
+.inner:
+  %j = phi i64 [ 0, %.outer.preheader ], [ %j.next, %.inner ]
+  %2 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 0, i64 %j
+  %3 = load i32, i32* %2
+  %4 = load i32, i32* %1
+  %5 = add nsw i32 %4, %3
+  %6 = getelementptr inbounds %struct.s, %struct.s* %obj, i64 0, i32 2, i64 %i, i64 %j
+  %7 = load i32, i32* %6
+  %8 = add nsw i32 %5, %7
+  store i32 %8, i32* %6  
+  %j.next = add nuw nsw i64 %j, 1
+  %exitcond.inner = icmp eq i64 %j.next, 32
+  br i1 %exitcond.inner, label %.outer, label %.inner
+}