[LSR] Pointers with different address spaces are considered incompatible.

[uabelho]

Summary

Function isCompatibleIVType is already used as a guard before the call to

SE.getMinusSCEV(OperExpr, PrevExpr);

in LSRInstance::ChainInstruction. getMinusSCEV requires the expressions
to be of the same type, so we now consider two pointers with different
address spaces to be incompatible, since it is possible that the pointers
in fact have different sizes.

[uabelho]

Very similar to https://reviews.llvm.org/D29717 but in LSR instead of LoopVectorize.

As I wrote in the commit message I haven't been able to reproduce this on an in-tree
target. If anyone knows a target where pointers in different address spaces have different
sizes, then I can probably fix a reproducer exposing a failed assertion in SCEV.

[uabelho]

After a hint from my colleague Ka-Ka (thanks!) I managed to reproduce the crash on "amdgcn--".

I'll update the patch with a test case tomorrow.

[uabelho]

Added test case. Without the fix the test case makes the compiler crash in SE.getMinusSCEV(OperExpr, PrevExpr);

[qcolombet]

LGTM

lib/Transforms/Scalar/LoopStrengthReduce.cpp

if (TruncInst *Trunc = dyn_cast<TruncInst>(Oper))
     return Trunc->getOperand(0);
   return Oper;
 }
 
 /// Return true if we allow an IV chain to include both types.
 static bool isCompatibleIVType(Value *LVal, Value *RVal) {
   Type *LType = LVal->getType();
   Type *RType = RVal->getType();
-  return (LType == RType) || (LType->isPointerTy() && RType->isPointerTy());
+  return (LType == RType) || (LType->isPointerTy() && RType->isPointerTy() &&
+                              // Different address spaces means (possibly)
+                              // different types of the pointer implementation,
+                              // e.g. i16 vs i32 so disallow that.
+                              (LType->getPointerAddressSpace() ==
+                               RType->getPointerAddressSpace()));
 }
 
 /// Return an approximation of this SCEV expression's "base", or NULL for any
 /// constant. Returning the expression itself is conservative. Returning a
 /// deeper subexpression is more precise and valid as long as it isn't less
 /// complex than another subexpression. For expressions involving multiple
 /// unscaled values, we need to return the pointer-type SCEVUnknown. This avoids
 /// forming chains across objects, such as: PrevOper==a[i], IVOper==b[i],