Fix crash when looking up the addrspace of GEPs with vector types

[arsenm]

I'm kind of surprised this was broken. It would probably be better to come up with a testcase where this breaks in a more fundamental pass like instcombine

include/llvm/IR/Instructions.h

890 ↗

(On Diff #13128)

This one looks correct already, and this change should be unnecessary. getPointerAddressSpace() already calls getScalarType()

include/llvm/IR/Operator.h

405–406

This is a correct fix, but this should use Ty->getPointerAddressSpace() like everywhere else

[dotdash]

Updated the patch to use getPointerAddressSpace(), couldn't find a different
pass that triggers the bug though, so the test still uses mergefunc

[arsenm]

LGTM. I would expect you could trigger this by running instcombine with a GEP that ends up getting some kind of constant folding

[dotdash]

Had another look, still couldn't find anything. The code is either using GetElementPtrInst instead of GEPOperator, calls getType()->getPointerAddressSpace() or checks for vector types beforehand, so it doesn't trigger the bug. Alias analysis seemed like it could work, but I didn't manage to get a testcase with that either (and I'm too clueless about that code).

Anyway, could someone please land this for me? I don't have commit access. Thanks!

[arsenm]

Committed r216930

include/llvm/IR/Operator.h

public:
   /// PointerType.
   Type *getPointerOperandType() const {
     return getPointerOperand()->getType();
   }
 
   /// getPointerAddressSpace - Method to return the address space of the
   /// pointer operand.
   unsigned getPointerAddressSpace() const {
-    return cast<PointerType>(getPointerOperandType())->getAddressSpace();
+    return getPointerOperandType()->getPointerAddressSpace();
   }
 
   unsigned getNumIndices() const {  // Note: always non-negative
     return getNumOperands() - 1;
   }
 
   bool hasIndices() const {
     return getNumOperands() > 1;
   }

test/Transforms/MergeFunc/vector-GEP-crash.ll

+; RUN: opt -mergefunc -disable-output < %s
+; This used to cause a crash when compairing the GEPs
+
+define void @foo(<2 x i64*>) {
+  %tmp = getelementptr <2 x i64*> %0, <2 x i64> <i64 0, i64 0>
+  ret void
+}
+
+define void @bar(<2 x i64*>) {
+  %tmp = getelementptr <2 x i64*> %0, <2 x i64> <i64 0, i64 0>
+  ret void
+}