[InstSimplify] Fix SimplifyGEPInst when GEP index type is vector type.

[huihuiz]

Summary

The following folds

"gep (gep V, C), (sub 0, V) -> C"
"gep (gep V, C), (xor V, -1) -> C-1"

are only valid when GEP index type is scalar type.

This patch bailout early when GEP index type is vector type. Explicitly call
TypeSize::getFixedSize to assert on places where scalable type doesn't make sense.

[huihuiz]

Take test.ll, run opt -S -instsimplify -o -
Then you will get the compiler warning: Compiler has made implicit assumption that TypeSize is not scalable.

define <vscale x 16 x i8*> @getelementptr_src_ty_not_vscale(<vscale x 16 x i8*> %a, <vscale x 16 x i64> %offset) {
  %v = getelementptr i8, <vscale x 16 x i8*> %a, <vscale x 16 x i64> %offset
  ret <vscale x 16 x i8*> %v
}

For this case SrcTy is i8

[efriedma]

There are two different kinds of "scalable" involved when you're dealing with GEPs: whether you're indexing over a scalable type, and whether the result is a scalable vector of pointers. Mixing the two together into a single boolean doesn't really make sense.

In this context, I'm not sure why it would matter whether the result is a vector of pointers.

[huihuiz]

My initial thought was wrong.
Spent sometime look again, we are nearly impossible to generate incorrect result without the code fixes in this patch.
But comparing a scalable size with a fixed size still doesn't make sense.

E.g., We are comparing scalable type size %offset to a fixed size 'IdxWidth' returned by DataLayout::getIndexSizeInBits

define <vscale x 1 x i8*> @gep(i8* %a, <vscale x 1 x i32> %offset) {
  %v = getelementptr i8, i8* %a, <vscale x 1 x i32> %offset
  ret <vscale x 1 x i8*> %v
}

Please help take a look, let me know what you think?

[efriedma]

llvm/lib/Analysis/InstructionSimplify.cpp

4150

This is supposed to be comparing the size of the scalar types, I think? Scalar types can't be scalable.

That said, it looks like the code isn't actually doing that at the moment.

[huihuiz]

Thanks Eli for the feedback!

Update the patch to address the folds in question, reject if GEP index type is vector type.

[efriedma]

llvm/lib/Analysis/InstructionSimplify.cpp

4143

I think the transform is still valid, it just isn't implemented.

llvm/lib/Analysis/InstructionSimplify.cpp

     // getelementptr P, 0 -> P.
     if (match(Ops[1], m_Zero()) && Ops[0]->getType() == GEPTy)
       return Ops[0];
 
     Type *Ty = SrcTy;
     if (!IsScalableVec && Ty->isSized()) {
       Value *P;
       uint64_t C;
-      uint64_t TyAllocSize = Q.DL.getTypeAllocSize(Ty);
+      uint64_t TyAllocSize = Q.DL.getTypeAllocSize(Ty).getFixedSize();
       // getelementptr P, N -> P if P points to a type of zero size.
       if (TyAllocSize == 0 && Ops[0]->getType() == GEPTy)
         return Ops[0];
 
       // The following transforms are only safe if the ptrtoint cast
       // doesn't truncate the pointers.
       if (Ops[1]->getType()->getScalarSizeInBits() ==
           Q.DL.getPointerSizeInBits(AS)) {

                   m_SDiv(m_Sub(m_Value(P), m_PtrToInt(m_Specific(Ops[0]))),
                          m_SpecificInt(TyAllocSize))))
           if (Value *R = PtrToIntOrZero(P))
             return R;
       }
     }
   }
 
-  if (!IsScalableVec && Q.DL.getTypeAllocSize(LastType) == 1 &&
+  if (!IsScalableVec && Q.DL.getTypeAllocSize(LastType).getFixedSize() == 1 &&
       all_of(Ops.slice(1).drop_back(1),
              [](Value *Idx) { return match(Idx, m_Zero()); })) {
     unsigned IdxWidth =
         Q.DL.getIndexSizeInBits(Ops[0]->getType()->getPointerAddressSpace());
-    if (Q.DL.getTypeSizeInBits(Ops.back()->getType()) == IdxWidth) {
+    // Skip if offset type size is not a compile-time constant.
+    TypeSize OffsetTypeSize = Q.DL.getTypeSizeInBits(Ops.back()->getType());
+    if (!OffsetTypeSize.isScalable() &&
+        OffsetTypeSize.getFixedSize() == IdxWidth) {
       APInt BasePtrOffset(IdxWidth, 0);
       Value *StrippedBasePtr =
           Ops[0]->stripAndAccumulateInBoundsConstantOffsets(Q.DL,
                                                             BasePtrOffset);
 
       // gep (gep V, C), (sub 0, V) -> C
       if (match(Ops.back(),
                 m_Sub(m_Zero(), m_PtrToInt(m_Specific(StrippedBasePtr))))) {