[InstCombine] Handle vector compares in foldGEPIcmp()

[JesperAntonsson]

Summary

This is to fix PR38984 "InstCombine assertion at vector gep/icmp folding":
https://bugs.llvm.org/show_bug.cgi?id=38984

[lebedev.ri]

lib/Transforms/InstCombine/InstCombineCompares.cpp

978–980

Can't you use Constant::getIntegerValue(I.getType(), ICmpInst::isTrueWhenEqual(Cond))?

[JesperAntonsson]

Roman Lebedev commented with an elegant way to create varying integer values, so this patch makes use of that. Changed name of patch to reflect that we now handle rather than ignore vector icmps.

[lebedev.ri]

LG with a nit.

lib/Transforms/InstCombine/InstCombineCompares.cpp

976–983

There are all these Constant::get()/ConstantInt::get() overloads that don't require APInt.
Have you checked they aren't sufficient?

[JesperAntonsson]

Used an overloaded getter with simpler interface.

lib/Transforms/InstCombine/InstCombineCompares.cpp

if (GEPLHS->getNumOperands() == GEPRHS->getNumOperands()) {
             // Irreconcilable differences.
             NumDifferences = 2;
             break;
           } else {
             if (NumDifferences++) break;
             DiffOperand = i;
           }
         }
 
-      if (NumDifferences == 0)   // SAME GEP?
+      if (NumDifferences == 0) {  // SAME GEP?
+        Value *Replacement = Builder.getInt1(ICmpInst::isTrueWhenEqual(Cond));
+        // Ignore vector compares.
+        if (Replacement->getType() == I.getType())
           return replaceInstUsesWith(I, // No comparison is needed here.
-                             Builder.getInt1(ICmpInst::isTrueWhenEqual(Cond)));
-
+                                     Replacement);
+      }
       else if (NumDifferences == 1 && GEPsInBounds) {
         Value *LHSV = GEPLHS->getOperand(DiffOperand);
         Value *RHSV = GEPRHS->getOperand(DiffOperand);
         // Make sure we do a signed comparison here.
         return new ICmpInst(ICmpInst::getSignedPredicate(Cond), LHSV, RHSV);
       }
     }
 

test/Transforms/InstCombine/pr38984.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -instcombine -S | FileCheck %s
+target datalayout = "p:16:16"
+
+@offsets = external dso_local global [4 x i16], align 1
+
+define void @PR38984() {
+; CHECK-LABEL: @PR38984(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret void
+;
+entry:
+  %0 = load i16, i16* getelementptr ([4 x i16], [4 x i16]* @offsets, i16 0, i16 undef), align 1
+  %1 = insertelement <4 x i16> undef, i16 %0, i32 3
+  %2 = sub <4 x i16> zeroinitializer, %1
+  %3 = sext <4 x i16> %2 to <4 x i32>
+  %4 = getelementptr inbounds i64, i64* null, <4 x i32> %3
+  %5 = ptrtoint <4 x i64*> %4 to <4 x i32>
+  %6 = getelementptr inbounds i64, i64* null, <4 x i16> %2
+  %7 = ptrtoint <4 x i64*> %6 to <4 x i32>
+  %8 = icmp eq <4 x i32> %5, %7
+  %9 = select <4 x i1> %8, <4 x i16> zeroinitializer, <4 x i16> <i16 1, i16 1, i16 1, i16 1>
+  %10 = sext <4 x i16> %9 to <4 x i32>
+  ret void
+}