[InstCombine] Fix IC trying to create a xor of pointer types

[aemerson]

Summary

[InstCombine] Fix IC trying to create a xor of pointer types.

We were previously crashing here because icmp operands can be pointers as well as integers, and the code didn't check before building a xor. I've fixed this by doing a ptrtoint convert on the assumption that ptrtoints are free.

rdar://42473741

[spatel]

I've fixed this by doing a ptrtoint convert on the assumption that ptrtoints are free.

I don't think we can make that assumption as a target-independent transform. Restrict this fold to integer types instead?

[aemerson]

Sure, done.

[spatel]

LGTM

llvm/trunk/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

Value *InstCombiner::foldXorOfICmps(ICmpInst *LHS, ICmpInst *RHS) {
 
   // TODO: This can be generalized to compares of non-signbits using
   // decomposeBitTestICmp(). It could be enhanced more by using (something like)
   // foldLogOpOfMaskedICmps().
   ICmpInst::Predicate PredL = LHS->getPredicate(), PredR = RHS->getPredicate();
   Value *LHS0 = LHS->getOperand(0), *LHS1 = LHS->getOperand(1);
   Value *RHS0 = RHS->getOperand(0), *RHS1 = RHS->getOperand(1);
   if ((LHS->hasOneUse() || RHS->hasOneUse()) &&
-      LHS0->getType() == RHS0->getType()) {
+      LHS0->getType() == RHS0->getType() &&
+      LHS0->getType()->isIntOrIntVectorTy()) {
     // (X > -1) ^ (Y > -1) --> (X ^ Y) < 0
     // (X <  0) ^ (Y <  0) --> (X ^ Y) < 0
     if ((PredL == CmpInst::ICMP_SGT && match(LHS1, m_AllOnes()) &&
          PredR == CmpInst::ICMP_SGT && match(RHS1, m_AllOnes())) ||
         (PredL == CmpInst::ICMP_SLT && match(LHS1, m_Zero()) &&
          PredR == CmpInst::ICMP_SLT && match(RHS1, m_Zero()))) {
       Value *Zero = ConstantInt::getNullValue(LHS0->getType());
       return Builder.CreateICmpSLT(Builder.CreateXor(LHS0, RHS0), Zero);

llvm/trunk/test/Transforms/InstCombine/xor-icmps.ll

 ; CHECK-NEXT:    ret i1 true
 ;
   %C = icmp eq i8 %A, %B
   %D = icmp ne i8 %B, %A
   %E = xor i1 %C, %D
   ret i1 %E
 }
 
+define i1 @xor_icmp_ptr(i8* %c, i8* %d) {
+; CHECK-LABEL: @xor_icmp_ptr(
+; CHECK-NEXT:    [[CMP:%.*]] = icmp slt i8* [[C:%.*]], null
+; CHECK-NEXT:    [[CMP1:%.*]] = icmp slt i8* [[D:%.*]], null
+; CHECK-NEXT:    [[XOR:%.*]] = xor i1 [[CMP]], [[CMP1]]
+; CHECK-NEXT:    ret i1 [[XOR]]
+;
+  %cmp = icmp slt i8* %c, null
+  %cmp1 = icmp slt i8* %d, null
+  %xor = xor i1 %cmp, %cmp1
+  ret i1 %xor
+}
+