[InstCombine][RFC] Canonicalize constant mask in masked merge mattern

[lebedev.ri]

Summary

Masked merge has a pattern of: ((x ^ y) & M) ^ y.
Once PR6773 fold is done, that pattern will be produced.

But, if the mask is constant, there is no difference between ((x ^ y) & M) ^ y and ((x ^ y) & ~M) ^ x and ((x ^ y) & ~(~M)) ^ y,
I think the constant mask should be canonicalized.

I propose the following:

• if this is scalar:
  • if inverted mask has less bits set (Population count) than the original mask, DO invert
  • if zero-extended value of inverted mask is smaller than of the original mask, DO invert
  • else DON'T invert.
• if it is vector: *. apply that ^ scalar predicate to each non-undef element.
  • • count number of elements where the predicate is true.
    • for undef elements the predicate is false.
  • if there are more elements where the predicate is true, than elements where predicate is false, DO invert
  • else DON'T invert.

These are strict less/more comparisons, not less or equal/more or equal.
This ensures that the canonicalization is stable, and won't be undone / won't cause cycles.

https://rise4fun.com/Alive/eox

Thoughts?

[lebedev.ri]

Tidy up comments and the isNonCanonicalMask().

[lebedev.ri]

Drop duplicate tests.
I have just realized that non-const mask is not canonicalized yet too https://godbolt.org/g/NKPHGF, so i'll rework this slightly..

[lebedev.ri]

Rebased ontop of less controversial D45664, reducing the diff's size.

[lebedev.ri]

nullptr-init Mc too.
Does not appear to matter, but i have already hit the bug when
i forgot to do that to M, so better safe than sorry.

[lebedev.ri]

Rebased, slightly cleanup the matcher by using m_CombineAnd().

[spatel]

See my comment in D45733. Sorry that I didn't get a chance to review/discuss this sooner.

I doubt the premise that this pattern should exist in the first place, so I suspect we don't want to add logic to transform it. But if it does exist, then and/and/or is the best canonical form for the IR - it's better for bit-tracking analysis and better for codegen.

[lebedev.ri]

In D45655#1071003, @spatel wrote:

See my comment in D45733. Sorry that I didn't get a chance to review/discuss this sooner.

I doubt the premise that this pattern should exist in the first place, so I suspect we don't want to add logic to transform it. But if it does exist, then and/and/or is the best canonical form for the IR - it's better for bit-tracking analysis and better for codegen.

After further disscussion in D45733, yes, probably.
One more reason - the pattern could be (x & C1) | (y & C2), where C1 and C2 don't have common bits set, so the result will have ~(C1|C2) bits unset.
And we would not be able to efficiently fold that to ((x^y)&m)^y.
So yeah, let's unfold it for constants.

lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

if (Value *AndICmp = SimplifyBinOp(Instruction::And, LHS, RHS, SQ)) {
         return Builder.CreateAnd(LHS, RHS);
       }
     }
   }
 
   return nullptr;
 }
 
+/// This is cst_pred_ty, but with bias - if more than half of elements matched,
+/// then the result is true. undef is skipped, not counted as a match.
+template <typename Predicate> struct _cst_biased_pred_ty : public Predicate {
+  template <typename ITy> bool match(ITy *V) {
+    if (const auto *CI = dyn_cast<ConstantInt>(V))
+      return this->isValue(CI->getValue());
+    if (V->getType()->isVectorTy()) {
+      if (const auto *C = dyn_cast<Constant>(V)) {
+        if (const auto *CI = dyn_cast_or_null<ConstantInt>(C->getSplatValue()))
+          return this->isValue(CI->getValue());
+
+        // Non-splat vector constant: check each element for a match.
+        unsigned NumElts = V->getType()->getVectorNumElements();
+        assert(NumElts != 0 && "Constant vector with no elements?");
+        unsigned matched = 0;
+        for (unsigned i = 0; i != NumElts; ++i) {
+          Constant *Elt = C->getAggregateElement(i);
+          if (!Elt)
+            return false;
+          if (isa<UndefValue>(Elt))
+            continue; // Do *NOT* increase 'matched'!
+          auto *CI = dyn_cast<ConstantInt>(Elt);
+          if (!CI)
+            return false;
+          if (this->isValue(CI->getValue()))
+            ++matched;
+        }
+        const unsigned notMatched = NumElts - matched;
+        return matched > notMatched;
+      }
+    }
+    return false;
+  }
+};
+
+/// Never invert if it will increase count of set bits.
+/// If it decreases the count of set bits - do invert.
+/// Else, if it decreases the zero-extended value of the mask - do invert.
+/// NOTE: We must use </>, not <=/>= to prevent [un]doing our own changes.
+static bool isNonCanonicalMask(const APInt &m) {
+  const unsigned setBits = m.countPopulation();
+  const unsigned unsetBits = m.getBitWidth() - setBits;
+  if (setBits != unsetBits)
+    return setBits > unsetBits;
+
+  assert(setBits == unsetBits);
+  return (~m).getZExtValue() < m.getZExtValue();
+}
+
+struct is_noncanonical_mask {
+  bool isValue(const APInt &C) { return isNonCanonicalMask(C); }
+};
+inline _cst_biased_pred_ty<is_noncanonical_mask> _NonCanonicalMask() {
+  return _cst_biased_pred_ty<is_noncanonical_mask>();
+}
+
 /// If we have a masked merge, in the form of:
 ///   |        A  |  |B|
 ///   ((x ^ y) & M) ^ y
 ///    |  D  |
-/// If A has one use, and M is inverted, we can convert it into:
+/// If A has one use, and M is inverted, or is is non-canonical constant,
+/// we can convert it into:
 ///    |  D  |
 ///   ((x ^ y) & ~M) ^ x
 static Instruction *invertMaskInMaskedMerge(BinaryOperator &I,
                                             InstCombiner::BuilderTy &Builder) {
-  Value *D, *M, *B, *X, *Y;
-  if (!(match(&I, m_c_Xor(m_OneUse(m_And(m_Value(D), m_Not(m_Value(M)))),
-                          m_Value(B))) &&
+  Value *D, *B, *X, *Y;
+  Value *M = nullptr;
+  Constant *Mc;
+  if (!(match(&I,
+              m_c_Xor(m_OneUse(m_And(m_Value(D), m_CombineOr(m_Not(m_Value(M)),
+                                                             m_Constant(Mc)))),
+        ((Mc && match(Mc, _NonCanonicalMask())) || !Mc) &&
         match(D, m_Xor(m_Value(X), m_Value(Y))) &&
         match(B, m_CombineOr(m_Specific(X), m_Specific(Y)))))
     return nullptr;
 
-  // De-invert the mask and swap the value in B part.
+  // We either have the inverted non-constant mask, or non-canonical constant.
+  assert((!M && Mc) || (M && !Mc));
+  // If we have non-canonical constant mask, invert it. Else just use the mask.
+  if (!M)
+    M = Builder.CreateNot(Mc);
+
+  // Use canonical mask and swap the value in B part.
   Value *NewA = Builder.CreateAnd(D, M);
   Value *NewB = (B == Y) ? X : Y;
   return BinaryOperator::CreateXor(NewA, NewB);
 }
 
 // FIXME: We use commutative matchers (m_c_*) for some, but not all, matches
 // here. We should standardize that construct where it is needed or choose some
 // other way to ensure that commutated variants of patterns are not missed.

test/Transforms/InstCombine/invert-const-mask-in-masked-merge-scalar.ll

;
   %n1 = and i3 %n0, 2
   %r  = xor i3 %n1, %y
   ret i3 %r
 }
 
 define i3 @t_3_3 (i3 %x, i3 %y) {
 ; CHECK-LABEL: @t_3_3(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i3 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i3 [[N0]], 3
-; CHECK-NEXT:    [[R:%.*]] = xor i3 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i3 [[N0]], -4
+; CHECK-NEXT:    [[R:%.*]] = xor i3 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i3 [[R]]
 ;
   %n0 = xor i3 %x, %y
   %n1 = and i3 %n0, 3
   %r  = xor i3 %n1, %y
   ret i3 %r
 }
 
 define i3 @t_3_4 (i3 %x, i3 %y) {
 ; CHECK-LABEL: @t_3_4(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i3 [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    [[N1:%.*]] = and i3 [[N0]], -4
 ; CHECK-NEXT:    [[R:%.*]] = xor i3 [[N1]], [[Y]]
 ; CHECK-NEXT:    ret i3 [[R]]
 ;
   %n0 = xor i3 %x, %y
   %n1 = and i3 %n0, -4
   %r  = xor i3 %n1, %y
   ret i3 %r
 }
 
 define i3 @t_3_5 (i3 %x, i3 %y) {
 ; CHECK-LABEL: @t_3_5(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i3 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i3 [[N0]], -3
-; CHECK-NEXT:    [[R:%.*]] = xor i3 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i3 [[N0]], 2
+; CHECK-NEXT:    [[R:%.*]] = xor i3 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i3 [[R]]
 ;
   %n0 = xor i3 %x, %y
   %n1 = and i3 %n0, -3
   %r  = xor i3 %n1, %y
   ret i3 %r
 }
 
 define i3 @t_3_6 (i3 %x, i3 %y) {
 ; CHECK-LABEL: @t_3_6(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i3 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i3 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i3 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i3 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i3 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i3 [[R]]
 ;
   %n0 = xor i3 %x, %y
   %n1 = and i3 %n0, -2
   %r  = xor i3 %n1, %y
   ret i3 %r
 }
 
;
   %n1 = and i4 %n0, 6
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_7 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_7(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], 7
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], -8
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, 7
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_8 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_8(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
 ; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -8
 ; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -8
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_9 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_9(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -7
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 6
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -7
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_10 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_10(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -6
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 5
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -6
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_11 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_11(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -5
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 4
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -5
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_12 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_12(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -4
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 3
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -4
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_13 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_13(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -3
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 2
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -3
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @t_4_14 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @t_4_14(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -2
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 ; ============================================================================ ;
 ; Commutativity
 ; ============================================================================ ;
 
 ; Used to make sure that the IR complexity sorting does not interfere.
 declare i4 @gen4()
 
 define i4 @c_1_0_0 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @c_1_0_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %y, %x ; swapped order
   %n1 = and i4 %n0, -2
   %r  = xor i4 %n1, %y
   ret i4 %r
 }
 
 define i4 @c_0_1_0 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @c_0_1_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[X]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -2
   %r  = xor i4 %n1, %x ; %x instead of %y
   ret i4 %r
 }
 
 define i4 @c_0_0_1 () {
 ; CHECK-LABEL: @c_0_0_1(
 ; CHECK-NEXT:    [[X:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[Y:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X]], [[Y]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[Y]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %x  = call i4 @gen4()
   %y  = call i4 @gen4()
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -2
   %r  = xor i4 %y, %n1 ; swapped order
   ret i4 %r
 }
 
 define i4 @c_1_1_0 (i4 %x, i4 %y) {
 ; CHECK-LABEL: @c_1_1_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[N1]], [[X]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %n0 = xor i4 %y, %x ; swapped order
   %n1 = and i4 %n0, -2
   %r  = xor i4 %n1, %x ; %x instead of %y
   ret i4 %r
 }
 
 define i4 @c_1_0_1 (i4 %x) {
 ; CHECK-LABEL: @c_1_0_1(
 ; CHECK-NEXT:    [[Y:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[Y]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[Y]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %y  = call i4 @gen4()
   %n0 = xor i4 %y, %x ; swapped order
   %n1 = and i4 %n0, -2
   %r  = xor i4 %y, %n1 ; swapped order
   ret i4 %r
 }
 
 define i4 @c_0_1_1 (i4 %y) {
 ; CHECK-LABEL: @c_0_1_1(
 ; CHECK-NEXT:    [[X:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[X]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[X]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %x  = call i4 @gen4()
   %n0 = xor i4 %x, %y
   %n1 = and i4 %n0, -2
   %r  = xor i4 %x, %n1 ; swapped order, %x instead of %y
   ret i4 %r
 }
 
 define i4 @c_1_1_1 () {
 ; CHECK-LABEL: @c_1_1_1(
 ; CHECK-NEXT:    [[X:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[Y:%.*]] = call i4 @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor i4 [[Y]], [[X]]
-; CHECK-NEXT:    [[N1:%.*]] = and i4 [[N0]], -2
-; CHECK-NEXT:    [[R:%.*]] = xor i4 [[X]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i4 [[N0]], 1
+; CHECK-NEXT:    [[R:%.*]] = xor i4 [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret i4 [[R]]
 ;
   %x  = call i4 @gen4()
   %y  = call i4 @gen4()
   %n0 = xor i4 %y, %x ; swapped order
   %n1 = and i4 %n0, -2
   %r  = xor i4 %x, %n1 ; swapped order, %x instead of %y
   ret i4 %r

test/Transforms/InstCombine/invert-const-mask-in-masked-merge-vector.ll

 ; non-canonical, than canonical elements. Undef is considered canonical.
 
 ; All the heavy lifting has already been done in scalar.ll testfile,
 ; so here we can simply test just a few simple cases.
 
 define <2 x i4> @splat (<2 x i4> %x, <2 x i4> %y) {
 ; CHECK-LABEL: @splat(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %n0 = xor <2 x i4> %x, %y
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %n1, %y
   ret <2 x i4> %r
 }
 
 define <3 x i4> @splat_undef (<3 x i4> %x, <3 x i4> %y) {
 ; CHECK-LABEL: @splat_undef(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <3 x i4> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <3 x i4> [[N0]], <i4 -2, i4 undef, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <3 x i4> [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <3 x i4> [[N0]], <i4 1, i4 undef, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <3 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <3 x i4> [[R]]
 ;
   %n0 = xor <3 x i4> %x, %y
   %n1 = and <3 x i4> %n0, <i4 -2, i4 undef, i4 -2>
   %r  = xor <3 x i4> %n1, %y
   ret <3 x i4> %r
 }
 
 define <3 x i4> @nonsplat (<3 x i4> %x, <3 x i4> %y) {
 ; CHECK-LABEL: @nonsplat(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <3 x i4> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <3 x i4> [[N0]], <i4 -2, i4 1, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <3 x i4> [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <3 x i4> [[N0]], <i4 1, i4 -2, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <3 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <3 x i4> [[R]]
 ;
   %n0 = xor <3 x i4> %x, %y
   %n1 = and <3 x i4> %n0, <i4 -2, i4 1, i4 -2> ; 2 of 3 -> do invert
   %r  = xor <3 x i4> %n1, %y
   ret <3 x i4> %r
 }
 
 ; ============================================================================ ;
 ; Commutativity
 ; ============================================================================ ;
 
 ; Used to make sure that the IR complexity sorting does not interfere.
 declare <2 x i4> @gen4()
 
 define <2 x i4> @c_1_0_0 (<2 x i4> %x, <2 x i4> %y) {
 ; CHECK-LABEL: @c_1_0_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[N1]], [[Y]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %n0 = xor <2 x i4> %y, %x ; swapped order
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %n1, %y
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_0_1_0 (<2 x i4> %x, <2 x i4> %y) {
 ; CHECK-LABEL: @c_0_1_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[X:%.*]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[N1]], [[X]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %n0 = xor <2 x i4> %x, %y
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %n1, %x ; %x instead of %y
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_0_0_1 () {
 ; CHECK-LABEL: @c_0_0_1(
 ; CHECK-NEXT:    [[X:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[Y:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[X]], [[Y]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[Y]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %x  = call <2 x i4> @gen4()
   %y  = call <2 x i4> @gen4()
   %n0 = xor <2 x i4> %x, %y
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %y, %n1 ; swapped order
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_1_1_0 (<2 x i4> %x, <2 x i4> %y) {
 ; CHECK-LABEL: @c_1_1_0(
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[Y:%.*]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[N1]], [[X]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %n0 = xor <2 x i4> %y, %x ; swapped order
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %n1, %x ; %x instead of %y
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_1_0_1 (<2 x i4> %x) {
 ; CHECK-LABEL: @c_1_0_1(
 ; CHECK-NEXT:    [[Y:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[Y]], [[X:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[Y]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[X]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %y  = call <2 x i4> @gen4()
   %n0 = xor <2 x i4> %y, %x ; swapped order
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %y, %n1 ; swapped order
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_0_1_1 (<2 x i4> %y) {
 ; CHECK-LABEL: @c_0_1_1(
 ; CHECK-NEXT:    [[X:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[X]], [[Y:%.*]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[X]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %x  = call <2 x i4> @gen4()
   %n0 = xor <2 x i4> %x, %y
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %x, %n1 ; swapped order, %x instead of %y
   ret <2 x i4> %r
 }
 
 define <2 x i4> @c_1_1_1 () {
 ; CHECK-LABEL: @c_1_1_1(
 ; CHECK-NEXT:    [[X:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[Y:%.*]] = call <2 x i4> @gen4()
 ; CHECK-NEXT:    [[N0:%.*]] = xor <2 x i4> [[Y]], [[X]]
-; CHECK-NEXT:    [[N1:%.*]] = and <2 x i4> [[N0]], <i4 -2, i4 -2>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[X]], [[N1]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i4> [[N0]], <i4 1, i4 1>
+; CHECK-NEXT:    [[R:%.*]] = xor <2 x i4> [[TMP1]], [[Y]]
 ; CHECK-NEXT:    ret <2 x i4> [[R]]
 ;
   %x  = call <2 x i4> @gen4()
   %y  = call <2 x i4> @gen4()
   %n0 = xor <2 x i4> %y, %x ; swapped order
   %n1 = and <2 x i4> %n0, <i4 -2, i4 -2>
   %r  = xor <2 x i4> %x, %n1 ; swapped order, %x instead of %y
   ret <2 x i4> %r