[DagCombine] De Morgan laws: 'nand' logic with an inverted operand

[lebedev.ri]

Summary

As discussed in D46031, it seems we want to do this.
Depends on tests in D46072.

[lebedev.ri]

Hmm, and once i fix the vector handling, the same @test_andnotpd / @test_andnotps
in test/CodeGen/X86/{sse,sse2,avx}-schedule.ll break,
just like in the motivational case specified in D46031 :/

[lebedev.ri]

• Rebased ontop of revised tests
• Fix vector handling
• the same @test_andnotpd / @test_andnotps in test/CodeGen/X86/{sse,sse2,avx}-schedule.ll broke, just like in the motivational case specified in D46031 :/

[lebedev.ri]

• Rebased ontop of revised tests.
• Handle nor pattern too.
• Still not sure how to handle vandnpd/vandnps breakage.

[spatel]

In D46073#1080839, @lebedev.ri wrote:

• Rebased ontop of revised tests.
• Handle nor pattern too.
• Still not sure how to handle vandnpd/vandnps breakage.

I lost track of where we are in the masked merge odyssey. :)
I think this is the next step, but we have to avoid regressing the x86 andnps tests?
Can we use TLI.hasAndNot() to predicate the transform? Ie, if you have 'andn', then there's no point trying to eliminate a 'not' that will get merged with another instruction?

[lebedev.ri]

In D46073#1085463, @spatel wrote:

In D46073#1080839, @lebedev.ri wrote:

• Rebased ontop of revised tests.
• Handle nor pattern too.
• Still not sure how to handle vandnpd/vandnps breakage.

I lost track of where we are in the masked merge odyssey. :)

We are *almost* there..

I think this is the next step

I think we could handle D46031 first.

but we have to avoid regressing the x86 andnps tests?

Yes.

Can we use TLI.hasAndNot() to predicate the transform? Ie, if you have 'andn', then there's no point trying to eliminate a 'not' that will get merged with another instruction?

[lebedev.ri]

In D46073#1085463, @spatel wrote:

In D46073#1080839, @lebedev.ri wrote:

• Rebased ontop of revised tests.
• Handle nor pattern too.
• Still not sure how to handle vandnpd/vandnps breakage.

I lost track of where we are in the masked merge odyssey. :)
I think this is the next step, but we have to avoid regressing the x86 andnps tests?

Can we use TLI.hasAndNot() to predicate the transform? Ie, if you have 'andn', then there's no point trying to eliminate a 'not' that will get merged with another instruction?

I think this would need to be a bit more complex. E.g.:

• if this not is used in and, and the other hand of and ...
  • ... is ok as per hasAndNot() (i.e. in x86's case, not an immediate), then there is no immediate benefit of simplification. [A]
  • ... is not ok as per hasAndNot() (i.e. in x86's case, e.g. an immediate), then we shouldn't at least degrade the code by simplifying. [B]
• else, if it is used elsewhere (not in and), then we shouldn't at least degrade the code by simplifying. [B]
• We have two patterns. (Let's look at the inner and/or):
  • ~(~A & B) --> (A | ~B) <- the original form *may* [00] or may not [01] get folded into andn, the transformed variant is unlikely to be folded (ORN: logical OR NOT (Thumb only)).
  • ~(~A | B) --> (A & ~B) <- the original form is unlikely to be folded into orn, the transformed variant *may* [10] or may not [11] to be folded. [1]

So we have 8 variants: (X being the other hand)

from	to	delta
OP X, not(and(not(A), B))	OP X, or (A, not(B))	3 -> 2, transform
OP X, not(andn(A, B))	OP X, or (A, not(B))	2 -> 2, keep
andn X, and(not(A), B)	and X, or (A, not(B))	2 -> 2, keep
andn X, andn(A, B)	and X, or (A, not(B))	1 -> 2, keep!
OP X, not(or(not(A), B))	OP X, and (A, not(B))	3 -> 2, transform
OP X, not(or(not(A), B))	OP X, andn (A, B)	3 -> 1, transform!
andn X, or(not(A), B)	and X, and (A, not(B))	2 -> 2, keep
andn X, or(not(A), B)	and X, andn (A, B)	2 -> 1, transform

In other words there seems to be only one case where we certainly should *not* transform.
Though we may want to also evaluate whether X is actually NOT(Y), and whether this transform would allow to fold that NOT into and.
Also, the inverse transform (the opposite of this differential) needs to be evaluated, if it helps to fold into andn.

[lebedev.ri]

This comment was removed by lebedev.ri.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

private:
 
     SDValue visitShiftByConstant(SDNode *N, ConstantSDNode *Amt);
 
     SDValue foldSelectOfConstants(SDNode *N);
     SDValue foldVSelectOfConstants(SDNode *N);
     SDValue foldBinOpIntoSelect(SDNode *BO);
     bool SimplifySelectOps(SDNode *SELECT, SDValue LHS, SDValue RHS);
     SDValue SimplifyBinOpWithSameOpcodeHands(SDNode *N);
+    SDValue SimplifyUsingDeMorganLaws(SDNode *N);
     SDValue SimplifySelect(const SDLoc &DL, SDValue N0, SDValue N1, SDValue N2);
     SDValue SimplifySelectCC(const SDLoc &DL, SDValue N0, SDValue N1,
                              SDValue N2, SDValue N3, ISD::CondCode CC,
                              bool NotExtCompare = false);
     SDValue foldSelectCCToShiftAnd(const SDLoc &DL, SDValue N0, SDValue N1,
                                    SDValue N2, SDValue N3, ISD::CondCode CC);
     SDValue foldLogicOfSetCCs(bool IsAnd, SDValue N0, SDValue N1,
                               const SDLoc &DL);
SDValue DAGCombiner::MatchLoadCombine(SDNode *N) {
 
   // Transfer chain users from old loads to the new load.
   for (LoadSDNode *L : Loads)
     DAG.ReplaceAllUsesOfValueWith(SDValue(L, 1), SDValue(NewLoad.getNode(), 1));
 
   return NeedsBswap ? DAG.getNode(ISD::BSWAP, SDLoc(N), VT, NewLoad) : NewLoad;
 }
 
+// Apply DeMorgan's Law for 'nand'/'nor' logic with an inverted operand.
+// xor (and (xor A, -1), B), -1 -> or  (xor B, -1), A
+// xor (or  (xor A, -1), B), -1 -> and (xor B, -1), A
+SDValue DAGCombiner::SimplifyUsingDeMorganLaws(SDNode *N) {
+  assert(N->getOpcode() == ISD::XOR);
+
+  SDValue N0 = N->getOperand(0);
+  SDValue N1 = N->getOperand(1);
+  EVT VT = N0.getValueType();
+
+  if (!isAllOnesConstantOrAllOnesSplatConstant(N1))
+    return SDValue();
+
+  unsigned Opcode;
+  SDValue A, B;
+
+  auto matchAB = [&Opcode, &A, &B](SDValue BinOp, bool swapXors) -> bool {
+    if (!(BinOp.getOpcode() == ISD::AND || BinOp.getOpcode() == ISD::OR) ||
+        !BinOp.hasOneUse())
+      return false;
+    SDValue X0 = BinOp->getOperand(0);
+    SDValue X1 = BinOp->getOperand(1);
+    if (X0.getOpcode() != ISD::XOR || swapXors)
+      std::swap(X1, X0);
+    if (X0.getOpcode() != ISD::XOR || !X0.hasOneUse() ||
+        !isAllOnesConstantOrAllOnesSplatConstant(X0->getOperand(1)))
+      return false;
+    Opcode = BinOp.getOpcode();
+    A = X0->getOperand(0);
+    B = X1;
+    return true;
+  };
+
+  if (!(matchAB(N0, false) || matchAB(N0, true) || matchAB(N1, false) ||
+        matchAB(N1, true)))
+    return SDValue();
+
+  SDLoc DL(N);
+
+  unsigned InvertedOpcode = Opcode == ISD::AND ? ISD::OR : ISD::AND;
+  SDValue NotB = DAG.getNOT(DL, B, VT);
+  return DAG.getNode(InvertedOpcode, DL, VT, NotB, A);
+}
+
 // If the target has andn, bsl, or a similar bit-select instruction,
 // we want to unfold masked merge, with canonical pattern of:
 //   |        A  |  |B|
 //   ((x ^ y) & m) ^ y
 //    |  D  |
 // Into:
 //   (x & m) | (y & ~m)
 SDValue DAGCombiner::unfoldMaskedMerge(SDNode *N) {
return DAG.getNode(ISD::ROTL, DL, VT, DAG.getConstant(~1, DL, VT),
                        N0.getOperand(1));
   }
 
   // Simplify: xor (op x...), (op y...)  -> (op (xor x, y))
   if (N0.getOpcode() == N1.getOpcode())
     if (SDValue Tmp = SimplifyBinOpWithSameOpcodeHands(N))
       return Tmp;
 
+  if (SDValue Tmp = SimplifyUsingDeMorganLaws(N))
+    return Tmp;
+
   // Unfold  ((x ^ y) & m) ^ y  into  (x & m) | (y & ~m)  if profitable
   if (SDValue MM = unfoldMaskedMerge(N))
     return MM;
 
   // Simplify the expression using non-local knowledge.
   if (SimplifyDemandedBits(SDValue(N, 0)))
     return SDValue(N, 0);
 

test/CodeGen/AArch64/demorgan-extra.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-unknown-linux-gnu -o - %s | FileCheck %s
 
 ; There is a identical twin test in test/Transforms/InstCombine/demorgan-extra.ll
 ; Please keep them in sync! :)
 
 declare void @use32(i32)
 declare i32 @gen32()
 
 ; ============================================================================ ;
 
 ; ~(~A & B) --> (A | ~B)
 
 define i32 @demorgan_nand(i32 %A, i32 %B) nounwind {
 ; CHECK-LABEL: demorgan_nand:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    and w8, w0, w1
-; CHECK-NEXT:    orn w0, w8, w1
+; CHECK-NEXT:    orn w0, w0, w1
 ; CHECK-NEXT:    ret
   %notx = xor i32 %A, -1
   %c = and i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_const(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_const:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    mov w8, #-43
 ; CHECK-NEXT:    orr w0, w0, w8
 ; CHECK-NEXT:    ret
   %notx = xor i32 %A, -1
   %c = and i32 %notx, 42
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define <2 x i32> @demorgan_nand_vec(<2 x i32> %A, <2 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    bic v0.8b, v1.8b, v0.8b
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    orn v0.8b, v0.8b, v1.8b
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, %B
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nand_vec_undef(<3 x i32> %A, <3 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_undef:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    bic v0.16b, v1.16b, v0.16b
 ; CHECK-NEXT:    mvn v0.16b, v0.16b
 ; CHECK-NEXT:    ret
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = and <3 x i32> %notx, %B
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nand_vec_splatconst(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_splatconst:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    movi v1.2s, #42
-; CHECK-NEXT:    bic v0.8b, v1.8b, v0.8b
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    mvni v1.2s, #42
+; CHECK-NEXT:    orr v0.8b, v0.8b, v1.8b
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, <i32 42, i32 42>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nand_vec_const_undef(<3 x i32> %A) nounwind {
; CHECK-NEXT:    ret
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nand_vec_const(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_const:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x8, .LCPI6_0
 ; CHECK-NEXT:    ldr d1, [x8, :lo12:.LCPI6_0]
-; CHECK-NEXT:    bic v0.8b, v1.8b, v0.8b
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    orr v0.8b, v0.8b, v1.8b
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, <i32 42, i32 31>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nand_vec_128bit(<4 x i32> %A, <4 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_128bit:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    bic v0.16b, v1.16b, v0.16b
-; CHECK-NEXT:    mvn v0.16b, v0.16b
+; CHECK-NEXT:    orn v0.16b, v0.16b, v1.16b
 ; CHECK-NEXT:    ret
   %notx = xor <4 x i32> %A, <i32 -1, i32 -1, i32 -1, i32 -1>
   %c = and <4 x i32> %notx, %B
   %notc = xor <4 x i32> %c, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nand_vec_128bit_undef(<4 x i32> %A, <4 x i32> %B) nounwind {

 }
 
 define i32 @demorgan_nand_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_commutative:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    stp x19, x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
-; CHECK-NEXT:    and w8, w19, w0
-; CHECK-NEXT:    orn w0, w8, w0
+; CHECK-NEXT:    orn w0, w19, w0
 ; CHECK-NEXT:    ldp x19, x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %B = call i32 @gen32()
   %notx = xor i32 %A, -1
   %c = and i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_extraxor(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_extraxor:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x20, [sp, #-32]! // 8-byte Folded Spill
 ; CHECK-NEXT:    stp x19, x30, [sp, #16] // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    mov w20, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    eor w8, w20, w0
-; CHECK-NEXT:    and w9, w19, w8
+; CHECK-NEXT:    orn w0, w19, w8
 ; CHECK-NEXT:    ldp x19, x30, [sp, #16] // 8-byte Folded Reload
-; CHECK-NEXT:    orn w0, w9, w8
 ; CHECK-NEXT:    ldr x20, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = and i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_extraxor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_extraxor_commutative:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x20, [sp, #-32]! // 8-byte Folded Spill
 ; CHECK-NEXT:    stp x19, x30, [sp, #16] // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    mov w20, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    eor w8, w20, w0
-; CHECK-NEXT:    and w9, w19, w8
+; CHECK-NEXT:    orn w0, w19, w8
 ; CHECK-NEXT:    ldp x19, x30, [sp, #16] // 8-byte Folded Reload
-; CHECK-NEXT:    orn w0, w9, w8
 ; CHECK-NEXT:    ldr x20, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = and i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1

 
 ; ============================================================================ ;
 
 ; ~(~A | B) --> (A & ~B)
 
 define i32 @demorgan_nor(i32 %A, i32 %B) nounwind {
 ; CHECK-LABEL: demorgan_nor:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    orn w8, w1, w0
-; CHECK-NEXT:    mvn w0, w8
+; CHECK-NEXT:    bic w0, w0, w1
 ; CHECK-NEXT:    ret
   %notx = xor i32 %A, -1
   %c = or i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_const(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_const:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    mov w8, #-43
 ; CHECK-NEXT:    and w0, w0, w8
 ; CHECK-NEXT:    ret
   %notx = xor i32 %A, -1
   %c = or i32 %notx, 42
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define <2 x i32> @demorgan_nor_vec(<2 x i32> %A, <2 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    orn v0.8b, v1.8b, v0.8b
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    bic v0.8b, v0.8b, v1.8b
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, %B
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nor_vec_undef(<3 x i32> %A, <3 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_undef:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    orn v0.16b, v1.16b, v0.16b
 ; CHECK-NEXT:    mvn v0.16b, v0.16b
 ; CHECK-NEXT:    ret
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = or <3 x i32> %notx, %B
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nor_vec_splatconst(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_splatconst:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mvn v0.8b, v0.8b
-; CHECK-NEXT:    orr v0.2s, #42
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    bic v0.2s, #42
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, <i32 42, i32 42>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nor_vec_const_undef(<3 x i32> %A) nounwind {
; CHECK-NEXT:    ret
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nor_vec_const(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_const:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x8, .LCPI22_0
 ; CHECK-NEXT:    ldr d1, [x8, :lo12:.LCPI22_0]
-; CHECK-NEXT:    orn v0.8b, v1.8b, v0.8b
-; CHECK-NEXT:    mvn v0.8b, v0.8b
+; CHECK-NEXT:    and v0.8b, v0.8b, v1.8b
 ; CHECK-NEXT:    ret
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, <i32 42, i32 31>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nor_vec_128bit(<4 x i32> %A, <4 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_128bit:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    orn v0.16b, v1.16b, v0.16b
-; CHECK-NEXT:    mvn v0.16b, v0.16b
+; CHECK-NEXT:    bic v0.16b, v0.16b, v1.16b
 ; CHECK-NEXT:    ret
   %notx = xor <4 x i32> %A, <i32 -1, i32 -1, i32 -1, i32 -1>
   %c = or <4 x i32> %notx, %B
   %notc = xor <4 x i32> %c, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nor_vec_128bit_undef(<4 x i32> %A, <4 x i32> %B) nounwind {

 }
 
 define i32 @demorgan_nor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_commutative:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    stp x19, x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
-; CHECK-NEXT:    orn w8, w0, w19
-; CHECK-NEXT:    mvn w0, w8
+; CHECK-NEXT:    bic w0, w19, w0
 ; CHECK-NEXT:    ldp x19, x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %B = call i32 @gen32()
   %notx = xor i32 %A, -1
   %c = or i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_extraxor(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_extraxor:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x20, [sp, #-32]! // 8-byte Folded Spill
 ; CHECK-NEXT:    stp x19, x30, [sp, #16] // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    mov w20, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    eor w8, w20, w0
-; CHECK-NEXT:    orn w8, w8, w19
+; CHECK-NEXT:    bic w0, w19, w8
 ; CHECK-NEXT:    ldp x19, x30, [sp, #16] // 8-byte Folded Reload
-; CHECK-NEXT:    mvn w0, w8
 ; CHECK-NEXT:    ldr x20, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = or i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_extraxor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_extraxor_commutative:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    str x20, [sp, #-32]! // 8-byte Folded Spill
 ; CHECK-NEXT:    stp x19, x30, [sp, #16] // 8-byte Folded Spill
 ; CHECK-NEXT:    mov w19, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    mov w20, w0
 ; CHECK-NEXT:    bl gen32
 ; CHECK-NEXT:    eor w8, w20, w0
-; CHECK-NEXT:    orn w8, w8, w19
+; CHECK-NEXT:    bic w0, w19, w8
 ; CHECK-NEXT:    ldp x19, x30, [sp, #16] // 8-byte Folded Reload
-; CHECK-NEXT:    mvn w0, w8
 ; CHECK-NEXT:    ldr x20, [sp], #32 // 8-byte Folded Reload
 ; CHECK-NEXT:    ret
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = or i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1

test/CodeGen/AArch64/unfold-masked-merge-scalar-variablemask.ll

; CHECK-NEXT:    ret
   %mx = and i32 %mask, %x
   %my = and i32 %notmask, -1
   %r = or i32 %mx, %my
   ret i32 %r
 }
 define i32 @in_constant_varx_mone(i32 %x, i32 %y, i32 %mask) {
 ; CHECK-LABEL: in_constant_varx_mone:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    and w8, w0, w2
-; CHECK-NEXT:    orn w0, w8, w2
+; CHECK-NEXT:    orn w0, w0, w2
 ; CHECK-NEXT:    ret
   %n0 = xor i32 %x, -1 ; %x
   %n1 = and i32 %n0, %mask
   %r = xor i32 %n1, -1
   ret i32 %r
 }
 define i32 @out_constant_varx_mone_invmask(i32 %x, i32 %y, i32 %mask) {
 ; CHECK-LABEL: out_constant_varx_mone_invmask:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    bic w8, w0, w2
 ; CHECK-NEXT:    orr w0, w8, w2
 ; CHECK-NEXT:    ret
   %notmask = xor i32 %mask, -1
   %mx = and i32 %notmask, %x
   %my = and i32 %mask, -1
   %r = or i32 %mx, %my
   ret i32 %r
 }
 define i32 @in_constant_varx_mone_invmask(i32 %x, i32 %y, i32 %mask) {
 ; CHECK-LABEL: in_constant_varx_mone_invmask:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    bic w8, w0, w2
-; CHECK-NEXT:    orr w0, w8, w2
+; CHECK-NEXT:    orr w0, w2, w0
 ; CHECK-NEXT:    ret
   %notmask = xor i32 %mask, -1
   %n0 = xor i32 %x, -1 ; %x
   %n1 = and i32 %n0, %notmask
   %r = xor i32 %n1, -1
   ret i32 %r
 }
 define i32 @out_constant_varx_42(i32 %x, i32 %y, i32 %mask) {

test/CodeGen/X86/avx-schedule.ll

; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %3 = call <8 x float> @llvm.x86.avx.addsub.ps.256(<8 x float> %1, <8 x float> %2)
   ret <8 x float> %3
 }
 declare <8 x float> @llvm.x86.avx.addsub.ps.256(<8 x float>, <8 x float>) nounwind readnone
 
 define <4 x double> @test_andnotpd(<4 x double> %a0, <4 x double> %a1, <4 x double> *%a2) {
 ; GENERIC-LABEL: test_andnotpd:
 ; GENERIC:       # %bb.0:
-; GENERIC-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; GENERIC-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; GENERIC-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; GENERIC-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; GENERIC-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; GENERIC-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; GENERIC-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; GENERIC-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; SANDY-LABEL: test_andnotpd:
 ; SANDY:       # %bb.0:
-; SANDY-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; SANDY-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; SANDY-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; SANDY-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; SANDY-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; SANDY-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; SANDY-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; SANDY-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; SANDY-NEXT:    retq # sched: [1:1.00]
 ;
 ; HASWELL-LABEL: test_andnotpd:
 ; HASWELL:       # %bb.0:
-; HASWELL-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; HASWELL-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; HASWELL-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; HASWELL-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; HASWELL-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; HASWELL-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; HASWELL-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; HASWELL-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; HASWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-LABEL: test_andnotpd:
 ; BROADWELL:       # %bb.0:
-; BROADWELL-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; BROADWELL-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [7:1.00]
+; BROADWELL-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; BROADWELL-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; BROADWELL-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; BROADWELL-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; BROADWELL-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [7:1.00]
 ; BROADWELL-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; BROADWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-LABEL: test_andnotpd:
 ; SKYLAKE:       # %bb.0:
-; SKYLAKE-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:0.33]
-; SKYLAKE-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; SKYLAKE-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [4:0.50]
+; SKYLAKE-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; SKYLAKE-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [4:0.50]
 ; SKYLAKE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-LABEL: test_andnotpd:
 ; SKX:       # %bb.0:
-; SKX-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:0.33]
-; SKX-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; SKX-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.33]
+; SKX-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [4:0.33]
+; SKX-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.33]
+; SKX-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.33]
+; SKX-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; SKX-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [4:0.33]
 ; SKX-NEXT:    retq # sched: [7:1.00]
 ;
 ; BTVER2-LABEL: test_andnotpd:
 ; BTVER2:       # %bb.0:
-; BTVER2-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; BTVER2-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [6:2.00]
+; BTVER2-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [2:2.00]
+; BTVER2-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; BTVER2-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; BTVER2-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [6:2.00]
 ; BTVER2-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:2.00]
 ; BTVER2-NEXT:    retq # sched: [4:1.00]
 ;
 ; ZNVER1-LABEL: test_andnotpd:
 ; ZNVER1:       # %bb.0:
-; ZNVER1-NEXT:    vandnpd %ymm1, %ymm0, %ymm0 # sched: [1:0.25]
-; ZNVER1-NEXT:    vandnpd (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; ZNVER1-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; ZNVER1-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.25]
+; ZNVER1-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; ZNVER1-NEXT:    vaddpd %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %1 = bitcast <4 x double> %a0 to <4 x i64>
   %2 = bitcast <4 x double> %a1 to <4 x i64>
   %3 = xor <4 x i64> %1, <i64 -1, i64 -1, i64 -1, i64 -1>
   %4 = and <4 x i64> %3, %2
   %5 = load <4 x double>, <4 x double> *%a2, align 32
   %6 = bitcast <4 x double> %5 to <4 x i64>
   %7 = xor <4 x i64> %4, <i64 -1, i64 -1, i64 -1, i64 -1>
   %8 = and <4 x i64> %6, %7
   %9 = bitcast <4 x i64> %8 to <4 x double>
   %10 = fadd <4 x double> %a1, %9
   ret <4 x double> %10
 }
 
 define <8 x float> @test_andnotps(<8 x float> %a0, <8 x float> %a1, <8 x float> *%a2) {
 ; GENERIC-LABEL: test_andnotps:
 ; GENERIC:       # %bb.0:
-; GENERIC-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; GENERIC-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; GENERIC-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; GENERIC-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; GENERIC-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; GENERIC-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; GENERIC-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; GENERIC-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; SANDY-LABEL: test_andnotps:
 ; SANDY:       # %bb.0:
-; SANDY-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; SANDY-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; SANDY-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; SANDY-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; SANDY-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; SANDY-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; SANDY-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; SANDY-NEXT:    retq # sched: [1:1.00]
 ;
 ; HASWELL-LABEL: test_andnotps:
 ; HASWELL:       # %bb.0:
-; HASWELL-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; HASWELL-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
+; HASWELL-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; HASWELL-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; HASWELL-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; HASWELL-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; HASWELL-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:1.00]
 ; HASWELL-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; HASWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-LABEL: test_andnotps:
 ; BROADWELL:       # %bb.0:
-; BROADWELL-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; BROADWELL-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [7:1.00]
+; BROADWELL-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:1.00]
+; BROADWELL-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; BROADWELL-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; BROADWELL-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; BROADWELL-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [7:1.00]
 ; BROADWELL-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; BROADWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-LABEL: test_andnotps:
 ; SKYLAKE:       # %bb.0:
-; SKYLAKE-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:0.33]
-; SKYLAKE-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; SKYLAKE-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [4:0.50]
+; SKYLAKE-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; SKYLAKE-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [4:0.50]
 ; SKYLAKE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-LABEL: test_andnotps:
 ; SKX:       # %bb.0:
-; SKX-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:0.33]
-; SKX-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; SKX-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.33]
+; SKX-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [4:0.33]
+; SKX-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.33]
+; SKX-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.33]
+; SKX-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; SKX-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [4:0.33]
 ; SKX-NEXT:    retq # sched: [7:1.00]
 ;
 ; BTVER2-LABEL: test_andnotps:
 ; BTVER2:       # %bb.0:
-; BTVER2-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:1.00]
-; BTVER2-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [6:2.00]
+; BTVER2-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [2:2.00]
+; BTVER2-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:1.00]
+; BTVER2-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:1.00]
+; BTVER2-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [6:2.00]
 ; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:2.00]
 ; BTVER2-NEXT:    retq # sched: [4:1.00]
 ;
 ; ZNVER1-LABEL: test_andnotps:
 ; ZNVER1:       # %bb.0:
-; ZNVER1-NEXT:    vandnps %ymm1, %ymm0, %ymm0 # sched: [1:0.25]
-; ZNVER1-NEXT:    vandnps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
+; ZNVER1-NEXT:    vxorps %xmm2, %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vcmptrueps %ymm2, %ymm2, %ymm2 # sched: [3:1.00]
+; ZNVER1-NEXT:    vxorps %ymm2, %ymm1, %ymm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vorps %ymm0, %ymm2, %ymm0 # sched: [1:0.25]
+; ZNVER1-NEXT:    vandps (%rdi), %ymm0, %ymm0 # sched: [8:0.50]
 ; ZNVER1-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # sched: [3:1.00]
 ; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %1 = bitcast <8 x float> %a0 to <4 x i64>
   %2 = bitcast <8 x float> %a1 to <4 x i64>
   %3 = xor <4 x i64> %1, <i64 -1, i64 -1, i64 -1, i64 -1>
   %4 = and <4 x i64> %3, %2
   %5 = load <8 x float>, <8 x float> *%a2, align 32
   %6 = bitcast <8 x float> %5 to <4 x i64>

test/CodeGen/X86/demorgan-extra.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -o - %s | FileCheck %s
 
 ; There is a identical twin test in test/Transforms/InstCombine/demorgan-extra.ll
 ; Please keep them in sync! :)
 
 declare void @use32(i32)
 declare i32 @gen32()
 
 ; ============================================================================ ;
 
 ; ~(~A & B) --> (A | ~B)
 
 define i32 @demorgan_nand(i32 %A, i32 %B) nounwind {
 ; CHECK-LABEL: demorgan_nand:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    notl %edi
-; CHECK-NEXT:    andl %esi, %edi
-; CHECK-NEXT:    notl %edi
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    notl %esi
+; CHECK-NEXT:    orl %edi, %esi
+; CHECK-NEXT:    movl %esi, %eax
 ; CHECK-NEXT:    retq
   %notx = xor i32 %A, -1
   %c = and i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_const(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_const:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    orl $-43, %edi
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    retq
   %notx = xor i32 %A, -1
   %c = and i32 %notx, 42
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define <2 x i32> @demorgan_nand_vec(<2 x i32> %A, <2 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movaps {{.*#+}} xmm2 = [4294967295,4294967295]
-; CHECK-NEXT:    xorps %xmm2, %xmm0
-; CHECK-NEXT:    andps %xmm1, %xmm0
-; CHECK-NEXT:    xorps %xmm2, %xmm0
+; CHECK-NEXT:    xorps {{.*}}(%rip), %xmm1
+; CHECK-NEXT:    orps %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, %B
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nand_vec_undef(<3 x i32> %A, <3 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_undef:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pcmpeqd %xmm2, %xmm2
 ; CHECK-NEXT:    pandn %xmm1, %xmm0
 ; CHECK-NEXT:    pxor %xmm2, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = and <3 x i32> %notx, %B
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nand_vec_splatconst(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_splatconst:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andnps {{.*}}(%rip), %xmm0
-; CHECK-NEXT:    xorps {{.*}}(%rip), %xmm0
+; CHECK-NEXT:    orps {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, <i32 42, i32 42>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nand_vec_const_undef(<3 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_const_undef:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pcmpeqd %xmm1, %xmm1
 ; CHECK-NEXT:    pandn {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    pxor %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = and <3 x i32> %notx, <i32 42, i32 undef, i32 42>
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nand_vec_const(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_const:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    andnps {{.*}}(%rip), %xmm0
-; CHECK-NEXT:    xorps {{.*}}(%rip), %xmm0
+; CHECK-NEXT:    orps {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = and <2 x i32> %notx, <i32 42, i32 31>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nand_vec_128bit(<4 x i32> %A, <4 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nand_vec_128bit:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pcmpeqd %xmm2, %xmm2
-; CHECK-NEXT:    pandn %xmm1, %xmm0
-; CHECK-NEXT:    pxor %xmm2, %xmm0
+; CHECK-NEXT:    pxor %xmm1, %xmm2
+; CHECK-NEXT:    por %xmm2, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <4 x i32> %A, <i32 -1, i32 -1, i32 -1, i32 -1>
   %c = and <4 x i32> %notx, %B
   %notc = xor <4 x i32> %c, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nand_vec_128bit_undef(<4 x i32> %A, <4 x i32> %B) nounwind {

 }
 
 define i32 @demorgan_nand_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_commutative:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    retq
   %B = call i32 @gen32()
   %notx = xor i32 %A, -1
   %c = and i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_extraxor(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_extraxor:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    movl %eax, %ebp
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    xorl %ebp, %eax
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = and i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nand_extraxor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nand_extraxor_commutative:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    movl %eax, %ebp
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    xorl %ebp, %eax
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1

 
 ; ============================================================================ ;
 
 ; ~(~A | B) --> (A & ~B)
 
 define i32 @demorgan_nor(i32 %A, i32 %B) nounwind {
 ; CHECK-LABEL: demorgan_nor:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    notl %edi
-; CHECK-NEXT:    orl %esi, %edi
-; CHECK-NEXT:    notl %edi
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    notl %esi
+; CHECK-NEXT:    andl %edi, %esi
+; CHECK-NEXT:    movl %esi, %eax
 ; CHECK-NEXT:    retq
   %notx = xor i32 %A, -1
   %c = or i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_const(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_const:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    andl $-43, %edi
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    retq
   %notx = xor i32 %A, -1
   %c = or i32 %notx, 42
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define <2 x i32> @demorgan_nor_vec(<2 x i32> %A, <2 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movaps {{.*#+}} xmm2 = [4294967295,4294967295]
-; CHECK-NEXT:    xorps %xmm2, %xmm0
-; CHECK-NEXT:    orps %xmm1, %xmm0
-; CHECK-NEXT:    xorps %xmm2, %xmm0
+; CHECK-NEXT:    xorps {{.*}}(%rip), %xmm1
+; CHECK-NEXT:    andps %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, %B
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nor_vec_undef(<3 x i32> %A, <3 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_undef:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pcmpeqd %xmm2, %xmm2
 ; CHECK-NEXT:    pxor %xmm2, %xmm0
 ; CHECK-NEXT:    por %xmm1, %xmm0
 ; CHECK-NEXT:    pxor %xmm2, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = or <3 x i32> %notx, %B
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nor_vec_splatconst(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_splatconst:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movaps {{.*#+}} xmm1 = [4294967295,4294967295]
-; CHECK-NEXT:    xorps %xmm1, %xmm0
-; CHECK-NEXT:    orps {{.*}}(%rip), %xmm0
-; CHECK-NEXT:    xorps %xmm1, %xmm0
+; CHECK-NEXT:    andps {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, <i32 42, i32 42>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <3 x i32> @demorgan_nor_vec_const_undef(<3 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_const_undef:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pcmpeqd %xmm1, %xmm1
 ; CHECK-NEXT:    pxor %xmm1, %xmm0
 ; CHECK-NEXT:    por {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    pxor %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <3 x i32> %A, <i32 -1, i32 undef, i32 -1>
   %c = or <3 x i32> %notx, <i32 42, i32 undef, i32 42>
   %notc = xor <3 x i32> %c, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %notc
 }
 
 define <2 x i32> @demorgan_nor_vec_const(<2 x i32> %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_const:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movaps {{.*#+}} xmm1 = [4294967295,4294967295]
-; CHECK-NEXT:    xorps %xmm1, %xmm0
-; CHECK-NEXT:    orps {{.*}}(%rip), %xmm0
-; CHECK-NEXT:    xorps %xmm1, %xmm0
+; CHECK-NEXT:    andps {{.*}}(%rip), %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <2 x i32> %A, <i32 -1, i32 -1>
   %c = or <2 x i32> %notx, <i32 42, i32 31>
   %notc = xor <2 x i32> %c, <i32 -1, i32 -1>
   ret <2 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nor_vec_128bit(<4 x i32> %A, <4 x i32> %B) nounwind {
 ; CHECK-LABEL: demorgan_nor_vec_128bit:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    pcmpeqd %xmm2, %xmm2
-; CHECK-NEXT:    pxor %xmm2, %xmm0
-; CHECK-NEXT:    por %xmm1, %xmm0
-; CHECK-NEXT:    pxor %xmm2, %xmm0
+; CHECK-NEXT:    andnps %xmm0, %xmm1
+; CHECK-NEXT:    movaps %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %notx = xor <4 x i32> %A, <i32 -1, i32 -1, i32 -1, i32 -1>
   %c = or <4 x i32> %notx, %B
   %notc = xor <4 x i32> %c, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %notc
 }
 
 define <4 x i32> @demorgan_nor_vec_128bit_undef(<4 x i32> %A, <4 x i32> %B) nounwind {

 }
 
 define i32 @demorgan_nor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_commutative:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    retq
   %B = call i32 @gen32()
   %notx = xor i32 %A, -1
   %c = or i32 %B, %notx ; swapped
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_extraxor(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_extraxor:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    movl %eax, %ebp
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    xorl %ebp, %eax
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1
   %c = or i32 %notx, %B
   %notc = xor i32 %c, -1
   ret i32 %notc
 }
 
 define i32 @demorgan_nor_extraxor_commutative(i32 %A) nounwind {
 ; CHECK-LABEL: demorgan_nor_extraxor_commutative:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movl %edi, %ebx
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    movl %eax, %ebp
 ; CHECK-NEXT:    callq gen32
 ; CHECK-NEXT:    xorl %ebp, %eax
-; CHECK-NEXT:    notl %ebx
-; CHECK-NEXT:    orl %ebx, %eax
 ; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    andl %ebx, %eax
 ; CHECK-NEXT:    addq $8, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
 ; CHECK-NEXT:    retq
   %V = call i32 @gen32()
   %Z = call i32 @gen32()
   %B = xor i32 %V, %Z ; not with -1
   %notx = xor i32 %A, -1

test/CodeGen/X86/sse-schedule.ll

; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %6 = and <4 x i32> %3, %5
   %7 = bitcast <4 x i32> %6 to <4 x float>
   ret <4 x float> %7
 }
 
 define <4 x float> @test_andnotps(<4 x float> %a0, <4 x float> %a1, <4 x float> *%a2) {
 ; GENERIC-LABEL: test_andnotps:
 ; GENERIC:       # %bb.0:
-; GENERIC-NEXT:    andnps %xmm1, %xmm0 # sched: [1:1.00]
-; GENERIC-NEXT:    andnps (%rdi), %xmm0 # sched: [7:1.00]
+; GENERIC-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; GENERIC-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; GENERIC-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; GENERIC-NEXT:    pand (%rdi), %xmm0 # sched: [7:0.50]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; ATOM-LABEL: test_andnotps:
 ; ATOM:       # %bb.0:
-; ATOM-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.50]
-; ATOM-NEXT:    andnps (%rdi), %xmm0 # sched: [1:1.00]
-; ATOM-NEXT:    nop # sched: [1:0.50]
-; ATOM-NEXT:    nop # sched: [1:0.50]
-; ATOM-NEXT:    nop # sched: [1:0.50]
-; ATOM-NEXT:    nop # sched: [1:0.50]
+; ATOM-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; ATOM-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; ATOM-NEXT:    por %xmm2, %xmm0 # sched: [1:0.50]
+; ATOM-NEXT:    pand (%rdi), %xmm0 # sched: [1:1.00]
 ; ATOM-NEXT:    retq # sched: [79:39.50]
 ;
 ; SLM-LABEL: test_andnotps:
 ; SLM:       # %bb.0:
-; SLM-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.50]
-; SLM-NEXT:    andnps (%rdi), %xmm0 # sched: [4:1.00]
+; SLM-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SLM-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; SLM-NEXT:    por %xmm2, %xmm0 # sched: [1:0.50]
+; SLM-NEXT:    pand (%rdi), %xmm0 # sched: [4:1.00]
 ; SLM-NEXT:    retq # sched: [4:1.00]
 ;
 ; SANDY-SSE-LABEL: test_andnotps:
 ; SANDY-SSE:       # %bb.0:
-; SANDY-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:1.00]
-; SANDY-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [7:1.00]
+; SANDY-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SANDY-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SANDY-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; SANDY-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [7:0.50]
 ; SANDY-SSE-NEXT:    retq # sched: [1:1.00]
 ;
 ; SANDY-LABEL: test_andnotps:
 ; SANDY:       # %bb.0:
-; SANDY-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; SANDY-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [7:1.00]
+; SANDY-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SANDY-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.33]
+; SANDY-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.33]
+; SANDY-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SANDY-NEXT:    retq # sched: [1:1.00]
 ;
 ; HASWELL-SSE-LABEL: test_andnotps:
 ; HASWELL-SSE:       # %bb.0:
-; HASWELL-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:1.00]
-; HASWELL-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [7:1.00]
+; HASWELL-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; HASWELL-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; HASWELL-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; HASWELL-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [7:0.50]
 ; HASWELL-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; HASWELL-LABEL: test_andnotps:
 ; HASWELL:       # %bb.0:
-; HASWELL-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; HASWELL-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [7:1.00]
+; HASWELL-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; HASWELL-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.33]
+; HASWELL-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.33]
+; HASWELL-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; HASWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-SSE-LABEL: test_andnotps:
 ; BROADWELL-SSE:       # %bb.0:
-; BROADWELL-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:1.00]
-; BROADWELL-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [6:1.00]
+; BROADWELL-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; BROADWELL-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; BROADWELL-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; BROADWELL-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [6:0.50]
 ; BROADWELL-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-LABEL: test_andnotps:
 ; BROADWELL:       # %bb.0:
-; BROADWELL-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; BROADWELL-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
+; BROADWELL-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BROADWELL-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.33]
+; BROADWELL-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.33]
+; BROADWELL-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [6:0.50]
 ; BROADWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-SSE-LABEL: test_andnotps:
 ; SKYLAKE-SSE:       # %bb.0:
-; SKYLAKE-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.33]
-; SKYLAKE-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [7:0.50]
+; SKYLAKE-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SKYLAKE-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SKYLAKE-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; SKYLAKE-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [7:0.50]
 ; SKYLAKE-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-LABEL: test_andnotps:
 ; SKYLAKE:       # %bb.0:
-; SKYLAKE-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:0.33]
-; SKYLAKE-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
+; SKYLAKE-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SKYLAKE-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SKYLAKE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-SSE-LABEL: test_andnotps:
 ; SKX-SSE:       # %bb.0:
-; SKX-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.33]
-; SKX-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [7:0.50]
+; SKX-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SKX-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SKX-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.33]
+; SKX-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [7:0.50]
 ; SKX-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-LABEL: test_andnotps:
 ; SKX:       # %bb.0:
-; SKX-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:0.33]
-; SKX-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
+; SKX-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SKX-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.33]
+; SKX-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.33]
+; SKX-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SKX-NEXT:    retq # sched: [7:1.00]
 ;
 ; BTVER2-SSE-LABEL: test_andnotps:
 ; BTVER2-SSE:       # %bb.0:
-; BTVER2-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.50]
-; BTVER2-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [6:1.00]
+; BTVER2-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [6:1.00]
 ; BTVER2-SSE-NEXT:    retq # sched: [4:1.00]
 ;
 ; BTVER2-LABEL: test_andnotps:
 ; BTVER2:       # %bb.0:
-; BTVER2-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:0.50]
-; BTVER2-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
+; BTVER2-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.50]
+; BTVER2-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.50]
+; BTVER2-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
 ; BTVER2-NEXT:    retq # sched: [4:1.00]
 ;
 ; ZNVER1-SSE-LABEL: test_andnotps:
 ; ZNVER1-SSE:       # %bb.0:
-; ZNVER1-SSE-NEXT:    andnps %xmm1, %xmm0 # sched: [1:0.25]
-; ZNVER1-SSE-NEXT:    andnps (%rdi), %xmm0 # sched: [8:0.50]
+; ZNVER1-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    por %xmm2, %xmm0 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    pand (%rdi), %xmm0 # sched: [8:0.50]
 ; ZNVER1-SSE-NEXT:    retq # sched: [1:0.50]
 ;
 ; ZNVER1-LABEL: test_andnotps:
 ; ZNVER1:       # %bb.0:
-; ZNVER1-NEXT:    vandnps %xmm1, %xmm0, %xmm0 # sched: [1:0.25]
-; ZNVER1-NEXT:    vandnps (%rdi), %xmm0, %xmm0 # sched: [8:0.50]
+; ZNVER1-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpxor %xmm2, %xmm1, %xmm1 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpor %xmm0, %xmm1, %xmm0 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [8:0.50]
 ; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %1 = bitcast <4 x float> %a0 to <4 x i32>
   %2 = bitcast <4 x float> %a1 to <4 x i32>
   %3 = xor <4 x i32> %1, <i32 -1, i32 -1, i32 -1, i32 -1>
   %4 = and <4 x i32> %3, %2
   %5 = load <4 x float>, <4 x float> *%a2, align 16
   %6 = bitcast <4 x float> %5 to <4 x i32>
   %7 = xor <4 x i32> %4, <i32 -1, i32 -1, i32 -1, i32 -1>

test/CodeGen/X86/sse2-schedule.ll

; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %7 = bitcast <4 x i32> %6 to <2 x double>
   %8 = fadd <2 x double> %a1, %7
   ret <2 x double> %8
 }
 
 define <2 x double> @test_andnotpd(<2 x double> %a0, <2 x double> %a1, <2 x double> *%a2) {
 ; GENERIC-LABEL: test_andnotpd:
 ; GENERIC:       # %bb.0:
-; GENERIC-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:1.00]
-; GENERIC-NEXT:    andnpd (%rdi), %xmm0 # sched: [7:1.00]
-; GENERIC-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; GENERIC-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; GENERIC-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; GENERIC-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; GENERIC-NEXT:    pand (%rdi), %xmm2 # sched: [7:0.50]
+; GENERIC-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; GENERIC-NEXT:    movapd %xmm2, %xmm0 # sched: [1:1.00]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; ATOM-LABEL: test_andnotpd:
 ; ATOM:       # %bb.0:
-; ATOM-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.50]
-; ATOM-NEXT:    andnpd (%rdi), %xmm0 # sched: [1:1.00]
-; ATOM-NEXT:    addpd %xmm1, %xmm0 # sched: [6:3.00]
+; ATOM-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; ATOM-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; ATOM-NEXT:    por %xmm0, %xmm2 # sched: [1:0.50]
+; ATOM-NEXT:    pand (%rdi), %xmm2 # sched: [1:1.00]
+; ATOM-NEXT:    addpd %xmm1, %xmm2 # sched: [6:3.00]
+; ATOM-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.50]
 ; ATOM-NEXT:    retq # sched: [79:39.50]
 ;
 ; SLM-LABEL: test_andnotpd:
 ; SLM:       # %bb.0:
-; SLM-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.50]
-; SLM-NEXT:    andnpd (%rdi), %xmm0 # sched: [4:1.00]
-; SLM-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; SLM-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SLM-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; SLM-NEXT:    por %xmm0, %xmm2 # sched: [1:0.50]
+; SLM-NEXT:    pand (%rdi), %xmm2 # sched: [4:1.00]
+; SLM-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; SLM-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.50]
 ; SLM-NEXT:    retq # sched: [4:1.00]
 ;
 ; SANDY-SSE-LABEL: test_andnotpd:
 ; SANDY-SSE:       # %bb.0:
-; SANDY-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:1.00]
-; SANDY-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [7:1.00]
-; SANDY-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; SANDY-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SANDY-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SANDY-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; SANDY-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [7:0.50]
+; SANDY-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; SANDY-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:1.00]
 ; SANDY-SSE-NEXT:    retq # sched: [1:1.00]
 ;
 ; SANDY-LABEL: test_andnotpd:
 ; SANDY:       # %bb.0:
-; SANDY-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; SANDY-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [7:1.00]
+; SANDY-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SANDY-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.33]
+; SANDY-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.33]
+; SANDY-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SANDY-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [3:1.00]
 ; SANDY-NEXT:    retq # sched: [1:1.00]
 ;
 ; HASWELL-SSE-LABEL: test_andnotpd:
 ; HASWELL-SSE:       # %bb.0:
-; HASWELL-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:1.00]
-; HASWELL-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [7:1.00]
-; HASWELL-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; HASWELL-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; HASWELL-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; HASWELL-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; HASWELL-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [7:0.50]
+; HASWELL-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; HASWELL-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:1.00]
 ; HASWELL-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; HASWELL-LABEL: test_andnotpd:
 ; HASWELL:       # %bb.0:
-; HASWELL-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; HASWELL-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [7:1.00]
+; HASWELL-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; HASWELL-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.33]
+; HASWELL-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.33]
+; HASWELL-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; HASWELL-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [3:1.00]
 ; HASWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-SSE-LABEL: test_andnotpd:
 ; BROADWELL-SSE:       # %bb.0:
-; BROADWELL-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:1.00]
-; BROADWELL-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [6:1.00]
-; BROADWELL-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; BROADWELL-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; BROADWELL-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; BROADWELL-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; BROADWELL-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [6:0.50]
+; BROADWELL-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; BROADWELL-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:1.00]
 ; BROADWELL-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; BROADWELL-LABEL: test_andnotpd:
 ; BROADWELL:       # %bb.0:
-; BROADWELL-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:1.00]
-; BROADWELL-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
+; BROADWELL-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BROADWELL-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.33]
+; BROADWELL-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.33]
+; BROADWELL-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [6:0.50]
 ; BROADWELL-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [3:1.00]
 ; BROADWELL-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-SSE-LABEL: test_andnotpd:
 ; SKYLAKE-SSE:       # %bb.0:
-; SKYLAKE-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.33]
-; SKYLAKE-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [7:0.50]
-; SKYLAKE-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [4:0.50]
+; SKYLAKE-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SKYLAKE-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SKYLAKE-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; SKYLAKE-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [7:0.50]
+; SKYLAKE-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [4:0.50]
+; SKYLAKE-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.33]
 ; SKYLAKE-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKYLAKE-LABEL: test_andnotpd:
 ; SKYLAKE:       # %bb.0:
-; SKYLAKE-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:0.33]
-; SKYLAKE-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
+; SKYLAKE-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SKYLAKE-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.33]
+; SKYLAKE-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SKYLAKE-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [4:0.50]
 ; SKYLAKE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-SSE-LABEL: test_andnotpd:
 ; SKX-SSE:       # %bb.0:
-; SKX-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.33]
-; SKX-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [7:0.50]
-; SKX-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [4:0.33]
+; SKX-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; SKX-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.33]
+; SKX-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.33]
+; SKX-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [7:0.50]
+; SKX-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [4:0.33]
+; SKX-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.33]
 ; SKX-SSE-NEXT:    retq # sched: [7:1.00]
 ;
 ; SKX-LABEL: test_andnotpd:
 ; SKX:       # %bb.0:
-; SKX-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:0.33]
-; SKX-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
+; SKX-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; SKX-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.33]
+; SKX-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.33]
+; SKX-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [7:0.50]
 ; SKX-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [4:0.33]
 ; SKX-NEXT:    retq # sched: [7:1.00]
 ;
 ; BTVER2-SSE-LABEL: test_andnotpd:
 ; BTVER2-SSE:       # %bb.0:
-; BTVER2-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.50]
-; BTVER2-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [6:1.00]
-; BTVER2-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; BTVER2-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.50]
+; BTVER2-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [6:1.00]
+; BTVER2-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; BTVER2-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.50]
 ; BTVER2-SSE-NEXT:    retq # sched: [4:1.00]
 ;
 ; BTVER2-LABEL: test_andnotpd:
 ; BTVER2:       # %bb.0:
-; BTVER2-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:0.50]
-; BTVER2-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
+; BTVER2-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.50]
+; BTVER2-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.50]
+; BTVER2-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.50]
+; BTVER2-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [6:1.00]
 ; BTVER2-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [3:1.00]
 ; BTVER2-NEXT:    retq # sched: [4:1.00]
 ;
 ; ZNVER1-SSE-LABEL: test_andnotpd:
 ; ZNVER1-SSE:       # %bb.0:
-; ZNVER1-SSE-NEXT:    andnpd %xmm1, %xmm0 # sched: [1:0.25]
-; ZNVER1-SSE-NEXT:    andnpd (%rdi), %xmm0 # sched: [8:0.50]
-; ZNVER1-SSE-NEXT:    addpd %xmm1, %xmm0 # sched: [3:1.00]
+; ZNVER1-SSE-NEXT:    pcmpeqd %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    pxor %xmm1, %xmm2 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    por %xmm0, %xmm2 # sched: [1:0.25]
+; ZNVER1-SSE-NEXT:    pand (%rdi), %xmm2 # sched: [8:0.50]
+; ZNVER1-SSE-NEXT:    addpd %xmm1, %xmm2 # sched: [3:1.00]
+; ZNVER1-SSE-NEXT:    movapd %xmm2, %xmm0 # sched: [1:0.25]
 ; ZNVER1-SSE-NEXT:    retq # sched: [1:0.50]
 ;
 ; ZNVER1-LABEL: test_andnotpd:
 ; ZNVER1:       # %bb.0:
-; ZNVER1-NEXT:    vandnpd %xmm1, %xmm0, %xmm0 # sched: [1:0.25]
-; ZNVER1-NEXT:    vandnpd (%rdi), %xmm0, %xmm0 # sched: [8:0.50]
+; ZNVER1-NEXT:    vpcmpeqd %xmm2, %xmm2, %xmm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpxor %xmm2, %xmm1, %xmm2 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpor %xmm0, %xmm2, %xmm0 # sched: [1:0.25]
+; ZNVER1-NEXT:    vpand (%rdi), %xmm0, %xmm0 # sched: [8:0.50]
 ; ZNVER1-NEXT:    vaddpd %xmm0, %xmm1, %xmm0 # sched: [3:1.00]
 ; ZNVER1-NEXT:    retq # sched: [1:0.50]
   %1 = bitcast <2 x double> %a0 to <4 x i32>
   %2 = bitcast <2 x double> %a1 to <4 x i32>
   %3 = xor <4 x i32> %1, <i32 -1, i32 -1, i32 -1, i32 -1>
   %4 = and <4 x i32> %3, %2
   %5 = load <2 x double>, <2 x double> *%a2, align 16
   %6 = bitcast <2 x double> %5 to <4 x i32>

test/CodeGen/X86/unfold-masked-merge-scalar-variablemask.ll

; CHECK-BMI-NEXT:    retq
   %mx = and i32 %mask, %x
   %my = and i32 %notmask, -1
   %r = or i32 %mx, %my
   ret i32 %r
 }
 define i32 @in_constant_varx_mone(i32 %x, i32 %y, i32 %mask) {
 ; CHECK-NOBMI-LABEL: in_constant_varx_mone:
 ; CHECK-NOBMI:       # %bb.0:
-; CHECK-NOBMI-NEXT:    notl %edi
-; CHECK-NOBMI-NEXT:    andl %edx, %edi
-; CHECK-NOBMI-NEXT:    notl %edi
-; CHECK-NOBMI-NEXT:    movl %edi, %eax
+; CHECK-NOBMI-NEXT:    notl %edx
+; CHECK-NOBMI-NEXT:    orl %edi, %edx
+; CHECK-NOBMI-NEXT:    movl %edx, %eax
 ; CHECK-NOBMI-NEXT:    retq
 ;
 ; CHECK-BMI-LABEL: in_constant_varx_mone:
 ; CHECK-BMI:       # %bb.0:
-; CHECK-BMI-NEXT:    andl %edx, %edi
 ; CHECK-BMI-NEXT:    notl %edx
 ; CHECK-BMI-NEXT:    orl %edi, %edx
 ; CHECK-BMI-NEXT:    movl %edx, %eax
 ; CHECK-BMI-NEXT:    retq
   %n0 = xor i32 %x, -1 ; %x
   %n1 = and i32 %n0, %mask
   %r = xor i32 %n1, -1
   ret i32 %r
; CHECK-BMI-NEXT:    retq
   %mx = and i32 %notmask, %x
   %my = and i32 %mask, -1
   %r = or i32 %mx, %my
   ret i32 %r
 }
 define i32 @in_constant_varx_mone_invmask(i32 %x, i32 %y, i32 %mask) {
 ; CHECK-NOBMI-LABEL: in_constant_varx_mone_invmask:
 ; CHECK-NOBMI:       # %bb.0:
-; CHECK-NOBMI-NEXT:    notl %edx
-; CHECK-NOBMI-NEXT:    notl %edi
-; CHECK-NOBMI-NEXT:    andl %edx, %edi
-; CHECK-NOBMI-NEXT:    notl %edi
+; CHECK-NOBMI-NEXT:    orl %edx, %edi
 ; CHECK-NOBMI-NEXT:    movl %edi, %eax
 ; CHECK-NOBMI-NEXT:    retq
 ;
 ; CHECK-BMI-LABEL: in_constant_varx_mone_invmask:
 ; CHECK-BMI:       # %bb.0:
-; CHECK-BMI-NEXT:    andnl %edi, %edx, %eax
-; CHECK-BMI-NEXT:    orl %edx, %eax
+; CHECK-BMI-NEXT:    orl %edx, %edi
+; CHECK-BMI-NEXT:    movl %edi, %eax
 ; CHECK-BMI-NEXT:    retq
   %notmask = xor i32 %mask, -1
   %n0 = xor i32 %x, -1 ; %x
   %n1 = and i32 %n0, %notmask
   %r = xor i32 %n1, -1
   ret i32 %r
 }
 define i32 @out_constant_varx_42(i32 %x, i32 %y, i32 %mask) {