[InstCombine] Canonicalize "and, add", "or, add", "xor, add"

[emgullufsen]

Summary

Canonicalize

((x + C1) & C2) --> ((x & C2) + C1)     
((x + C1) ^ C2) --> ((x ^ C2) + C1)     
((x + C1) | C2) --> ((x | C2) + C1)

for suitable constants C1 and C2.

Alive2 proofs: add, or --> or, add
add, xor --> xor, add
add, and --> and, add

[emgullufsen]

Hadn't run clang-format

[emgullufsen]

I wasn't sure whether to subsume D130080 or make this revision a child of that one.

[emgullufsen]

Update on top of new negative multiple use tests in and-xor-or.ll

[emgullufsen]

Updated on top of vector tests

[emgullufsen]

Added clarifying comment in test

[emgullufsen]

clang-format

[emgullufsen]

Needed to run update_test_checks.py against add.ll

[foad]

Looks good overall.

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1769

The way you've written it is fine. Just remove the comment.

1774

Can you just fold the ! and >= into < please

1783

This can be llvm_unreachable

llvm/test/Transforms/InstCombine/integer-round-up-pow2-alignment.ll

13 ↗

(On Diff #451667)

These changes are not significant - it's just update_test_checks choosing a different name for its variables. Could you regenerate the checks for this file first and commit that, and then rebase the current patch?

[bcl5980]

We need to keep nsw/nuw for add I think.
And it's better to provider alive proof link also.

llvm/test/Transforms/InstCombine/add.ll

726 ↗

(On Diff #451667)

I think this code want to get better address pattern for load/store. But don't know why we do this before with this comment.
@spatel this test case is commited on dab8515370c7fd43782e72c46333e801454219cc
do you remember the history about the test ?

[hiraditya]

nit: please edit the summary with

// (X + C2) | C
// (X + C2) ^ C
// (X + C2) & C

[spatel]

llvm/test/Transforms/InstCombine/add.ll

726 ↗

(On Diff #451667)

I didn't find any direct reason, but f82c55fa082711f520 inverted the transform and updated the comment. This patch needs to be rebased, so we are not confusing the diffs.

[foad]

In D131142#3743081, @bcl5980 wrote:

We need to keep nsw/nuw for add I think.

Agreed, good point.

[emgullufsen]

Rebase & Address comments

[emgullufsen]

This patch now does not affect integer-round-up-pow2-alignment.ll (since those changes merged via rGf82c55fa082711f520a7359393b483956b69bf08). So, the NFC changes (changes of variable names) that update_test_checks would make don't have to be addressed at this time (I don't think). I did pull those NFC changes to this test out anyway as D132564, but as said I don't think this needs to be messed with just now.

[spatel]

This seems right, but can you verify with Alive2 and add the links to the patch summary.
If I modeled the code correctly, it looks like this for or/xor:
https://alive2.llvm.org/ce/z/BhAeCl
...and needs a small adjustment for 'and'.

[emgullufsen]

In D131142#3752110, @spatel wrote:

This seems right, but can you verify with Alive2 and add the links to the patch summary.
If I modeled the code correctly, it looks like this for or/xor:
https://alive2.llvm.org/ce/z/BhAeCl
...and needs a small adjustment for 'and'.

Oh shoot I had meant to include alive2 proofs - sorry for that (and thank you for or/xor proofs) - will get patch summary updated.

[spatel]

LGTM

[emgullufsen]

Sorry for the failed test - I ran update_test_checks against test/Transforms/InstCombine/freeze.ll,
and the changes I verified using Alive2 https://alive2.llvm.org/ce/z/4ABLcz

[spatel]

To confirm, you just missed updating a test, right?
Still LGTM - the extra test diff is an improvement. :)

[emgullufsen]

In D131142#3752627, @spatel wrote:

To confirm, you just missed updating a test, right?
Still LGTM - the extra test diff is an improvement. :)

Yes I just missed updating a test that I should have noticed, that actually caused bot failure. But freeze.ll seems to be the only one I missed - check-llvm results look good now.

[emgullufsen]

Thanks for reverting @reames - when you have a second maybe you could please have a look over this updated patch?

[reames]

In D131142#3752707, @emgullufsen wrote:

Thanks for reverting @reames - when you have a second maybe you could please have a look over this updated patch?

No need. I reverted because of a broken test, if that's been fixed, your LGTM stands.

[emgullufsen]

In D131142#3752708, @reames wrote:

In D131142#3752707, @emgullufsen wrote:

Thanks for reverting @reames - when you have a second maybe you could please have a look over this updated patch?

No need. I reverted because of a broken test, if that's been fixed, your LGTM stands.

Cool I see - I didn't think I needed to ask for review from reverting dev from Developer Policy, but just wanted to be sure I guess. Thanks again to you and to @spatel for help with proofs and review. Will be more careful with tests in the future.

[bcl5980]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1810

I still insist we should add the nsw/nuw flags here as this transform is really easy case to pass the flags.

[spatel]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1810

Yes - if we can keep flags, we should do that. @emgullufsen - can you create tests/patch for that?

[emgullufsen]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1810

Sorry to leave this unseen for so long - certainly I am happy to create tests/patch to account for nsw/nuw flags, thanks @bcl5980 & @spatel

[emgullufsen]

In D131142#3743081, @bcl5980 wrote:

We need to keep nsw/nuw for add I think.
And it's better to provider alive proof link also.

Sorry to miss provisioning for carrying through nsw/nuw flags before, I am preparing tests/patch to fix this and will add as reviewer.

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

if (!isa<Constant>(X) && !isa<Constant>(Y) && !isa<Constant>(Z) &&
         !X->hasOneUse()) {
       Value *YZ = Builder.CreateBinOp(Opcode, Y, Z);
       return BinaryOperator::Create(Opcode, YZ, X);
     }
   }
 
   return nullptr;
 }
 
+// Match
+// (X + C2) | C
+// (X + C2) ^ C
+// (X + C2) & C
+// and convert to do the bitwise logic first:
+// (X | C) + C2
+// (X ^ C) + C2
+// (X & C) + C2
+// iff bits affected by logic op are lower than last bit affected by math op
+static Instruction *canonicalizeLogicFirst(BinaryOperator &I,
+                                           InstCombiner::BuilderTy &Builder) {
+  Type *Ty = I.getType();
+  Instruction::BinaryOps OpC = I.getOpcode();
+  Value *Op0 = I.getOperand(0);
+  Value *Op1 = I.getOperand(1);
+  Value *X;
+  const APInt *C, *C2;
+
+  if (!(match(Op0, m_OneUse(m_Add(m_Value(X), m_APInt(C2)))) &&
+        match(Op1, m_APInt(C))))
+    return nullptr;
+
+  unsigned Width = Ty->getScalarSizeInBits();
+  unsigned LastOneMath = Width - C2->countTrailingZeros();
+
+  switch (OpC) {
+  case Instruction::And:
+    if (C->countLeadingOnes() < LastOneMath)
+      return nullptr;
+    break;
+  case Instruction::Xor:
+  case Instruction::Or:
+    if (C->countLeadingZeros() < LastOneMath)
+      return nullptr;
+    break;
+  default:
+    llvm_unreachable("Unexpected BinaryOp!");
+  }
+
+  Value *NewBinOp = Builder.CreateBinOp(OpC, X, ConstantInt::get(Ty, *C));
+  return BinaryOperator::CreateAdd(NewBinOp, ConstantInt::get(Ty, *C2));
+}
+
 // FIXME: We use commutative matchers (m_c_*) for some, but not all, matches
 // here. We should standardize that construct where it is needed or choose some
 // other way to ensure that commutated variants of patterns are not missed.
 Instruction *InstCombinerImpl::visitAnd(BinaryOperator &I) {
   Type *Ty = I.getType();
 
   if (Value *V = simplifyAndInst(I.getOperand(0), I.getOperand(1),
                                  SQ.getWithInstruction(&I)))
if (match(Op0, m_Add(m_Value(X), m_APInt(AddC)))) {
       // the constant we are adding has no bits set below that bit, then the
       // add is flipping a single bit. Example:
       // (X + 4) & 4 --> (X & 4) ^ 4
       if (Op0->hasOneUse() && C->isPowerOf2() && (*AddC & (*C - 1)) == 0) {
         assert((*C & *AddC) != 0 && "Expected common bit");
         Value *NewAnd = Builder.CreateAnd(X, Op1);
         return BinaryOperator::CreateXor(NewAnd, Op1);
       }
-
-      // If all bits affected by the add are included in a high-bit-mask, do the
-      // mask op before the add. Example:
-      // (X + 16) & -4 --> (X & -4) + 16
-      if (Op0->hasOneUse() && C->isNegatedPowerOf2() && *AddC == (*AddC & *C)) {
-        Value *NewAnd = Builder.CreateAnd(X, Op1);
-        return BinaryOperator::CreateAdd(NewAnd, ConstantInt::get(Ty, *AddC));
-      }
     }
 
     // ((C1 OP zext(X)) & C2) -> zext((C1 OP X) & C2) if C2 fits in the
     // bitwidth of X and OP behaves well when given trunc(C1) and X.
     auto isNarrowableBinOpcode = [](BinaryOperator *B) {
       switch (B->getOpcode()) {
       case Instruction::Xor:
       case Instruction::Or:
Instruction *InstCombinerImpl::visitAnd(BinaryOperator &I) {
   PHINode *PN = nullptr;
   Value *Start = nullptr, *Step = nullptr;
   if (matchSimpleRecurrence(&I, PN, Start, Step) && DT.dominates(Step, PN))
     return replaceInstUsesWith(I, Builder.CreateAnd(Start, Step));
 
   if (Instruction *R = reassociateForUses(I, Builder))
     return R;
 
+  if (Instruction *Canonicalized = canonicalizeLogicFirst(I, Builder))
+    return Canonicalized;
+
   return nullptr;
 }
 
 Instruction *InstCombinerImpl::matchBSwapOrBitReverse(Instruction &I,
                                                       bool MatchBSwaps,
                                                       bool MatchBitReversals) {
   SmallVector<Instruction *, 4> Insts;
   if (!recognizeBSwapOrBitReverseIdiom(&I, MatchBSwaps, MatchBitReversals,
if (match(C, m_OneUse(m_c_And(m_Specific(A), m_Value()))) ||
         match(C, m_OneUse(m_c_And(m_Specific(B), m_Value()))))
       return BinaryOperator::CreateOr(
           Builder.CreateOr(C, Builder.CreateAnd(A, B)), D);
   }
 
   if (Instruction *R = reassociateForUses(I, Builder))
     return R;
 
+  if (Instruction *Canonicalized = canonicalizeLogicFirst(I, Builder))
+    return Canonicalized;
+
   return nullptr;
 }
 
 /// A ^ B can be specified using other logic ops in a variety of patterns. We
 /// can fold these early and efficiently by morphing an existing instruction.
 static Instruction *foldXorToXor(BinaryOperator &I,
                                  InstCombiner::BuilderTy &Builder) {
   assert(I.getOpcode() == Instruction::Xor);
if (match(&I, m_c_Xor(m_OneUse(m_Xor(m_CombineAnd(m_Value(X),
                                                     m_Unless(m_ConstantExpr())),
                                        m_ImmConstant(C1))),
                         m_Value(Y))))
     return BinaryOperator::CreateXor(Builder.CreateXor(X, Y), C1);
 
   if (Instruction *R = reassociateForUses(I, Builder))
     return R;
 
+  if (Instruction *Canonicalized = canonicalizeLogicFirst(I, Builder))
+    return Canonicalized;
+
   return nullptr;
 }

llvm/test/Transforms/InstCombine/and-xor-or.ll

;
   %t6 = and i32 %t4, %t5
   %t7 = icmp ne i32 %t6, 0
   ret i1 %t7
 }
 
 define i32 @canonicalize_logic_first_or0(i32 %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_or0
 ; CHECK-SAME: (i32 [[X:%.*]]) {
-; CHECK-NEXT:    [[A:%.*]] = add i32 [[X]], 112
-; CHECK-NEXT:    [[R:%.*]] = or i32 [[A]], 15
+; CHECK-NEXT:    [[TMP1:%.*]] = or i32 [[X]], 15
+; CHECK-NEXT:    [[R:%.*]] = add i32 [[TMP1]], 112
 ; CHECK-NEXT:    ret i32 [[R]]
 ;
   %a = add i32 %x, 112 ; 01110000
   %r = or i32 %a, 15   ; 00001111
   ret i32 %r
 }
 
 define <2 x i32> @canonicalize_logic_first_or_vector0(<2 x i32> %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_or_vector0
 ; CHECK-SAME: (<2 x i32> [[X:%.*]]) {
-; CHECK-NEXT:    [[A:%.*]] = add <2 x i32> [[X]], <i32 112, i32 112>
-; CHECK-NEXT:    [[R:%.*]] = or <2 x i32> [[A]], <i32 15, i32 15>
+; CHECK-NEXT:    [[TMP1:%.*]] = or <2 x i32> [[X]], <i32 15, i32 15>
+; CHECK-NEXT:    [[R:%.*]] = add <2 x i32> [[TMP1]], <i32 112, i32 112>
 ; CHECK-NEXT:    ret <2 x i32> [[R]]
 ;
   %a = add <2 x i32> <i32 112, i32 112>, %x ; <0x00000070, 0x00000070>
   %r = or <2 x i32> <i32 15, i32 15>, %a    ; <0x0000000F, 0x0000000F>
   ret <2 x i32> %r
 }
 
 ; elementwise these constants should be ok to canonicalize logic op then math
;
   %a = add i32 %x, 112 ; 01110000
   %r = or i32 %a, 16   ; 00010000
   ret i32 %r
 }
 
 define i8 @canonicalize_logic_first_and0(i8 %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_and0
 ; CHECK-SAME: (i8 [[X:%.*]]) {
-; CHECK-NEXT:    [[B:%.*]] = add i8 [[X]], 48
-; CHECK-NEXT:    [[R:%.*]] = and i8 [[B]], -10
+; CHECK-NEXT:    [[TMP1:%.*]] = and i8 [[X]], -10
+; CHECK-NEXT:    [[R:%.*]] = add i8 [[TMP1]], 48
 ; CHECK-NEXT:    ret i8 [[R]]
 ;
   %b = add i8 %x, 48    ;  00110000
   %r = and i8 %b, -10   ;  11110110
   ret i8 %r
 }
 
 define <2 x i8> @canonicalize_logic_first_and_vector0(<2 x i8> %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_and_vector0
 ; CHECK-SAME: (<2 x i8> [[X:%.*]]) {
-; CHECK-NEXT:    [[A:%.*]] = add <2 x i8> [[X]], <i8 48, i8 48>
-; CHECK-NEXT:    [[R:%.*]] = and <2 x i8> [[A]], <i8 -10, i8 -10>
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i8> [[X]], <i8 -10, i8 -10>
+; CHECK-NEXT:    [[R:%.*]] = add <2 x i8> [[TMP1]], <i8 48, i8 48>
 ; CHECK-NEXT:    ret <2 x i8> [[R]]
 ;
   %a = add <2 x i8> <i8 48, i8 48>, %x
   %r = and <2 x i8> <i8 -10, i8 -10>, %a
   ret <2 x i8> %r
 }
 
 ; element-wise the constants match constraints
;
   %b = add i8 %x, 48    ;  00110000
   %r = and i8 %b, -26   ;  11100110
   ret i8 %r
 }
 
 define i8 @canonicalize_logic_first_xor_0(i8 %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_xor_0
 ; CHECK-SAME: (i8 [[X:%.*]]) {
-; CHECK-NEXT:    [[A:%.*]] = add i8 [[X]], 96
-; CHECK-NEXT:    [[R:%.*]] = xor i8 [[A]], 31
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i8 [[X]], 31
+; CHECK-NEXT:    [[R:%.*]] = add i8 [[TMP1]], 96
 ; CHECK-NEXT:    ret i8 [[R]]
 ;
   %a = add i8 %x, 96  ; 01100000
   %r = xor i8 %a, 31  ; 00011111
   ret i8 %r
 }
 
 define <2 x i32> @canonicalize_logic_first_xor_vector0(<2 x i32> %x) {
 ; CHECK-LABEL: define {{[^@]+}}@canonicalize_logic_first_xor_vector0
 ; CHECK-SAME: (<2 x i32> [[X:%.*]]) {
-; CHECK-NEXT:    [[A:%.*]] = add <2 x i32> [[X]], <i32 -8388608, i32 -8388608>
-; CHECK-NEXT:    [[R:%.*]] = xor <2 x i32> [[A]], <i32 32783, i32 32783>
+; CHECK-NEXT:    [[TMP1:%.*]] = xor <2 x i32> [[X]], <i32 32783, i32 32783>
+; CHECK-NEXT:    [[R:%.*]] = add <2 x i32> [[TMP1]], <i32 -8388608, i32 -8388608>
 ; CHECK-NEXT:    ret <2 x i32> [[R]]
 ;
   %a = add <2 x i32> <i32 -8388608, i32 -8388608>, %x ; <0xFF800000, 0xFF800000>
   %r = xor <2 x i32> <i32 32783, i32 32783>, %a       ; <0x0000800F, 0x0000800F>
   ret <2 x i32> %r
 }
 
 ; elementwise these constants do obey constraints required to canonicalize

llvm/test/Transforms/InstCombine/freeze.ll

;
   %cond.fr = freeze i1 %cond
   ret i1 %cond.fr
 }
 
 define i32 @early_freeze_test3(i32 %v1) {
 ; CHECK-LABEL: @early_freeze_test3(
 ; CHECK-NEXT:    [[V1_FR:%.*]] = freeze i32 [[V1:%.*]]
 ; CHECK-NEXT:    [[V2:%.*]] = shl i32 [[V1_FR]], 1
-; CHECK-NEXT:    [[V3:%.*]] = add i32 [[V2]], 2
-; CHECK-NEXT:    [[V4:%.*]] = or i32 [[V3]], 1
+; CHECK-NEXT:    [[V4:%.*]] = add i32 [[V2]], 3
 ; CHECK-NEXT:    ret i32 [[V4]]
 ;
   %v2 = shl i32 %v1, 1
   %v3 = add nuw i32 %v2, 2
   %v4 = or i32 %v3, 1
   %v4.fr = freeze i32 %v4
   ret i32 %v4.fr
 }

llvm/test/Transforms/InstCombine/sub.ll

;
   %a1 = add i8 %x, 4
   %a2 = and i8 %x, 7
   %r = sub i8 %a1, %a2
   ret i8 %r
 }
 
 define <2 x i8> @sub_mask_lowbits_splat_extra_use(<2 x i8> %x, <2 x i8>* %p) {
 ; CHECK-LABEL: @sub_mask_lowbits_splat_extra_use(
-; CHECK-NEXT:    [[A1:%.*]] = add <2 x i8> [[X:%.*]], <i8 -64, i8 -64>
-; CHECK-NEXT:    [[A2:%.*]] = and <2 x i8> [[X]], <i8 10, i8 10>
+; CHECK-NEXT:    [[A2:%.*]] = and <2 x i8> [[X:%.*]], <i8 10, i8 10>
 ; CHECK-NEXT:    store <2 x i8> [[A2]], <2 x i8>* [[P:%.*]], align 2
-; CHECK-NEXT:    [[R:%.*]] = and <2 x i8> [[A1]], <i8 -11, i8 -11>
+; CHECK-NEXT:    [[TMP1:%.*]] = and <2 x i8> [[X]], <i8 -11, i8 -11>
+; CHECK-NEXT:    [[R:%.*]] = add <2 x i8> [[TMP1]], <i8 -64, i8 -64>
 ; CHECK-NEXT:    ret <2 x i8> [[R]]
 ;
   %a1 = add <2 x i8> %x, <i8 192, i8 192> ; 0xc0
   %a2 = and <2 x i8> %x, <i8 10, i8 10>   ; 0x0a
   store <2 x i8> %a2, <2 x i8>* %p
   %r = sub <2 x i8> %a1, %a2
   ret <2 x i8> %r
 }