[InstCombine] Narrow type of logical operation chains in certain cases

[mnadeem]

Summary

Reassociate chains of
extend(X) | (extend(Y) | Z) --> (extend(X) | extend(Y)) | Z
which will then get optimized to --> extend(X | Y) | Z

Allows performing operations with a higher vector factor when vectorized.
https://alive2.llvm.org/ce/z/ha4JJC

[lebedev.ri]

Please

1. Reduce the tests - they should only contain the minimal needed pattern - two extends and two logical ops.
2. Add tests with extra uses - this fold can not be performed when the existing logical ops have other uses.

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1650–1653

1654–1656

You should create the inner op with Builder,
and outer BinaryOperator::Create, and just return latter.

[mnadeem]

Address comments.

[lebedev.ri]

Please precommit the tests

[mnadeem]

rebase on precommited tests

[lebedev.ri]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1653

Wait, why are you checking that I, the root instruction, is a single-use?

[spatel]

I think I looked at this as a problem for -reassociate a long time ago. It makes changes on larger patterns like:
https://alive2.llvm.org/ce/z/UD56tJ
...but misses the shorter sequences.

(and this patch would not help match that example in -instcombine alone IIUC)

I think this patch also misses cases where we have a logic op with a wide constant:
https://alive2.llvm.org/ce/z/Ls8hXZ

Do you plan to extend it to deal with those patterns too?

[mnadeem]

remove base instruction's check for one use

[mnadeem]

In D109807#3008300, @spatel wrote:

I think I looked at this as a problem for -reassociate a long time ago. It makes changes on larger patterns like:
https://alive2.llvm.org/ce/z/UD56tJ
...but misses the shorter sequences.

(and this patch would not help match that example in -instcombine alone IIUC)

I think this patch also misses cases where we have a logic op with a wide constant:
https://alive2.llvm.org/ce/z/Ls8hXZ

Do you plan to extend it to deal with those patterns too?

I shouldn't have given this revision a general title, I was only targeting a specific case i.e. extend(X) | (extend(Y) | Z)

But your first case can probably be handled in instcombine like this (I havent tested this though):

// %conv1 = sext i4 %c to i6
// %conv2 = sext i4 %d to i6
// %or1 = or i6 %conv1, %a
// %or2 = or i6 %conv2, %b
// %or3 = or i6 %or1, %or2
// to
// %conv1 = sext i4 %c to i6
// %conv2 = sext i4 %d to i6
// %or1 = or i6 %conv1, %conv2 --> will later be converted to extend (or i4 ...)
// %or2 = or i6 %a, %b
// %or3 = or i6 %or1, %or2
Instruction *InstCombinerImpl::reassosNestedCastedBitwiseLogic(BinaryOperator &I) {
  auto LogicOpc = I.getOpcode();
  assert(I.isBitwiseLogicOp() && "Unexpected opcode for bitwise logic instcombine");

  auto *Op0 = dyn_cast<BinaryOperator>(I.getOperand(0));
  auto *Op1 = dyn_cast<BinaryOperator>(I.getOperand(1));
  if (!Op0 || !Op1)
    return nullptr;
  if (Op0->getOpcode() != Op1->getOpcode() || Op0->getOpcode() != LogicOpc)
    return nullptr;
  if (match(Op0, m_OneUse(m_c_BinOp(m_CombineAnd(m_ZExtOrSExt(m_Value()), m_Value(A)), m_Value(B)))) &&
      match(Op1, m_OneUse(m_c_BinOp(m_CombineAnd(m_ZExtOrSExt(m_Value()), m_Value(C)), m_Value(D))))) {
    Value *OrWithExtends = Builder.CreateBinOp(LogicOpc, A, Z);
    Value *OrWithOutExtends = Builder.CreateBinOp(LogicOpc, B, D);
    return BinaryOperator::Create(LogicOpc, NewOp, Z);
  }
  return nullptr;
}

[spatel]

In D109807#3011145, @mnadeem wrote:

I shouldn't have given this revision a general title, I was only targeting a specific case i.e. extend(X) | (extend(Y) | Z)

No problem - I was just wondering if you had a more general solution in mind. We handle some of the most basic reassociation patterns in instcombine already, so this patch seems fine to me.

if (match(Op0, m_OneUse(m_c_BinOp(m_CombineAnd(m_ZExtOrSExt(m_Value()), m_Value(A)), m_Value(B)))) &&
    match(Op1, m_OneUse(m_c_BinOp(m_CombineAnd(m_ZExtOrSExt(m_Value()), m_Value(C)), m_Value(D))))) {

On the other hand, this is probably going too far. We can keep adding logic ops to the chain to move the casts further and further apart, and there's no realistic way to bring them back together in instcombine. That's the job of the -reassociate pass.

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1628–1633

It's not clear to me why this swap is necessary. Do you have a test case where a logic binop has a cast as operand 0 and a binop as operand 1? Complexity-based canonicalization is supposed to prevent that. See InstCombinerImpl::SimplifyAssociativeOrCommutative().

llvm/test/Transforms/InstCombine/and-xor-or.ll

509–510

I think we're missing tests:

1. Negative test with different logic opcodes.
2. Negative test with different cast opcodes.
3. Test with different cast source types.
4. Test with multiple uses of cast instruction(s).
5. Tests where the first cast is operand 1 of the logic op (notice in the original tests that the operands are commuted from where they started - search around the test directory for "thwart complexity-based canonicalization" for ways to prevent that).

[mnadeem]

Added more tests as per comments.

[mnadeem]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1628–1633

I think you have it the other way around, the code currently only checks for the cast to be on the LHS.
This swap handles the case when the Op0 (higher complexity) is another binop.

[spatel]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1628–1633

Ah, I see.

I wonder if it would be easier to read if we just match this in its "natural" form then:
(extend(X) | Y) | extend(Z) --> (extend(X) | extend(Z)) | Y

You would hoist this code above the current bailout for !Cast0 if you did it that way.

llvm/test/Transforms/InstCombine/and-xor-or.ll

509–510

I don't see an example of "3" here yet. I'm imagining something like this:

define i64 @f(i64 %a, i8 %b, i16 %c) {
  %conv = zext i8 %b to i64
  %conv2 = sext i16 %c to i64
  %xor = xor i64 %conv, %a
  %xor2 = xor i64 %xor, %conv2
  ret i64 %xor2
}

533

This test doesn't add much value. In InstCombine, I don't think we ever care if the instruction that is being replaced has multiple uses.

551–552

Similar to test comment above: this does not add value vs. the previous test if it is just an extra use of the final value.

573–575

That seems like a reasonable canonicalization (and the 'and' test below shows a potential win)...
But we should note that this is intentional in a code comment.

But what happens if "%conv" is trunc or fptoui or some other non-ext cast?

Please add test(s) with those patterns.

[mnadeem]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1628–1633

Relying on the exit below to keep things simple:

if (!SrcTy->isIntOrIntVectorTy())
  return nullptr;

I can hoist the code but then I would have to add tests for floating point conversions.

llvm/test/Transforms/InstCombine/and-xor-or.ll

509–510

added zext_xor_chain_diffSrcTy above it is similar to your snippet but with both zext casts.

533

Removed the test

551–552

Removed the test

573–575

Added the comments in one of the other tests above and also in the c code.

But what happens if "%conv" is trunc or fptoui or some other non-ext cast?

added zext_trunc_and_chain etc at the bottom. Is this what you meant?

[spatel]

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

1646

This code comment is not accurate - we allow non-extend casts for one of the operands. See comment on the tests below.

llvm/test/Transforms/InstCombine/and-xor-or.ll

573–575

Sure, that is one example of different cast/type.

But there's an inconsistency here. I don't think you intended it, and I don't think we want it: why are we reassociating any cast as long as the source type is an int or int vector, but not allowing FP casts/types?

It's fine if this patch only deals with pairs of extends or it allows any cast ops, but we're in some in-between state currently if I'm seeing it correctly. We want to avoid an arbitrary line in canonicalization logic.

Please add these tests:

define i32 @zext_bitcast_int_vec_and_chain(i32 %a, i16 %b, <2 x i16> %c) {
  %conv = zext i16 %b to i32
  %conv2 = bitcast <2 x i16> %c to i32
  %and = and i32 %conv, %a
  %and2 = and i32 %and, %conv2
  ret i32 %and2
}

define i32 @zext_bitcast_fp_vec_and_chain(i32 %a, i16 %b, <2 x half> %c) {
  %conv = zext i16 %b to i32
  %conv2 = bitcast <2 x half> %c to i32
  %and = and i32 %conv, %a
  %and2 = and i32 %and, %conv2
  ret i32 %and2
}

[lebedev.ri]

This review seems to be stuck/dead, consider abandoning if no longer relevant.

[mnadeem]

Removing from reviewer's ready to review list for now. Will come back to this patch when/if time permits.

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

 
 /// Fold {and,or,xor} (cast X), Y.
 Instruction *InstCombinerImpl::foldCastedBitwiseLogic(BinaryOperator &I) {
   auto LogicOpc = I.getOpcode();
   assert(I.isBitwiseLogicOp() && "Unexpected opcode for bitwise logic folding");
 
   Value *Op0 = I.getOperand(0), *Op1 = I.getOperand(1);
   CastInst *Cast0 = dyn_cast<CastInst>(Op0);
+  if (!Cast0) {
+    std::swap(Op0, Op1);
+    Cast0 = dyn_cast<CastInst>(Op0);
     if (!Cast0)
       return nullptr;
+  }
 
   // This must be a cast from an integer or integer vector source type to allow
   // transformation of the logic operation to the source type.
   Type *DestTy = I.getType();
   Type *SrcTy = Cast0->getSrcTy();
   if (!SrcTy->isIntOrIntVectorTy())
     return nullptr;
 
   if (Instruction *Ret = foldLogicCastConstant(I, Cast0, Builder))
     return Ret;
 
+  // Reassociate chains of Ops {and,or,xor} where one side is an extend.
+  // extend(X) | (extend(Y) | Z) --> (extend(X) | extend(Y)) | Z
+  // (extend(X) | extend(Y)) may then be further optimized to narrow the
+  // operation to smaller types. This canonicalization is done regardless
+  // of whether we can narrow the type or not.
+  Value *Y, *Z;
+  if (match(Op1, m_OneUse(m_c_BinOp(
+                     m_CombineAnd(m_ZExtOrSExt(m_Value()), m_Value(Y)),
+                     m_Value(Z)))) &&
+      cast<BinaryOperator>(Op1)->getOpcode() == LogicOpc) {
+    Value *NewOp = Builder.CreateBinOp(LogicOpc, Op0, Y);
+    return BinaryOperator::Create(LogicOpc, NewOp, Z);
+  }
+
   CastInst *Cast1 = dyn_cast<CastInst>(Op1);
   if (!Cast1)
     return nullptr;
 
   // Both operands of the logic operation are casts. The casts must be of the
   // same type for reduction.
   auto CastOpcode = Cast0->getOpcode();
   if (CastOpcode != Cast1->getOpcode() || SrcTy != Cast1->getSrcTy())

llvm/test/Transforms/InstCombine/and-xor-or.ll

 }
 
 ; Reassociate chains of extend(X) | (extend(Y) | Z).
 ; Check that logical op is performed on a smaller type and then extended.
 
 define i64 @sext_or_chain_complexity(i64 %a0, i16 %b, i16 %c) {
 ; CHECK-LABEL: @sext_or_chain_complexity(
 ; CHECK-NEXT:    [[A:%.*]] = shl i64 [[A0:%.*]], 1
-; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[OR:%.*]] = or i64 [[A]], [[CONV]]
-; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = or i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[A]], [[TMP2]]
 ; CHECK-NEXT:    ret i64 [[OR2]]
 ;
   %a = add i64 %a0, %a0    ; thwart complexity-based canonicalization
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %or = or i64 %a, %conv
   %or2 = or i64 %or, %conv2
   ret i64 %or2
 }
 
 define i64 @sext_or_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @sext_or_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[OR:%.*]] = or i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = or i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[OR2]]
 ;
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %or = or i64 %conv, %a
   %or2 = or i64 %or, %conv2
   ret i64 %or2
 }
 
 define i64 @zext_or_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_or_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = zext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[OR:%.*]] = or i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = or i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = zext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[OR2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = zext i16 %c to i64
   %or = or i64 %conv, %a
   %or2 = or i64 %or, %conv2
   ret i64 %or2
 }
 
 define i64 @sext_and_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @sext_and_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[AND:%.*]] = and i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[AND]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[AND2]]
 ;
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %and = and i64 %conv, %a
   %and2 = and i64 %and, %conv2
   ret i64 %and2
 }
 
 define i64 @zext_and_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_and_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = zext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[AND:%.*]] = and i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[AND]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = zext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[AND2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = zext i16 %c to i64
   %and = and i64 %conv, %a
   %and2 = and i64 %and, %conv2
   ret i64 %and2
 }
 
 define i64 @sext_xor_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @sext_xor_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[XOR:%.*]] = xor i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[XOR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[XOR2]]
 ;
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %xor = xor i64 %conv, %a
   %xor2 = xor i64 %xor, %conv2
   ret i64 %xor2
 }
 
 define i64 @zext_xor_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_xor_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = zext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[XOR:%.*]] = xor i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[XOR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = zext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[XOR2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = zext i16 %c to i64
   %xor = xor i64 %conv, %a
   %xor2 = xor i64 %xor, %conv2
   ret i64 %xor2
 }
 
 ; Variation with different cast source types.
 ; Only one test to show the canonicalization, this can potentially
 ; be done in a smaller type.
 define i64 @zext_xor_chain_diffSrcTy(i64 %a, i8 %b, i16 %c) {
 ; CHECK-LABEL: @zext_xor_chain_diffSrcTy(
 ; CHECK-NEXT:    [[CONV:%.*]] = zext i8 [[B:%.*]] to i64
 ; CHECK-NEXT:    [[CONV2:%.*]] = zext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[XOR:%.*]] = xor i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[XOR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i64 [[CONV2]], [[CONV]]
+; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[TMP1]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[XOR2]]
 ;
   %conv = zext i8 %b to i64
   %conv2 = zext i16 %c to i64
   %xor = xor i64 %conv, %a
   %xor2 = xor i64 %xor, %conv2
   ret i64 %xor2
 }
 
 ; Tests with multiple uses.
 define i64 @sext_or_chain_two_uses1_negative(i64 %a, i16 %b, i16 %c, i64 %d) {
 ; CHECK-LABEL: @sext_or_chain_two_uses1_negative(
 ; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
 ; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
 ; CHECK-NEXT:    [[OR:%.*]] = or i64 [[CONV]], [[A:%.*]]
 ; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
 ; CHECK-NEXT:    [[USE:%.*]] = udiv i64 [[OR]], [[D:%.*]]
 ; CHECK-NEXT:    [[RETVAL:%.*]] = udiv i64 [[OR2]], [[USE]]
 ; CHECK-NEXT:    ret i64 [[RETVAL]]
 ;
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   ; %or has two uses
   %or = or i64 %conv, %a
   %or2 = or i64 %or, %conv2
   %use = udiv i64 %or, %d
   %retval = udiv i64 %or2, %use
   ret i64 %retval
 }
 
 ; The extension has multiple uses but we can still perform the logical op
 ; in a smaller type due to the canonicalization.
 define i64 @sext_or_chain_two_uses2(i64 %a, i16 %b, i16 %c, i64 %d) {
 ; CHECK-LABEL: @sext_or_chain_two_uses2(
 ; CHECK-NEXT:    [[CONV:%.*]] = sext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[OR:%.*]] = or i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = or i16 [[C:%.*]], [[B]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    [[USE:%.*]] = udiv i64 [[CONV]], [[D:%.*]]
 ; CHECK-NEXT:    [[RETVAL:%.*]] = udiv i64 [[OR2]], [[USE]]
 ; CHECK-NEXT:    ret i64 [[RETVAL]]
 ;
   %conv = sext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %or = or i64 %conv, %a
   %or2 = or i64 %or, %conv2
   %use = udiv i64 %conv, %d
   %retval = udiv i64 %or2, %use
   ret i64 %retval
 }
 
 ; Negative test with different logic opcode.
 define i64 @zext_xor_and_chain_negative(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_xor_and_chain_negative(
 ; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
 ; CHECK-NEXT:    [[CONV2:%.*]] = zext i16 [[C:%.*]] to i64
 ; CHECK-NEXT:    [[AND:%.*]] = and i64 [[CONV]], [[A:%.*]]
 ; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[AND]], [[CONV2]]
 ; CHECK-NEXT:    ret i64 [[XOR2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = zext i16 %c to i64
   %and = and i64 %conv, %a
   %xor2 = xor i64 %and, %conv2
   ret i64 %xor2
 }
 
 ; Tests with different cast opcodes to show the canonicalization.
 define i64 @zext_sext_xor_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_sext_xor_chain(
 ; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
 ; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[XOR:%.*]] = xor i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[XOR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i64 [[CONV2]], [[CONV]]
+; CHECK-NEXT:    [[XOR2:%.*]] = xor i64 [[TMP1]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[XOR2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %xor = xor i64 %conv, %a
   %xor2 = xor i64 %xor, %conv2
   ret i64 %xor2
 }
 
 define i64 @zext_sext_or_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_sext_or_chain(
 ; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
 ; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[OR:%.*]] = or i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[OR]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = or i64 [[CONV2]], [[CONV]]
+; CHECK-NEXT:    [[OR2:%.*]] = or i64 [[TMP1]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[OR2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %or = or i64 %conv, %a
   %or2 = or i64 %or, %conv2
   ret i64 %or2
 }
 
 define i64 @zext_sext_and_chain(i64 %a, i16 %b, i16 %c) {
 ; CHECK-LABEL: @zext_sext_and_chain(
-; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i64
-; CHECK-NEXT:    [[CONV2:%.*]] = sext i16 [[C:%.*]] to i64
-; CHECK-NEXT:    [[AND:%.*]] = and i64 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[AND]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i16 [[C:%.*]], [[B:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = zext i16 [[TMP1]] to i64
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[TMP2]], [[A:%.*]]
 ; CHECK-NEXT:    ret i64 [[AND2]]
 ;
   %conv = zext i16 %b to i64
   %conv2 = sext i16 %c to i64
   %and = and i64 %conv, %a
   %and2 = and i64 %and, %conv2
   ret i64 %and2
 }
;
   %and2 = and i32 %and, %conv2
   ret i32 %and2
 }
 
 define i32 @zext_trunc_and_chain(i32 %a, i16 %b, i64 %c) {
 ; CHECK-LABEL: @zext_trunc_and_chain(
 ; CHECK-NEXT:    [[CONV:%.*]] = zext i16 [[B:%.*]] to i32
 ; CHECK-NEXT:    [[CONV2:%.*]] = trunc i64 [[C:%.*]] to i32
-; CHECK-NEXT:    [[AND:%.*]] = and i32 [[CONV]], [[A:%.*]]
-; CHECK-NEXT:    [[AND2:%.*]] = and i32 [[AND]], [[CONV2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i32 [[CONV2]], [[CONV]]
+; CHECK-NEXT:    [[AND2:%.*]] = and i32 [[TMP1]], [[A:%.*]]
 ; CHECK-NEXT:    ret i32 [[AND2]]
 ;
   %conv = zext i16 %b to i32
   %conv2 = trunc i64 %c to i32
   %and = and i32 %conv, %a
   %and2 = and i32 %and, %conv2
   ret i32 %and2
 }