[ARM] Materialise some boolean values to avoid a branch

[rogfer01]

Summary

This patch combines some cases of ARMISD::CMOV for integers that arise in comparisons of the form

a != b ? x : 0
a == b ? 0 : x

and that currently (e.g. in Thumb1) are emitted as branches.

[efriedma]

A lot of these combines look very similar to the work done for ISD::ADDCARRY/SUBCARRY; see https://reviews.llvm.org/D29872 . Can we reuse that work rather than add a bunch of ARM-specific code?

[rogfer01]

Hi @efriedma,

thanks for pointing me to the ADDCARRY/SUBCARRY nodes. I tried to use them but I ran into serious problems (maybe due to some inexperience on my side working with SelectionDAG). First if I want to use them I assume I have to make them legal for the ARM back end, this means that I have to lower them somehow. After the short discussion in the list about the semantics of SUBCARRY I made ISD::ADDCARRY to lower to ARMISD::ADDC and ISD::SUBCARRY to lower to ARMISD::SUBC plus doing an extra (ISD::ADD 1, c) to the carry result, to change from borrow semantics (ISD::SUBCARRY semantics) to ARM carry semantics.

One side effect of doing this is that many operations that were previously legalized using ISD::ADD and ISD::ADDC like i64 add/sub are now legalized using ISD::ADDCARRY. And one generic combiner for ISD::ADDCARRY when it has a zero carry transforms it into ISD::UADDO (similarly ISD::USUBO for ISD::SUBCARRY without borrow). This, unfortunately, leads to very poor (and probably wrong) code involving msr / mrs instructions.

I tried to fix some of these issues but still it is very easy that sometimes the carry value is moved to some general purpose register and the backend decides to use mrs. For instance a case like this

define i64 @f3(i32 %al, i32 %bl) {
entry:
    ; unsigned wide add
    %aw = zext i32 %al to i64
    %bw = zext i32 %bl to i64
    %cw = add i64 %aw, %bw
    ; ch == carry bit
    %ch = lshr i64 %cw, 32
	%dw = add i64 %ch, %bw
	ret i64 %dw
}

leads to something like this before instruction selection

SelectionDAG has 13 nodes:
  t0: ch = EntryToken
  t4: i32,ch = CopyFromReg t0, Register:i32 %vreg1
  t17: ch,glue = CopyToReg t0, Register:i32 %R0, t28
  t19: ch,glue = CopyToReg t17, Register:i32 %R1, t28:1, t17:1
      t2: i32,ch = CopyFromReg t0, Register:i32 %vreg0
    t29: i32,i32 = ARMISD::ADDC t2, t4
  t28: i32,i32 = ARMISD::ADDE t4, Constant:i32<0>, t29:1
  t20: ch = ARMISD::RET_FLAG t19, Register:i32 %R0, Register:i32 %R1, t19:1

but the t19: ch,glue = CopyToReg t17, Register:i32 %R1, t28:1, t17:1 ends being emitted as mrs r1, apsr, below is the final code generated by llc -mtriple=armv6t2-eabi which I think is really wrong as the apsr register has more bits than the carry.

f3:
        adds    r0, r0, r1
        adcs    r0, r1, #0
        mrs     r1, apsr
        bx      lr

So my understanding is that using these nodes, while something that has to be done in the future, is not an easy addition to this change.

The upside is that it is possible to express the original change of this patch using the ADDCARRY and SUBCARRY nodes, so this may still be worth the effort.

Do you agree with my analysis or I am missing something very obvious here. I can upload a phab with the current change as it is if it helps.

Thank you very much,

[efriedma]

If you're seeing references to apsr, you aren't lowering ADDCARRY correctly.

ARMISD::ADDE has three operands; two integers, and apsr. It has two results: an integer, and the resulting apsr. If you're lowering ADDCARRY to ARMISD::ADDE, you actually need three operations: one to convert the boolean carry input to an apsr, one ARMISD::ADDE to perform the actual operation, and one operation to convert the result apsr to a boolean. See LowerADDSUBCARRY for how the x86 backend does this.

(Afterwards, you use a target dagcombine to eliminate the redundant operations.)

[rogfer01]

Hi Eli,

to make this more manageable I opened D35192 in which I only use ADDCARRY and SUBCARRY. Once we're happy with that one I will update this change to use the new nodes.

Regards,

[rogfer01]

ChangeLog:

• Updated the patch to use ADDCARRY / SUBCARRY nodes.

[rogfer01]

lib/Target/ARM/ARMISelLowering.cpp

10010–10020

I think this may make the ADDC combiner above redundant as ADDC x, -1 will usually become SUBC x, 1. I'll try to see if I can remove the ADDC one.

[rogfer01]

Friendly ping :)

[samparker]

lib/Target/ARM/ARMISelLowering.cpp

12231

early exit when not integer instead?

12261

I don't follow how this sequence better? What makes this more efficient than a cmp and conditional move? On a Thumb1Only target, even a cmp and branch will only be 3 cycles, I think.

12310

Same as above, I guess I'm missing something... could you explain why this is better please?

[rogfer01]

lib/Target/ARM/ARMISelLowering.cpp

12231

Sadly there is some code after this block that does not require integer (if I'm reading it correctly).

12261

Consider a case like

int test(int a, int b) {
  return a != b;
}

ToT is currently generating (--target=arm -mcpu=cortex-m0 -O2)

test:
	.fnstart
@ BB#0:                                 @ %entry
	mov	r2, r0
	movs	r0, #1
	movs	r3, #0
	cmp	r2, r1
	bne	.LBB0_2
@ BB#1:                                 @ %entry
	mov	r0, r3
.LBB0_2:                                @ %entry
	bx	lr

with this combiner we can generate

test:
	.fnstart
@ BB#0:                                 @ %entry
	subs	r0, r0, r1
	subs	r1, r0, #1
	sbcs	r0, r1
	bx	lr

So, not counting bx, we go from a sequence of 5/6 to 3.

12310

Consider a case like

int test(int a, int b) {
  return a == b;
}

ToT is currently generating (--target=arm -mcpu=cortex-m0 -O2)

test:
	.fnstart
@ BB#0:                                 @ %entry
	mov	r2, r0
	movs	r0, #1
	movs	r3, #0
	cmp	r2, r1
	beq	.LBB0_2
@ BB#1:                                 @ %entry
	mov	r0, r3
.LBB0_2:                                @ %entry
	bx	lr

But the code above is in practice like

int test(int a, int b) {
  return a == b ? 1 : 0;
}

where k = 0 (2^k = 1).

So with this change we can generate

test:
	.fnstart
@ BB#0:                                 @ %entry
	subs	r1, r0, r1
	movs	r0, #0
	subs	r0, r0, r1
	adcs	r0, r1
	bx	lr

We go from 5/6 instructions to always 4 (not counting the bx).

When k != 0 (e.g. return a == b ? 0 : k) the improvement is more modest from 5/6 to always 5, due to the LSL

12352–12355

Ignore these comments, this ended here accidentally. I'll remove them in the next update.

[samparker]

lib/Target/ARM/ARMISelLowering.cpp

12231

Couldn't you move the original combiner to live above this? Doesn't look like the known bits will be operating on floats.

12261

aaah, awesome, thanks!

12310

thanks for the great explanation!

test/CodeGen/ARM/atomic-cmpxchg.ll

29

Sorry, I really should of checked the tested before I asked you to explain... so thanks again for taking the time.

test/CodeGen/ARM/cmn.ll

11

Why change the input?

test/CodeGen/ARM/cmpxchg-O0.ll

23

Is this still beneficial when conditional moves are available? This test makes it look like an extra instruction is used.

test/CodeGen/ARM/select-imm.ll

155

I'd say that it's also worth checking that branches aren't generated for these tests.

test/CodeGen/Thumb/branchless-cmp.ll

3

Same here, it would be nice to ensure that there isn't a cmp and a br generated.

test/CodeGen/Thumb2/thumb2-cmn.ll

9–10

Why the input change for these tests?

[rogfer01]

ChangeLog:

• Reordered combiner and added early exit for non-integers.
• Constrained the CLZ combiner to T32 because it is not clear whether it is profitable in A32 (where conditional moves are more accessible)
• Added negative tests to make sure no branches are emitted where we don't want them.

test/CodeGen/ARM/cmn.ll

11

By doing an artificial select we avoid triggering the new combiner while forcing the previous cmn to appear.

test/CodeGen/ARM/cmpxchg-O0.ll

23

I've just reduced the scope of this change to only Thumb1 for this case as it is less obvious for Arm that this is an improvement.

[efriedma]

lib/Target/ARM/ARMISelLowering.cpp

12252

What are you trying to do here? With Thumb2, the relevant instructions are basically identical in ARM mode vs. Thumb mode, so doing this transform exclusively in Thumb mode doesn't really make sense.

[rogfer01]

lib/Target/ARM/ARMISelLowering.cpp

12252

Mmmh, I was trying to emit this in ARM because it is not always beneficial in cmpxcg-O0 below. We're going from 2 instructions to 3.

For Thumb 2 I understand there is an IT block (which the test does not check) so we stay in 3 instructions.

(I'm aware that the metric "number of instructions" is a bit poor)

I may be missing something here.

[rogfer01]

lib/Target/ARM/ARMISelLowering.cpp

12252

Also I noticed that if I'm not doing this in ARM any more checking for V5TOps is unnecessary.

[efriedma]

"it" is more like an instruction prefix than an actual instruction from the perspective of the CPU; it gets decoded into the same thing as an ARM mode conditional instruction. It's not an improvement to replace "it" with an actual instruction.

[rogfer01]

@efriedma Ah I see. If I get you right, the initial change was more sensible. Also Sam's concerns were on a file that is explicitly marked as -O0.

I will go back the original change.

[rogfer01]

ChangeLog:

• Reinstate the transformation for ARM as well.

[efriedma]

This patch would be easier to review if you would commit the changes to add new tests and RUN lines separately.

test/CodeGen/Thumb/long-setcc.ll

25–26

Could we fix this test to include some higher-quality CHECK lines, so it's clear what we're actually generating?

[rogfer01]

ChangeLog

• Split tests in D41122
• Improve test long-setcc.ll

[rogfer01]

ChangeLog:

• I forgot to update the Thumb counterpart of long-setcc.ll in the last update

[rogfer01]

ChangeLog

• New tests in D41122 are now updated in this change to show the change in codegen

[rogfer01]

Ping. Modulo the bugs we may find with ADDCARRY / SUBCARRY further thoughts on this?

[efriedma]

The use of clz is very similar to what the PowerPC backend does; I wonder if we can pick up any additional tricks from there.

lib/Target/ARM/ARMISelLowering.cpp

7443

When do we hit this case? I would expect that normally DAGCombiner::visitADDCARRY would combine this away.

10010–10020

Did you end up checking this?

12245

Do we generate ARMISD::CMOV for values which aren't integers?

[rogfer01]

lib/Target/ARM/ARMISelLowering.cpp

7443

In a testcase like

define i32 @test1a(i32 %a, i32 %b) {
entry:
  %cmp = icmp ne i32 %a, %b
  %cond = zext i1 %cmp to i32
  ret i32 %cond
}

without this we generate

	subs	r0, r0, r1
	movs	r1, #1
	subs	r2, r1, #1
	mov	r2, r0
	sbcs	r2, r1
	sbcs	r0, r2
	bx	lr

with it we can generate

	subs	r0, r0, r1
	subs	r1, r0, #1
	sbcs	r0, r1
	bx	lr

Alternatively I could queue a combine to the newly created ISD::SUBCARRY in line 12270 below. I think DCI.CombineTo should be able to do that. Does this sound right?

10010–10020

Yep, neither is redundant apparently, but I will check with more detail why it happens.

12245

In cases like this

float a;
int b;
a = (b & 0x1) != 0;

there is a moment where the DAG contains this node

t19: f32 = ARMISD::CMOV ConstantFP:f32<0.000000e+00>, ConstantFP:f32<1.000000e+00>, Constant:i32<1>, Register:i32 %cpsr, t18

[efriedma]

lib/Target/ARM/ARMISelLowering.cpp

7443

If we're not correctly adding new nodes to the DAGCombiner queue in a custom DAGCombine, we should fix that.

12245

Huh.

I guess we're getting lucky that the code below to create an AssertZext doesn't crash.

[rogfer01]

ChangeLog:

• Change ConvertBooleanCarryToCarryFlag to use ARMISD::SUBC x, 1 instead of ARMISD::ADDC x, ~0, this way a single combiner in PerformAddcSubcCombine is enough.
• Do not use ISD::SUBCARRY if the third operand is a zero. While the generic combiner can lower this to ISD::USUBO it may not have a chance to run before we lower that ISD::SUBCARRY.

[rogfer01]

ChangeLog:

• Remove whitespace and unused variables in LowerADDSUBCARRY

[rogfer01]

@efriedma @samparker any further comments on this change? We can add more clz-like tricks in later changes if needed.

[rogfer01]

Ping :-)

[samparker]

test/CodeGen/ARM/select-imm.ll

235

Why do we have a sxtb for T1 and not T2?

[rogfer01]

test/CodeGen/ARM/select-imm.ll

235

This comes from the load byte + sext in that testcase.

In Thumb2 we can coalesce the load byte + sext in a single ldrsb.w Rt, [Rn, #0] while in Thumb1 we can't do that (at least directly) because there ldsrb is of the form ldsrb Rt, [Rn, Rm] so a ldrb Rt, [Rn, #0] and then a sxtb Rt, Rn are used instead.

[samparker]

No other points from me, lets get this monster in. (fingers crossed)

test/CodeGen/ARM/select-imm.ll

235

Ok cheers, I didn't realise we didn't have ldrsb in thumb-1. But still odd this doesn't get combined away, guess it must be because the load has two users.

[rogfer01]

ChangeLog:

• Rebase with ToT

[rogfer01]

Let's see how this one goes :)

Thanks @samparker and @efriedma

lib/Target/ARM/ARMISelLowering.cpp

 }
 
 static SDValue LowerADDSUBCARRY(SDValue Op, SelectionDAG &DAG) {
   SDNode *N = Op.getNode();
   EVT VT = N->getValueType(0);
   SDVTList VTs = DAG.getVTList(VT, MVT::i32);
 
   SDValue Carry = Op.getOperand(2);
+
+  // Let the target independent generic combiner work for us.
+  if (isNullConstant(Carry))
+    return SDValue();
+
   EVT CarryVT = Carry.getValueType();
 
   SDLoc DL(Op);
 
   APInt NegOne = APInt::getAllOnesValue(CarryVT.getScalarSizeInBits());
 
   SDValue Result;
   if (Op.getOpcode() == ISD::ADDCARRY) {
if (N->getOpcode() == ARMISD::ADDC) {
     SDValue RHS = N->getOperand(1);
     if (LHS->getOpcode() == ARMISD::ADDE &&
         isNullConstant(LHS->getOperand(0)) &&
         isNullConstant(LHS->getOperand(1)) && isAllOnesConstant(RHS)) {
       return DCI.CombineTo(N, SDValue(N, 0), LHS->getOperand(2));
     }
   }
 
+  if (N->getOpcode() == ARMISD::SUBC) {
+    // (SUBC (ADDE 0, 0, C), 1) -> C
+    SDValue LHS = N->getOperand(0);
+    SDValue RHS = N->getOperand(1);
+    if (LHS->getOpcode() == ARMISD::ADDE &&
+        isNullConstant(LHS->getOperand(0)) &&
+        isNullConstant(LHS->getOperand(1)) && isOneConstant(RHS)) {
+      return DCI.CombineTo(N, SDValue(N, 0), LHS->getOperand(2));
+    }
+  }
+
   if (Subtarget->isThumb1Only()) {
     SDValue RHS = N->getOperand(1);
     if (ConstantSDNode *C = dyn_cast<ConstantSDNode>(RHS)) {
       int32_t imm = C->getSExtValue();
       if (imm < 0 && imm > std::numeric_limits<int>::min()) {
         SDLoc DL(N);
         RHS = DAG.getConstant(-imm, DL, MVT::i32);
         unsigned Opcode = (N->getOpcode() == ARMISD::ADDC) ? ARMISD::SUBC
ARMTargetLowering::PerformCMOVCombine(SDNode *N, SelectionDAG &DAG) const {
   } else if (CC == ARMCC::EQ && TrueVal == RHS) {
     SDValue ARMcc;
     SDValue NewCmp = getARMCmp(LHS, RHS, ISD::SETNE, ARMcc, DAG, dl);
     Res = DAG.getNode(ARMISD::CMOV, dl, VT, LHS, FalseVal, ARMcc,
                       N->getOperand(3), NewCmp);
   }
 
   // (cmov F T ne CPSR (cmpz (cmov 0 1 CC CPSR Cmp) 0))
   // -> (cmov F T CC CPSR Cmp)
   if (CC == ARMCC::NE && LHS.getOpcode() == ARMISD::CMOV && LHS->hasOneUse()) {
     auto *LHS0C = dyn_cast<ConstantSDNode>(LHS->getOperand(0));
     auto *LHS1C = dyn_cast<ConstantSDNode>(LHS->getOperand(1));
     auto *RHSC = dyn_cast<ConstantSDNode>(RHS);
     if ((LHS0C && LHS0C->getZExtValue() == 0) &&
         (LHS1C && LHS1C->getZExtValue() == 1) &&
         (RHSC && RHSC->getZExtValue() == 0)) {
       return DAG.getNode(ARMISD::CMOV, dl, VT, FalseVal, TrueVal,
                          LHS->getOperand(2), LHS->getOperand(3),
                          LHS->getOperand(4));
     }
   }
 
+  if (!VT.isInteger())
+      return SDValue();
+
+  // Materialize a boolean comparison for integers so we can avoid branching.
+  if (isNullConstant(FalseVal)) {
+    if (CC == ARMCC::EQ && isOneConstant(TrueVal)) {
+      if (!Subtarget->isThumb1Only() && Subtarget->hasV5TOps()) {
+        // If x == y then x - y == 0 and ARM's CLZ will return 32, shifting it
+        // right 5 bits will make that 32 be 1, otherwise it will be 0.
+        // CMOV 0, 1, ==, (CMPZ x, y) -> SRL (CTLZ (SUB x, y)), 5
+        SDValue Sub = DAG.getNode(ISD::SUB, dl, VT, LHS, RHS);
+        Res = DAG.getNode(ISD::SRL, dl, VT, DAG.getNode(ISD::CTLZ, dl, VT, Sub),
+                          DAG.getConstant(5, dl, MVT::i32));
+      } else {
+        // CMOV 0, 1, ==, (CMPZ x, y) ->
+        //     (ADDCARRY (SUB x, y), t:0, t:1)
+        // where t = (SUBCARRY 0, (SUB x, y), 0)
+        //
+        // The SUBCARRY computes 0 - (x - y) and this will give a borrow when
+        // x != y. In other words, a carry C == 1 when x == y, C == 0
+        // otherwise.
+        // The final ADDCARRY computes
+        //     x - y + (0 - (x - y)) + C == C
+        SDValue Sub = DAG.getNode(ISD::SUB, dl, VT, LHS, RHS);
+        SDVTList VTs = DAG.getVTList(VT, MVT::i32);
+        SDValue Neg = DAG.getNode(ISD::SUBCARRY, dl, VTs, FalseVal, Sub,
+                                  DAG.getConstant(0, dl, MVT::i32));
+        // ISD::SUBCARRY returns a borrow but we want the carry here
+        // actually.
+        SDValue Carry =
+            DAG.getNode(ISD::SUB, dl, MVT::i32,
+                        DAG.getConstant(1, dl, MVT::i32), Neg.getValue(1));
+        Res = DAG.getNode(ISD::ADDCARRY, dl, VTs, Sub, Neg, Carry);
+      }
+    } else if (CC == ARMCC::NE && LHS != RHS &&
+               (!Subtarget->isThumb1Only() || isPowerOf2Constant(TrueVal))) {
+      // This seems pointless but will allow us to combine it further below.
+      // CMOV 0, z, !=, (CMPZ x, y) -> CMOV (SUB x, y), z, !=, (CMPZ x, y)
+      SDValue Sub = DAG.getNode(ISD::SUB, dl, VT, LHS, RHS);
+      Res = DAG.getNode(ARMISD::CMOV, dl, VT, Sub, TrueVal, ARMcc,
+                        N->getOperand(3), Cmp);
+    }
+  } else if (isNullConstant(TrueVal)) {
+    if (CC == ARMCC::EQ && LHS != RHS &&
+        (!Subtarget->isThumb1Only() || isPowerOf2Constant(FalseVal))) {
+      // This seems pointless but will allow us to combine it further below
+      // Note that we change == for != as this is the dual for the case above.
+      // CMOV z, 0, ==, (CMPZ x, y) -> CMOV (SUB x, y), z, !=, (CMPZ x, y)
+      SDValue Sub = DAG.getNode(ISD::SUB, dl, VT, LHS, RHS);
+      Res = DAG.getNode(ARMISD::CMOV, dl, VT, Sub, FalseVal,
+                        DAG.getConstant(ARMCC::NE, dl, MVT::i32),
+                        N->getOperand(3), Cmp);
+    }
+  }
+
+  // On Thumb1, the DAG above may be further combined if z is a power of 2
+  // (z == 2 ^ K).
+  // CMOV (SUB x, y), z, !=, (CMPZ x, y) ->
+  //       merge t3, t4
+  // where t1 = (SUBCARRY (SUB x, y), z, 0)
+  //       t2 = (SUBCARRY (SUB x, y), t1:0, t1:1)
+  //       t3 = if K != 0 then (SHL t2:0, K) else t2:0
+  //       t4 = (SUB 1, t2:1)   [ we want a carry, not a borrow ]
+  const APInt *TrueConst;
+  if (Subtarget->isThumb1Only() && CC == ARMCC::NE &&
+      (FalseVal.getOpcode() == ISD::SUB) && (FalseVal.getOperand(0) == LHS) &&
+      (FalseVal.getOperand(1) == RHS) &&
+      (TrueConst = isPowerOf2Constant(TrueVal))) {
+    SDVTList VTs = DAG.getVTList(VT, MVT::i32);
+    unsigned ShiftAmount = TrueConst->logBase2();
+    if (ShiftAmount)
+      TrueVal = DAG.getConstant(1, dl, VT);
+    SDValue Subc = DAG.getNode(ISD::SUBCARRY, dl, VTs, FalseVal, TrueVal,
+                               DAG.getConstant(0, dl, MVT::i32));
+    Res = DAG.getNode(ISD::SUBCARRY, dl, VTs, FalseVal, Subc, Subc.getValue(1));
+    // Make it a carry, not a borrow.
+    SDValue Carry = DAG.getNode(
+        ISD::SUB, dl, VT, DAG.getConstant(1, dl, MVT::i32), Res.getValue(1));
+    Res = DAG.getNode(ISD::MERGE_VALUES, dl, VTs, Res, Carry);
+
+    if (ShiftAmount)
+      Res = DAG.getNode(ISD::SHL, dl, VT, Res,
+                        DAG.getConstant(ShiftAmount, dl, MVT::i32));
+  }
+
   if (Res.getNode()) {
     KnownBits Known;
     DAG.computeKnownBits(SDValue(N,0), Known);
     // Capture demanded bits information that would be otherwise lost.
     if (Known.Zero == 0xfffffffe)
       Res = DAG.getNode(ISD::AssertZext, dl, MVT::i32, Res,
                         DAG.getValueType(MVT::i1));
     else if (Known.Zero == 0xffffff00)
       Res = DAG.getNode(ISD::AssertZext, dl, MVT::i32, Res,
                         DAG.getValueType(MVT::i8));
     else if (Known.Zero == 0xffff0000)
       Res = DAG.getNode(ISD::AssertZext, dl, MVT::i32, Res,
                         DAG.getValueType(MVT::i16));
   }
 
   return Res;
 }
 
 SDValue ARMTargetLowering::PerformDAGCombine(SDNode *N,
                                              DAGCombinerInfo &DCI) const {
   switch (N->getOpcode()) {
   default: break;
   case ARMISD::ADDE:    return PerformADDECombine(N, DCI, Subtarget);
   case ARMISD::UMLAL:   return PerformUMLALCombine(N, DCI.DAG, Subtarget);
   case ISD::ADD:        return PerformADDCombine(N, DCI, Subtarget);
   case ISD::SUB:        return PerformSUBCombine(N, DCI);
   case ISD::MUL:        return PerformMULCombine(N, DCI, Subtarget);
   case ISD::OR:         return PerformORCombine(N, DCI, Subtarget);
   case ISD::XOR:        return PerformXORCombine(N, DCI, Subtarget);
   case ISD::AND:        return PerformANDCombine(N, DCI, Subtarget);
   case ARMISD::ADDC:
   case ARMISD::SUBC:    return PerformAddcSubcCombine(N, DCI, Subtarget);
   case ARMISD::SUBE:    return PerformAddeSubeCombine(N, DCI.DAG, Subtarget);
   case ARMISD::BFI:     return PerformBFICombine(N, DCI);

test/CodeGen/ARM/and-load-combine.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=armv7 %s -o - | FileCheck %s --check-prefix=ARM
 ; RUN: llc -mtriple=armv7eb %s -o - | FileCheck %s --check-prefix=ARMEB
 ; RUN: llc -mtriple=armv6m %s -o - | FileCheck %s --check-prefix=THUMB1
 ; RUN: llc -mtriple=thumbv8m.main %s -o - | FileCheck %s --check-prefix=THUMB2
 
 define arm_aapcscc zeroext i1 @cmp_xor8_short_short(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_xor8_short_short:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r0, [r0]
 ; ARM-NEXT:    ldrh r1, [r1]
-; ARM-NEXT:    eor r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    eor r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_xor8_short_short:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r0, [r0]
 ; ARMEB-NEXT:    ldrh r1, [r1]
-; ARMEB-NEXT:    eor r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    eor r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_xor8_short_short:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r0, [r0]
-; THUMB1-NEXT:    ldrh r2, [r1]
-; THUMB1-NEXT:    eors r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB0_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB0_2: @ %entry
+; THUMB1-NEXT:    ldrh r1, [r1]
+; THUMB1-NEXT:    eors r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_xor8_short_short:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ldrh r1, [r1]
 ; THUMB2-NEXT:    eors r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                     i16* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %1 = load i16, i16* %b, align 2
   %xor2 = xor i16 %1, %0
   %2 = and i16 %xor2, 255
   %cmp = icmp eq i16 %2, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_xor8_short_int(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_xor8_short_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r0, [r0]
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    eor r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    eor r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_xor8_short_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r0, [r0]
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    eor r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    eor r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_xor8_short_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    eors r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB1_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB1_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    eors r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_xor8_short_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    eors r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                   i32* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %conv = zext i16 %0 to i32
   %1 = load i32, i32* %b, align 4
   %xor = xor i32 %1, %conv
   %and = and i32 %xor, 255
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_xor8_int_int(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_xor8_int_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r0, [r0]
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    eor r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    eor r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_xor8_int_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r0, [r0]
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    eor r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    eor r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_xor8_int_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    eors r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB2_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB2_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    eors r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_xor8_int_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    eors r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                 i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %xor = xor i32 %1, %0
   %and = and i32 %xor, 255
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_xor16(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_xor16:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r0, [r0]
-; ARM-NEXT:    movw r2, #65535
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    eor r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, r2
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    eor r0, r1, r0
+; ARM-NEXT:    uxth r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_xor16:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r0, [r0]
-; ARMEB-NEXT:    movw r2, #65535
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    eor r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, r2
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    eor r0, r1, r0
+; ARMEB-NEXT:    uxth r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_xor16:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    eors r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #16
-; THUMB1-NEXT:    beq .LBB3_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB3_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    eors r1, r0
+; THUMB1-NEXT:    uxth r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_xor16:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    eors r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #16
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxth r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                          i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %xor = xor i32 %1, %0
   %and = and i32 %xor, 65535
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_or8_short_short(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_or8_short_short:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r0, [r0]
 ; ARM-NEXT:    ldrh r1, [r1]
-; ARM-NEXT:    orr r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    orr r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_or8_short_short:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r0, [r0]
 ; ARMEB-NEXT:    ldrh r1, [r1]
-; ARMEB-NEXT:    orr r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    orr r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_or8_short_short:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r0, [r0]
-; THUMB1-NEXT:    ldrh r2, [r1]
-; THUMB1-NEXT:    orrs r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB4_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB4_2: @ %entry
+; THUMB1-NEXT:    ldrh r1, [r1]
+; THUMB1-NEXT:    orrs r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_or8_short_short:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ldrh r1, [r1]
 ; THUMB2-NEXT:    orrs r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                    i16* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %1 = load i16, i16* %b, align 2
   %or2 = or i16 %1, %0
   %2 = and i16 %or2, 255
   %cmp = icmp eq i16 %2, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_or8_short_int(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_or8_short_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r0, [r0]
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    orr r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    orr r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_or8_short_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r0, [r0]
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    orr r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    orr r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_or8_short_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    orrs r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB5_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB5_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    orrs r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_or8_short_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    orrs r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                  i32* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %conv = zext i16 %0 to i32
   %1 = load i32, i32* %b, align 4
   %or = or i32 %1, %conv
   %and = and i32 %or, 255
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_or8_int_int(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_or8_int_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r0, [r0]
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    orr r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    orr r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_or8_int_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r0, [r0]
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    orr r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    orr r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_or8_int_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    orrs r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB6_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB6_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    orrs r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_or8_int_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    orrs r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %or = or i32 %1, %0
   %and = and i32 %or, 255
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_or16(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_or16:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r0, [r0]
-; ARM-NEXT:    movw r2, #65535
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    orr r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, r2
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    orr r0, r1, r0
+; ARM-NEXT:    uxth r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_or16:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r0, [r0]
-; ARMEB-NEXT:    movw r2, #65535
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    orr r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, r2
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    orr r0, r1, r0
+; ARMEB-NEXT:    uxth r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_or16:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    orrs r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #16
-; THUMB1-NEXT:    beq .LBB7_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB7_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    orrs r1, r0
+; THUMB1-NEXT:    uxth r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_or16:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    orrs r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #16
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxth r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                         i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %or = or i32 %1, %0
   %and = and i32 %or, 65535
   %cmp = icmp eq i32 %and, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_and8_short_short(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_and8_short_short:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r1, [r1]
 ; ARM-NEXT:    ldrh r0, [r0]
-; ARM-NEXT:    and r1, r0, r1
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    and r0, r0, r1
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_and8_short_short:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r1, [r1]
 ; ARMEB-NEXT:    ldrh r0, [r0]
-; ARMEB-NEXT:    and r1, r0, r1
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    and r0, r0, r1
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_and8_short_short:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r1, [r1]
-; THUMB1-NEXT:    ldrh r2, [r0]
-; THUMB1-NEXT:    ands r2, r1
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB8_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB8_2: @ %entry
+; THUMB1-NEXT:    ldrh r0, [r0]
+; THUMB1-NEXT:    ands r0, r1
+; THUMB1-NEXT:    uxtb r1, r0
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_and8_short_short:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r1, [r1]
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ands r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                     i16* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %1 = load i16, i16* %b, align 2
   %and3 = and i16 %0, 255
   %2 = and i16 %and3, %1
   %cmp = icmp eq i16 %2, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_and8_short_int(i16* nocapture readonly %a,
 ; ARM-LABEL: cmp_and8_short_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldrh r0, [r0]
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    and r1, r1, r0
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    and r0, r1, r0
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_and8_short_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldrh r0, [r0]
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    and r1, r1, r0
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    and r0, r1, r0
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_and8_short_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldrh r0, [r0]
-; THUMB1-NEXT:    ldr r2, [r1]
-; THUMB1-NEXT:    ands r2, r0
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB9_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB9_2: @ %entry
+; THUMB1-NEXT:    ldr r1, [r1]
+; THUMB1-NEXT:    ands r1, r0
+; THUMB1-NEXT:    uxtb r1, r1
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_and8_short_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldrh r0, [r0]
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    ands r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                   i32* nocapture readonly %b) {
 entry:
   %0 = load i16, i16* %a, align 2
   %1 = load i32, i32* %b, align 4
   %2 = and i16 %0, 255
   %and = zext i16 %2 to i32
   %and1 = and i32 %1, %and
   %cmp = icmp eq i32 %and1, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_and8_int_int(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_and8_int_int:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r1, [r1]
 ; ARM-NEXT:    ldr r0, [r0]
-; ARM-NEXT:    and r1, r0, r1
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, #255
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    and r0, r0, r1
+; ARM-NEXT:    uxtb r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_and8_int_int:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r1, [r1]
 ; ARMEB-NEXT:    ldr r0, [r0]
-; ARMEB-NEXT:    and r1, r0, r1
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, #255
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    and r0, r0, r1
+; ARMEB-NEXT:    uxtb r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_and8_int_int:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r1, [r1]
-; THUMB1-NEXT:    ldr r2, [r0]
-; THUMB1-NEXT:    ands r2, r1
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #24
-; THUMB1-NEXT:    beq .LBB10_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB10_2: @ %entry
+; THUMB1-NEXT:    ldr r0, [r0]
+; THUMB1-NEXT:    ands r0, r1
+; THUMB1-NEXT:    uxtb r1, r0
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_and8_int_int:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ands r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #24
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxtb r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                                 i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %and = and i32 %0, 255
   %and1 = and i32 %and, %1
   %cmp = icmp eq i32 %and1, 0
   ret i1 %cmp
 }
 
 define arm_aapcscc zeroext i1 @cmp_and16(i32* nocapture readonly %a,
 ; ARM-LABEL: cmp_and16:
 ; ARM:       @ %bb.0: @ %entry
 ; ARM-NEXT:    ldr r1, [r1]
-; ARM-NEXT:    movw r2, #65535
 ; ARM-NEXT:    ldr r0, [r0]
-; ARM-NEXT:    and r1, r0, r1
-; ARM-NEXT:    mov r0, #0
-; ARM-NEXT:    tst r1, r2
-; ARM-NEXT:    movweq r0, #1
+; ARM-NEXT:    and r0, r0, r1
+; ARM-NEXT:    uxth r0, r0
+; ARM-NEXT:    clz r0, r0
+; ARM-NEXT:    lsr r0, r0, #5
 ; ARM-NEXT:    bx lr
 ;
 ; ARMEB-LABEL: cmp_and16:
 ; ARMEB:       @ %bb.0: @ %entry
 ; ARMEB-NEXT:    ldr r1, [r1]
-; ARMEB-NEXT:    movw r2, #65535
 ; ARMEB-NEXT:    ldr r0, [r0]
-; ARMEB-NEXT:    and r1, r0, r1
-; ARMEB-NEXT:    mov r0, #0
-; ARMEB-NEXT:    tst r1, r2
-; ARMEB-NEXT:    movweq r0, #1
+; ARMEB-NEXT:    and r0, r0, r1
+; ARMEB-NEXT:    uxth r0, r0
+; ARMEB-NEXT:    clz r0, r0
+; ARMEB-NEXT:    lsr r0, r0, #5
 ; ARMEB-NEXT:    bx lr
 ;
 ; THUMB1-LABEL: cmp_and16:
 ; THUMB1:       @ %bb.0: @ %entry
 ; THUMB1-NEXT:    ldr r1, [r1]
-; THUMB1-NEXT:    ldr r2, [r0]
-; THUMB1-NEXT:    ands r2, r1
-; THUMB1-NEXT:    movs r0, #1
-; THUMB1-NEXT:    movs r1, #0
-; THUMB1-NEXT:    lsls r2, r2, #16
-; THUMB1-NEXT:    beq .LBB11_2
-; THUMB1-NEXT:  @ %bb.1: @ %entry
-; THUMB1-NEXT:    mov r0, r1
-; THUMB1-NEXT:  .LBB11_2: @ %entry
+; THUMB1-NEXT:    ldr r0, [r0]
+; THUMB1-NEXT:    ands r0, r1
+; THUMB1-NEXT:    uxth r1, r0
+; THUMB1-NEXT:    movs r0, #0
+; THUMB1-NEXT:    subs r0, r0, r1
+; THUMB1-NEXT:    adcs r0, r1
 ; THUMB1-NEXT:    bx lr
 ;
 ; THUMB2-LABEL: cmp_and16:
 ; THUMB2:       @ %bb.0: @ %entry
 ; THUMB2-NEXT:    ldr r1, [r1]
 ; THUMB2-NEXT:    ldr r0, [r0]
 ; THUMB2-NEXT:    ands r0, r1
-; THUMB2-NEXT:    lsls r0, r0, #16
-; THUMB2-NEXT:    mov.w r0, #0
-; THUMB2-NEXT:    it eq
-; THUMB2-NEXT:    moveq r0, #1
+; THUMB2-NEXT:    uxth r0, r0
+; THUMB2-NEXT:    clz r0, r0
+; THUMB2-NEXT:    lsrs r0, r0, #5
 ; THUMB2-NEXT:    bx lr
                                          i32* nocapture readonly %b) {
 entry:
   %0 = load i32, i32* %a, align 4
   %1 = load i32, i32* %b, align 4
   %and = and i32 %0, 65535
   %and1 = and i32 %and, %1
   %cmp = icmp eq i32 %and1, 0

test/CodeGen/ARM/atomic-cmpxchg.ll

 entry:
   %0 = cmpxchg i8* %addr, i8 %desired, i8 %new monotonic monotonic
   %1 = extractvalue { i8, i1 } %0, 1
   ret i1 %1
 }
 
 ; CHECK-ARM-LABEL: test_cmpxchg_res_i8
 ; CHECK-ARM: bl __sync_val_compare_and_swap_1
-; CHECK-ARM: mov [[REG:r[0-9]+]], #0
-; CHECK-ARM: cmp r0, {{r[0-9]+}}
-; CHECK-ARM: moveq [[REG]], #1
-; CHECK-ARM: mov r0, [[REG]]
+; CHECK-ARM: sub r0, r0, {{r[0-9]+}}
+; CHECK-ARM: rsbs [[REG:r[0-9]+]], r0, #0
+; CHECK-ARM: adc r0, r0, [[REG]]
 
 ; CHECK-THUMB-LABEL: test_cmpxchg_res_i8
 ; CHECK-THUMB: bl __sync_val_compare_and_swap_1
 ; CHECK-THUMB-NOT: mov [[R1:r[0-7]]], r0
-; CHECK-THUMB: movs [[R1:r[0-7]]], r0
-; CHECK-THUMB: movs r0, #1
-; CHECK-THUMB: movs [[R2:r[0-9]+]], #0
-; CHECK-THUMB: cmp [[R1]], {{r[0-9]+}}
-; CHECK-THUMB: beq
-; CHECK-THUMB: movs r0, [[R2]]
+; CHECK-THUMB: subs [[R1:r[0-7]]], r0, {{r[0-9]+}}
+; CHECK-THUMB: movs r0, #0
+; CHECK-THUMB: subs r0, r0, [[R1]]
+; CHECK-THUMB: adcs r0, [[R1]]
 
 ; CHECK-ARMV6-LABEL: test_cmpxchg_res_i8:
 ; CHECK-ARMV6-NEXT:  .fnstart
 ; CHECK-ARMV6-NEXT: uxtb [[DESIRED:r[0-9]+]], r1
 ; CHECK-ARMV6-NEXT: [[TRY:.LBB[0-9_]+]]:
 ; CHECK-ARMV6-NEXT: ldrexb [[LD:r[0-9]+]], [r0]
 ; CHECK-ARMV6-NEXT: cmp [[LD]], [[DESIRED]]
 ; CHECK-ARMV6-NEXT: movne [[RES:r[0-9]+]], #0
 ; CHECK-ARMV6-NEXT: bxne lr
 ; CHECK-ARMV6-NEXT: strexb [[SUCCESS:r[0-9]+]], r2, [r0]
 ; CHECK-ARMV6-NEXT: cmp [[SUCCESS]], #0
 ; CHECK-ARMV6-NEXT: moveq [[RES]], #1
 ; CHECK-ARMV6-NEXT: bxeq lr
 ; CHECK-ARMV6-NEXT: b [[TRY]]
 
 ; CHECK-THUMBV6-LABEL: test_cmpxchg_res_i8:
 ; CHECK-THUMBV6:       mov [[EXPECTED:r[0-9]+]], r1
 ; CHECK-THUMBV6-NEXT:  bl __sync_val_compare_and_swap_1
-; CHECK-THUMBV6-NEXT:  mov [[RES:r[0-9]+]], r0
-; CHECK-THUMBV6-NEXT:  movs r0, #1
-; CHECK-THUMBV6-NEXT:  movs [[ZERO:r[0-9]+]], #0
-; CHECK-THUMBV6-NEXT:  cmp [[RES]], [[EXPECTED]]
-; CHECK-THUMBV6-NEXT:  beq [[END:.LBB[0-9_]+]]
-; CHECK-THUMBV6-NEXT:  mov r0, [[ZERO]]
-; CHECK-THUMBV6-NEXT: [[END]]:
-; CHECK-THUMBV6-NEXT:  pop {{.*}}pc}
+; CHECK-THUMBV6-NEXT:  subs [[R1:r[0-7]]], r0, {{r[0-9]+}}
+; CHECK-THUMBV6-NEXT:  movs r0, #0
+; CHECK-THUMBV6-NEXT:  subs r0, r0, [[R1]]
+; CHECK-THUMBV6-NEXT:  adcs r0, [[R1]]
 
 ; CHECK-ARMV7-LABEL: test_cmpxchg_res_i8:
 ; CHECK-ARMV7-NEXT: .fnstart
 ; CHECK-ARMV7-NEXT: uxtb [[DESIRED:r[0-9]+]], r1
 ; CHECK-ARMV7-NEXT: b [[TRY:.LBB[0-9_]+]]
 ; CHECK-ARMV7-NEXT: [[HEAD:.LBB[0-9_]+]]:
 ; CHECK-ARMV7-NEXT: strexb [[SUCCESS:r[0-9]+]], r2, [r0]
 ; CHECK-ARMV7-NEXT: cmp [[SUCCESS]], #0

test/CodeGen/ARM/cmn.ll

 ; RUN: llc < %s -mtriple thumbv7-apple-ios | FileCheck %s
 ; <rdar://problem/7569620>
 
 define i32 @compare_i_gt(i32 %a) {
 entry:
 ; CHECK:     compare_i_gt
 ; CHECK-NOT: mvn
 ; CHECK:     cmn
   %cmp = icmp sgt i32 %a, -78
   %. = zext i1 %cmp to i32
   ret i32 %.
 }
 
 define i32 @compare_r_eq(i32 %a, i32 %b) {
 entry:
 ; CHECK: compare_r_eq
-; CHECK: cmn
+; CHECK: rsbs r1, r1, #0
+; CHECK: subs r0, r0, r1
+; CHECK: clz  r0, r0
+; CHECK: lsrs r0, r0, #5
   %sub = sub nsw i32 0, %b
   %cmp = icmp eq i32 %a, %sub
   %. = zext i1 %cmp to i32
   ret i32 %.
 }

test/CodeGen/ARM/cmp.ll

 ; RUN: llc -mtriple=armv7 %s -o - | FileCheck %s
 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s --check-prefix=CHECK-T2
 
 define i1 @f1(i32 %a, i32 %b) {
 ; CHECK-LABEL: f1:
-; CHECK: mov r2, #0
-; CHECK: cmp r0, r1
-; CHECK: movwne r2, #1
-; CHECK: mov r0, r2
-; CHECK-T2: mov{{.*}} r2, #0
-; CHECK-T2: cmp r0, r1
-; CHECK-T2: movne r2, #1
-; CHECK-T2: mov r0, r2
+; CHECK: subs r0, r0, r1
+; CHECK: movwne r0, #1
+; CHECK-T2: subs    r0, r0, r1
+; CHECK-T2: it      ne
+; CHECK-T2: movne   r0, #1
     %tmp = icmp ne i32 %a, %b
     ret i1 %tmp
 }
 
 define i1 @f2(i32 %a, i32 %b) {
 ; CHECK-LABEL: f2:
-; CHECK: mov r2, #0
-; CHECK: cmp r0, r1
-; CHECK: movweq r2, #1
-; CHECK: mov r0, r2
-; CHECK-T2: mov{{.*}} r2, #0
-; CHECK-T2: cmp r0, r1
-; CHECK-T2: moveq r2, #1
-; CHECK-T2: mov r0, r2
+; CHECK: sub     r0, r0, r1
+; CHECK: clz     r0, r0
+; CHECK: lsr     r0, r0, #5
+; CHECK-T2: subs    r0, r0, r1
+; CHECK-T2: clz     r0, r0
+; CHECK-T2: lsrs    r0, r0, #5
     %tmp = icmp eq i32 %a, %b
     ret i1 %tmp
 }
 
 define i1 @f6(i32 %a, i32 %b) {
 ; CHECK-LABEL: f6:
-; CHECK: mov r2, #0
-; CHECK: cmp {{.*}}, r1, lsl #5
-; CHECK: movweq r2, #1
-; CHECK: mov r0, r2
-; CHECK-T2: mov{{.*}} r2, #0
-; CHECK-T2: cmp.w r0, r1, lsl #5
-; CHECK-T2: moveq r2, #1
-; CHECK-T2: mov r0, r2
+; CHECK: sub     r0, r0, r1, lsl #5
+; CHECK: clz     r0, r0
+; CHECK: lsr     r0, r0, #5
+; CHECK-T2: sub.w   r0, r0, r1, lsl #5
+; CHECK-T2: clz     r0, r0
+; CHECK-T2: lsrs    r0, r0, #5
     %tmp = shl i32 %b, 5
     %tmp1 = icmp eq i32 %a, %tmp
     ret i1 %tmp1
 }
 
 define i1 @f7(i32 %a, i32 %b) {
 ; CHECK-LABEL: f7:
-; CHECK: mov r2, #0
-; CHECK: cmp r0, r1, lsr #6
-; CHECK: movwne r2, #1
-; CHECK: mov r0, r2
-; CHECK-T2: mov{{.*}} r2, #0
-; CHECK-T2: cmp.w r0, r1, lsr #6
-; CHECK-T2: movne r2, #1
-; CHECK-T2: mov r0, r2
+; CHECK: sub     r2, r0, r1, lsr #6
+; CHECK: cmp     r0, r1, lsr #6
+; CHECK: movwne  r2, #1
+; CHECK: mov     r0, r2
+; CHECK-T2: sub.w   r2, r0, r1, lsr #6
+; CHECK-T2: cmp.w   r0, r1, lsr #6
+; CHECK-T2: it      ne
+; CHECK-T2: movne   r2, #1
+; CHECK-T2: mov     r0, r2
     %tmp = lshr i32 %b, 6
     %tmp1 = icmp ne i32 %a, %tmp
     ret i1 %tmp1
 }
 
 define i1 @f8(i32 %a, i32 %b) {
 ; CHECK-LABEL: f8:
-; CHECK: mov r2, #0
-; CHECK: cmp r0, r1, asr #7
-; CHECK: movweq r2, #1
-; CHECK: mov r0, r2
-; CHECK-T2: mov{{.*}} r2, #0
-; CHECK-T2: cmp.w r0, r1, asr #7
-; CHECK-T2: moveq r2, #1
-; CHECK-T2: mov r0, r2
+; CHECK: sub     r0, r0, r1, asr #7
+; CHECK: clz     r0, r0
+; CHECK: lsr     r0, r0, #5
+; CHECK-T2: sub.w   r0, r0, r1, asr #7
+; CHECK-T2: clz     r0, r0
+; CHECK-T2: lsrs    r0, r0, #5
     %tmp = ashr i32 %b, 7
     %tmp1 = icmp eq i32 %a, %tmp
     ret i1 %tmp1
 }
 
 define i1 @f9(i32 %a) {
 ; CHECK-LABEL: f9:
-; CHECK: mov r1, #0
-; CHECK: cmp r0, r0, ror #8
-; CHECK: movwne r1, #1
-; CHECK: mov r0, r1
-; CHECK-T2: mov{{.*}} r1, #0
-; CHECK-T2: cmp.w r0, r0, ror #8
-; CHECK-T2: movne r1, #1
-; CHECK-T2: mov r0, r1
+; CHECK: sub     r1, r0, r0, ror #8
+; CHECK: cmp     r0, r0, ror #8
+; CHECK: movwne  r1, #1
+; CHECK: mov     r0, r1
+; CHECK-T2: sub.w   r1, r0, r0, ror #8
+; CHECK-T2: cmp.w   r0, r0, ror #8
+; CHECK-T2: it      ne
+; CHECK-T2: movne   r1, #1
+; CHECK-T2: mov     r0, r1
     %l8 = shl i32 %a, 24
     %r8 = lshr i32 %a, 8
     %tmp = or i32 %l8, %r8
     %tmp1 = icmp ne i32 %a, %tmp
     ret i1 %tmp1
 }
 
 ; CHECK-LABEL: swap_cmp_shl

test/CodeGen/ARM/cmpxchg-O0.ll

 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     ldrexb [[OLD:r[0-9]+]], [r0]
 ; CHECK:     cmp [[OLD]], [[DESIRED]]
 ; CHECK:     bne [[DONE:.LBB[0-9]+_[0-9]+]]
 ; CHECK:     strexb [[STATUS:r[0-9]+]], r2, [r0]
 ; CHECK:     cmp{{(\.w)?}} [[STATUS]], #0
 ; CHECK:     bne [[RETRY]]
 ; CHECK: [[DONE]]:
-; CHECK:     cmp{{(\.w)?}} [[OLD]], [[DESIRED]]
-; CHECK:     {{moveq|movweq}} {{r[0-9]+}}, #1
+; Materialisation of a boolean is done with sub/clz/lsr
+; CHECK:     sub{{(s)?}} [[CMP1:r[0-9]+]], [[OLD]], [[DESIRED]]
+; CHECK:     clz [[CMP2:r[0-9]+]], [[CMP1]]
+; CHECK:     lsr{{(s)?}} {{r[0-9]+}}, [[CMP2]], #5
 ; CHECK:     dmb ish
   %res = cmpxchg i8* %addr, i8 %desired, i8 %new seq_cst monotonic
   ret { i8, i1 } %res
 }
 
 define { i16, i1 } @test_cmpxchg_16(i16* %addr, i16 %desired, i16 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_16:
 ; CHECK:     dmb ish
 ; CHECK:     uxth [[DESIRED:r[0-9]+]], [[DESIRED]]
 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     ldrexh [[OLD:r[0-9]+]], [r0]
 ; CHECK:     cmp [[OLD]], [[DESIRED]]
 ; CHECK:     bne [[DONE:.LBB[0-9]+_[0-9]+]]
 ; CHECK:     strexh [[STATUS:r[0-9]+]], r2, [r0]
 ; CHECK:     cmp{{(\.w)?}} [[STATUS]], #0
 ; CHECK:     bne [[RETRY]]
 ; CHECK: [[DONE]]:
-; CHECK:     cmp{{(\.w)?}} [[OLD]], [[DESIRED]]
-; CHECK:     {{moveq|movweq}} {{r[0-9]+}}, #1
+; Materialisation of a boolean is done with sub/clz/lsr
+; CHECK:     sub{{(s)?}} [[CMP1:r[0-9]+]], [[OLD]], [[DESIRED]]
+; CHECK:     clz [[CMP2:r[0-9]+]], [[CMP1]]
+; CHECK:     lsr{{(s)?}} {{r[0-9]+}}, [[CMP2]], #5
 ; CHECK:     dmb ish
   %res = cmpxchg i16* %addr, i16 %desired, i16 %new seq_cst monotonic
   ret { i16, i1 } %res
 }
 
 define { i32, i1 } @test_cmpxchg_32(i32* %addr, i32 %desired, i32 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_32:
 ; CHECK:     dmb ish
 ; CHECK-NOT:     uxt
 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     ldrex [[OLD:r[0-9]+]], [r0]
 ; CHECK:     cmp [[OLD]], [[DESIRED]]
 ; CHECK:     bne [[DONE:.LBB[0-9]+_[0-9]+]]
 ; CHECK:     strex [[STATUS:r[0-9]+]], r2, [r0]
 ; CHECK:     cmp{{(\.w)?}} [[STATUS]], #0
 ; CHECK:     bne [[RETRY]]
 ; CHECK: [[DONE]]:
-; CHECK:     cmp{{(\.w)?}} [[OLD]], [[DESIRED]]
-; CHECK:     {{moveq|movweq}} {{r[0-9]+}}, #1
+; Materialisation of a boolean is done with sub/clz/lsr
+; CHECK:     sub{{(s)?}} [[CMP1:r[0-9]+]], [[OLD]], [[DESIRED]]
+; CHECK:     clz [[CMP2:r[0-9]+]], [[CMP1]]
+; CHECK:     lsr{{(s)?}} {{r[0-9]+}}, [[CMP2]], #5
 ; CHECK:     dmb ish
   %res = cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst monotonic
   ret { i32, i1 } %res
 }
 
 define { i64, i1 } @test_cmpxchg_64(i64* %addr, i64 %desired, i64 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_64:
 ; CHECK:     dmb ish

test/CodeGen/ARM/fp16-promote.ll

 ; instructions anyway.
 ; CHECK-ALL-LABEL: test_fcmp_une:
 ; CHECK-FP16: vcvtb.f32.f16
 ; CHECK-FP16: vcvtb.f32.f16
 ; CHECK-LIBCALL: bl __aeabi_h2f
 ; CHECK-LIBCALL: bl __aeabi_h2f
 ; CHECK-VFP: vcmp.f32
 ; CHECK-NOVFP: bl __aeabi_fcmpeq
-; CHECK-FP16: vmrs APSR_nzcv, fpscr
-; CHECK-ALL: movw{{ne|eq}}
+; CHECK-VFP-NEXT: vmrs APSR_nzcv, fpscr
+; CHECK-VFP-NEXT: movwne
+; CHECK-NOVFP-NEXT: clz r0, r0
+; CHECK-NOVFP-NEXT: lsr r0, r0, #5
 define i1 @test_fcmp_une(half* %p, half* %q) #0 {
   %a = load half, half* %p, align 2
   %b = load half, half* %q, align 2
   %r = fcmp une half %a, %b
   ret i1 %r
 }
 
 ; CHECK-ALL-LABEL: test_fcmp_ueq:

test/CodeGen/ARM/long-setcc.ll

 ; RUN: llc -mtriple=arm-eabi < %s | FileCheck %s
 
 define i1 @t1(i64 %x) {
 ; CHECK-LABEL: t1:
 ; CHECK: lsr  r0, r1, #31
   %B = icmp slt i64 %x, 0
   ret i1 %B
 }
 
 define i1 @t2(i64 %x) {
 ; CHECK-LABEL: t2:
-; CHECK: mov  r0, #0
-; CHECK: cmp  r1, #0
-; CHECK: moveq  r0, #1
+; CHECK: rsbs r0, r1, #0
+; CHECK: adc  r0, r1, r0
   %tmp = icmp ult i64 %x, 4294967296
   ret i1 %tmp
 }
 
 define i1 @t3(i32 %x) {
 ; CHECK-LABEL: t3:
 ; CHECK: mov  r0, #0
   %tmp = icmp ugt i32 %x, -1
   ret i1 %tmp
 }
 
 ; CHECK-NOT: cmp
 

test/CodeGen/ARM/select-imm.ll

 ; ARM: orr [[R1b:r[0-9]+]], [[R1]], #256
 ; ARM: movgt {{r[0-1]}}, #123
 
 ; ARMT2-LABEL: t1:
 ; ARMT2: movw [[R:r[0-1]]], #357
 ; ARMT2: movwgt [[R]], #123
 
 ; THUMB1-LABEL: t1:
-; THUMB1: mov     r1, r0
-; THUMB1: movs    r2, #255
-; THUMB1: adds    r2, #102
-; THUMB1: movs    r0, #123
-; THUMB1: cmp     r1, #1
+; THUMB1: cmp r{{[0-9]+}}, #1
 ; THUMB1: bgt
 
 ; THUMB2-LABEL: t1:
 ; THUMB2: movw [[R:r[0-1]]], #357
 ; THUMB2: movgt [[R]], #123
 
   %0 = icmp sgt i32 %c, 1
   %1 = select i1 %0, i32 123, i32 357
; THUMB2: movwgt [[R]], #357
   %0 = icmp sgt i32 %c, 1
   %1 = select i1 %0, i32 357, i32 123
   ret i32 %1
 }
 
 define i32 @t3(i32 %a) nounwind readnone {
 entry:
 ; ARM-LABEL: t3:
-; ARM: mov [[R:r[0-1]]], #0
-; ARM: moveq [[R]], #1
+; ARM: rsbs r1, r0, #0
+; ARM: adc  r0, r0, r1
 
 ; ARMT2-LABEL: t3:
-; ARMT2: mov [[R:r[0-1]]], #0
-; ARMT2: movweq [[R]], #1
+; ARMT2: clz r0, r0
+; ARMT2: lsr r0, r0, #5
 
 ; THUMB1-LABEL: t3:
-; THUMB1: mov     r1, r0
-; THUMB1: movs    r0, #1
-; THUMB1: movs    r2, #0
-; THUMB1: cmp     r1, #160
-; THUMB1: beq
+; THUMB1: movs r1, #0
+; THUMB1: subs r1, r1, r0
+; THUMB1: adcs r0, r1
 
 ; THUMB2-LABEL: t3:
-; THUMB2: mov{{(s|\.w)}} [[R:r[0-1]]], #0
-; THUMB2: moveq [[R]], #1
+; THUMB2: clz r0, r0
+; THUMB2: lsrs r0, r0, #5
   %0 = icmp eq i32 %a, 160
   %1 = zext i1 %0 to i32
   ret i32 %1
 }
 
 define i32 @t4(i32 %a, i32 %b, i32 %x) nounwind {
 entry:
 ; ARM-LABEL: t4:
; THUMB2: mvnlt [[R0:r[0-9]+]], #11141290
   ret i32 %1
 }
 
 ; rdar://9758317
 define i32 @t5(i32 %a) nounwind {
 entry:
 ; ARM-LABEL: t5:
 ; ARM-NOT: mov
-; ARM: cmp r0, #1
+; ARM: sub  r0, r0, #1
 ; ARM-NOT: mov
-; ARM: movne r0, #0
+; ARM: rsbs r1, r0, #0
+; ARM: adc  r0, r0, r1
 
 ; THUMB1-LABEL: t5:
-; THUMB1: mov     r1, r0
-; THUMB1: movs    r0, #0
-; THUMB1: cmp     r1, #1
-; THUMB1: bne
+; THUMB1-NOT: bne
+; THUMB1: movs r0, #0
+; THUMB1: subs r0, r0, r1
+; THUMB1: adcs r0, r1
 
 ; THUMB2-LABEL: t5:
 ; THUMB2-NOT: mov
-; THUMB2: cmp r0, #1
-; THUMB2: it ne
-; THUMB2: movne r0, #0
+; THUMB2: subs r0, #1
+; THUMB2: clz  r0, r0
+; THUMB2: lsrs r0, r0, #5
+
   %cmp = icmp eq i32 %a, 1
   %conv = zext i1 %cmp to i32
   ret i32 %conv
 }
 
 define i32 @t6(i32 %a) nounwind {
 entry:
 ; ARM-LABEL: t6:

 ; THUMB2: cmp r0, #0
 ; THUMB2: it ne
 ; THUMB2: movne r0, #1
   %tobool = icmp ne i32 %a, 0
   %lnot.ext = zext i1 %tobool to i32
   ret i32 %lnot.ext
 }
 
 define i32 @t7(i32 %a, i32 %b) nounwind readnone {
 entry:
 ; ARM-LABEL: t7:
-; ARM: mov     r2, #0
-; ARM: cmp     r0, r1
-; ARM: movne   r2, #1
-; ARM: lsl     r0, r2, #2
+; ARM: subs r0, r0, r1
+; ARM: movne   r0, #1
+; ARM: lsl     r0, r0, #2
 
 ; ARMT2-LABEL: t7:
-; ARMT2: mov     r2, #0
-; ARMT2: cmp     r0, r1
-; ARMT2: movwne  r2, #1
-; ARMT2: lsl     r0, r2, #2
+; ARMT2: subs r0, r0, r1
+; ARMT2: movwne r0, #1
+; ARMT2: lsl     r0, r0, #2
 
 ; THUMB1-LABEL: t7:
-; THUMB1: movs    r2, #1
-; THUMB1: movs    r3, #0
-; THUMB1: cmp     r0, r1
-; THUMB1: bne     .LBB6_2
-; THUMB1: mov     r2, r3
-; THUMB1: .LBB6_2:
-; THUMB1: lsls    r0, r2, #2
+; THUMB1: subs r0, r0, r1
+; THUMB1: subs r1, r0, #1
+; THUMB1: sbcs r0, r1
+; THUMB1: lsls r0, r0, #2
 
 ; THUMB2-LABEL: t7:
-; THUMB2: movs    r2, #0
-; THUMB2: cmp     r0, r1
-; THUMB2: it      ne
-; THUMB2: movne   r2, #1
-; THUMB2: lsls    r0, r2, #2
+; THUMB2: subs r0, r0, r1
+; THUMB2: it ne
+; THUMB2: movne r0, #1
+; THUMB2: lsls    r0, r0, #2
   %0 = icmp ne i32 %a, %b
   %1 = select i1 %0, i32 4, i32 0
   ret i32 %1
 }
 
 define void @t8(i32 %a) {
 entry:
 
 ; ARM scheduler emits icmp/zext before both calls, so isn't relevant
 
 ; ARMT2-LABEL: t8:
-; ARMT2: mov     r1, r0
-; ARMT2: mov     r0, #9
-; ARMT2: mov     r4, #0
-; ARMT2: cmp     r1, #5
-; ARMT2: movweq  r4, #1
-; ARMT2: bl      t7
+; ARMT2: bl t7
+; ARMT2: mov r1, r0
+; ARMT2: sub r0, r4, #5
+; ARMT2: clz r0, r0
+; ARMT2: lsr r0, r0, #5
 
 ; THUMB1-LABEL: t8:
-; THUMB1: mov     r1, r0
-; THUMB1: movs    r4, #1
-; THUMB1: movs    r0, #0
-; THUMB1: cmp     r1, #5
-; THUMB1: beq     .LBB7_2
-; THUMB1: mov     r4, r0
+; THUMB1: bl t7
+; THUMB1: mov r1, r0
+; THUMB1: subs r2, r4, #5
+; THUMB1: movs r0, #0
+; THUMB1: subs r0, r0, r2
+; THUMB1: adcs r0, r2
 
 ; THUMB2-LABEL: t8:
-; THUMB2: mov     r1, r0
-; THUMB2: movs    r4, #0
-; THUMB2: cmp     r1, #5
-; THUMB2: it      eq
-; THUMB2: moveq   r4, #1
+; THUMB2: bl t7
+; THUMB2: mov r1, r0
+; THUMB2: subs r0, r4, #5
+; THUMB2: clz r0, r0
+; THUMB2: lsrs r0, r0, #5
+
   %cmp = icmp eq i32 %a, 5
   %conv = zext i1 %cmp to i32
   %call = tail call i32 @t7(i32 9, i32 %a)
   tail call i32 @t7(i32 %conv, i32 %call)
   ret void
 }
 
 define void @t9(i8* %a, i8 %b) {
 entry:
 
 ; ARM scheduler emits icmp/zext before both calls, so isn't relevant
 
 ; ARMT2-LABEL: t9:
-; ARMT2: cmp     r4, r4
-; ARMT2: movweq  r0, #1
+; ARMT2: bl f
+; ARMT2: uxtb r0, r4
+; ARMT2: cmp  r0, r0
+; ARMT2: add  r1, r4, #1
+; ARMT2: mov  r2, r0
+; ARMT2: add  r2, r2, #1
+; ARMT2: add  r1, r1, #1
+; ARMT2: uxtb r3, r2
+; ARMT2: cmp  r3, r0
 
 ; THUMB1-LABEL: t9:
-; THUMB1: cmp     r4, r4
-; THUMB1: beq     .LBB8_2
-; THUMB1: mov     r0, r1
+; THUMB1: movs  r2, #0
+; THUMB1: subs  r1, r2, #0
+; THUMB1: adcs  r1, r2
 
 ; THUMB2-LABEL: t9:
-; THUMB2: cmp     r4, r4
-; THUMB2: it      eq
-; THUMB2: moveq   r0, #1
-
+; THUMB2: adds  r1, r4, #1
+; THUMB2: adds  r2, #1
+; THUMB2: adds  r1, #1
   %0 = load i8, i8* %a
   %conv = sext i8 %0 to i32
   %conv119 = zext i8 %0 to i32
   %conv522 = and i32 %conv, 255
   %cmp723 = icmp eq i32 %conv522, %conv119
   tail call void @f(i1 zeroext %cmp723)
   br i1 %cmp723, label %while.body, label %while.end
 
entry:
   %div = sdiv i32 %0, %1
   %mul = mul nsw i32 %div, %1
   %rem = srem i32 %0, %1
   %add = add nsw i32 %mul, %rem
   %cmp = icmp eq i32 %add, %0
   ret i1 %cmp
 
 ; ARM-LABEL: t10:
-; ARM: mov     r0, #0
-; ARM: cmn     r1, #3
-; ARM: moveq   r0, #1
+; ARM: rsbs r1, r0, #0
+; ARM: adc  r0, r0, r1
 
 ; ARMT2-LABEL: t10:
-; ARMT2: mov     r0, #0
-; ARMT2: cmn     r1, #3
-; ARMT2: movweq  r0, #1
+; ARMT2: clz r0, r0
+; ARMT2: lsr r0, r0, #5
 
 ; THUMB1-LABEL: t10:
-; THUMB1: movs    r0, #1
-; THUMB1: movs    r1, #0
-; THUMB1: cmp     r2, r5
-; THUMB1: beq     .LBB9_2
-; THUMB1: mov     r0, r1
+; THUMB1: movs r0, #0
+; THUMB1: subs r0, r0, r1
+; THUMB1: adcs r0, r1
 
 ; THUMB2-LABEL: t10:
-; THUMB2: adds    r0, #3
-; THUMB2: mov.w   r0, #0
-; THUMB2: it      eq
-; THUMB2: moveq   r0, #1
+; THUMB2: clz r0, r0
+; THUMB2: lsrs r0, r0, #5
 
 ; V8MBASE-LABEL: t10:
 ; V8MBASE-NOT: movs r0, #0
 ; V8MBASE: movs r0, #7
 }
 
 define i1 @t11() {
 entry:
entry:
   %clear9 = and i32 %set3, -4096
   %set10 = or i32 %clear9, %rem
   store i32 %set10, i32* %bit
   %clear12 = and i32 %set10, 4095
   %cmp = icmp eq i32 %clear12, 3
   ret i1 %cmp
 
 ; ARM-LABEL: t11:
-; ARM: mov     r0, #0
-; ARM: cmp     r1, #3
-; ARM: moveq   r0, #1
+; ARM: rsbs r1, r0, #0
+; ARM: adc  r0, r0, r1
 
 ; ARMT2-LABEL: t11:
-; ARMT2: mov     r0, #0
-; ARMT2: cmp     r1, #3
-; ARMT2: movweq  r0, #1
+; ARMT2: clz r0, r0
+; ARMT2: lsr r0, r0, #5
 
 ; THUMB1-LABEL: t11:
 ; THUMB1-NOT: movs r0, #0
 ; THUMB1: movs r0, #5
 
 ; THUMB2-LABEL: t11:
-; THUMB2: movs    r0, #0
-; THUMB2: cmp     r1, #3
-; THUMB2: it      eq
-; THUMB2: moveq   r0, #1
+; THUMB2: clz r0, r0
+; THUMB2: lsrs r0, r0, #5
 
 ; V8MBASE-LABEL: t11:
 ; V8MBASE-NOT: movs r0, #0
-; V8MBASE: movw  r0, #40960
+; V8MBASE: movw r0, #40960
 }
+

test/CodeGen/ARM/setcc-logic.ll

; CHECK-NEXT:    bx lr
   ret i1 %and
 }
 
 ; PR32401 - https://bugs.llvm.org/show_bug.cgi?id=32401
 
 define zeroext i1 @and_eq(i32 %a, i32 %b, i32 %c, i32 %d) nounwind {
 ; CHECK-LABEL: and_eq:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    eor r2, r2, r3
-; CHECK-NEXT:    eor r0, r0, r1
-; CHECK-NEXT:    orrs r0, r0, r2
-; CHECK-NEXT:    mov r0, #0
-; CHECK-NEXT:    movweq r0, #1
-; CHECK-NEXT:    bx lr
+; CHECK: eor     r2, r2, r3
+; CHECK: eor     r0, r0, r1
+; CHECK: orr     r0, r0, r2
+; CHECK: clz     r0, r0
+; CHECK: lsr     r0, r0, #5
+; CHECK: bx      lr
   %cmp1 = icmp eq i32 %a, %b
   %cmp2 = icmp eq i32 %c, %d
   %and = and i1 %cmp1, %cmp2
   ret i1 %and
 }
 
 define zeroext i1 @or_ne(i32 %a, i32 %b, i32 %c, i32 %d) nounwind {
 ; CHECK-LABEL: or_ne:

test/CodeGen/Thumb/branchless-cmp.ll

-; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumb-eabi -mcpu=cortex-m0 %s -verify-machineinstrs -o - | FileCheck %s
 
 define i32 @test1a(i32 %a, i32 %b) {
-; CHECK-LABEL: test1a:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    bne .LBB0_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB0_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp ne i32 %a, %b
   %cond = zext i1 %cmp to i32
   ret i32 %cond
+; CHECK-LABEL: test1a:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs r0, r0, r1
+; CHECK-NEXT:  subs r1, r0, #1
+; CHECK-NEXT:  sbcs r0, r1
 }
 
 define i32 @test1b(i32 %a, i32 %b) {
-; CHECK-LABEL: test1b:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    beq .LBB1_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB1_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp eq i32 %a, %b
   %cond = zext i1 %cmp to i32
   ret i32 %cond
+; CHECK-LABEL: test1b:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs    r1, r0, r1
+; CHECK-NEXT:  movs    r0, #0
+; CHECK-NEXT:  subs    r0, r0, r1
+; CHECK-NEXT:  adcs    r0, r1
 }
 
 define i32 @test2a(i32 %a, i32 %b) {
-; CHECK-LABEL: test2a:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    beq .LBB2_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB2_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp eq i32 %a, %b
   %cond = zext i1 %cmp to i32
   ret i32 %cond
+; CHECK-LABEL: test2a:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs    r1, r0, r1
+; CHECK-NEXT:  movs    r0, #0
+; CHECK-NEXT:  subs    r0, r0, r1
+; CHECK-NEXT:  adcs    r0, r1
 }
 
 define i32 @test2b(i32 %a, i32 %b) {
-; CHECK-LABEL: test2b:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    bne .LBB3_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB3_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp ne i32 %a, %b
   %cond = zext i1 %cmp to i32
   ret i32 %cond
+; CHECK-LABEL: test2b:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs    r0, r0, r1
+; CHECK-NEXT:  subs    r1, r0, #1
+; CHECK-NEXT:  sbcs    r0, r1
 }
 
 define i32 @test3a(i32 %a, i32 %b) {
-; CHECK-LABEL: test3a:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    movs r3, #4
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    beq .LBB4_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB4_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp eq i32 %a, %b
   %cond = select i1 %cmp, i32 0, i32 4
   ret i32 %cond
+; CHECK-LABEL: test3a:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs    r0, r0, r1
+; CHECK-NEXT:  subs    r1, r0, #1
+; CHECK-NEXT:  sbcs    r0, r1
+; CHECK-NEXT:  lsls    r0, r0, #2
 }
 
 define i32 @test3b(i32 %a, i32 %b) {
-; CHECK-LABEL: test3b:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    movs r2, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r0, r1
-; CHECK-NEXT:    beq .LBB5_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r2, r3
-; CHECK-NEXT:  .LBB5_2: @ %entry
-; CHECK-NEXT:    lsls r0, r2, #2
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp eq i32 %a, %b
   %cond = select i1 %cmp, i32 4, i32 0
   ret i32 %cond
+; CHECK-LABEL: test3b:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:      subs  r0, r0, r1
+; CHECK-NEXT: movs  r1, #0
+; CHECK-NEXT: subs  r1, r1, r0
+; CHECK-NEXT: adcs  r1, r0
+; CHECK-NEXT: lsls  r0, r1, #2
 }
 
 ; FIXME: This one hasn't changed actually
 ; but could look like test3b
 define i32 @test4a(i32 %a, i32 %b) {
-; CHECK-LABEL: test4a:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    movs r3, #4
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    bne .LBB6_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r0, r3
-; CHECK-NEXT:  .LBB6_2: @ %entry
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp ne i32 %a, %b
   %cond = select i1 %cmp, i32 0, i32 4
   ret i32 %cond
+; CHECK-LABEL: test4a:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       mov     r2, r0
+; CHECK-NEXT:  movs    r0, #0
+; CHECK-NEXT:  movs    r3, #4
+; CHECK-NEXT:  cmp     r2, r1
+; CHECK-NEXT:  bne     .[[BRANCH:[A-Z0-9_]+]]
+; CHECK:       mov     r0, r3
+; CHECK:       .[[BRANCH]]:
 }
 
 define i32 @test4b(i32 %a, i32 %b) {
-; CHECK-LABEL: test4b:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    movs r2, #1
-; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    cmp r0, r1
-; CHECK-NEXT:    bne .LBB7_2
-; CHECK-NEXT:  @ %bb.1: @ %entry
-; CHECK-NEXT:    mov r2, r3
-; CHECK-NEXT:  .LBB7_2: @ %entry
-; CHECK-NEXT:    lsls r0, r2, #2
-; CHECK-NEXT:    bx lr
 entry:
   %cmp = icmp ne i32 %a, %b
   %cond = select i1 %cmp, i32 4, i32 0
   ret i32 %cond
+; CHECK-LABEL: test4b:
+; CHECK-NOT: b{{(ne)|(eq)}}
+; CHECK:       subs  r0, r0, r1
+; CHECK-NEXT:  subs  r1, r0, #1
+; CHECK-NEXT:  sbcs  r0, r1
+; CHECK-NEXT:  lsls  r0, r0, #2
 }
 

test/CodeGen/Thumb/constants.ll

 
 ; CHECK-T1-LABEL: @mov_and_add2
 ; CHECK-T2-LABEL: @mov_and_add2
 ; CHECK-T1: ldr r0,
 ; CHECK-T2: movw r0, #511
 define i32 @mov_and_add2() {
   ret i32 511
 }
+
+; CHECK-T1-LABEL: @test64
+; CHECK-T2-LABEL: @test64
+; CHECK-T1: movs r4, #0
+; CHECK-T1: mvns r5, r4
+; CHECK-T1: mov r0, r5
+; CHECK-T1: subs r0, #15
+; CHECK-T2: subs.w r0, r{{[0-9]+}}, #15
+; CHECK-T2-NEXT: sbc r1, r{{[0-9]+}}, #0
+define i32 @test64() {
+entry:
+  tail call void @fn1(i64 -1)
+  tail call void @fn1(i64 -1)
+  tail call void @fn1(i64 -16)
+  ret i32 0
+}
+declare void @fn1(i64) ;

test/CodeGen/Thumb/long-setcc.ll

 ; RUN: llc -mtriple=thumb-eabi < %s | FileCheck %s
 
 define i1 @t1(i64 %x) {
 ; CHECK-LABEL: t1:
-; CHECK: lsrs  r0, r1, #31
-  %B = icmp slt i64 %x, 0
-  ret i1 %B
+; CHECK: lsrs r0, r1, #31
+  %B = icmp slt i64 %x, 0
+  ret i1 %B
 }
 
 define i1 @t2(i64 %x) {
 ; CHECK-LABEL: t2:
-; CHECK: movs    r0, #1
-; CHECK: movs    r2, #0
-; CHECK: cmp     r1, #0
-; CHECK: beq     .LBB1_2
-  %tmp = icmp ult i64 %x, 4294967296
-  ret i1 %tmp
+; CHECK: movs r0, #0
+; CHECK: subs r0, r0, r1
+; CHECK: adcs r0, r1
+  %tmp = icmp ult i64 %x, 4294967296
+  ret i1 %tmp
 }
 
 define i1 @t3(i32 %x) {
 ; CHECK-LABEL: t3:
-; CHECK: movs  r0, #0
-  %tmp = icmp ugt i32 %x, -1
-  ret i1 %tmp
+; CHECK: movs r0, #0
+  %tmp = icmp ugt i32 %x, -1
+  ret i1 %tmp
 }
 
-
 ; CHECK-NOT: cmp
+

test/CodeGen/Thumb2/float-cmp.ll

 }
 define i1 @cmp_f_ord(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_ord:
 ; NONE: bl __aeabi_fcmpun
 ; HARD: vcmpe.f32
 ; HARD: movvc r0, #1
   %1 = fcmp ord float %a, %b
   ret i1 %1
-}define i1 @cmp_f_ueq(float %a, float %b) {
+}
+define i1 @cmp_f_ueq(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_ueq:
 ; NONE: bl __aeabi_fcmpeq
 ; NONE: bl __aeabi_fcmpun
 ; HARD: vcmp.f32
 ; HARD: moveq r0, #1
 ; HARD: movvs r0, #1
   %1 = fcmp ueq float %a, %b
   ret i1 %1
 }
 define i1 @cmp_f_ugt(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_ugt:
 ; NONE: bl __aeabi_fcmple
-; NONE: cmp r0, #0
-; NONE-NEXT: it eq
+; NONE-NEXT: clz r0, r0
+; NONE-NEXT: lsrs r0, r0, #5
 ; HARD: vcmpe.f32
 ; HARD: movhi r0, #1
   %1 = fcmp ugt float %a, %b
   ret i1 %1
 }
 define i1 @cmp_f_uge(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_uge:
 ; NONE: bl __aeabi_fcmplt
-; NONE: cmp r0, #0
-; NONE-NEXT: it eq
+; NONE-NEXT: clz r0, r0
+; NONE-NEXT: lsrs r0, r0, #5
 ; HARD: vcmpe.f32
 ; HARD: movpl r0, #1
   %1 = fcmp uge float %a, %b
   ret i1 %1
 }
 define i1 @cmp_f_ult(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_ult:
 ; NONE: bl __aeabi_fcmpge
-; NONE: cmp r0, #0
-; NONE-NEXT: it eq
+; NONE-NEXT: clz r0, r0
+; NONE-NEXT: lsrs r0, r0, #5
 ; HARD: vcmpe.f32
 ; HARD: movlt r0, #1
   %1 = fcmp ult float %a, %b
   ret i1 %1
 }
 define i1 @cmp_f_ule(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_ule:
 ; NONE: bl __aeabi_fcmpgt
-; NONE: cmp r0, #0
-; NONE-NEXT: it eq
+; NONE-NEXT: clz r0, r0
+; NONE-NEXT: lsrs r0, r0, #5
 ; HARD: vcmpe.f32
 ; HARD: movle r0, #1
   %1 = fcmp ule float %a, %b
   ret i1 %1
 }
 define i1 @cmp_f_une(float %a, float %b) {
 ; CHECK-LABEL: cmp_f_une:
 ; NONE: bl __aeabi_fcmpeq

test/CodeGen/Thumb2/thumb2-cmn.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests could be improved by 'movs r0, #0' being rematerialized below the
 ; test as 'mov.w r0, #0'.
 
-define i1 @f1(i32 %a, i32 %b) {
+define i32 @f1(i32 %a, i32 %b) {
     %nb = sub i32 0, %b
     %tmp = icmp ne i32 %a, %nb
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f1:
 ; CHECK:  cmn {{.*}}, r1
 
-define i1 @f2(i32 %a, i32 %b) {
+define i32 @f2(i32 %a, i32 %b) {
     %nb = sub i32 0, %b
     %tmp = icmp ne i32 %nb, %a
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f2:
 ; CHECK:  cmn {{.*}}, r1
 
-define i1 @f3(i32 %a, i32 %b) {
+define i32 @f3(i32 %a, i32 %b) {
     %nb = sub i32 0, %b
     %tmp = icmp eq i32 %a, %nb
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f3:
 ; CHECK:  cmn {{.*}}, r1
 
-define i1 @f4(i32 %a, i32 %b) {
+define i32 @f4(i32 %a, i32 %b) {
     %nb = sub i32 0, %b
     %tmp = icmp eq i32 %nb, %a
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f4:
 ; CHECK:  cmn {{.*}}, r1
 
-define i1 @f5(i32 %a, i32 %b) {
+define i32 @f5(i32 %a, i32 %b) {
     %tmp = shl i32 %b, 5
     %nb = sub i32 0, %tmp
     %tmp1 = icmp eq i32 %nb, %a
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f5:
 ; CHECK:  cmn.w {{.*}}, r1, lsl #5
 
-define i1 @f6(i32 %a, i32 %b) {
+define i32 @f6(i32 %a, i32 %b) {
     %tmp = lshr i32 %b, 6
     %nb = sub i32 0, %tmp
     %tmp1 = icmp ne i32 %nb, %a
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f6:
 ; CHECK:  cmn.w {{.*}}, r1, lsr #6
 
-define i1 @f7(i32 %a, i32 %b) {
+define i32 @f7(i32 %a, i32 %b) {
     %tmp = ashr i32 %b, 7
     %nb = sub i32 0, %tmp
     %tmp1 = icmp eq i32 %a, %nb
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f7:
 ; CHECK:  cmn.w {{.*}}, r1, asr #7
 
-define i1 @f8(i32 %a, i32 %b) {
+define i32 @f8(i32 %a, i32 %b) {
     %l8 = shl i32 %a, 24
     %r8 = lshr i32 %a, 8
     %tmp = or i32 %l8, %r8
     %nb = sub i32 0, %tmp
     %tmp1 = icmp ne i32 %a, %nb
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f8:
 ; CHECK:  cmn.w {{.*}}, {{.*}}, ror #8
 
 
 define void @f9(i32 %a, i32 %b) nounwind optsize {
   tail call void asm sideeffect "cmn.w     r0, r1", ""() nounwind, !srcloc !0
   ret void
 }
 
 !0 = !{i32 81}
 
 ; CHECK-LABEL: f9:
 ; CHECK:  cmn.w r0, r1

test/CodeGen/Thumb2/thumb2-cmn2.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; -0x000000bb = 4294967109
-define i1 @f1(i32 %a) {
+define i32 @f1(i32 %a) {
 ; CHECK-LABEL: f1:
 ; CHECK: adds {{r.*}}, #187
     %tmp = icmp ne i32 %a, 4294967109
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; -0x00aa00aa = 4283826006
-define i1 @f2(i32 %a) {
+define i32 @f2(i32 %a) {
 ; CHECK-LABEL: f2:
 ; CHECK: cmn.w {{r.*}}, #11141290
     %tmp = icmp eq i32 %a, 4283826006
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; -0xcc00cc00 = 872363008
-define i1 @f3(i32 %a) {
+define i32 @f3(i32 %a) {
 ; CHECK-LABEL: f3:
 ; CHECK: cmn.w {{r.*}}, #-872363008
     %tmp = icmp ne i32 %a, 872363008
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; -0x00110000 = 4293853184
-define i1 @f4(i32 %a) {
+define i32 @f4(i32 %a) {
 ; CHECK-LABEL: f4:
 ; CHECK: cmn.w {{r.*}}, #1114112
     %tmp = icmp eq i32 %a, 4293853184
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }

test/CodeGen/Thumb2/thumb2-cmp.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests would be improved by 'movs r0, #0' being rematerialized below the
 ; test as 'mov.w r0, #0'.
 
 ; 0x000000bb = 187
-define i1 @f1(i32 %a) {
+define i32 @f1(i32 %a) {
 ; CHECK-LABEL: f1:
 ; CHECK: cmp {{.*}}, #187
     %tmp = icmp ne i32 %a, 187
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; 0x00aa00aa = 11141290
-define i1 @f2(i32 %a) {
+define i32 @f2(i32 %a) {
 ; CHECK-LABEL: f2:
 ; CHECK: cmp.w {{.*}}, #11141290
     %tmp = icmp eq i32 %a, 11141290 
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; 0xcc00cc00 = 3422604288
-define i1 @f3(i32 %a) {
+define i32 @f3(i32 %a) {
 ; CHECK-LABEL: f3:
 ; CHECK: cmp.w {{.*}}, #-872363008
     %tmp = icmp ne i32 %a, 3422604288
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; 0xdddddddd = 3722304989
-define i1 @f4(i32 %a) {
+define i32 @f4(i32 %a) {
 ; CHECK-LABEL: f4:
 ; CHECK: cmp.w {{.*}}, #-572662307
     %tmp = icmp ne i32 %a, 3722304989
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; 0x00110000 = 1114112
-define i1 @f5(i32 %a) {
+define i32 @f5(i32 %a) {
 ; CHECK-LABEL: f5:
 ; CHECK: cmp.w {{.*}}, #1114112
     %tmp = icmp eq i32 %a, 1114112
-    ret i1 %tmp
+    %ret = select i1 %tmp, i32 42, i32 24
+    ret i32 %ret
 }
 
 ; Check that we don't do an invalid (a > b) --> !(a < b + 1) transform.
 ;
 ; CHECK-LABEL: f6:
 ; CHECK-NOT: cmp.w {{.*}}, #-2147483648
 ; CHECK: bx lr
 define i32 @f6(i32 %a) {
     %tmp = icmp sgt i32 %a, 2147483647
     br i1 %tmp, label %true, label %false
 true:
     ret i32 2
 false:
     ret i32 0
 }

test/CodeGen/Thumb2/thumb2-teq.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests would be improved by 'movs r0, #0' being rematerialized below the
 ; test as 'mov.w r0, #0'.
 
 ; 0x000000bb = 187
-define i1 @f2(i32 %a) {
+define i32 @f2(i32 %a) {
     %tmp = xor i32 %a, 187
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f2:
 ; CHECK:  teq.w {{.*}}, #187
 
 ; 0x00aa00aa = 11141290
-define i1 @f3(i32 %a) {
+define i32 @f3(i32 %a) {
     %tmp = xor i32 %a, 11141290 
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f3:
 ; CHECK:  teq.w {{.*}}, #11141290
 
 ; 0xcc00cc00 = 3422604288
-define i1 @f6(i32 %a) {
+define i32 @f6(i32 %a) {
     %tmp = xor i32 %a, 3422604288
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f6:
 ; CHECK:  teq.w {{.*}}, #-872363008
 
 ; 0xdddddddd = 3722304989
-define i1 @f7(i32 %a) {
+define i32 @f7(i32 %a) {
     %tmp = xor i32 %a, 3722304989
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f7:
 ; CHECK:  teq.w {{.*}}, #-572662307
 
+; 0x00110000 = 1114112
+define i32 @f10(i32 %a) {
+    %tmp = xor i32 %a, 1114112
+    %tmp1 = icmp eq i32 0, %tmp
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
+}
+; CHECK-LABEL: f10:
+; CHECK:  teq.w {{.*}}, #1114112
+
+; 0x000000bb = 187
+define i1 @f12(i32 %a) {
+    %tmp = xor i32 %a, 187
+    %tmp1 = icmp eq i32 0, %tmp
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f12:
+; CHECK:      eor r0, r0, #187
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0x00aa00aa = 11141290
+define i1 @f13(i32 %a) {
+    %tmp = xor i32 %a, 11141290 
+    %tmp1 = icmp eq i32 %tmp, 0
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f13:
+; CHECK:      eor r0, r0, #11141290
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0xcc00cc00 = 3422604288
+define i1 @f16(i32 %a) {
+    %tmp = xor i32 %a, 3422604288
+    %tmp1 = icmp eq i32 0, %tmp
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f16:
+; CHECK:      eor r0, r0, #-872363008
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
 ; 0xdddddddd = 3722304989
-define i1 @f8(i32 %a) {
+define i1 @f17(i32 %a) {
     %tmp = xor i32 %a, 3722304989
-    %tmp1 = icmp ne i32 0, %tmp
+    %tmp1 = icmp eq i32 %tmp, 0
     ret i1 %tmp1
 }
+; CHECK-LABEL: f17:
+; CHECK:      eor r0, r0, #-572662307
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
 
 ; 0x00110000 = 1114112
-define i1 @f10(i32 %a) {
+define i1 @f18(i32 %a) {
     %tmp = xor i32 %a, 1114112
     %tmp1 = icmp eq i32 0, %tmp
     ret i1 %tmp1
 }
-; CHECK-LABEL: f10:
-; CHECK:  teq.w {{.*}}, #1114112
+; CHECK-LABEL: f18:
+; CHECK:      eor r0, r0, #1114112
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
 

test/CodeGen/Thumb2/thumb2-teq2.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests would be improved by 'movs r0, #0' being rematerialized below the
 ; tst as 'mov.w r0, #0'.
 
-define i1 @f2(i32 %a, i32 %b) {
+define i32 @f2(i32 %a, i32 %b) {
 ; CHECK: f2
 ; CHECK: teq.w {{.*}}, r1
     %tmp = xor i32 %a, %b
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f4(i32 %a, i32 %b) {
+define i32 @f4(i32 %a, i32 %b) {
 ; CHECK: f4
 ; CHECK: teq.w  {{.*}}, r1
     %tmp = xor i32 %a, %b
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f6(i32 %a, i32 %b) {
+define i32 @f6(i32 %a, i32 %b) {
 ; CHECK: f6
 ; CHECK: teq.w  {{.*}}, r1, lsl #5
     %tmp = shl i32 %b, 5
     %tmp1 = xor i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f7(i32 %a, i32 %b) {
+define i32 @f7(i32 %a, i32 %b) {
 ; CHECK: f7
 ; CHECK: teq.w  {{.*}}, r1, lsr #6
     %tmp = lshr i32 %b, 6
     %tmp1 = xor i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f8(i32 %a, i32 %b) {
+define i32 @f8(i32 %a, i32 %b) {
 ; CHECK: f8
 ; CHECK: teq.w  {{.*}}, r1, asr #7
     %tmp = ashr i32 %b, 7
     %tmp1 = xor i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f9(i32 %a, i32 %b) {
+define i32 @f9(i32 %a, i32 %b) {
 ; CHECK: f9
 ; CHECK: teq.w  {{.*}}, {{.*}}, ror #8
     %l8 = shl i32 %a, 24
     %r8 = lshr i32 %a, 8
     %tmp = or i32 %l8, %r8
     %tmp1 = xor i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }

test/CodeGen/Thumb2/thumb2-tst.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests would be improved by 'movs r0, #0' being rematerialized below the
 ; tst as 'mov.w r0, #0'.
 
 ; 0x000000bb = 187
-define i1 @f2(i32 %a) {
+define i32 @f2(i32 %a) {
     %tmp = and i32 %a, 187
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f2:
 ; CHECK:  tst.w {{.*}}, #187
 
 ; 0x00aa00aa = 11141290
-define i1 @f3(i32 %a) {
+define i32 @f3(i32 %a) {
     %tmp = and i32 %a, 11141290 
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f3:
 ; CHECK:  tst.w {{.*}}, #11141290
 
 ; 0xcc00cc00 = 3422604288
-define i1 @f6(i32 %a) {
+define i32 @f6(i32 %a) {
     %tmp = and i32 %a, 3422604288
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f6:
 ; CHECK:  tst.w {{.*}}, #-872363008
 
 ; 0xdddddddd = 3722304989
-define i1 @f7(i32 %a) {
+define i32 @f7(i32 %a) {
     %tmp = and i32 %a, 3722304989
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f7:
 ; CHECK:  tst.w {{.*}}, #-572662307
 
 ; 0x00110000 = 1114112
-define i1 @f10(i32 %a) {
+define i32 @f10(i32 %a) {
     %tmp = and i32 %a, 1114112
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 ; CHECK-LABEL: f10:
 ; CHECK:  tst.w {{.*}}, #1114112
+
+; 0x000000bb = 187
+define i1 @f12(i32 %a) {
+    %tmp = and i32 %a, 187
+    %tmp1 = icmp eq i32 0, %tmp
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f12:
+; CHECK:    and r0, r0, #187
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0x00aa00aa = 11141290
+define i1 @f13(i32 %a) {
+    %tmp = and i32 %a, 11141290
+    %tmp1 = icmp eq i32 %tmp, 0
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f13:
+; CHECK:    and r0, r0, #11141290
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0xcc00cc00 = 3422604288
+define i1 @f16(i32 %a) {
+    %tmp = and i32 %a, 3422604288
+    %tmp1 = icmp eq i32 0, %tmp
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f16:
+; CHECK:    and r0, r0, #-872363008
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0xdddddddd = 3722304989
+define i1 @f17(i32 %a) {
+    %tmp = and i32 %a, 3722304989
+    %tmp1 = icmp eq i32 %tmp, 0
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f17:
+; CHECK:    bic r0, r0, #572662306
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+
+; 0x00110000 = 1114112
+define i1 @f18(i32 %a) {
+    %tmp = and i32 %a, 1114112
+    %tmp1 = icmp eq i32 0, %tmp
+    ret i1 %tmp1
+}
+; CHECK-LABEL: f18:
+; CHECK:    and r0, r0, #1114112
+; CHECK-NEXT: clz r0, r0
+; CHECK-NEXT: lsrs r0, r0, #5
+

test/CodeGen/Thumb2/thumb2-tst2.ll

 ; RUN: llc -mtriple=thumb-eabi -mcpu=arm1156t2-s -mattr=+thumb2 %s -o - | FileCheck %s
 
 ; These tests would be improved by 'movs r0, #0' being rematerialized below the
 ; tst as 'mov.w r0, #0'.
 
-define i1 @f2(i32 %a, i32 %b) {
+define i32 @f2(i32 %a, i32 %b) {
 ; CHECK-LABEL: f2:
 ; CHECK: tst {{.*}}, r1
     %tmp = and i32 %a, %b
     %tmp1 = icmp eq i32 %tmp, 0
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f4(i32 %a, i32 %b) {
+define i32 @f4(i32 %a, i32 %b) {
 ; CHECK-LABEL: f4:
 ; CHECK: tst {{.*}}, r1
     %tmp = and i32 %a, %b
     %tmp1 = icmp eq i32 0, %tmp
-    ret i1 %tmp1
+    %ret = select i1 %tmp1, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f6(i32 %a, i32 %b) {
+define i32 @f6(i32 %a, i32 %b) {
 ; CHECK-LABEL: f6:
 ; CHECK: tst.w {{.*}}, r1, lsl #5
     %tmp = shl i32 %b, 5
     %tmp1 = and i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f7(i32 %a, i32 %b) {
+define i32 @f7(i32 %a, i32 %b) {
 ; CHECK-LABEL: f7:
 ; CHECK: tst.w {{.*}}, r1, lsr #6
     %tmp = lshr i32 %b, 6
     %tmp1 = and i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f8(i32 %a, i32 %b) {
+define i32 @f8(i32 %a, i32 %b) {
 ; CHECK-LABEL: f8:
 ; CHECK: tst.w {{.*}}, r1, asr #7
     %tmp = ashr i32 %b, 7
     %tmp1 = and i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }
 
-define i1 @f9(i32 %a, i32 %b) {
+define i32 @f9(i32 %a, i32 %b) {
 ; CHECK-LABEL: f9:
 ; CHECK: tst.w {{.*}}, {{.*}}, ror #8
     %l8 = shl i32 %a, 24
     %r8 = lshr i32 %a, 8
     %tmp = or i32 %l8, %r8
     %tmp1 = and i32 %a, %tmp
     %tmp2 = icmp eq i32 %tmp1, 0
-    ret i1 %tmp2
+    %ret = select i1 %tmp2, i32 42, i32 24
+    ret i32 %ret
 }