[ARM] Support all versions of AND, ORR, EOR and BIC in optimizeCompareInstr

[fzhinkin]

Summary

Combine cmp with zero and all versions of AND, ORR, EOR and BIC instructions into S-suffixed versions.

Related issue: https://github.com/llvm/llvm-project/issues/57122

[fzhinkin]

llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp

3009

Didn't support shifts as it seems very unlikely to see these instructions here.

llvm/test/CodeGen/ARM/cmp-peephole.ll

6

I'll precommit new tests if there will be no objections.

[fzhinkin]

Supported shifts

[fzhinkin]

Added tests on zero comparison for equality.

[fzhinkin]

Rebase

[fzhinkin]

Built stage2 binaries and executed clang and llvm tests, all passed:

build.log.bz2538 KBDownload

[fzhinkin]

Rebase

[fzhinkin]

This comment was removed by fzhinkin.

[dmgreen]

Sounds like a good patch. Are all the cases covered by the tests? It looks like they are, but it's a bit hard to tell from the current file.

llvm/test/CodeGen/ARM/cmp-peephole.ll

6

Precommitting the tests sounds good.

[fzhinkin]

Rebase

llvm/test/CodeGen/ARM/cmp-peephole.ll

6

Done

[efriedma]

llvm/lib/Target/ARM/ARMInstrInfo.td

4888 ↗

(On Diff #458790)

I'm not sure how you use the shift's S-version in general. Do we have test coverage for something like void a(int x, int y, void z()) { if ((x >> 1) == y) z(); }?

(If the issue is that we're choosing CMPrsi over CMPri, you can probably use AddedComplexity to force the preferred form.)

[fzhinkin]

llvm/lib/Target/ARM/ARMInstrInfo.td

4888 ↗

(On Diff #458790)

Thank you for taking a look at it! It seems like there are no tests covering this particular case (and I didn't add it either :( ), and for code that looks like something you mentioned removal of these patterns will result in a worse code sequence (shift + cmp instead of single cmp). I'll roll back this part of the change.

The issue I tried to overcome by removing two ARMcmpZ patterns is following:
consider a code like return (a >> b) != 0 ? (a >> b) : 42 (https://godbolt.org/z/EfsTq3dP4).
Before instruction selection corresponding DAG will looks like:

  t0: ch = EntryToken
  t2: i32,ch = CopyFromReg t0, Register:i32 %0
  t4: i32,ch = CopyFromReg t0, Register:i32 %1
t5: i32 = srl t2, t4
    t16: glue = ARMISD::CMPZ t5, Constant:i32<0>
  t17: i32 = ARMISD::CMOV t5, Constant:i32<42>, Constant:i32<0>, Register:i32 $cpsr, t16
t12: ch,glue = CopyToReg t0, Register:i32 $r0, t17
t13: ch = ARMISD::RET_FLAG t12, Register:i32 $r0, t12:1

CMPZ t5 where t5 is i32 = srl t2, t4 will match pattern for ARMcmpZ, so after instr selection we'll get:

t0: ch = EntryToken
 t2: i32,ch = CopyFromReg t0, Register:i32 %0
 t4: i32,ch = CopyFromReg t0, Register:i32 %1
     t5: i32 = MOVsr t2, t4, TargetConstant:i32<3>, TargetConstant:i32<14>, Register:i32 $noreg, Register:i32 $noreg
       t6: i32 = MOVi TargetConstant:i32<0>, TargetConstant:i32<14>, Register:i32 $noreg, Register:i32 $noreg
     t16: i32,glue = CMPrsr t6, t2, t4, TargetConstant:i32<3>, TargetConstant:i32<14>, Register:i32 $noreg
   t17: i32 = MOVCCi t5, TargetConstant:i32<42>, TargetConstant:i32<0>, Register:i32 $cpsr, t16:1
 t12: ch,glue = CopyToReg t0, Register:i32 $r0, t17
 t13: ch = BX_RET TargetConstant:i32<14>, Register:i32 $noreg, Register:i32 $r0, t12, t12:1

CMPrsr (t16) is no longer related to t5 and isOptimizeCompareCandidate won't be able to replace it with S-version of t5.

I tried to use AddedComplexity, but it didn't help.
Probably, the better place to handle this case is ARMTargetLowering::getARMCmp (and there is a TODO suggesting to do that there, and I missed it previously).

[fzhinkin]

llvm/lib/Target/ARM/ARMInstrInfo.td

4888 ↗

(On Diff #458790)

A shameful part of why "AddedComplexity didn't help" is that I applied it incorrectly. :)
Otherwise it seems to work, I'll add test cases for code like void a(int x, int y, void z()) { if ((x >> 1) == y) z(); } and will come back with the update.

[fzhinkin]

Introduced additional patterns to allow handing of shift+cmp by the peephole

[samtebbs]

Looks pretty good to me, but maybe give the other reviewers some time for another look.

[fzhinkin]

In D131786#3802609, @samtebbs wrote:

Looks pretty good to me, but maybe give the other reviewers some time for another look.

Thank you!

@dmgreen @efriedma are you OK with this change?

[efriedma]

LGTM

llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp

static bool isOptimizeCompareCandidate(MachineInstr *MI, bool &IsThumb1) {
   case ARM::t2ADCrr:
   case ARM::t2ADCri:
   case ARM::t2SUBrr:
   case ARM::t2SUBri:
   case ARM::t2SBCrr:
   case ARM::t2SBCri:
   case ARM::ANDrr:
   case ARM::ANDri:
+  case ARM::ANDrsr:
+  case ARM::ANDrsi:
   case ARM::t2ANDrr:
   case ARM::t2ANDri:
+  case ARM::t2ANDrs:
   case ARM::ORRrr:
   case ARM::ORRri:
+  case ARM::ORRrsr:
+  case ARM::ORRrsi:
   case ARM::t2ORRrr:
   case ARM::t2ORRri:
+  case ARM::t2ORRrs:
   case ARM::EORrr:
   case ARM::EORri:
+  case ARM::EORrsr:
+  case ARM::EORrsi:
   case ARM::t2EORrr:
   case ARM::t2EORri:
+  case ARM::t2EORrs:
+  case ARM::BICri:
+  case ARM::BICrr:
+  case ARM::BICrsi:
+  case ARM::BICrsr:
+  case ARM::t2BICri:
+  case ARM::t2BICrr:
+  case ARM::t2BICrs:
   case ARM::t2LSRri:
   case ARM::t2LSRrr:
   case ARM::t2LSLri:
   case ARM::t2LSLrr:
     return true;
   }
 }
 
 /// optimizeCompareInstr - Convert the instruction supplying the argument to the
 /// comparison into one that sets the zero bit in the flags register;
 /// Remove a redundant Compare instruction if an earlier instruction can set the
 /// flags in the same way as Compare.
if (!isSafe) {
     for (MachineBasicBlock *Succ : MBB->successors())
       if (Succ->isLiveIn(ARM::CPSR))
         return false;
   }
 
   // Toggle the optional operand to CPSR (if it exists - in Thumb1 we always
   // set CPSR so this is represented as an explicit output)
   if (!IsThumb1) {
-    MI->getOperand(5).setReg(ARM::CPSR);
-    MI->getOperand(5).setIsDef(true);
+    unsigned CPSRRegNum = MI->getNumExplicitOperands() - 1;
+    MI->getOperand(CPSRRegNum).setReg(ARM::CPSR);
+    MI->getOperand(CPSRRegNum).setIsDef(true);
   }
   assert(!isPredicated(*MI) && "Can't use flags from predicated instruction");
   CmpInstr.eraseFromParent();
 
   // Modify the condition code of operands in OperandsToUpdate.
   // Since we have SUB(r1, r2) and CMP(r2, r1), the condition code needs to
   // be changed from r2 > r1 to r1 < r2, from r2 < r1 to r1 > r2, etc.
   for (unsigned i = 0, e = OperandsToUpdate.size(); i < e; i++)

llvm/test/CodeGen/ARM/cmp-peephole.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=armv7a < %s | FileCheck %s --check-prefix=ARM
+; RUN: llc -mtriple=armv6m < %s | FileCheck %s --check-prefix=THUMB
+; RUN: llc -mtriple=armv7m < %s | FileCheck %s --check-prefix=THUMB2
+
+define i1 @cmp_ne_zero_and_rr(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_and_rr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    ands r0, r0, r1
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_rr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_rr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    ands r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %and = and i32 %a, %b
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_ri(i32 %a) {
+; ARM-LABEL: cmp_ne_zero_and_ri:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    ands r0, r0, #42
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_ri:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    movs r1, #42
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_ri:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    ands r0, r0, #42
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %and = and i32 %a, 42
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_rsr(i32 %a, i32 %b, i32 %c) {
+; ARM-LABEL: cmp_ne_zero_and_rsr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    ands r0, r0, r1, lsl r2
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_rsr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r2
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_rsr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    lsls r1, r2
+; THUMB2-NEXT:    ands r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, %c
+  %and = and i32 %sh, %a
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_rsi(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_and_rsi:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    ands r0, r0, r1, lsl #17
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_rsi:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r1, #17
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_rsi:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    ands.w r0, r0, r1, lsl #17
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, 17
+  %and = and i32 %sh, %a
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_or_rr(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_or_rr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    orrs r0, r0, r1
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_or_rr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    orrs r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_or_rr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    orrs r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %or = or i32 %a, %b
+  %res = icmp ne i32 %or, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_or_ri(i32 %a) {
+; ARM-LABEL: cmp_ne_zero_or_ri:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    orrs r0, r0, #42
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_or_ri:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    movs r1, #42
+; THUMB-NEXT:    orrs r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_or_ri:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    orrs r0, r0, #42
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %or = or i32 %a, 42
+  %res = icmp ne i32 %or, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_or_rsr(i32 %a, i32 %b, i32 %c) {
+; ARM-LABEL: cmp_ne_zero_or_rsr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    orrs r0, r0, r1, lsl r2
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_or_rsr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r2
+; THUMB-NEXT:    orrs r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_or_rsr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    lsls r1, r2
+; THUMB2-NEXT:    orrs r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, %c
+  %or = or i32 %sh, %a
+  %res = icmp ne i32 %or, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_or_rsi(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_or_rsi:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    orrs r0, r0, r1, lsl #17
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_or_rsi:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r1, #17
+; THUMB-NEXT:    orrs r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_or_rsi:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    orrs.w r0, r0, r1, lsl #17
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, 17
+  %or = or i32 %sh, %a
+  %res = icmp ne i32 %or, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_xor_rr(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_xor_rr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    eors r0, r0, r1
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_xor_rr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    eors r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_xor_rr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    eors r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %xor = xor i32 %a, %b
+  %res = icmp ne i32 %xor, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_xor_ri(i32 %a) {
+; ARM-LABEL: cmp_ne_zero_xor_ri:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    subs r0, r0, #42
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_xor_ri:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    subs r0, #42
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_xor_ri:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    subs r0, #42
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %xor = xor i32 %a, 42
+  %res = icmp ne i32 %xor, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_xor_rsr(i32 %a, i32 %b, i32 %c) {
+; ARM-LABEL: cmp_ne_zero_xor_rsr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    eors r0, r0, r1, lsl r2
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_xor_rsr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r2
+; THUMB-NEXT:    eors r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_xor_rsr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    lsls r1, r2
+; THUMB2-NEXT:    eors r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, %c
+  %xor = xor i32 %sh, %a
+  %res = icmp ne i32 %xor, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_xor_rsi(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_xor_rsi:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    eors r0, r0, r1, lsl #17
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_xor_rsi:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    lsls r1, r1, #17
+; THUMB-NEXT:    eors r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_xor_rsi:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    eors.w r0, r0, r1, lsl #17
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %sh = shl i32 %b, 17
+  %xor = xor i32 %sh, %a
+  %res = icmp ne i32 %xor, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_not_rr(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_and_not_rr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    bics r0, r0, r1
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_not_rr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    bics r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_not_rr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    bics r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %not = xor i32 %b, -1
+  %and = and i32 %a, %not
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_not_ri(i32 %a) {
+; ARM-LABEL: cmp_ne_zero_and_not_ri:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    bics r0, r0, #42
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_not_ri:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    movs r1, #42
+; THUMB-NEXT:    bics r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_not_ri:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    bics r0, r0, #42
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %not = xor i32 42, -1
+  %and = and i32 %a, %not
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_not_rsr(i32 %a, i32 %b, i32 %c) {
+; ARM-LABEL: cmp_ne_zero_and_not_rsr:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    mvn r1, r1
+; ARM-NEXT:    ands r0, r0, r1, lsl r2
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_not_rsr:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    mvns r1, r1
+; THUMB-NEXT:    lsls r1, r2
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_not_rsr:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    mvns r1, r1
+; THUMB2-NEXT:    lsls r1, r2
+; THUMB2-NEXT:    ands r0, r1
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %not = xor i32 %b, -1
+  %sh = shl i32 %not, %c
+  %and = and i32 %sh, %a
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
+
+define i1 @cmp_ne_zero_and_not_rsi(i32 %a, i32 %b) {
+; ARM-LABEL: cmp_ne_zero_and_not_rsi:
+; ARM:       @ %bb.0:
+; ARM-NEXT:    mvn r1, r1
+; ARM-NEXT:    ands r0, r0, r1, lsl #17
+; ARM-NEXT:    movwne r0, #1
+; ARM-NEXT:    bx lr
+;
+; THUMB-LABEL: cmp_ne_zero_and_not_rsi:
+; THUMB:       @ %bb.0:
+; THUMB-NEXT:    mvns r1, r1
+; THUMB-NEXT:    lsls r1, r1, #17
+; THUMB-NEXT:    ands r0, r1
+; THUMB-NEXT:    subs r1, r0, #1
+; THUMB-NEXT:    sbcs r0, r1
+; THUMB-NEXT:    bx lr
+;
+; THUMB2-LABEL: cmp_ne_zero_and_not_rsi:
+; THUMB2:       @ %bb.0:
+; THUMB2-NEXT:    mvns r1, r1
+; THUMB2-NEXT:    ands.w r0, r0, r1, lsl #17
+; THUMB2-NEXT:    it ne
+; THUMB2-NEXT:    movne r0, #1
+; THUMB2-NEXT:    bx lr
+  %not = xor i32 %b, -1
+  %sh = shl i32 %not, 17
+  %and = and i32 %sh, %a
+  %res = icmp ne i32 %and, 0
+  ret i1 %res
+}
 No newline at end of file

llvm/test/CodeGen/ARM/consthoist-icmpimm.ll

 ; CHECKV7M-NEXT:    .save {r7, lr}
 ; CHECKV7M-NEXT:    push {r7, lr}
 ; CHECKV7M-NEXT:    mov r12, r0
 ; CHECKV7M-NEXT:    ldr r0, [sp, #16]
 ; CHECKV7M-NEXT:    lsls r0, r0, #31
 ; CHECKV7M-NEXT:    ldrd lr, r0, [sp, #8]
 ; CHECKV7M-NEXT:    beq .LBB6_2
 ; CHECKV7M-NEXT:  @ %bb.1: @ %then
-; CHECKV7M-NEXT:    orr.w r2, r3, r2, lsr #17
-; CHECKV7M-NEXT:    orr.w r1, r1, r12, lsr #17
-; CHECKV7M-NEXT:    cmp r2, #0
+; CHECKV7M-NEXT:    orrs.w r2, r3, r2, lsr #17
 ; CHECKV7M-NEXT:    mov r2, r0
 ; CHECKV7M-NEXT:    it ne
 ; CHECKV7M-NEXT:    movne r2, lr
-; CHECKV7M-NEXT:    cmp r1, #0
+; CHECKV7M-NEXT:    orrs.w r1, r1, r12, lsr #17
 ; CHECKV7M-NEXT:    it ne
 ; CHECKV7M-NEXT:    movne r0, lr
 ; CHECKV7M-NEXT:    add r0, r2
 ; CHECKV7M-NEXT:    pop {r7, pc}
 ; CHECKV7M-NEXT:  .LBB6_2: @ %else
-; CHECKV7M-NEXT:    orr.w r1, r3, r2, lsr #17
-; CHECKV7M-NEXT:    cmp r1, #0
+; CHECKV7M-NEXT:    orrs.w r1, r3, r2, lsr #17
 ; CHECKV7M-NEXT:    it ne
 ; CHECKV7M-NEXT:    movne r0, lr
 ; CHECKV7M-NEXT:    pop {r7, pc}
 ;
 ; CHECKV7A-LABEL: icmp64_uge_m2:
 ; CHECKV7A:       @ %bb.0:
 ; CHECKV7A-NEXT:    .save {r4, lr}
 ; CHECKV7A-NEXT:    push {r4, lr}
 ; CHECKV7A-NEXT:    ldr r4, [sp, #16]
 ; CHECKV7A-NEXT:    mov r12, r0
 ; CHECKV7A-NEXT:    ldrd lr, r0, [sp, #8]
 ; CHECKV7A-NEXT:    lsls r4, r4, #31
 ; CHECKV7A-NEXT:    beq .LBB6_2
 ; CHECKV7A-NEXT:  @ %bb.1: @ %then
-; CHECKV7A-NEXT:    orr.w r2, r3, r2, lsr #17
-; CHECKV7A-NEXT:    orr.w r1, r1, r12, lsr #17
-; CHECKV7A-NEXT:    cmp r2, #0
+; CHECKV7A-NEXT:    orrs.w r2, r3, r2, lsr #17
 ; CHECKV7A-NEXT:    mov r2, r0
 ; CHECKV7A-NEXT:    it ne
 ; CHECKV7A-NEXT:    movne r2, lr
-; CHECKV7A-NEXT:    cmp r1, #0
+; CHECKV7A-NEXT:    orrs.w r1, r1, r12, lsr #17
 ; CHECKV7A-NEXT:    it ne
 ; CHECKV7A-NEXT:    movne r0, lr
 ; CHECKV7A-NEXT:    add r0, r2
 ; CHECKV7A-NEXT:    pop {r4, pc}
 ; CHECKV7A-NEXT:  .LBB6_2: @ %else
-; CHECKV7A-NEXT:    orr.w r1, r3, r2, lsr #17
-; CHECKV7A-NEXT:    cmp r1, #0
+; CHECKV7A-NEXT:    orrs.w r1, r3, r2, lsr #17
 ; CHECKV7A-NEXT:    it ne
 ; CHECKV7A-NEXT:    movne r0, lr
 ; CHECKV7A-NEXT:    pop {r4, pc}
   br i1 %c, label %then, label %else
 then:
   %c1 = icmp uge i64 %x, 131072
   %c2 = icmp uge i64 %y, 131072
   %s1 = select i1 %c1, i32 %a, i32 %b

llvm/test/CodeGen/ARM/hoist-and-by-const-from-lshr-in-eqcmp-zero.ll

 ; A few negative tests
 ;------------------------------------------------------------------------------;
 
 define i1 @negative_scalar_i8_bitsinmiddle_slt(i8 %x, i8 %y) nounwind {
 ; ARM6-LABEL: negative_scalar_i8_bitsinmiddle_slt:
 ; ARM6:       @ %bb.0:
 ; ARM6-NEXT:    uxtb r1, r1
 ; ARM6-NEXT:    mov r2, #24
-; ARM6-NEXT:    and r1, r0, r2, lsr r1
+; ARM6-NEXT:    ands r0, r0, r2, lsr r1
 ; ARM6-NEXT:    mov r0, #0
-; ARM6-NEXT:    cmp r1, #0
 ; ARM6-NEXT:    movmi r0, #1
 ; ARM6-NEXT:    bx lr
 ;
 ; ARM78-LABEL: negative_scalar_i8_bitsinmiddle_slt:
 ; ARM78:       @ %bb.0:
 ; ARM78-NEXT:    uxtb r1, r1
 ; ARM78-NEXT:    mov r2, #24
-; ARM78-NEXT:    and r1, r0, r2, lsr r1
+; ARM78-NEXT:    ands r0, r0, r2, lsr r1
 ; ARM78-NEXT:    mov r0, #0
-; ARM78-NEXT:    cmp r1, #0
 ; ARM78-NEXT:    movwmi r0, #1
 ; ARM78-NEXT:    bx lr
 ;
 ; THUMB6-LABEL: negative_scalar_i8_bitsinmiddle_slt:
 ; THUMB6:       @ %bb.0:
 ; THUMB6-NEXT:    uxtb r1, r1
 ; THUMB6-NEXT:    movs r2, #24
 ; THUMB6-NEXT:    lsrs r2, r1

llvm/test/CodeGen/ARM/icmp-shift-opt.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=armv7 %s -o - | FileCheck %s
 
 ; Optimize expanded SRL/SHL used as an input of
 ; SETCC comparing it with zero by removing rotation.
 ;
 ; See https://bugs.llvm.org/show_bug.cgi?id=50197
 define i64 @opt_setcc_lt_power_of_2(i64 %a) nounwind {
 ; CHECK-LABEL: opt_setcc_lt_power_of_2:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:  .LBB0_1: @ %loop
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    adds r0, r0, #1
 ; CHECK-NEXT:    adc r1, r1, #0
-; CHECK-NEXT:    orr r2, r1, r0, lsr #16
-; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    orrs r2, r1, r0, lsr #16
 ; CHECK-NEXT:    bne .LBB0_1
 ; CHECK-NEXT:  @ %bb.2: @ %exit
 ; CHECK-NEXT:    bx lr
   br label %loop
 
 loop:
   %phi.a = phi i64 [ %a, %0 ], [ %inc, %loop ]
   %inc = add i64 %phi.a, 1
; CHECK-NEXT:    bx lr
    %srl = lshr i64 %a, 17
    %cmp = icmp eq i64 %srl, 0
    ret i1 %cmp
 }
 
 define i1 @opt_setcc_srl_ne_zero(i64 %a) nounwind {
 ; CHECK-LABEL: opt_setcc_srl_ne_zero:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    orr r0, r1, r0, lsr #17
-; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    orrs r0, r1, r0, lsr #17
 ; CHECK-NEXT:    movwne r0, #1
 ; CHECK-NEXT:    bx lr
    %srl = lshr i64 %a, 17
    %cmp = icmp ne i64 %srl, 0
    ret i1 %cmp
 }
 
 define i1 @opt_setcc_shl_eq_zero(i64 %a) nounwind {
 ; CHECK-LABEL: opt_setcc_shl_eq_zero:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    orr r0, r0, r1, lsl #17
 ; CHECK-NEXT:    clz r0, r0
 ; CHECK-NEXT:    lsr r0, r0, #5
 ; CHECK-NEXT:    bx lr
    %shl = shl i64 %a, 17
    %cmp = icmp eq i64 %shl, 0
    ret i1 %cmp
 }
 
 define i1 @opt_setcc_shl_ne_zero(i64 %a) nounwind {
 ; CHECK-LABEL: opt_setcc_shl_ne_zero:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    orr r0, r0, r1, lsl #17
-; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    orrs r0, r0, r1, lsl #17
 ; CHECK-NEXT:    movwne r0, #1
 ; CHECK-NEXT:    bx lr
    %shl = shl i64 %a, 17
    %cmp = icmp ne i64 %shl, 0
    ret i1 %cmp
 }
 
 ; Negative test: optimization should not be applied if shift has multiple users.

 define i1 @opt_setcc_shl_ne_zero_i128(i128 %a) nounwind {
 ; CHECK-LABEL: opt_setcc_shl_ne_zero_i128:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    orr r3, r1, r3
 ; CHECK-NEXT:    orr r0, r2, r0
 ; CHECK-NEXT:    orr r2, r0, r3
 ; CHECK-NEXT:    orr r0, r0, r1
 ; CHECK-NEXT:    lsr r0, r0, #15
-; CHECK-NEXT:    orr r0, r0, r2, lsl #17
-; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    orrs r0, r0, r2, lsl #17
 ; CHECK-NEXT:    movwne r0, #1
 ; CHECK-NEXT:    bx lr
   %shl = shl i128 %a, 17
   %cmp = icmp ne i128 %shl, 0
   ret i1 %cmp
 }
 
 declare void @use(i64 %a)

llvm/test/CodeGen/ARM/sadd_sat.ll

 ; CHECK-T1-NEXT:    pop {r4, pc}
 ;
 ; CHECK-T2-LABEL: func2:
 ; CHECK-T2:       @ %bb.0:
 ; CHECK-T2-NEXT:    adds r0, r0, r2
 ; CHECK-T2-NEXT:    eor.w r12, r1, r3
 ; CHECK-T2-NEXT:    adc.w r2, r1, r3
 ; CHECK-T2-NEXT:    eors r1, r2
-; CHECK-T2-NEXT:    bic.w r1, r1, r12
-; CHECK-T2-NEXT:    cmp r1, #0
-; CHECK-T2-NEXT:    mov.w r1, #-2147483648
+; CHECK-T2-NEXT:    bics.w r1, r1, r12
 ; CHECK-T2-NEXT:    it mi
 ; CHECK-T2-NEXT:    asrmi r0, r2, #31
+; CHECK-T2-NEXT:    mov.w r1, #-2147483648
 ; CHECK-T2-NEXT:    it mi
 ; CHECK-T2-NEXT:    eormi.w r2, r1, r2, asr #31
 ; CHECK-T2-NEXT:    mov r1, r2
 ; CHECK-T2-NEXT:    bx lr
 ;
 ; CHECK-ARM-LABEL: func2:
 ; CHECK-ARM:       @ %bb.0:
 ; CHECK-ARM-NEXT:    adds r0, r0, r2
 ; CHECK-ARM-NEXT:    eor r12, r1, r3
 ; CHECK-ARM-NEXT:    adc r2, r1, r3
 ; CHECK-ARM-NEXT:    eor r1, r1, r2
-; CHECK-ARM-NEXT:    bic r1, r1, r12
-; CHECK-ARM-NEXT:    cmp r1, #0
-; CHECK-ARM-NEXT:    mov r1, #-2147483648
+; CHECK-ARM-NEXT:    bics r1, r1, r12
 ; CHECK-ARM-NEXT:    asrmi r0, r2, #31
+; CHECK-ARM-NEXT:    mov r1, #-2147483648
 ; CHECK-ARM-NEXT:    eormi r2, r1, r2, asr #31
 ; CHECK-ARM-NEXT:    mov r1, r2
 ; CHECK-ARM-NEXT:    bx lr
   %tmp = call i64 @llvm.sadd.sat.i64(i64 %x, i64 %y)
   ret i64 %tmp
 }
 
 define signext i16 @func16(i16 signext %x, i16 signext %y) nounwind {

llvm/test/CodeGen/ARM/sadd_sat_plus.ll

 ; CHECK-T2:       @ %bb.0:
 ; CHECK-T2-NEXT:    ldr r2, [sp]
 ; CHECK-T2-NEXT:    ldr.w r12, [sp, #4]
 ; CHECK-T2-NEXT:    adds r0, r0, r2
 ; CHECK-T2-NEXT:    adc.w r2, r1, r12
 ; CHECK-T2-NEXT:    eor.w r3, r1, r12
 ; CHECK-T2-NEXT:    eors r1, r2
 ; CHECK-T2-NEXT:    bics r1, r3
-; CHECK-T2-NEXT:    cmp r1, #0
-; CHECK-T2-NEXT:    mov.w r1, #-2147483648
 ; CHECK-T2-NEXT:    it mi
 ; CHECK-T2-NEXT:    asrmi r0, r2, #31
+; CHECK-T2-NEXT:    mov.w r1, #-2147483648
 ; CHECK-T2-NEXT:    it mi
 ; CHECK-T2-NEXT:    eormi.w r2, r1, r2, asr #31
 ; CHECK-T2-NEXT:    mov r1, r2
 ; CHECK-T2-NEXT:    bx lr
 ;
 ; CHECK-ARM-LABEL: func64:
 ; CHECK-ARM:       @ %bb.0:
 ; CHECK-ARM-NEXT:    ldr r12, [sp]
 ; CHECK-ARM-NEXT:    ldr r2, [sp, #4]
 ; CHECK-ARM-NEXT:    adds r0, r0, r12
 ; CHECK-ARM-NEXT:    eor r3, r1, r2
 ; CHECK-ARM-NEXT:    adc r2, r1, r2
 ; CHECK-ARM-NEXT:    eor r1, r1, r2
-; CHECK-ARM-NEXT:    bic r1, r1, r3
-; CHECK-ARM-NEXT:    cmp r1, #0
-; CHECK-ARM-NEXT:    mov r1, #-2147483648
+; CHECK-ARM-NEXT:    bics r1, r1, r3
 ; CHECK-ARM-NEXT:    asrmi r0, r2, #31
+; CHECK-ARM-NEXT:    mov r1, #-2147483648
 ; CHECK-ARM-NEXT:    eormi r2, r1, r2, asr #31
 ; CHECK-ARM-NEXT:    mov r1, r2
 ; CHECK-ARM-NEXT:    bx lr
   %a = mul i64 %y, %z
   %tmp = call i64 @llvm.sadd.sat.i64(i64 %x, i64 %z)
   ret i64 %tmp
 }
 

llvm/test/CodeGen/Thumb2/LowOverheadLoops/arm_cmplx_dot_prod_f32.ll

 ; CHECK-NEXT:    vcmla.f32 q0, q4, q3, #90
 ; CHECK-NEXT:    le lr, .LBB0_2
 ; CHECK-NEXT:  @ %bb.3: @ %while.end
 ; CHECK-NEXT:    vcmla.f32 q0, q1, q2, #0
 ; CHECK-NEXT:    movs r4, #6
 ; CHECK-NEXT:    vcmla.f32 q0, q1, q2, #90
 ; CHECK-NEXT:    vldrw.u32 q1, [r1, #-16]
 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #-16]
-; CHECK-NEXT:    and.w r2, r4, r2, lsl #1
+; CHECK-NEXT:    ands.w r2, r4, r2, lsl #1
 ; CHECK-NEXT:    vcmla.f32 q0, q2, q1, #0
 ; CHECK-NEXT:    vcmla.f32 q0, q2, q1, #90
-; CHECK-NEXT:    cbz r2, .LBB0_8
+; CHECK-NEXT:    beq .LBB0_8
 ; CHECK-NEXT:  @ %bb.4: @ %while.body9
 ; CHECK-NEXT:    vctp.32 r2
 ; CHECK-NEXT:    cmp r2, #4
 ; CHECK-NEXT:    vpstttt
 ; CHECK-NEXT:    vldrwt.u32 q1, [r1]
 ; CHECK-NEXT:    vldrwt.u32 q2, [r0]
 ; CHECK-NEXT:    vcmlat.f32 q0, q2, q1, #0
 ; CHECK-NEXT:    vcmlat.f32 q0, q2, q1, #90

llvm/test/CodeGen/Thumb2/mve-laneinterleaving.ll

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    vmov.f32 s8, s2
 ; CHECK-NEXT:    vmov.f32 s2, s3
 ; CHECK-NEXT:    vmov.f32 s10, s7
 ; CHECK-NEXT:    vmov r10, s8
 ; CHECK-NEXT:    vmov.f32 s8, s6
-; CHECK-NEXT:    vmov r6, s2
+; CHECK-NEXT:    vmov r7, s2
 ; CHECK-NEXT:    vmov.f32 s2, s1
 ; CHECK-NEXT:    vmov.f32 s6, s5
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    asr.w r0, r10, #31
-; CHECK-NEXT:    asrs r7, r6, #31
+; CHECK-NEXT:    asrs r5, r7, #31
 ; CHECK-NEXT:    adds.w r4, r10, r2
+; CHECK-NEXT:    eor.w r6, r10, r2
 ; CHECK-NEXT:    adc r3, r0, #0
 ; CHECK-NEXT:    asrl r4, r3, r2
 ; CHECK-NEXT:    subs r0, r4, r2
 ; CHECK-NEXT:    sbc lr, r3, #0
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    umull r0, r8, r0, r2
-; CHECK-NEXT:    adds r4, r6, r3
-; CHECK-NEXT:    eor.w r1, r6, r3
-; CHECK-NEXT:    adc r5, r7, #0
-; CHECK-NEXT:    eor.w r7, r10, r2
+; CHECK-NEXT:    adds r4, r7, r3
+; CHECK-NEXT:    eor.w r1, r7, r3
+; CHECK-NEXT:    adc r5, r5, #0
 ; CHECK-NEXT:    asrl r4, r5, r3
-; CHECK-NEXT:    orr.w r7, r7, r10, asr #31
 ; CHECK-NEXT:    subs r4, r4, r3
-; CHECK-NEXT:    orr.w r1, r1, r6, asr #31
 ; CHECK-NEXT:    sbc r5, r5, #0
-; CHECK-NEXT:    cmp r7, #0
+; CHECK-NEXT:    orrs.w r6, r6, r10, asr #31
 ; CHECK-NEXT:    umull r4, r12, r4, r3
 ; CHECK-NEXT:    csetm r9, eq
-; CHECK-NEXT:    movs r7, #0
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    bfi r7, r9, #0, #8
+; CHECK-NEXT:    orrs.w r1, r1, r7, asr #31
+; CHECK-NEXT:    mov.w r6, #0
 ; CHECK-NEXT:    csetm r1, eq
-; CHECK-NEXT:    bfi r7, r1, #8, #8
+; CHECK-NEXT:    bfi r6, r9, #0, #8
 ; CHECK-NEXT:    mla r5, r5, r3, r12
-; CHECK-NEXT:    rsbs r1, r6, #0
-; CHECK-NEXT:    vmsr p0, r7
+; CHECK-NEXT:    bfi r6, r1, #8, #8
+; CHECK-NEXT:    rsbs r1, r7, #0
 ; CHECK-NEXT:    mla r7, lr, r2, r8
 ; CHECK-NEXT:    lsll r4, r5, r1
 ; CHECK-NEXT:    rsb.w r1, r10, #0
 ; CHECK-NEXT:    lsll r0, r7, r1
 ; CHECK-NEXT:    vmov lr, s2
 ; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    lsll r0, r7, r2
 ; CHECK-NEXT:    lsll r4, r5, r3
-; CHECK-NEXT:    mov.w r12, #0
+; CHECK-NEXT:    vmsr p0, r6
 ; CHECK-NEXT:    vmov q3[2], q3[0], r0, r4
+; CHECK-NEXT:    mov.w r12, #0
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    adds.w r2, lr, r1
 ; CHECK-NEXT:    asr.w r0, lr, #31
 ; CHECK-NEXT:    adc r3, r0, #0
 ; CHECK-NEXT:    asrl r2, r3, r1
 ; CHECK-NEXT:    subs r0, r2, r1
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    sbc r7, r3, #0
 ; CHECK-NEXT:    vmov r3, s4
 ; CHECK-NEXT:    umull r0, r6, r0, r1
 ; CHECK-NEXT:    asrs r5, r2, #31
 ; CHECK-NEXT:    adds r4, r2, r3
 ; CHECK-NEXT:    adc r5, r5, #0
 ; CHECK-NEXT:    asrl r4, r5, r3
 ; CHECK-NEXT:    subs r4, r4, r3
 ; CHECK-NEXT:    sbc r8, r5, #0
 ; CHECK-NEXT:    mla r5, r7, r1, r6
 ; CHECK-NEXT:    eor.w r6, lr, r1
-; CHECK-NEXT:    orr.w r6, r6, lr, asr #31
+; CHECK-NEXT:    orrs.w r6, r6, lr, asr #31
 ; CHECK-NEXT:    eor.w r7, r2, r3
-; CHECK-NEXT:    cmp r6, #0
-; CHECK-NEXT:    orr.w r7, r7, r2, asr #31
 ; CHECK-NEXT:    csetm r6, eq
-; CHECK-NEXT:    cmp r7, #0
+; CHECK-NEXT:    orrs.w r7, r7, r2, asr #31
 ; CHECK-NEXT:    csetm r7, eq
 ; CHECK-NEXT:    rsb.w lr, lr, #0
 ; CHECK-NEXT:    bfi r12, r7, #0, #8
 ; CHECK-NEXT:    lsll r0, r5, lr
 ; CHECK-NEXT:    bfi r12, r6, #8, #8
 ; CHECK-NEXT:    umull r4, r6, r4, r3
 ; CHECK-NEXT:    lsll r0, r5, r1
 ; CHECK-NEXT:    rsbs r1, r2, #0