[X86] Canonicalize SGT/UGT compares with constants to use SGE/UGE to reduce the number of EFLAGs reads. (PR48760)

[RKSimon]

Summary

This demonstrates a possible fix for PR48760 - for compares with constants, canonicalize the SGT/UGT condition code to use SGE/UGE which should reduce the number of EFLAGs bits we need to read.

As discussed on PR48760, some EFLAG bits are treated independently which can require additional uops to merge together for certain CMOVcc/SETcc/etc. modes.

I've limited this to cases where the constant increment doesn't result in a larger encoding or additional i64 constant materializations.

[RKSimon]

Address the atomic flags regressions by implementing a similar CC tweak in combineSetCCAtomicArith - I'm happy to pull the atomic change out into another patch although not all paths would be tested until this patch had landed.

[RKSimon]

rebase

[RKSimon]

ping?

[lebedev.ri]

Some thoughts.

Some min/max-like patterns lose out because the comparison constant is now different from the select value - do we have any opinions on how critical this is?

This does seem not ideal. Can we even check that the cmp has a select with such an operand by now?

llvm/lib/Target/X86/X86ISelLowering.cpp

23462–23463

Why do we handle inverse case of decrementing in combineSetCCAtomicArith() but not here?

23465–23470

This comment reads as-if it's about the previous condition.
I'd suggest adding something like "note that we can't change GT to GE for tautological comparisons,
where incrementing the limit would cause an overflow" before if (!Op1Val.isNullValue() &&

llvm/test/CodeGen/X86/sdiv_fix_sat.ll

474–480

This is the clamp regression i guess?

631–632

same elesewhere in the file

[craig.topper]

In D101074#2722222, @lebedev.ri wrote:

Some thoughts.

Some min/max-like patterns lose out because the comparison constant is now different from the select value - do we have any opinions on how critical this is?

This does seem not ideal. Can we even check that the cmp has a select with such an operand by now?

Can we fix that in combineSelect where we canonicalize min/max patterns to use sign flag?

[RKSimon]

Add SETLT/SETULT handling

[lebedev.ri]

In D101074#2819404, @RKSimon wrote:

Add SETLT/SETULT handling

Thanks.
I think this looks ok to me.
What about the regressions?

llvm/lib/Target/X86/X86ISelLowering.cpp

42074

APInt::isMinValue()?

[pengfei]

llvm/test/CodeGen/X86/lack-of-signed-truncation-check.ll

601–602

Is it possible that we happen to exceed IMM16?

[RKSimon]

Use APInt::isMinValue() for consistency

Added fold for select(seteq(X,Y),A,select(setgt(X,Y), A, B)) -> select(setge(X,Y), A, B) folds for SETCC after type legalization - this could be added separately if required.

This just leaves a few regressions where we create 2 64-bit constants instead of 1

[RKSimon]

llvm/test/CodeGen/X86/lack-of-signed-truncation-check.ll

601–602

No - we check for min/max values before decrementing/incrementing to ensure we don't wrap the value.

[RKSimon]

llvm/test/CodeGen/X86/ctpop-combine.ll

45 ↗

(On Diff #352693)

Regression - we go from a CF read to CF+ZF reads - it looks in most cases the LT->LE conversions are going to cause this - I think I'm going to just handle the GT->GE cases initially.

[RKSimon]

The equivalent less-than fold to reduce EFLAGS dependency is SLE/ULE -> SLT/ULT - which we don't typically need to do because TargetLowering.SimplifySetCC already canonicalizes to that - sorry I was on automatic when was asked to support SLT/ULT (and it'd been far too long since I looked at the patch...)

[pengfei]

llvm/test/CodeGen/X86/lack-of-signed-truncation-check.ll

601–602

I just saw you check int8 and int32. But I cannot create a case for the int16 boundary value due to its promoted to int32.

[RKSimon]

llvm/test/CodeGen/X86/lack-of-signed-truncation-check.ll

601–602

Yes i8 and i32 immediates are special cases because the width of the immediate might not match the width of the operand type - but if we're using an i16 immediate on i32/i64 it will always be extended to i32 immediate.

[RKSimon]

Remove the unnecessary SLT/ULT handling + avoid the remaining i64 immediate comparison regressions.

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

41891

This can be pushed first which should give us most of the sdiv_fix_sat.ll + udiv_fix_sat.ll codegen improvements

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

23472

Should this be inside emitFlagsForSetcc? That way setcc+brcond works. LowerBRCOND doesn't call LowerSETCC.

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

23472

Yes, I was hoping to move it inside TranslateX86CC later (see TODO) - as it felt like this caused far too many diffs as an initial patch - I'll can try again now that the SLT/ULT handling has been dropped.

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

23472

Yes there's a huge amount of test case churn that I'd prefer to do as a follow up patch for review - and just stick with the TODO while we handle the select/cmov cases first.

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

23472

Ok.

[RKSimon]

Does anyone have any more comments?

[lebedev.ri]

What about the case where the active bit width decreases, e.g. -129 is i9, incremented it is -128, which is i8.

llvm/test/CodeGen/X86/sdiv_fix_sat.ll

474–480

And it's still here. Do we believe that the addition materialization cost is hidden
by the cmp improvement?
Can't we not do this if the SETCC is used by a SELECT with one hand matching the unchanged immediate?

[RKSimon]

rebase and permit reduction from i32 to i8 immediate encoding

[lebedev.ri]

llvm/lib/Target/X86/X86ISelLowering.cpp

23476

Comment maybe needs updatind.

23486–23487

I've spent way too much time trying to understand this block,
which means this is too compilcated and may or may not be incorrect.
I would suggest something like

if(Op1ValPlusOne.isSignedIntN(32) && (!Op1Val.isSignedIntN(8) || Op1ValPlusOne.isSignedIntN(8))

(isSignedIntN(N+1) guarantees that isSignedIntN(N))

[lebedev.ri]

llvm/lib/Target/X86/X86ISelLowering.cpp

23481

Actually, what about: -2147483649 is i33, incremented it is -2147483648, i32.

[RKSimon]

Cleaned up the isSignedIntN logic

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

23481

Sorry - missed this when I was getting back to speed - I'll add a test

[RKSimon]

rebase with the sgt i33 -> sge i32 test

[lebedev.ri]

LGTM, thank you, sorry for so many back and forth here.
Any other comments?

llvm/lib/Target/X86/X86ISelLowering.cpp

23476

Still needs updating

[spatel]

llvm/lib/Target/X86/X86ISelLowering.cpp

23473

I didn't make the connection from SGT -> SGE to SETG -> SETGE to ZF/OF/SF or SETA -> SETAE to ZF/CF and then to an actual perf difference until re-reading the comments in PR48760 and opening the x86 manual...and I'm still not entirely clear about it. :) This deserves more explanation in the code comment.

Maybe something like:
The "GE" conditions map to less EFLAGS bits than their "GT" counterparts. Specifically, the GE conditions don't read the ZF. This may translate to less uops depending on uarch implementation.

llvm/lib/Target/X86/X86ISelLowering.cpp

SDValue X86TargetLowering::LowerSETCC(SDValue Op, SelectionDAG &DAG) const {
 
   // Handle f128 first, since one possible outcome is a normal integer
   // comparison which gets handled by emitFlagsForSetcc.
   if (Op0.getValueType() == MVT::f128) {
     softenSetCCOperands(DAG, MVT::f128, Op0, Op1, CC, dl, Op0, Op1, Chain,
                         Op.getOpcode() == ISD::STRICT_FSETCCS);
 
     // If softenSetCCOperands returned a scalar, use it.
     if (!Op1.getNode()) {
       assert(Op0.getValueType() == Op.getValueType() &&
              "Unexpected setcc expansion!");
       if (IsStrict)
         return DAG.getMergeValues({Op0, Chain}, dl);
       return Op0;
     }
   }
 
   if (Op0.getSimpleValueType().isInteger()) {
+    // Attempt to canonicalize SGT/UGT -> SGE/UGE compares with constant which
+    // reduces the number of EFLAGs bit reads (the GE conditions don't read ZF),
+    // this may translate to less uops depending on uarch implementation. The
+    // equivalent for SLE/ULE -> SLT/ULT isn't likely to happen as we already
+    // canonicalize to that CondCode.
+    // NOTE: Only do this if incrementing the constant doesn't increase the bit
+    // encoding size - so it must either already be a i8 or i32 immediate, or it
+    // shrinks down to that. We don't do this for any i64's to avoid additional
+    // constant materializations.
+    // TODO: Can we move this to TranslateX86CC to handle jumps/branches too?
+    if (auto *Op1C = dyn_cast<ConstantSDNode>(Op1)) {
+      const APInt &Op1Val = Op1C->getAPIntValue();
+      if (!Op1Val.isNullValue()) {
+        // Ensure the constant+1 doesn't overflow.
+        if ((CC == ISD::CondCode::SETGT && !Op1Val.isMaxSignedValue()) ||
+            (CC == ISD::CondCode::SETUGT && !Op1Val.isMaxValue())) {
+          APInt Op1ValPlusOne = Op1Val + 1;
+          if (Op1ValPlusOne.isSignedIntN(32) &&
+              (!Op1Val.isSignedIntN(8) || Op1ValPlusOne.isSignedIntN(8))) {
+            Op1 = DAG.getConstant(Op1ValPlusOne, dl, Op0.getValueType());
+            CC = CC == ISD::CondCode::SETGT ? ISD::CondCode::SETGE
+                                            : ISD::CondCode::SETUGE;
+          }
+        }
+      }
+    }
+
     SDValue X86CC;
     SDValue EFLAGS = emitFlagsForSetcc(Op0, Op1, CC, dl, DAG, X86CC);
     SDValue Res = DAG.getNode(X86ISD::SETCC, dl, MVT::i8, X86CC, EFLAGS);
     return IsStrict ? DAG.getMergeValues({Res, Chain}, dl) : Res;
   }
 
   // Handle floating point.
   X86::CondCode CondCode = TranslateX86CC(CC, dl, /*IsFP*/ true, Op0, Op1, DAG);
if (RHS.getOpcode() == ISD::SELECT && RHS.getOperand(1) == LHS &&
           Cond1 == InnerSetCC.getOperand(1)) {
         ISD::CondCode NewCC;
         switch (CC == ISD::SETEQ ? InnerCC : CC) {
         case ISD::SETGT:  NewCC = ISD::SETGE; break;
         case ISD::SETLT:  NewCC = ISD::SETLE; break;
         case ISD::SETUGT: NewCC = ISD::SETUGE; break;
         case ISD::SETULT: NewCC = ISD::SETULE; break;
         default: NewCC = ISD::SETCC_INVALID; break;
         }
         if (NewCC != ISD::SETCC_INVALID) {
           Cond = DAG.getSetCC(DL, CondVT, Cond0, Cond1, NewCC);
           return DAG.getSelect(DL, VT, Cond, LHS, RHS.getOperand(2));
         }
       }
     }
   }
 
if (Opc != ISD::ATOMIC_LOAD_ADD && Opc != ISD::ATOMIC_LOAD_SUB)
     return SDValue();
 
   SDValue OpRHS = CmpLHS.getOperand(2);
   auto *OpRHSC = dyn_cast<ConstantSDNode>(OpRHS);
   if (!OpRHSC)
     return SDValue();
 
   APInt Addend = OpRHSC->getAPIntValue();
   if (Opc == ISD::ATOMIC_LOAD_SUB)
     Addend = -Addend;
 
   auto *CmpRHSC = dyn_cast<ConstantSDNode>(CmpRHS);
   if (!CmpRHSC)
     return SDValue();
 
   APInt Comparison = CmpRHSC->getAPIntValue();
   APInt NegAddend = -Addend;
 
+  // See if we can adjust the CC to make the comparison match the negated
+  // addend.
+  if (Comparison != NegAddend) {
+    APInt IncComparison = Comparison + 1;
+    if (IncComparison == NegAddend) {
+      if (CC == X86::COND_A && !Comparison.isMaxValue()) {
+        Comparison = IncComparison;
+        CC = X86::COND_AE;
+      } else if (CC == X86::COND_LE && !Comparison.isMaxSignedValue()) {
+        Comparison = IncComparison;
+        CC = X86::COND_L;
+      }
+    }
+    APInt DecComparison = Comparison - 1;
+    if (DecComparison == NegAddend) {
+      if (CC == X86::COND_AE && !Comparison.isMinValue()) {
+        Comparison = DecComparison;
+        CC = X86::COND_A;
+      } else if (CC == X86::COND_L && !Comparison.isMinSignedValue()) {
+        Comparison = DecComparison;
+        CC = X86::COND_LE;
+      }
+    }
+  }
+
   // If the addend is the negation of the comparison value, then we can do
   // a full comparison by emitting the atomic arithmetic as a locked sub.
   if (Comparison == NegAddend) {
     // The CC is fine, but we need to rewrite the LHS of the comparison as an
     // atomic sub.
     auto *AN = cast<AtomicSDNode>(CmpLHS.getNode());
     auto AtomicSub = DAG.getAtomic(
         ISD::ATOMIC_LOAD_SUB, SDLoc(CmpLHS), CmpVT,

llvm/test/CodeGen/X86/2008-09-11-CoalescerBug2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686--
 ; RUN: llc -pre-RA-sched=source < %s -mtriple=i686-unknown-linux -mcpu=corei7 | FileCheck %s --check-prefix=SOURCE-SCHED
 ; PR2748
 
 @g_73 = external dso_local global i32
 @g_5 = external dso_local global i32
 
 define i32 @func_44(i16 signext %p_46) nounwind {
 ; SOURCE-SCHED-LABEL: func_44:
 ; SOURCE-SCHED:       # %bb.0: # %entry
 ; SOURCE-SCHED-NEXT:    subl $12, %esp
 ; SOURCE-SCHED-NEXT:    movl g_5, %eax
 ; SOURCE-SCHED-NEXT:    sarl %eax
 ; SOURCE-SCHED-NEXT:    xorl %ecx, %ecx
-; SOURCE-SCHED-NEXT:    cmpl $1, %eax
-; SOURCE-SCHED-NEXT:    setg %cl
+; SOURCE-SCHED-NEXT:    cmpl $2, %eax
+; SOURCE-SCHED-NEXT:    setge %cl
 ; SOURCE-SCHED-NEXT:    movb g_73, %dl
 ; SOURCE-SCHED-NEXT:    xorl %eax, %eax
 ; SOURCE-SCHED-NEXT:    subb {{[0-9]+}}(%esp), %al
 ; SOURCE-SCHED-NEXT:    testb %dl, %dl
 ; SOURCE-SCHED-NEXT:    jne .LBB0_2
 ; SOURCE-SCHED-NEXT:  # %bb.1: # %bb11
 ; SOURCE-SCHED-NEXT:    movzbl %al, %eax
 ; SOURCE-SCHED-NEXT:    divb %dl

llvm/test/CodeGen/X86/atomic-eflags-reuse.ll

 ; FASTINCDEC-NEXT:    movl %esi, %eax
 ; FASTINCDEC-NEXT:    lock decq (%rdi)
 ; FASTINCDEC-NEXT:    cmovgel %edx, %eax
 ; FASTINCDEC-NEXT:    retq
 ;
 ; SLOWINCDEC-LABEL: test_sub_1_cmov_sle:
 ; SLOWINCDEC:       # %bb.0: # %entry
 ; SLOWINCDEC-NEXT:    movl %esi, %eax
-; SLOWINCDEC-NEXT:    lock addq $-1, (%rdi)
+; SLOWINCDEC-NEXT:    lock subq $1, (%rdi)
 ; SLOWINCDEC-NEXT:    cmovgel %edx, %eax
 ; SLOWINCDEC-NEXT:    retq
 entry:
   %tmp0 = atomicrmw sub i64* %p, i64 1 seq_cst
   %tmp1 = icmp sle i64 %tmp0, 0
   %tmp2 = select i1 %tmp1, i32 %a0, i32 %a1
   ret i32 %tmp2
 }

 ; CHECK-NEXT:    retq
 entry:
   %tmp0 = atomicrmw sub i64* %p, i64 1 seq_cst
   %tmp1 = icmp ugt i64 %tmp0, 1
   %tmp2 = zext i1 %tmp1 to i8
   ret i8 %tmp2
 }
 
-; FIXME: This test canonicalizes in a way that hides the fact that the
-; comparison can be folded into the atomic subtract.
 define i8 @test_sub_1_cmp_1_setcc_sle(i64* %p) #0 {
-; CHECK-LABEL: test_sub_1_cmp_1_setcc_sle:
-; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    movq $-1, %rax
-; CHECK-NEXT:    lock xaddq %rax, (%rdi)
-; CHECK-NEXT:    cmpq $2, %rax
-; CHECK-NEXT:    setl %al
-; CHECK-NEXT:    retq
+; FASTINCDEC-LABEL: test_sub_1_cmp_1_setcc_sle:
+; FASTINCDEC:       # %bb.0: # %entry
+; FASTINCDEC-NEXT:    lock decq (%rdi)
+; FASTINCDEC-NEXT:    setle %al
+; FASTINCDEC-NEXT:    retq
+;
+; SLOWINCDEC-LABEL: test_sub_1_cmp_1_setcc_sle:
+; SLOWINCDEC:       # %bb.0: # %entry
+; SLOWINCDEC-NEXT:    lock subq $1, (%rdi)
+; SLOWINCDEC-NEXT:    setle %al
+; SLOWINCDEC-NEXT:    retq
 entry:
   %tmp0 = atomicrmw sub i64* %p, i64 1 seq_cst
   %tmp1 = icmp sle i64 %tmp0, 1
   %tmp2 = zext i1 %tmp1 to i8
   ret i8 %tmp2
 }
 
 define i8 @test_sub_3_cmp_3_setcc_eq(i64* %p) #0 {
 ; CHECK-LABEL: test_sub_3_cmp_3_setcc_eq:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lock subq $3, (%rdi)
 ; CHECK-NEXT:    sete %al
 ; CHECK-NEXT:    retq
 entry:
   %tmp0 = atomicrmw sub i64* %p, i64 3 seq_cst
   %tmp1 = icmp eq i64 %tmp0, 3
   %tmp2 = zext i1 %tmp1 to i8
   ret i8 %tmp2
 }
 
-; FIXME: This test canonicalizes in a way that hides the fact that the
-; comparison can be folded into the atomic subtract.
 define i8 @test_sub_3_cmp_3_setcc_uge(i64* %p) #0 {
 ; CHECK-LABEL: test_sub_3_cmp_3_setcc_uge:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    movq $-3, %rax
-; CHECK-NEXT:    lock xaddq %rax, (%rdi)
-; CHECK-NEXT:    cmpq $2, %rax
-; CHECK-NEXT:    seta %al
+; CHECK-NEXT:    lock subq $3, (%rdi)
+; CHECK-NEXT:    setae %al
 ; CHECK-NEXT:    retq
 entry:
   %tmp0 = atomicrmw sub i64* %p, i64 3 seq_cst
   %tmp1 = icmp uge i64 %tmp0, 3
   %tmp2 = zext i1 %tmp1 to i8
   ret i8 %tmp2
 }
 
 attributes #0 = { nounwind }

llvm/test/CodeGen/X86/cmov.ll

 
 
 ; Should compile to setcc | -2.
 ; rdar://6668608
 define i32 @test5(i32* nocapture %P) nounwind readonly {
 ; CHECK-LABEL: test5:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    cmpl $41, (%rdi)
-; CHECK-NEXT:    setg %al
+; CHECK-NEXT:    cmpl $42, (%rdi)
+; CHECK-NEXT:    setge %al
 ; CHECK-NEXT:    orl $-2, %eax
 ; CHECK-NEXT:    retq
 entry:
   %0 = load i32, i32* %P, align 4
   %1 = icmp sgt i32 %0, 41
   %iftmp.0.0 = select i1 %1, i32 -1, i32 -2
   ret i32 %iftmp.0.0
 }
; CHECK-NEXT:    retq
   %d = select i1 %c, i8 %a, i8 %b
   ret i8 %d
 }
 
 define i64 @test8(i64 %0, i64 %1, i64 %2) {
 ; CHECK-LABEL: test8:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq %rsi, %rax
-; CHECK-NEXT:    movabsq $-2147483649, %rcx # imm = 0xFFFFFFFF7FFFFFFF
-; CHECK-NEXT:    cmpq %rcx, %rdi
-; CHECK-NEXT:    cmovleq %rdx, %rax
+; CHECK-NEXT:    cmpq $-2147483648, %rdi # imm = 0x80000000
+; CHECK-NEXT:    cmovlq %rdx, %rax
 ; CHECK-NEXT:    retq
   %4 = icmp sgt i64 %0, -2147483649
   %5 = select i1 %4, i64 %1, i64 %2
   ret i64 %5
 }
 
 define i32 @smin(i32 %x) {
 ; CHECK-LABEL: smin:

llvm/test/CodeGen/X86/lack-of-signed-truncation-check.ll

 
 ; Adding not a constant
 define i1 @add_ugecmp_bad_i16_i8_add(i16 %x, i16 %y) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i16_i8_add:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    addw {{[0-9]+}}(%esp), %ax
 ; X86-NEXT:    movzwl %ax, %eax
-; X86-NEXT:    cmpl $255, %eax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $256, %eax # imm = 0x100
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i16_i8_add:
 ; X64:       # %bb.0:
 ; X64-NEXT:    addl %esi, %edi
 ; X64-NEXT:    movzwl %di, %eax
-; X64-NEXT:    cmpl $255, %eax
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $256, %eax # imm = 0x100
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i16 %x, %y
   %tmp1 = icmp uge i16 %tmp0, 256 ; 1U << 8
   ret i1 %tmp1
 }
 
 ; Comparing not with a constant
 define i1 @add_ugecmp_bad_i16_i8_cmp(i16 %x, i16 %y) nounwind {

 
 ; First constant is not power of two
 define i1 @add_ugecmp_bad_i16_i8_c0notpoweroftwo(i16 %x) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i16_i8_c0notpoweroftwo:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $192, %eax
 ; X86-NEXT:    addl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movzwl %ax, %eax
-; X86-NEXT:    cmpl $255, %eax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $256, %eax # imm = 0x100
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i16_i8_c0notpoweroftwo:
 ; X64:       # %bb.0:
 ; X64-NEXT:    addl $192, %edi
 ; X64-NEXT:    movzwl %di, %eax
-; X64-NEXT:    cmpl $255, %eax
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $256, %eax # imm = 0x100
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i16 %x, 192 ; (1U << (8-1)) + (1U << (8-1-1))
   %tmp1 = icmp uge i16 %tmp0, 256 ; 1U << 8
   ret i1 %tmp1
 }
 
 ; Second constant is not power of two
 define i1 @add_ugecmp_bad_i16_i8_c1notpoweroftwo(i16 %x) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i16_i8_c1notpoweroftwo:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    subl $-128, %eax
 ; X86-NEXT:    movzwl %ax, %eax
-; X86-NEXT:    cmpl $767, %eax # imm = 0x2FF
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $768, %eax # imm = 0x300
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i16_i8_c1notpoweroftwo:
 ; X64:       # %bb.0:
 ; X64-NEXT:    subl $-128, %edi
 ; X64-NEXT:    movzwl %di, %eax
-; X64-NEXT:    cmpl $767, %eax # imm = 0x2FF
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $768, %eax # imm = 0x300
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i16 %x, 128 ; 1U << (8-1)
   %tmp1 = icmp uge i16 %tmp0, 768 ; (1U << 8)) + (1U << (8+1))
   ret i1 %tmp1
 }
 
 ; Magic check fails, 64 << 1 != 256
 define i1 @add_ugecmp_bad_i16_i8_magic(i16 %x) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i16_i8_magic:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    addl $64, %eax
 ; X86-NEXT:    movzwl %ax, %eax
-; X86-NEXT:    cmpl $255, %eax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $256, %eax # imm = 0x100
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i16_i8_magic:
 ; X64:       # %bb.0:
 ; X64-NEXT:    addl $64, %edi
 ; X64-NEXT:    movzwl %di, %eax
-; X64-NEXT:    cmpl $255, %eax
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $256, %eax # imm = 0x100
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i16 %x, 64 ; 1U << (8-1-1)
   %tmp1 = icmp uge i16 %tmp0, 256 ; 1U << 8
   ret i1 %tmp1
 }
 
 ; Bad 'destination type'
 define i1 @add_ugecmp_bad_i16_i4(i16 %x) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i16_i4:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    addl $8, %eax
-; X86-NEXT:    cmpw $15, %ax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpw $16, %ax
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i16_i4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    addl $8, %edi
-; X64-NEXT:    cmpw $15, %di
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpw $16, %di
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i16 %x, 8 ; 1U << (4-1)
   %tmp1 = icmp uge i16 %tmp0, 16 ; 1U << 4
   ret i1 %tmp1
 }
 
 ; Bad storage type
 define i1 @add_ugecmp_bad_i24_i8(i24 %x) nounwind {
 ; X86-LABEL: add_ugecmp_bad_i24_i8:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    subl $-128, %eax
 ; X86-NEXT:    andl $16777215, %eax # imm = 0xFFFFFF
-; X86-NEXT:    cmpl $255, %eax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $256, %eax # imm = 0x100
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: add_ugecmp_bad_i24_i8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    subl $-128, %edi
 ; X64-NEXT:    andl $16777215, %edi # imm = 0xFFFFFF
-; X64-NEXT:    cmpl $255, %edi
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $256, %edi # imm = 0x100
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %tmp0 = add i24 %x, 128 ; 1U << (8-1)
   %tmp1 = icmp uge i24 %tmp0, 256 ; 1U << 8
   ret i1 %tmp1
 }
 
 ; Slightly more canonical variant
 define i1 @add_ugtcmp_bad_i16_i8(i16 %x) nounwind {
 ; CHECK-LABEL: add_ugtcmp_bad_i16_i8:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    ret{{[l|q]}}
   %tmp0 = add i16 %x, 128 ; 1U << (8-1)
   %tmp1 = icmp ugt i16 %tmp0, -1 ; when we +1 it, it will wrap to 0
   ret i1 %tmp1
 }

llvm/test/CodeGen/X86/mul-constant-result.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown | FileCheck %s --check-prefix=X86
 
 ; Incremental updates of the instruction depths should be enough for this test
 ; case.
 ; RUN: llc < %s -mtriple=x86_64-unknown -mcpu=haswell -machine-combiner-inc-threshold=0| FileCheck %s --check-prefix=X64-HSW
 
 ; Function Attrs: norecurse nounwind readnone uwtable
 define i32 @mult(i32, i32) local_unnamed_addr #0 {
 ; X86-LABEL: mult:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    .cfi_offset %esi, -8
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    cmpl $1, %edx
+; X86-NEXT:    cmpl $2, %edx
 ; X86-NEXT:    movl $1, %eax
 ; X86-NEXT:    movl $1, %esi
-; X86-NEXT:    jg .LBB0_2
+; X86-NEXT:    jge .LBB0_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl %edx, %esi
 ; X86-NEXT:  .LBB0_2:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    testl %edx, %edx
 ; X86-NEXT:    je .LBB0_4
 ; X86-NEXT:  # %bb.3:
 ; X86-NEXT:    movl %esi, %eax

 ; X86-NEXT:    shll $5, %eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-HSW-LABEL: mult:
 ; X64-HSW:       # %bb.0:
 ; X64-HSW-NEXT:    # kill: def $edi killed $edi def $rdi
-; X64-HSW-NEXT:    cmpl $1, %esi
+; X64-HSW-NEXT:    cmpl $2, %esi
 ; X64-HSW-NEXT:    movl $1, %ecx
 ; X64-HSW-NEXT:    movl %esi, %eax
-; X64-HSW-NEXT:    cmovgl %ecx, %eax
+; X64-HSW-NEXT:    cmovgel %ecx, %eax
 ; X64-HSW-NEXT:    testl %esi, %esi
 ; X64-HSW-NEXT:    cmovel %ecx, %eax
 ; X64-HSW-NEXT:    decl %edi
 ; X64-HSW-NEXT:    cmpl $31, %edi
 ; X64-HSW-NEXT:    ja .LBB0_3
 ; X64-HSW-NEXT:  # %bb.1:
 ; X64-HSW-NEXT:    jmpq *.LJTI0_0(,%rdi,8)
 ; X64-HSW-NEXT:  .LBB0_2:

llvm/test/CodeGen/X86/or-branch.ll

 ; JUMP2-NEXT:    retl
 ; JUMP2-NEXT:  .LBB0_3: # %cond_true
 ; JUMP2-NEXT:    jmp bar@PLT # TAILCALL
 ;
 ; JUMP1-LABEL: foo:
 ; JUMP1:       # %bb.0: # %entry
 ; JUMP1-NEXT:    cmpl $0, {{[0-9]+}}(%esp)
 ; JUMP1-NEXT:    setne %al
-; JUMP1-NEXT:    cmpl $4, {{[0-9]+}}(%esp)
-; JUMP1-NEXT:    setg %cl
+; JUMP1-NEXT:    cmpl $5, {{[0-9]+}}(%esp)
+; JUMP1-NEXT:    setge %cl
 ; JUMP1-NEXT:    testb %al, %cl
 ; JUMP1-NEXT:    jne .LBB0_1
 ; JUMP1-NEXT:  # %bb.2: # %cond_true
 ; JUMP1-NEXT:    jmp bar@PLT # TAILCALL
 ; JUMP1-NEXT:  .LBB0_1: # %UnifiedReturnBlock
 ; JUMP1-NEXT:    retl
 entry:
   %tmp1 = icmp eq i32 %X, 0

 ; If the branch is unpredictable, don't add another branch
 ; regardless of whether they are expensive or not.
 
 define void @unpredictable(i32 %X, i32 %Y, i32 %Z) nounwind {
 ; JUMP2-LABEL: unpredictable:
 ; JUMP2:       # %bb.0: # %entry
 ; JUMP2-NEXT:    cmpl $0, {{[0-9]+}}(%esp)
 ; JUMP2-NEXT:    setne %al
-; JUMP2-NEXT:    cmpl $4, {{[0-9]+}}(%esp)
-; JUMP2-NEXT:    setg %cl
+; JUMP2-NEXT:    cmpl $5, {{[0-9]+}}(%esp)
+; JUMP2-NEXT:    setge %cl
 ; JUMP2-NEXT:    testb %al, %cl
 ; JUMP2-NEXT:    jne .LBB1_1
 ; JUMP2-NEXT:  # %bb.2: # %cond_true
 ; JUMP2-NEXT:    jmp bar@PLT # TAILCALL
 ; JUMP2-NEXT:  .LBB1_1: # %UnifiedReturnBlock
 ; JUMP2-NEXT:    retl
 ;
 ; JUMP1-LABEL: unpredictable:
 ; JUMP1:       # %bb.0: # %entry
 ; JUMP1-NEXT:    cmpl $0, {{[0-9]+}}(%esp)
 ; JUMP1-NEXT:    setne %al
-; JUMP1-NEXT:    cmpl $4, {{[0-9]+}}(%esp)
-; JUMP1-NEXT:    setg %cl
+; JUMP1-NEXT:    cmpl $5, {{[0-9]+}}(%esp)
+; JUMP1-NEXT:    setge %cl
 ; JUMP1-NEXT:    testb %al, %cl
 ; JUMP1-NEXT:    jne .LBB1_1
 ; JUMP1-NEXT:  # %bb.2: # %cond_true
 ; JUMP1-NEXT:    jmp bar@PLT # TAILCALL
 ; JUMP1-NEXT:  .LBB1_1: # %UnifiedReturnBlock
 ; JUMP1-NEXT:    retl
 entry:
   %tmp1 = icmp eq i32 %X, 0

llvm/test/CodeGen/X86/pr45995-2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O3  --x86-asm-syntax=intel -mtriple=x86_64-grtev4-linux-gnu -march=x86-64 -mcpu=skylake-avx512 -mattr=fma,avx512f < %s | FileCheck %s
 
 define <4 x i1> @selecter(i64 %0) {
 ; CHECK-LABEL: selecter:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xor eax, eax
-; CHECK-NEXT:    cmp rdi, 1
-; CHECK-NEXT:    setg al
+; CHECK-NEXT:    cmp rdi, 2
+; CHECK-NEXT:    setge al
 ; CHECK-NEXT:    lea eax, [rax + 2*rax]
 ; CHECK-NEXT:    kmovd k0, eax
 ; CHECK-NEXT:    vpmovm2d xmm0, k0
 ; CHECK-NEXT:    ret
   %2 = icmp slt i64 0, %0
   %3 = select i1 %2, <4 x i1> <i1 true, i1 true, i1 false, i1 false>, <4 x i1> zeroinitializer
   %4 = insertvalue [4 x <4 x i1>] zeroinitializer, <4 x i1> %3, 0
   %5 = icmp slt i64 1, %0

llvm/test/CodeGen/X86/pr5145.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-- < %s | FileCheck %s
 @sc8 = external dso_local global i8
 
 define void @atomic_maxmin_i8() {
 ; CHECK-LABEL: atomic_maxmin_i8:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movb sc8(%rip), %al
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_1: # %atomicrmw.start
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmpb $5, %al
+; CHECK-NEXT:    cmpb $6, %al
 ; CHECK-NEXT:    movzbl %al, %eax
 ; CHECK-NEXT:    movl $5, %ecx
-; CHECK-NEXT:    cmovgl %eax, %ecx
+; CHECK-NEXT:    cmovgel %eax, %ecx
 ; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    lock cmpxchgb %cl, sc8(%rip)
 ; CHECK-NEXT:    jne .LBB0_1
 ; CHECK-NEXT:  # %bb.2: # %atomicrmw.end
 ; CHECK-NEXT:    movb sc8(%rip), %al
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_3: # %atomicrmw.start2
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    cmpb $7, %al
 ; CHECK-NEXT:    movzbl %al, %eax
 ; CHECK-NEXT:    movl $6, %ecx
 ; CHECK-NEXT:    cmovll %eax, %ecx
 ; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    lock cmpxchgb %cl, sc8(%rip)
 ; CHECK-NEXT:    jne .LBB0_3
 ; CHECK-NEXT:  # %bb.4: # %atomicrmw.end1
 ; CHECK-NEXT:    movb sc8(%rip), %al
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_5: # %atomicrmw.start8
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    cmpb $7, %al
+; CHECK-NEXT:    cmpb $8, %al
 ; CHECK-NEXT:    movzbl %al, %eax
 ; CHECK-NEXT:    movl $7, %ecx
-; CHECK-NEXT:    cmoval %eax, %ecx
+; CHECK-NEXT:    cmovael %eax, %ecx
 ; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    lock cmpxchgb %cl, sc8(%rip)
 ; CHECK-NEXT:    jne .LBB0_5
 ; CHECK-NEXT:  # %bb.6: # %atomicrmw.end7
 ; CHECK-NEXT:    movb sc8(%rip), %al
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_7: # %atomicrmw.start14
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1

llvm/test/CodeGen/X86/sadd_sat.ll

 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    addb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    movzbl %al, %ecx
 ; X86-NEXT:    cmpb $7, %al
 ; X86-NEXT:    movl $7, %eax
 ; X86-NEXT:    cmovll %ecx, %eax
-; X86-NEXT:    cmpb $-8, %al
+; X86-NEXT:    cmpb $-7, %al
 ; X86-NEXT:    movl $248, %ecx
-; X86-NEXT:    cmovgl %eax, %ecx
+; X86-NEXT:    cmovgel %eax, %ecx
 ; X86-NEXT:    movsbl %cl, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: func3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    addb %sil, %dil
 ; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    cmpb $7, %al
 ; X64-NEXT:    movl $7, %ecx
 ; X64-NEXT:    cmovll %eax, %ecx
-; X64-NEXT:    cmpb $-8, %cl
+; X64-NEXT:    cmpb $-7, %cl
 ; X64-NEXT:    movl $248, %eax
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    movsbl %al, %eax
 ; X64-NEXT:    retq
   %tmp = call i4 @llvm.sadd.sat.i4(i4 %x, i4 %y);
   ret i4 %tmp;
 }
 
 define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
 ; X86-LABEL: vec:

llvm/test/CodeGen/X86/sadd_sat_plus.ll

 ; X86-NEXT:    mulb {{[0-9]+}}(%esp)
 ; X86-NEXT:    shlb $4, %al
 ; X86-NEXT:    sarb $4, %al
 ; X86-NEXT:    addb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    movzbl %al, %ecx
 ; X86-NEXT:    cmpb $7, %al
 ; X86-NEXT:    movl $7, %eax
 ; X86-NEXT:    cmovll %ecx, %eax
-; X86-NEXT:    cmpb $-8, %al
+; X86-NEXT:    cmpb $-7, %al
 ; X86-NEXT:    movl $248, %ecx
-; X86-NEXT:    cmovgl %eax, %ecx
+; X86-NEXT:    cmovgel %eax, %ecx
 ; X86-NEXT:    movsbl %cl, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: func4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    mulb %dl
 ; X64-NEXT:    shlb $4, %al
 ; X64-NEXT:    sarb $4, %al
 ; X64-NEXT:    addb %dil, %al
 ; X64-NEXT:    movzbl %al, %eax
 ; X64-NEXT:    cmpb $7, %al
 ; X64-NEXT:    movl $7, %ecx
 ; X64-NEXT:    cmovll %eax, %ecx
-; X64-NEXT:    cmpb $-8, %cl
+; X64-NEXT:    cmpb $-7, %cl
 ; X64-NEXT:    movl $248, %eax
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    movsbl %al, %eax
 ; X64-NEXT:    retq
   %a = mul i4 %y, %z
   %tmp = call i4 @llvm.sadd.sat.i4(i4 %x, i4 %a)
   ret i4 %tmp
 }

llvm/test/CodeGen/X86/sdiv_fix_sat.ll

 ; X64-NEXT:    xorb %sil, %cl
 ; X64-NEXT:    testl %edx, %edx
 ; X64-NEXT:    setne %dl
 ; X64-NEXT:    testb %cl, %dl
 ; X64-NEXT:    cmovel %eax, %edi
 ; X64-NEXT:    cmpl $65535, %edi # imm = 0xFFFF
 ; X64-NEXT:    movl $65535, %ecx # imm = 0xFFFF
 ; X64-NEXT:    cmovll %edi, %ecx
-; X64-NEXT:    cmpl $-65536, %ecx # imm = 0xFFFF0000
+; X64-NEXT:    cmpl $-65535, %ecx # imm = 0xFFFF0001
 ; X64-NEXT:    movl $-65536, %eax # imm = 0xFFFF0000
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi

 ; X86-NEXT:    xorb %bl, %cl
 ; X86-NEXT:    testl %edx, %edx
 ; X86-NEXT:    setne %dl
 ; X86-NEXT:    testb %cl, %dl
 ; X86-NEXT:    cmovel %eax, %edi
 ; X86-NEXT:    cmpl $65535, %edi # imm = 0xFFFF
 ; X86-NEXT:    movl $65535, %ecx # imm = 0xFFFF
 ; X86-NEXT:    cmovll %edi, %ecx
-; X86-NEXT:    cmpl $-65536, %ecx # imm = 0xFFFF0000
+; X86-NEXT:    cmpl $-65535, %ecx # imm = 0xFFFF0001
 ; X86-NEXT:    movl $-65536, %eax # imm = 0xFFFF0000
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    retl
   %tmp = call i16 @llvm.sdiv.fix.sat.i16(i16 %x, i16 %y, i32 7)
   ret i16 %tmp

 ; X64-NEXT:    xorb %sil, %cl
 ; X64-NEXT:    testl %edx, %edx
 ; X64-NEXT:    setne %dl
 ; X64-NEXT:    testb %cl, %dl
 ; X64-NEXT:    cmovel %eax, %edi
 ; X64-NEXT:    cmpl $16383, %edi # imm = 0x3FFF
 ; X64-NEXT:    movl $16383, %ecx # imm = 0x3FFF
 ; X64-NEXT:    cmovll %edi, %ecx
-; X64-NEXT:    cmpl $-16384, %ecx # imm = 0xC000
+; X64-NEXT:    cmpl $-16383, %ecx # imm = 0xC001
 ; X64-NEXT:    movl $-16384, %eax # imm = 0xC000
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func2:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi

 ; X86-NEXT:    xorb %bl, %cl
 ; X86-NEXT:    testl %edx, %edx
 ; X86-NEXT:    setne %dl
 ; X86-NEXT:    testb %cl, %dl
 ; X86-NEXT:    cmovel %eax, %edi
 ; X86-NEXT:    cmpl $16383, %edi # imm = 0x3FFF
 ; X86-NEXT:    movl $16383, %ecx # imm = 0x3FFF
 ; X86-NEXT:    cmovll %edi, %ecx
-; X86-NEXT:    cmpl $-16384, %ecx # imm = 0xC000
+; X86-NEXT:    cmpl $-16383, %ecx # imm = 0xC001
 ; X86-NEXT:    movl $-16384, %eax # imm = 0xC000
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    retl
   %x2 = sext i8 %x to i15
   %y2 = sext i8 %y to i15
   %tmp = call i15 @llvm.sdiv.fix.sat.i15(i15 %x2, i15 %y2, i32 14)

 ; X64-NEXT:    setne %dl
 ; X64-NEXT:    testb %cl, %dl
 ; X64-NEXT:    cmovel %eax, %esi
 ; X64-NEXT:    movswl %si, %eax
 ; X64-NEXT:    cmpl $16383, %eax # imm = 0x3FFF
 ; X64-NEXT:    movl $16383, %ecx # imm = 0x3FFF
 ; X64-NEXT:    cmovll %esi, %ecx
 ; X64-NEXT:    movswl %cx, %eax
-; X64-NEXT:    cmpl $-16384, %eax # imm = 0xC000
+; X64-NEXT:    cmpl $-16383, %eax # imm = 0xC001
 ; X64-NEXT:    movl $49152, %eax # imm = 0xC000
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx

 ; X86-NEXT:    setne %cl
 ; X86-NEXT:    testb %ch, %cl
 ; X86-NEXT:    cmovel %eax, %edi
 ; X86-NEXT:    movswl %di, %eax
 ; X86-NEXT:    cmpl $16383, %eax # imm = 0x3FFF
 ; X86-NEXT:    movl $16383, %ecx # imm = 0x3FFF
 ; X86-NEXT:    cmovll %edi, %ecx
 ; X86-NEXT:    movswl %cx, %eax
-; X86-NEXT:    cmpl $-16384, %eax # imm = 0xC000
+; X86-NEXT:    cmpl $-16383, %eax # imm = 0xC001
 ; X86-NEXT:    movl $49152, %eax # imm = 0xC000
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    retl
   %y2 = sext i8 %y to i15
   %y3 = shl i15 %y2, 7
   %tmp = call i15 @llvm.sdiv.fix.sat.i15(i15 %x, i15 %y3, i32 4)
   %tmp2 = sext i15 %tmp to i16

 ; X64-NEXT:    xorb %dl, %cl
 ; X64-NEXT:    testb %bl, %bl
 ; X64-NEXT:    setne %dl
 ; X64-NEXT:    testb %cl, %dl
 ; X64-NEXT:    cmovel %eax, %edi
 ; X64-NEXT:    cmpb $7, %dil
 ; X64-NEXT:    movl $7, %ecx
 ; X64-NEXT:    cmovll %edi, %ecx
-; X64-NEXT:    cmpb $-8, %cl
+; X64-NEXT:    cmpb $-7, %cl
 ; X64-NEXT:    movl $248, %eax
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func4:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %dl

 ; X86-NEXT:    xorb %dl, %dh
 ; X86-NEXT:    testb %cl, %cl
 ; X86-NEXT:    setne %cl
 ; X86-NEXT:    testb %dh, %cl
 ; X86-NEXT:    cmovel %esi, %eax
 ; X86-NEXT:    cmpb $7, %al
 ; X86-NEXT:    movl $7, %ecx
 ; X86-NEXT:    cmovll %eax, %ecx
-; X86-NEXT:    cmpb $-8, %cl
+; X86-NEXT:    cmpb $-7, %cl
 ; X86-NEXT:    movl $248, %eax
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    # kill: def $al killed $al killed $eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    retl
   %tmp = call i4 @llvm.sdiv.fix.sat.i4(i4 %x, i4 %y, i32 2)
   ret i4 %tmp
 }
 
 define i64 @func5(i64 %x, i64 %y) nounwind {

 ; X64-NEXT:    cmoveq {{[-0-9]+}}(%r{{[sb]}}p), %rbx # 8-byte Folded Reload
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    testq %rbp, %rbp
 ; X64-NEXT:    cmovnsq %rax, %rbp
 ; X64-NEXT:    movq $-1, %rcx
 ; X64-NEXT:    cmovgq %rcx, %rbx
 ; X64-NEXT:    testq %rbp, %rbp
 ; X64-NEXT:    cmovnsq %rbp, %rcx
-; X64-NEXT:    cmpq $-2, %rbp
-; X64-NEXT:    cmovleq %rax, %rbx
+; X64-NEXT:    cmpq $-1, %rbp
+; X64-NEXT:    cmovlq %rax, %rbx
 ; X64-NEXT:    shrdq $1, %rcx, %rbx
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    addq $24, %rsp
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    popq %r12
 ; X64-NEXT:    popq %r13
 ; X64-NEXT:    popq %r14
 ; X64-NEXT:    popq %r15

 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    cmpl $2147483647, %edx # imm = 0x7FFFFFFF
 ; X86-NEXT:    movl $2147483647, %edx # imm = 0x7FFFFFFF
 ; X86-NEXT:    cmovbl %ecx, %edx
 ; X86-NEXT:    testl %ecx, %ecx
 ; X86-NEXT:    movl $-1, %ecx
 ; X86-NEXT:    cmovsl %ecx, %esi
 ; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
 ; X86-NEXT:    cmovnel %eax, %esi
 ; X86-NEXT:    cmovnel {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
-; X86-NEXT:    cmpl $-2147483648, %edx # imm = 0x80000000
+; X86-NEXT:    cmpl $-2147483647, %edx # imm = 0x80000001
 ; X86-NEXT:    movl $-2147483648, %eax # imm = 0x80000000
-; X86-NEXT:    cmoval %edx, %eax
+; X86-NEXT:    cmovael %edx, %eax
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    sarl $31, %ecx
 ; X86-NEXT:    andl %esi, %ecx
 ; X86-NEXT:    cmpl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    movl $-2147483648, %ebx # imm = 0x80000000
 ; X86-NEXT:    cmovsl %ebx, %edx
 ; X86-NEXT:    movl $0, %ebx
 ; X86-NEXT:    cmovsl %ebx, %esi
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    cmpl $-1, %edi

 ; X64-NEXT:    xorb %sil, %cl
 ; X64-NEXT:    testl %edx, %edx
 ; X64-NEXT:    setne %dl
 ; X64-NEXT:    testb %cl, %dl
 ; X64-NEXT:    cmovel %eax, %edi
 ; X64-NEXT:    cmpl $131071, %edi # imm = 0x1FFFF
 ; X64-NEXT:    movl $131071, %ecx # imm = 0x1FFFF
 ; X64-NEXT:    cmovll %edi, %ecx
-; X64-NEXT:    cmpl $-131072, %ecx # imm = 0xFFFE0000
+; X64-NEXT:    cmpl $-131071, %ecx # imm = 0xFFFE0001
 ; X64-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func6:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    movswl {{[0-9]+}}(%esp), %esi

 ; X86-NEXT:    xorb %bl, %cl
 ; X86-NEXT:    testl %edx, %edx
 ; X86-NEXT:    setne %dl
 ; X86-NEXT:    testb %cl, %dl
 ; X86-NEXT:    cmovel %eax, %edi
 ; X86-NEXT:    cmpl $131071, %edi # imm = 0x1FFFF
 ; X86-NEXT:    movl $131071, %ecx # imm = 0x1FFFF
 ; X86-NEXT:    cmovll %edi, %ecx
-; X86-NEXT:    cmpl $-131072, %ecx # imm = 0xFFFE0000
+; X86-NEXT:    cmpl $-131071, %ecx # imm = 0xFFFE0001
 ; X86-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    retl
   %x2 = sext i16 %x to i18
   %y2 = sext i16 %y to i18
   %tmp = call i18 @llvm.sdiv.fix.sat.i18(i18 %x2, i18 %y2, i32 7)
   ret i18 %tmp

 ; X64-NEXT:    cmpq %rdx, %r13
 ; X64-NEXT:    movl $4294967295, %eax # imm = 0xFFFFFFFF
 ; X64-NEXT:    cmovbq %r13, %rax
 ; X64-NEXT:    xorl %ecx, %ecx
 ; X64-NEXT:    testq %r14, %r14
 ; X64-NEXT:    cmovnsq %rdx, %r13
 ; X64-NEXT:    cmoveq %rax, %r13
 ; X64-NEXT:    cmovnsq %rcx, %r14
 ; X64-NEXT:    movabsq $-4294967296, %rcx # imm = 0xFFFFFFFF00000000
 ; X64-NEXT:    cmpq %rcx, %r13
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    cmovaq %r13, %rax
 ; X64-NEXT:    testq %r14, %r14
 ; X64-NEXT:    cmovsq %rcx, %r13
 ; X64-NEXT:    cmpq $-1, %r14
 ; X64-NEXT:    cmoveq %rax, %r13
 ; X64-NEXT:    movq %r13, %xmm0
 ; X64-NEXT:    movdqa %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill

llvm/test/CodeGen/X86/select.ll

; MCU-NEXT:    retl
   store i8 %conv, i8* %dst, align 2
   ret void
 }
 
 ; reproducer for pr29002
 define void @clamp(i32 %src, i16* %dst) {
 ; GENERIC-LABEL: clamp:
 ; GENERIC:       ## %bb.0:
-; GENERIC-NEXT:    cmpl $32767, %edi ## imm = 0x7FFF
+; GENERIC-NEXT:    cmpl $32768, %edi ## imm = 0x8000
 ; GENERIC-NEXT:    movl $32767, %eax ## imm = 0x7FFF
-; GENERIC-NEXT:    cmovlel %edi, %eax
+; GENERIC-NEXT:    cmovll %edi, %eax
 ; GENERIC-NEXT:    cmpl $-32768, %eax ## imm = 0x8000
 ; GENERIC-NEXT:    movl $32768, %ecx ## imm = 0x8000
 ; GENERIC-NEXT:    cmovgel %eax, %ecx
 ; GENERIC-NEXT:    movw %cx, (%rsi)
 ; GENERIC-NEXT:    retq
 ;
 ; ATOM-LABEL: clamp:
 ; ATOM:       ## %bb.0:
-; ATOM-NEXT:    cmpl $32767, %edi ## imm = 0x7FFF
+; ATOM-NEXT:    cmpl $32768, %edi ## imm = 0x8000
 ; ATOM-NEXT:    movl $32767, %eax ## imm = 0x7FFF
 ; ATOM-NEXT:    movl $32768, %ecx ## imm = 0x8000
-; ATOM-NEXT:    cmovlel %edi, %eax
+; ATOM-NEXT:    cmovll %edi, %eax
 ; ATOM-NEXT:    cmpl $-32768, %eax ## imm = 0x8000
 ; ATOM-NEXT:    cmovgel %eax, %ecx
 ; ATOM-NEXT:    movw %cx, (%rsi)
 ; ATOM-NEXT:    retq
 ;
 ; ATHLON-LABEL: clamp:
 ; ATHLON:       ## %bb.0:
 ; ATHLON-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; ATHLON-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; ATHLON-NEXT:    cmpl $32767, %ecx ## imm = 0x7FFF
+; ATHLON-NEXT:    cmpl $32768, %ecx ## imm = 0x8000
 ; ATHLON-NEXT:    movl $32767, %edx ## imm = 0x7FFF
-; ATHLON-NEXT:    cmovlel %ecx, %edx
+; ATHLON-NEXT:    cmovll %ecx, %edx
 ; ATHLON-NEXT:    cmpl $-32768, %edx ## imm = 0x8000
 ; ATHLON-NEXT:    movl $32768, %ecx ## imm = 0x8000
 ; ATHLON-NEXT:    cmovgel %edx, %ecx
 ; ATHLON-NEXT:    movw %cx, (%eax)
 ; ATHLON-NEXT:    retl
 ;
 ; MCU-LABEL: clamp:
 ; MCU:       # %bb.0:
-; MCU-NEXT:    cmpl $32767, %eax # imm = 0x7FFF
+; MCU-NEXT:    cmpl $32768, %eax # imm = 0x8000
 ; MCU-NEXT:    movl $32767, %ecx # imm = 0x7FFF
-; MCU-NEXT:    jg .LBB22_2
+; MCU-NEXT:    jge .LBB22_2
 ; MCU-NEXT:  # %bb.1:
 ; MCU-NEXT:    movl %eax, %ecx
 ; MCU-NEXT:  .LBB22_2:
 ; MCU-NEXT:    cmpl $-32768, %ecx # imm = 0x8000
 ; MCU-NEXT:    movl $32768, %eax # imm = 0x8000
 ; MCU-NEXT:    jl .LBB22_4
 ; MCU-NEXT:  # %bb.3:
 ; MCU-NEXT:    movl %ecx, %eax

llvm/test/CodeGen/X86/select_const.ll

; CHECK-NEXT:    retq
   ret i64 %sub
 }
 
 ; No LEA with 8-bit, but this shouldn't need branches or cmov.
 
 define i8 @sel_1_neg1(i32 %x) {
 ; CHECK-LABEL: sel_1_neg1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    cmpl $42, %edi
-; CHECK-NEXT:    setg %al
+; CHECK-NEXT:    cmpl $43, %edi
+; CHECK-NEXT:    setge %al
 ; CHECK-NEXT:    shlb $2, %al
 ; CHECK-NEXT:    decb %al
 ; CHECK-NEXT:    retq
   %cmp = icmp sgt i32 %x, 42
   %sel = select i1 %cmp, i8 3, i8 -1
   ret i8 %sel
 }
 

 }
 
 ; If the comparison is available, the predicate can be inverted.
 
 define i32 @sel_1_neg1_32(i32 %x) {
 ; CHECK-LABEL: sel_1_neg1_32:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    cmpl $42, %edi
-; CHECK-NEXT:    setg %al
+; CHECK-NEXT:    cmpl $43, %edi
+; CHECK-NEXT:    setge %al
 ; CHECK-NEXT:    leal -1(%rax,%rax,8), %eax
 ; CHECK-NEXT:    retq
   %cmp = icmp sgt i32 %x, 42
   %sel = select i1 %cmp, i32 8, i32 -1
   ret i32 %sel
 }
 
 define i32 @sel_neg1_1_32(i32 %x) {
; CHECK-NEXT:    retq
   ret i64 %sel
 }
 
 ; This doesn't need a branch, but don't do the wrong thing if subtraction of the constants overflows.
 
 define i8 @sel_67_neg125(i32 %x) {
 ; CHECK-LABEL: sel_67_neg125:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    cmpl $42, %edi
+; CHECK-NEXT:    cmpl $43, %edi
 ; CHECK-NEXT:    movl $67, %ecx
 ; CHECK-NEXT:    movl $131, %eax
-; CHECK-NEXT:    cmovgl %ecx, %eax
+; CHECK-NEXT:    cmovgel %ecx, %eax
 ; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    retq
   %cmp = icmp sgt i32 %x, 42
   %sel = select i1 %cmp, i8 67, i8 -125
   ret i8 %sel
 }
 
 

llvm/test/CodeGen/X86/setcc-logic.ll

; CHECK-NEXT:    retq
   %c = or <4 x i1> %a, %b
   ret <4 x i1> %c
 }
 
 define zeroext i1 @ne_neg1_and_ne_zero(i64 %x) nounwind {
 ; CHECK-LABEL: ne_neg1_and_ne_zero:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    incq %rdi
-; CHECK-NEXT:    cmpq $1, %rdi
-; CHECK-NEXT:    seta %al
+; CHECK-NEXT:    cmpq $2, %rdi
+; CHECK-NEXT:    setae %al
 ; CHECK-NEXT:    retq
   %cmp1 = icmp ne i64 %x, -1
   %cmp2 = icmp ne i64 %x, 0
   %and = and i1 %cmp1, %cmp2
   ret i1 %and
 }
 
 ; PR32401 - https://bugs.llvm.org/show_bug.cgi?id=32401

llvm/test/CodeGen/X86/setcc.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-apple-darwin | FileCheck %s
 ; rdar://7329206
 
 define zeroext i16 @t1(i16 zeroext %x) nounwind readnone ssp {
 ; CHECK-LABEL: t1:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    cmpw $26, %di
-; CHECK-NEXT:    seta %al
+; CHECK-NEXT:    cmpw $27, %di
+; CHECK-NEXT:    setae %al
 ; CHECK-NEXT:    shll $5, %eax
 ; CHECK-NEXT:    retq
   %t0 = icmp ugt i16 %x, 26
   %if = select i1 %t0, i16 32, i16 0
   ret i16 %if
 }
 
 define zeroext i16 @t2(i16 zeroext %x) nounwind readnone ssp {

llvm/test/CodeGen/X86/smul_fix_sat.ll

 ; X64-LABEL: func:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movslq %esi, %rax
 ; X64-NEXT:    movslq %edi, %rcx
 ; X64-NEXT:    imulq %rax, %rcx
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    shrq $32, %rax
 ; X64-NEXT:    shrdl $2, %eax, %ecx
-; X64-NEXT:    cmpl $1, %eax
+; X64-NEXT:    cmpl $2, %eax
 ; X64-NEXT:    movl $2147483647, %edx # imm = 0x7FFFFFFF
-; X64-NEXT:    cmovlel %ecx, %edx
+; X64-NEXT:    cmovll %ecx, %edx
 ; X64-NEXT:    cmpl $-2, %eax
 ; X64-NEXT:    movl $-2147483648, %eax # imm = 0x80000000
 ; X64-NEXT:    cmovgel %edx, %eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    imull {{[0-9]+}}(%esp)
 ; X86-NEXT:    shrdl $2, %edx, %eax
-; X86-NEXT:    cmpl $1, %edx
+; X86-NEXT:    cmpl $2, %edx
 ; X86-NEXT:    movl $2147483647, %ecx # imm = 0x7FFFFFFF
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    cmpl $-2, %edx
 ; X86-NEXT:    movl $-2147483648, %ecx # imm = 0x80000000
 ; X86-NEXT:    cmovll %ecx, %eax
 ; X86-NEXT:    retl
   %tmp = call i32 @llvm.smul.fix.sat.i32(i32 %x, i32 %y, i32 2)
   ret i32 %tmp
 }
 
 define i64 @func2(i64 %x, i64 %y) nounwind {
 ; X64-LABEL: func2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    imulq %rsi
 ; X64-NEXT:    shrdq $2, %rdx, %rax
-; X64-NEXT:    cmpq $1, %rdx
+; X64-NEXT:    cmpq $2, %rdx
 ; X64-NEXT:    movabsq $9223372036854775807, %rcx # imm = 0x7FFFFFFFFFFFFFFF
-; X64-NEXT:    cmovgq %rcx, %rax
+; X64-NEXT:    cmovgeq %rcx, %rax
 ; X64-NEXT:    cmpq $-2, %rdx
 ; X64-NEXT:    movabsq $-9223372036854775808, %rcx # imm = 0x8000000000000000
 ; X64-NEXT:    cmovlq %rcx, %rax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func2:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebp

 ; X86-NEXT:    movl %ebx, %esi
 ; X86-NEXT:    sbbl $0, %esi
 ; X86-NEXT:    cmpl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    cmovnsl %ebx, %esi
 ; X86-NEXT:    cmovnsl %edi, %ebp
 ; X86-NEXT:    testl %esi, %esi
 ; X86-NEXT:    setg %bl
 ; X86-NEXT:    sete %bh
-; X86-NEXT:    cmpl $1, %ebp
-; X86-NEXT:    seta %dl
+; X86-NEXT:    cmpl $2, %ebp
+; X86-NEXT:    setae %dl
 ; X86-NEXT:    andb %bh, %dl
 ; X86-NEXT:    orb %bl, %dl
 ; X86-NEXT:    shrdl $2, %eax, %ecx
 ; X86-NEXT:    shrdl $2, %ebp, %eax
 ; X86-NEXT:    testb %dl, %dl
 ; X86-NEXT:    movl $2147483647, %edi # imm = 0x7FFFFFFF
 ; X86-NEXT:    cmovel %eax, %edi
 ; X86-NEXT:    movl $-1, %eax

 ; X64-NEXT:    imull %eax, %ecx
 ; X64-NEXT:    movl %ecx, %eax
 ; X64-NEXT:    shrb $2, %al
 ; X64-NEXT:    shrl $8, %ecx
 ; X64-NEXT:    movl %ecx, %edx
 ; X64-NEXT:    shlb $6, %dl
 ; X64-NEXT:    orb %al, %dl
 ; X64-NEXT:    movzbl %dl, %eax
-; X64-NEXT:    cmpb $1, %cl
+; X64-NEXT:    cmpb $2, %cl
 ; X64-NEXT:    movl $127, %edx
-; X64-NEXT:    cmovlel %eax, %edx
+; X64-NEXT:    cmovll %eax, %edx
 ; X64-NEXT:    cmpb $-2, %cl
 ; X64-NEXT:    movl $128, %eax
 ; X64-NEXT:    cmovgel %edx, %eax
 ; X64-NEXT:    sarb $4, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    shlb $4, %al
 ; X86-NEXT:    sarb $4, %al
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %cl
 ; X86-NEXT:    shlb $4, %cl
 ; X86-NEXT:    movsbl %cl, %ecx
 ; X86-NEXT:    movsbl %al, %eax
 ; X86-NEXT:    imull %ecx, %eax
 ; X86-NEXT:    movb %ah, %cl
 ; X86-NEXT:    shlb $6, %cl
 ; X86-NEXT:    shrb $2, %al
 ; X86-NEXT:    orb %cl, %al
 ; X86-NEXT:    movzbl %al, %ecx
-; X86-NEXT:    cmpb $1, %ah
+; X86-NEXT:    cmpb $2, %ah
 ; X86-NEXT:    movl $127, %edx
-; X86-NEXT:    cmovlel %ecx, %edx
+; X86-NEXT:    cmovll %ecx, %edx
 ; X86-NEXT:    cmpb $-2, %ah
 ; X86-NEXT:    movl $128, %eax
 ; X86-NEXT:    cmovgel %edx, %eax
 ; X86-NEXT:    sarb $4, %al
 ; X86-NEXT:    # kill: def $al killed $al killed $eax
 ; X86-NEXT:    retl
   %tmp = call i4 @llvm.smul.fix.sat.i4(i4 %x, i4 %y, i32 2)
   ret i4 %tmp
 }
 
 define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
 ; X64-LABEL: vec:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[3,3,3,3]
 ; X64-NEXT:    movd %xmm2, %eax
 ; X64-NEXT:    cltq
 ; X64-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[3,3,3,3]
 ; X64-NEXT:    movd %xmm2, %ecx
 ; X64-NEXT:    movslq %ecx, %rdx
 ; X64-NEXT:    imulq %rax, %rdx
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    shrq $32, %rcx
 ; X64-NEXT:    shrdl $2, %ecx, %edx
-; X64-NEXT:    cmpl $1, %ecx
+; X64-NEXT:    cmpl $2, %ecx
 ; X64-NEXT:    movl $2147483647, %eax # imm = 0x7FFFFFFF
-; X64-NEXT:    cmovgl %eax, %edx
+; X64-NEXT:    cmovgel %eax, %edx
 ; X64-NEXT:    cmpl $-2, %ecx
 ; X64-NEXT:    movl $-2147483648, %ecx # imm = 0x80000000
 ; X64-NEXT:    cmovll %ecx, %edx
 ; X64-NEXT:    movd %edx, %xmm2
 ; X64-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[2,3,2,3]
 ; X64-NEXT:    movd %xmm3, %edx
 ; X64-NEXT:    movslq %edx, %rdx
 ; X64-NEXT:    pshufd {{.*#+}} xmm3 = xmm0[2,3,2,3]
 ; X64-NEXT:    movd %xmm3, %esi
 ; X64-NEXT:    movslq %esi, %rsi
 ; X64-NEXT:    imulq %rdx, %rsi
 ; X64-NEXT:    movq %rsi, %rdx
 ; X64-NEXT:    shrq $32, %rdx
 ; X64-NEXT:    shrdl $2, %edx, %esi
-; X64-NEXT:    cmpl $1, %edx
-; X64-NEXT:    cmovgl %eax, %esi
+; X64-NEXT:    cmpl $2, %edx
+; X64-NEXT:    cmovgel %eax, %esi
 ; X64-NEXT:    cmpl $-2, %edx
 ; X64-NEXT:    cmovll %ecx, %esi
 ; X64-NEXT:    movd %esi, %xmm3
 ; X64-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm2[0],xmm3[1],xmm2[1]
 ; X64-NEXT:    movd %xmm1, %edx
 ; X64-NEXT:    movslq %edx, %rdx
 ; X64-NEXT:    movd %xmm0, %esi
 ; X64-NEXT:    movslq %esi, %rsi
 ; X64-NEXT:    imulq %rdx, %rsi
 ; X64-NEXT:    movq %rsi, %rdx
 ; X64-NEXT:    shrq $32, %rdx
 ; X64-NEXT:    shrdl $2, %edx, %esi
-; X64-NEXT:    cmpl $1, %edx
-; X64-NEXT:    cmovgl %eax, %esi
+; X64-NEXT:    cmpl $2, %edx
+; X64-NEXT:    cmovgel %eax, %esi
 ; X64-NEXT:    cmpl $-2, %edx
 ; X64-NEXT:    cmovll %ecx, %esi
 ; X64-NEXT:    movd %esi, %xmm2
 ; X64-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X64-NEXT:    movd %xmm1, %edx
 ; X64-NEXT:    movslq %edx, %rdx
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,1,1]
 ; X64-NEXT:    movd %xmm0, %esi
 ; X64-NEXT:    movslq %esi, %rsi
 ; X64-NEXT:    imulq %rdx, %rsi
 ; X64-NEXT:    movq %rsi, %rdx
 ; X64-NEXT:    shrq $32, %rdx
 ; X64-NEXT:    shrdl $2, %edx, %esi
-; X64-NEXT:    cmpl $1, %edx
-; X64-NEXT:    cmovgl %eax, %esi
+; X64-NEXT:    cmpl $2, %edx
+; X64-NEXT:    cmovgel %eax, %esi
 ; X64-NEXT:    cmpl $-2, %edx
 ; X64-NEXT:    cmovll %ecx, %esi
 ; X64-NEXT:    movd %esi, %xmm0
 ; X64-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
 ; X64-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm3[0]
 ; X64-NEXT:    movdqa %xmm2, %xmm0
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: vec:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    imull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrdl $2, %edx, %ecx
-; X86-NEXT:    cmpl $1, %edx
+; X86-NEXT:    cmpl $2, %edx
 ; X86-NEXT:    movl $2147483647, %ebp # imm = 0x7FFFFFFF
-; X86-NEXT:    cmovgl %ebp, %ecx
+; X86-NEXT:    cmovgel %ebp, %ecx
 ; X86-NEXT:    cmpl $-2, %edx
 ; X86-NEXT:    movl $-2147483648, %esi # imm = 0x80000000
 ; X86-NEXT:    cmovll %esi, %ecx
 ; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    imull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %edi
 ; X86-NEXT:    shrdl $2, %edx, %edi
-; X86-NEXT:    cmpl $1, %edx
-; X86-NEXT:    cmovgl %ebp, %edi
+; X86-NEXT:    cmpl $2, %edx
+; X86-NEXT:    cmovgel %ebp, %edi
 ; X86-NEXT:    cmpl $-2, %edx
 ; X86-NEXT:    cmovll %esi, %edi
 ; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    imull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %ebx
 ; X86-NEXT:    shrdl $2, %edx, %ebx
-; X86-NEXT:    cmpl $1, %edx
-; X86-NEXT:    cmovgl %ebp, %ebx
+; X86-NEXT:    cmpl $2, %edx
+; X86-NEXT:    cmovgel %ebp, %ebx
 ; X86-NEXT:    cmpl $-2, %edx
 ; X86-NEXT:    cmovll %esi, %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    imull {{[0-9]+}}(%esp)
 ; X86-NEXT:    shrdl $2, %edx, %eax
-; X86-NEXT:    cmpl $1, %edx
-; X86-NEXT:    cmovgl %ebp, %eax
+; X86-NEXT:    cmpl $2, %edx
+; X86-NEXT:    cmovgel %ebp, %eax
 ; X86-NEXT:    cmpl $-2, %edx
 ; X86-NEXT:    cmovll %esi, %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    movl %eax, 12(%edx)
 ; X86-NEXT:    movl %ebx, 8(%edx)
 ; X86-NEXT:    movl %edi, 4(%edx)
 ; X86-NEXT:    movl %ecx, (%edx)
 ; X86-NEXT:    movl %edx, %eax

 ; X86-NEXT:    subl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    movl %esi, %edi
 ; X86-NEXT:    sbbl $0, %edi
 ; X86-NEXT:    cmpl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    cmovnsl %esi, %edi
 ; X86-NEXT:    cmovnsl %ecx, %edx
 ; X86-NEXT:    shrdl $31, %edx, %eax
 ; X86-NEXT:    shrdl $31, %edi, %edx
-; X86-NEXT:    cmpl $1073741823, %edi # imm = 0x3FFFFFFF
+; X86-NEXT:    cmpl $1073741824, %edi # imm = 0x40000000
 ; X86-NEXT:    movl $2147483647, %ecx # imm = 0x7FFFFFFF
-; X86-NEXT:    cmovgl %ecx, %edx
+; X86-NEXT:    cmovgel %ecx, %edx
 ; X86-NEXT:    movl $-1, %ecx
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    xorl %ecx, %ecx
 ; X86-NEXT:    cmpl $-1073741824, %edi # imm = 0xC0000000
 ; X86-NEXT:    cmovll %ecx, %eax
 ; X86-NEXT:    movl $-2147483648, %ecx # imm = 0x80000000
 ; X86-NEXT:    cmovll %ecx, %edx
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
   %tmp = call i64 @llvm.smul.fix.sat.i64(i64 %x, i64 %y, i32 63)
   ret i64 %tmp
 }

llvm/test/CodeGen/X86/smul_fix_sat_constants.ll

 declare { i64, i1 } @llvm.smul.with.overflow.i64(i64, i64)
 
 define i64 @func() nounwind {
 ; X64-LABEL: func:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl $2, %ecx
 ; X64-NEXT:    movl $3, %eax
 ; X64-NEXT:    imulq %rcx
-; X64-NEXT:    cmpq $1, %rdx
+; X64-NEXT:    cmpq $2, %rdx
 ; X64-NEXT:    movabsq $9223372036854775807, %rax # imm = 0x7FFFFFFFFFFFFFFF
 ; X64-NEXT:    movl $1, %ecx
-; X64-NEXT:    cmovgq %rax, %rcx
+; X64-NEXT:    cmovgeq %rax, %rcx
 ; X64-NEXT:    cmpq $-2, %rdx
 ; X64-NEXT:    movabsq $-9223372036854775808, %rax # imm = 0x8000000000000000
 ; X64-NEXT:    cmovgeq %rcx, %rax
 ; X64-NEXT:    retq
   %tmp = call i64 @llvm.smul.fix.sat.i64(i64 3, i64 2, i32 2)
   ret i64 %tmp
 }
 

 
 define i64 @func3() nounwind {
 ; X64-LABEL: func3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $9223372036854775807, %rcx # imm = 0x7FFFFFFFFFFFFFFF
 ; X64-NEXT:    movl $2, %edx
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    imulq %rdx
-; X64-NEXT:    cmpq $1, %rdx
+; X64-NEXT:    cmpq $2, %rdx
 ; X64-NEXT:    movabsq $4611686018427387903, %rsi # imm = 0x3FFFFFFFFFFFFFFF
-; X64-NEXT:    cmovgq %rcx, %rsi
+; X64-NEXT:    cmovgeq %rcx, %rsi
 ; X64-NEXT:    cmpq $-2, %rdx
 ; X64-NEXT:    movabsq $-9223372036854775808, %rax # imm = 0x8000000000000000
 ; X64-NEXT:    cmovgeq %rsi, %rax
 ; X64-NEXT:    retq
   %tmp = call i64 @llvm.smul.fix.sat.i64(i64 9223372036854775807, i64 2, i32 2)
   ret i64 %tmp
 }
 

llvm/test/CodeGen/X86/srem-seteq.ll

 define i16 @test_srem_even(i16 %X) nounwind {
 ; X86-LABEL: test_srem_even:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $28087, {{[0-9]+}}(%esp), %eax # imm = 0x6DB7
 ; X86-NEXT:    addl $4680, %eax # imm = 0x1248
 ; X86-NEXT:    rorw %ax
 ; X86-NEXT:    movzwl %ax, %ecx
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $4680, %ecx # imm = 0x1248
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $4681, %ecx # imm = 0x1249
+; X86-NEXT:    setae %al
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_srem_even:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $28087, %edi, %eax # imm = 0x6DB7
 ; X64-NEXT:    addl $4680, %eax # imm = 0x1248
 ; X64-NEXT:    rorw %ax
 ; X64-NEXT:    movzwl %ax, %ecx
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $4680, %ecx # imm = 0x1248
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $4681, %ecx # imm = 0x1249
+; X64-NEXT:    setae %al
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %srem = srem i16 %X, 14
   %cmp = icmp ne i16 %srem, 0
   %ret = zext i1 %cmp to i16
   ret i16 %ret
 }
 

 
 ; 'NE' predicate is fine too.
 define i32 @test_srem_odd_setne(i32 %X) nounwind {
 ; X86-LABEL: test_srem_odd_setne:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $-858993459, {{[0-9]+}}(%esp), %ecx # imm = 0xCCCCCCCD
 ; X86-NEXT:    addl $429496729, %ecx # imm = 0x19999999
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $858993458, %ecx # imm = 0x33333332
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_srem_odd_setne:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $-858993459, %edi, %ecx # imm = 0xCCCCCCCD
 ; X64-NEXT:    addl $429496729, %ecx # imm = 0x19999999
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $858993458, %ecx # imm = 0x33333332
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %srem = srem i32 %X, 5
   %cmp = icmp ne i32 %srem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 
 ; The fold is only valid for positive divisors, negative-ones should be negated.
 define i32 @test_srem_negative_odd(i32 %X) nounwind {
 ; X86-LABEL: test_srem_negative_odd:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $-858993459, {{[0-9]+}}(%esp), %ecx # imm = 0xCCCCCCCD
 ; X86-NEXT:    addl $429496729, %ecx # imm = 0x19999999
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $858993458, %ecx # imm = 0x33333332
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_srem_negative_odd:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $-858993459, %edi, %ecx # imm = 0xCCCCCCCD
 ; X64-NEXT:    addl $429496729, %ecx # imm = 0x19999999
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $858993458, %ecx # imm = 0x33333332
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %srem = srem i32 %X, -5
   %cmp = icmp ne i32 %srem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 define i32 @test_srem_negative_even(i32 %X) nounwind {
 ; X86-LABEL: test_srem_negative_even:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $-1227133513, {{[0-9]+}}(%esp), %ecx # imm = 0xB6DB6DB7
 ; X86-NEXT:    addl $306783378, %ecx # imm = 0x12492492
 ; X86-NEXT:    rorl %ecx
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $306783378, %ecx # imm = 0x12492492
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $306783379, %ecx # imm = 0x12492493
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_srem_negative_even:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $-1227133513, %edi, %ecx # imm = 0xB6DB6DB7
 ; X64-NEXT:    addl $306783378, %ecx # imm = 0x12492492
 ; X64-NEXT:    rorl %ecx
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $306783378, %ecx # imm = 0x12492492
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $306783379, %ecx # imm = 0x12492493
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %srem = srem i32 %X, -14
   %cmp = icmp ne i32 %srem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 
 ;------------------------------------------------------------------------------;

llvm/test/CodeGen/X86/ssub_sat.ll

 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    subb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    movzbl %al, %ecx
 ; X86-NEXT:    cmpb $7, %al
 ; X86-NEXT:    movl $7, %eax
 ; X86-NEXT:    cmovll %ecx, %eax
-; X86-NEXT:    cmpb $-8, %al
+; X86-NEXT:    cmpb $-7, %al
 ; X86-NEXT:    movl $248, %ecx
-; X86-NEXT:    cmovgl %eax, %ecx
+; X86-NEXT:    cmovgel %eax, %ecx
 ; X86-NEXT:    movsbl %cl, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: func3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    subb %sil, %dil
 ; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    cmpb $7, %al
 ; X64-NEXT:    movl $7, %ecx
 ; X64-NEXT:    cmovll %eax, %ecx
-; X64-NEXT:    cmpb $-8, %cl
+; X64-NEXT:    cmpb $-7, %cl
 ; X64-NEXT:    movl $248, %eax
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    movsbl %al, %eax
 ; X64-NEXT:    retq
   %tmp = call i4 @llvm.ssub.sat.i4(i4 %x, i4 %y)
   ret i4 %tmp
 }
 
 define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
 ; X86-LABEL: vec:

llvm/test/CodeGen/X86/ssub_sat_plus.ll

 ; X86-NEXT:    mulb {{[0-9]+}}(%esp)
 ; X86-NEXT:    shlb $4, %al
 ; X86-NEXT:    sarb $4, %al
 ; X86-NEXT:    subb %al, %cl
 ; X86-NEXT:    movzbl %cl, %eax
 ; X86-NEXT:    cmpb $7, %cl
 ; X86-NEXT:    movl $7, %ecx
 ; X86-NEXT:    cmovll %eax, %ecx
-; X86-NEXT:    cmpb $-8, %cl
+; X86-NEXT:    cmpb $-7, %cl
 ; X86-NEXT:    movl $248, %eax
-; X86-NEXT:    cmovgl %ecx, %eax
+; X86-NEXT:    cmovgel %ecx, %eax
 ; X86-NEXT:    movsbl %al, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: func4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    mulb %dl
 ; X64-NEXT:    shlb $4, %al
 ; X64-NEXT:    sarb $4, %al
 ; X64-NEXT:    subb %al, %dil
 ; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    cmpb $7, %al
 ; X64-NEXT:    movl $7, %ecx
 ; X64-NEXT:    cmovll %eax, %ecx
-; X64-NEXT:    cmpb $-8, %cl
+; X64-NEXT:    cmpb $-7, %cl
 ; X64-NEXT:    movl $248, %eax
-; X64-NEXT:    cmovgl %ecx, %eax
+; X64-NEXT:    cmovgel %ecx, %eax
 ; X64-NEXT:    movsbl %al, %eax
 ; X64-NEXT:    retq
   %a = mul i4 %y, %z
   %tmp = call i4 @llvm.ssub.sat.i4(i4 %x, i4 %a)
   ret i4 %tmp
 }

llvm/test/CodeGen/X86/umul_fix_sat.ll

 ; X64-LABEL: func:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    movl %edi, %ecx
 ; X64-NEXT:    imulq %rax, %rcx
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    shrq $32, %rax
 ; X64-NEXT:    shrdl $2, %eax, %ecx
-; X64-NEXT:    cmpl $3, %eax
+; X64-NEXT:    cmpl $4, %eax
 ; X64-NEXT:    movl $-1, %eax
-; X64-NEXT:    cmovbel %ecx, %eax
+; X64-NEXT:    cmovbl %ecx, %eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    shrdl $2, %edx, %eax
-; X86-NEXT:    cmpl $3, %edx
+; X86-NEXT:    cmpl $4, %edx
 ; X86-NEXT:    movl $-1, %ecx
-; X86-NEXT:    cmoval %ecx, %eax
+; X86-NEXT:    cmovael %ecx, %eax
 ; X86-NEXT:    retl
   %tmp = call i32 @llvm.umul.fix.sat.i32(i32 %x, i32 %y, i32 2)
   ret i32 %tmp
 }
 
 define i64 @func2(i64 %x, i64 %y) nounwind {
 ; X64-LABEL: func2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    mulq %rsi
 ; X64-NEXT:    shrdq $2, %rdx, %rax
-; X64-NEXT:    cmpq $3, %rdx
+; X64-NEXT:    cmpq $4, %rdx
 ; X64-NEXT:    movq $-1, %rcx
-; X64-NEXT:    cmovaq %rcx, %rax
+; X64-NEXT:    cmovaeq %rcx, %rax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func2:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi

 ; X64-NEXT:    imull %esi, %eax
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    shrb $2, %cl
 ; X64-NEXT:    shrl $8, %eax
 ; X64-NEXT:    movl %eax, %edx
 ; X64-NEXT:    shlb $6, %dl
 ; X64-NEXT:    orb %cl, %dl
 ; X64-NEXT:    movzbl %dl, %ecx
-; X64-NEXT:    cmpb $3, %al
+; X64-NEXT:    cmpb $4, %al
 ; X64-NEXT:    movl $255, %eax
-; X64-NEXT:    cmovbel %ecx, %eax
+; X64-NEXT:    cmovbl %ecx, %eax
 ; X64-NEXT:    shrb $4, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    andb $15, %al
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %cl
 ; X86-NEXT:    movzbl %al, %edx
 ; X86-NEXT:    shlb $4, %cl
 ; X86-NEXT:    movzbl %cl, %eax
 ; X86-NEXT:    imull %edx, %eax
 ; X86-NEXT:    movb %ah, %cl
 ; X86-NEXT:    shlb $6, %cl
 ; X86-NEXT:    shrb $2, %al
 ; X86-NEXT:    orb %cl, %al
 ; X86-NEXT:    movzbl %al, %ecx
-; X86-NEXT:    cmpb $3, %ah
+; X86-NEXT:    cmpb $4, %ah
 ; X86-NEXT:    movl $255, %eax
-; X86-NEXT:    cmovbel %ecx, %eax
+; X86-NEXT:    cmovbl %ecx, %eax
 ; X86-NEXT:    shrb $4, %al
 ; X86-NEXT:    # kill: def $al killed $al killed $eax
 ; X86-NEXT:    retl
   %tmp = call i4 @llvm.umul.fix.sat.i4(i4 %x, i4 %y, i32 2)
   ret i4 %tmp
 }
 
 define <4 x i32> @vec(<4 x i32> %x, <4 x i32> %y) nounwind {
 ; X64-LABEL: vec:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[3,3,3,3]
 ; X64-NEXT:    movd %xmm2, %eax
 ; X64-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[3,3,3,3]
 ; X64-NEXT:    movd %xmm2, %ecx
 ; X64-NEXT:    imulq %rax, %rcx
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    shrq $32, %rax
 ; X64-NEXT:    shrdl $2, %eax, %ecx
-; X64-NEXT:    cmpl $3, %eax
+; X64-NEXT:    cmpl $4, %eax
 ; X64-NEXT:    movl $-1, %eax
-; X64-NEXT:    cmoval %eax, %ecx
+; X64-NEXT:    cmovael %eax, %ecx
 ; X64-NEXT:    movd %ecx, %xmm2
 ; X64-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[2,3,2,3]
 ; X64-NEXT:    movd %xmm3, %ecx
 ; X64-NEXT:    pshufd {{.*#+}} xmm3 = xmm0[2,3,2,3]
 ; X64-NEXT:    movd %xmm3, %edx
 ; X64-NEXT:    imulq %rcx, %rdx
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    shrq $32, %rcx
 ; X64-NEXT:    shrdl $2, %ecx, %edx
-; X64-NEXT:    cmpl $3, %ecx
-; X64-NEXT:    cmoval %eax, %edx
+; X64-NEXT:    cmpl $4, %ecx
+; X64-NEXT:    cmovael %eax, %edx
 ; X64-NEXT:    movd %edx, %xmm3
 ; X64-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm2[0],xmm3[1],xmm2[1]
 ; X64-NEXT:    movd %xmm1, %ecx
 ; X64-NEXT:    movd %xmm0, %edx
 ; X64-NEXT:    imulq %rcx, %rdx
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    shrq $32, %rcx
 ; X64-NEXT:    shrdl $2, %ecx, %edx
-; X64-NEXT:    cmpl $3, %ecx
-; X64-NEXT:    cmoval %eax, %edx
+; X64-NEXT:    cmpl $4, %ecx
+; X64-NEXT:    cmovael %eax, %edx
 ; X64-NEXT:    movd %edx, %xmm2
 ; X64-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X64-NEXT:    movd %xmm1, %ecx
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,1,1]
 ; X64-NEXT:    movd %xmm0, %edx
 ; X64-NEXT:    imulq %rcx, %rdx
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    shrq $32, %rcx
 ; X64-NEXT:    shrdl $2, %ecx, %edx
-; X64-NEXT:    cmpl $3, %ecx
-; X64-NEXT:    cmoval %eax, %edx
+; X64-NEXT:    cmpl $4, %ecx
+; X64-NEXT:    cmovael %eax, %edx
 ; X64-NEXT:    movd %edx, %xmm0
 ; X64-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
 ; X64-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm3[0]
 ; X64-NEXT:    movdqa %xmm2, %xmm0
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: vec:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    shrdl $2, %edx, %esi
-; X86-NEXT:    cmpl $3, %edx
+; X86-NEXT:    cmpl $4, %edx
 ; X86-NEXT:    movl $-1, %ecx
-; X86-NEXT:    cmoval %ecx, %esi
+; X86-NEXT:    cmovael %ecx, %esi
 ; X86-NEXT:    movl %ebp, %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %ebp
 ; X86-NEXT:    shrdl $2, %edx, %ebp
-; X86-NEXT:    cmpl $3, %edx
-; X86-NEXT:    cmoval %ecx, %ebp
+; X86-NEXT:    cmpl $4, %edx
+; X86-NEXT:    cmovael %ecx, %ebp
 ; X86-NEXT:    movl %ebx, %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl %eax, %ebx
 ; X86-NEXT:    shrdl $2, %edx, %ebx
-; X86-NEXT:    cmpl $3, %edx
-; X86-NEXT:    cmoval %ecx, %ebx
+; X86-NEXT:    cmpl $4, %edx
+; X86-NEXT:    cmovael %ecx, %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    shrdl $2, %edx, %eax
-; X86-NEXT:    cmpl $3, %edx
-; X86-NEXT:    cmoval %ecx, %eax
+; X86-NEXT:    cmpl $4, %edx
+; X86-NEXT:    cmovael %ecx, %eax
 ; X86-NEXT:    movl %eax, 12(%edi)
 ; X86-NEXT:    movl %ebx, 8(%edi)
 ; X86-NEXT:    movl %ebp, 4(%edi)
 ; X86-NEXT:    movl %esi, (%edi)
 ; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx

llvm/test/CodeGen/X86/urem-seteq-illegal-types.ll

 
 define i1 @test_urem_odd_setne(i4 %X) nounwind {
 ; X86-LABEL: test_urem_odd_setne:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    leal (%eax,%eax,2), %ecx
 ; X86-NEXT:    leal (%eax,%ecx,4), %eax
 ; X86-NEXT:    andb $15, %al
-; X86-NEXT:    cmpb $3, %al
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpb $4, %al
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_odd_setne:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    leal (%rdi,%rdi,2), %eax
 ; X64-NEXT:    leal (%rdi,%rax,4), %eax
 ; X64-NEXT:    andb $15, %al
-; X64-NEXT:    cmpb $3, %al
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpb $4, %al
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %urem = urem i4 %X, 5
   %cmp = icmp ne i4 %urem, 0
   ret i1 %cmp
 }
 
 define i1 @test_urem_negative_odd(i9 %X) nounwind {
 ; X86-LABEL: test_urem_negative_odd:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $307, {{[0-9]+}}(%esp), %eax # imm = 0x133
 ; X86-NEXT:    andl $511, %eax # imm = 0x1FF
-; X86-NEXT:    cmpw $1, %ax
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpw $2, %ax
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_negative_odd:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $307, %edi, %eax # imm = 0x133
 ; X64-NEXT:    andl $511, %eax # imm = 0x1FF
-; X64-NEXT:    cmpw $1, %ax
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpw $2, %ax
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %urem = urem i9 %X, -5
   %cmp = icmp ne i9 %urem, 0
   ret i1 %cmp
 }
 
 define <3 x i1> @test_urem_vec(<3 x i11> %X) nounwind {
 ; X86-LABEL: test_urem_vec:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $683, {{[0-9]+}}(%esp), %eax # imm = 0x2AB
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shll $10, %ecx
 ; X86-NEXT:    andl $2046, %eax # imm = 0x7FE
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    andl $2047, %eax # imm = 0x7FF
-; X86-NEXT:    cmpl $341, %eax # imm = 0x155
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $342, %eax # imm = 0x156
+; X86-NEXT:    setae %al
 ; X86-NEXT:    imull $1463, {{[0-9]+}}(%esp), %ecx # imm = 0x5B7
 ; X86-NEXT:    addl $-1463, %ecx # imm = 0xFA49
 ; X86-NEXT:    andl $2047, %ecx # imm = 0x7FF
-; X86-NEXT:    cmpl $292, %ecx # imm = 0x124
-; X86-NEXT:    seta %dl
+; X86-NEXT:    cmpl $293, %ecx # imm = 0x125
+; X86-NEXT:    setae %dl
 ; X86-NEXT:    imull $819, {{[0-9]+}}(%esp), %ecx # imm = 0x333
 ; X86-NEXT:    addl $-1638, %ecx # imm = 0xF99A
 ; X86-NEXT:    andl $2047, %ecx # imm = 0x7FF
-; X86-NEXT:    cmpw $1, %cx
-; X86-NEXT:    seta %cl
+; X86-NEXT:    cmpw $2, %cx
+; X86-NEXT:    setae %cl
 ; X86-NEXT:    retl
 ;
 ; SSE2-LABEL: test_urem_vec:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movd %esi, %xmm0
 ; SSE2-NEXT:    movd %edi, %xmm1
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
 ; SSE2-NEXT:    movd %edx, %xmm0

llvm/test/CodeGen/X86/urem-seteq.ll

 
 define i16 @test_urem_even(i16 %X) nounwind {
 ; X86-LABEL: test_urem_even:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $28087, {{[0-9]+}}(%esp), %eax # imm = 0x6DB7
 ; X86-NEXT:    rorw %ax
 ; X86-NEXT:    movzwl %ax, %ecx
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $4681, %ecx # imm = 0x1249
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $4682, %ecx # imm = 0x124A
+; X86-NEXT:    setae %al
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_even:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $28087, %edi, %eax # imm = 0x6DB7
 ; X64-NEXT:    rorw %ax
 ; X64-NEXT:    movzwl %ax, %ecx
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $4681, %ecx # imm = 0x1249
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $4682, %ecx # imm = 0x124A
+; X64-NEXT:    setae %al
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %urem = urem i16 %X, 14
   %cmp = icmp ne i16 %urem, 0
   %ret = zext i1 %cmp to i16
   ret i16 %ret
 }
 

 ;------------------------------------------------------------------------------;
 
 ; 'NE' predicate is fine too.
 define i32 @test_urem_odd_setne(i32 %X) nounwind {
 ; X86-LABEL: test_urem_odd_setne:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $-858993459, {{[0-9]+}}(%esp), %ecx # imm = 0xCCCCCCCD
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $858993460, %ecx # imm = 0x33333334
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_odd_setne:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $-858993459, %edi, %ecx # imm = 0xCCCCCCCD
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $858993459, %ecx # imm = 0x33333333
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $858993460, %ecx # imm = 0x33333334
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %urem = urem i32 %X, 5
   %cmp = icmp ne i32 %urem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 
 ; The fold is only valid for positive divisors, negative-ones should be negated.
 define i32 @test_urem_negative_odd(i32 %X) nounwind {
 ; X86-LABEL: test_urem_negative_odd:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $858993459, {{[0-9]+}}(%esp), %ecx # imm = 0x33333333
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $1, %ecx
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $2, %ecx
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_negative_odd:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $858993459, %edi, %ecx # imm = 0x33333333
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $1, %ecx
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $2, %ecx
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %urem = urem i32 %X, -5
   %cmp = icmp ne i32 %urem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 define i32 @test_urem_negative_even(i32 %X) nounwind {
 ; X86-LABEL: test_urem_negative_even:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $-920350135, {{[0-9]+}}(%esp), %ecx # imm = 0xC9249249
 ; X86-NEXT:    rorl %ecx
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    cmpl $1, %ecx
-; X86-NEXT:    seta %al
+; X86-NEXT:    cmpl $2, %ecx
+; X86-NEXT:    setae %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_negative_even:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $-920350135, %edi, %ecx # imm = 0xC9249249
 ; X64-NEXT:    rorl %ecx
 ; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    cmpl $1, %ecx
-; X64-NEXT:    seta %al
+; X64-NEXT:    cmpl $2, %ecx
+; X64-NEXT:    setae %al
 ; X64-NEXT:    retq
   %urem = urem i32 %X, -14
   %cmp = icmp ne i32 %urem, 0
   %ret = zext i1 %cmp to i32
   ret i32 %ret
 }
 
 ;------------------------------------------------------------------------------;

llvm/test/CodeGen/X86/vector-mulfix-legalize.ll

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pextrw $2, %xmm0, %eax
 ; CHECK-NEXT:    cwtl
 ; CHECK-NEXT:    leal (%rax,%rax,2), %ecx
 ; CHECK-NEXT:    movl %ecx, %edx
 ; CHECK-NEXT:    shrl $16, %edx
 ; CHECK-NEXT:    shldw $1, %cx, %dx
 ; CHECK-NEXT:    sarl $16, %ecx
-; CHECK-NEXT:    cmpl $16383, %ecx # imm = 0x3FFF
+; CHECK-NEXT:    cmpl $16384, %ecx # imm = 0x4000
 ; CHECK-NEXT:    movl $32767, %r8d # imm = 0x7FFF
-; CHECK-NEXT:    cmovgl %r8d, %edx
+; CHECK-NEXT:    cmovgel %r8d, %edx
 ; CHECK-NEXT:    cmpl $-16384, %ecx # imm = 0xC000
 ; CHECK-NEXT:    movl $32768, %ecx # imm = 0x8000
 ; CHECK-NEXT:    cmovll %ecx, %edx
 ; CHECK-NEXT:    pextrw $1, %xmm0, %esi
 ; CHECK-NEXT:    movswl %si, %edi
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    shrl $16, %eax
 ; CHECK-NEXT:    leal (%rdi,%rdi), %esi
 ; CHECK-NEXT:    shrdw $15, %ax, %si
 ; CHECK-NEXT:    sarl $15, %edi
-; CHECK-NEXT:    cmpl $16383, %edi # imm = 0x3FFF
-; CHECK-NEXT:    cmovgl %r8d, %esi
+; CHECK-NEXT:    cmpl $16384, %edi # imm = 0x4000
+; CHECK-NEXT:    cmovgel %r8d, %esi
 ; CHECK-NEXT:    cmpl $-16384, %edi # imm = 0xC000
 ; CHECK-NEXT:    cmovll %ecx, %esi
 ; CHECK-NEXT:    movd %xmm0, %eax
 ; CHECK-NEXT:    cwtl
 ; CHECK-NEXT:    movl %eax, %edi
 ; CHECK-NEXT:    shrl $16, %edi
 ; CHECK-NEXT:    shldw $1, %ax, %di
 ; CHECK-NEXT:    sarl $16, %eax
-; CHECK-NEXT:    cmpl $16383, %eax # imm = 0x3FFF
-; CHECK-NEXT:    cmovgl %r8d, %edi
+; CHECK-NEXT:    cmpl $16384, %eax # imm = 0x4000
+; CHECK-NEXT:    cmovgel %r8d, %edi
 ; CHECK-NEXT:    cmpl $-16384, %eax # imm = 0xC000
 ; CHECK-NEXT:    cmovll %ecx, %edi
 ; CHECK-NEXT:    movzwl %di, %eax
 ; CHECK-NEXT:    movd %eax, %xmm1
 ; CHECK-NEXT:    pinsrw $1, %esi, %xmm1
 ; CHECK-NEXT:    pinsrw $2, %edx, %xmm1
 ; CHECK-NEXT:    pextrw $3, %xmm0, %eax
 ; CHECK-NEXT:    cwtl
 ; CHECK-NEXT:    movl %eax, %edx
 ; CHECK-NEXT:    shrl $14, %edx
 ; CHECK-NEXT:    leal (,%rax,4), %esi
 ; CHECK-NEXT:    shrdw $15, %dx, %si
 ; CHECK-NEXT:    sarl $14, %eax
-; CHECK-NEXT:    cmpl $16383, %eax # imm = 0x3FFF
-; CHECK-NEXT:    cmovgl %r8d, %esi
+; CHECK-NEXT:    cmpl $16384, %eax # imm = 0x4000
+; CHECK-NEXT:    cmovgel %r8d, %esi
 ; CHECK-NEXT:    cmpl $-16384, %eax # imm = 0xC000
 ; CHECK-NEXT:    cmovll %ecx, %esi
 ; CHECK-NEXT:    pinsrw $3, %esi, %xmm1
 ; CHECK-NEXT:    movdqa %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %t = call <4 x i16> @llvm.smul.fix.sat.v4i16(<4 x i16> <i16 1, i16 2, i16 3, i16 4>, <4 x i16> %a, i32 15)
   ret <4 x i16> %t
 }
 
 
 define <4 x i16> @umulfixsat(<4 x i16> %a) {
 ; CHECK-LABEL: umulfixsat:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pextrw $2, %xmm0, %eax
 ; CHECK-NEXT:    leal (%rax,%rax,2), %eax
 ; CHECK-NEXT:    movl %eax, %edx
 ; CHECK-NEXT:    shrl $16, %edx
 ; CHECK-NEXT:    movl %edx, %ecx
 ; CHECK-NEXT:    shldw $1, %ax, %cx
-; CHECK-NEXT:    cmpl $32767, %edx # imm = 0x7FFF
+; CHECK-NEXT:    cmpl $32768, %edx # imm = 0x8000
 ; CHECK-NEXT:    movl $65535, %eax # imm = 0xFFFF
-; CHECK-NEXT:    cmoval %eax, %ecx
+; CHECK-NEXT:    cmovael %eax, %ecx
 ; CHECK-NEXT:    pextrw $1, %xmm0, %edx
 ; CHECK-NEXT:    addl %edx, %edx
 ; CHECK-NEXT:    movl %edx, %esi
 ; CHECK-NEXT:    shrl $16, %esi
 ; CHECK-NEXT:    movl %esi, %edi
 ; CHECK-NEXT:    shldw $1, %dx, %di
-; CHECK-NEXT:    cmpl $32767, %esi # imm = 0x7FFF
-; CHECK-NEXT:    cmoval %eax, %edi
+; CHECK-NEXT:    cmpl $32768, %esi # imm = 0x8000
+; CHECK-NEXT:    cmovael %eax, %edi
 ; CHECK-NEXT:    movd %xmm0, %edx
 ; CHECK-NEXT:    xorl %esi, %esi
 ; CHECK-NEXT:    shldw $1, %dx, %si
-; CHECK-NEXT:    movl $32767, %edx # imm = 0x7FFF
+; CHECK-NEXT:    movl $32768, %edx # imm = 0x8000
 ; CHECK-NEXT:    negl %edx
-; CHECK-NEXT:    cmoval %eax, %esi
+; CHECK-NEXT:    cmovael %eax, %esi
 ; CHECK-NEXT:    movzwl %si, %edx
 ; CHECK-NEXT:    movd %edx, %xmm1
 ; CHECK-NEXT:    pinsrw $1, %edi, %xmm1
 ; CHECK-NEXT:    pinsrw $2, %ecx, %xmm1
 ; CHECK-NEXT:    pextrw $3, %xmm0, %ecx
 ; CHECK-NEXT:    shll $2, %ecx
 ; CHECK-NEXT:    movl %ecx, %edx
 ; CHECK-NEXT:    shrl $16, %edx
 ; CHECK-NEXT:    movl %edx, %esi
 ; CHECK-NEXT:    shldw $1, %cx, %si
-; CHECK-NEXT:    cmpl $32767, %edx # imm = 0x7FFF
-; CHECK-NEXT:    cmoval %eax, %esi
+; CHECK-NEXT:    cmpl $32768, %edx # imm = 0x8000
+; CHECK-NEXT:    cmovael %eax, %esi
 ; CHECK-NEXT:    pinsrw $3, %esi, %xmm1
 ; CHECK-NEXT:    movdqa %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %t = call <4 x i16> @llvm.umul.fix.sat.v4i16(<4 x i16> <i16 1, i16 2, i16 3, i16 4>, <4 x i16> %a, i32 15)
   ret <4 x i16> %t
 }

llvm/test/CodeGen/X86/zext-sext.ll

 ; CHECK-NEXT:    movswl (%rax,%rsi,2), %eax
 ; CHECK-NEXT:    movl $1, %esi
 ; CHECK-NEXT:    imull %edx, %eax
 ; CHECK-NEXT:    xorl %edx, %edx
 ; CHECK-NEXT:    addl $2138875574, %eax # imm = 0x7F7CA6B6
 ; CHECK-NEXT:    cmpl $-8608074, %eax # imm = 0xFF7CA6B6
 ; CHECK-NEXT:    movslq %eax, %rdi
 ; CHECK-NEXT:    setl %dl
-; CHECK-NEXT:    cmpl $2138875573, %eax # imm = 0x7F7CA6B5
+; CHECK-NEXT:    cmpl $2138875574, %eax # imm = 0x7F7CA6B6
 ; CHECK-NEXT:    movq %rdi, %r8
 ; CHECK-NEXT:    leal -1(%rdx,%rdx), %edx
-; CHECK-NEXT:    cmovlel %edx, %esi
+; CHECK-NEXT:    cmovll %edx, %esi
 ; CHECK-NEXT:    subq %rax, %r8
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    cmpl $1, %esi
 ; CHECK-NEXT:    cmovneq %rax, %r8
 ; CHECK-NEXT:    testl %edi, %edi
 ; CHECK-NEXT:    cmovnsq %rax, %r8
 ; CHECK-NEXT:    movq (%rcx), %rax
 ; CHECK-NEXT:    subq %r8, %rdi