[DAGCombine][X86][AArch64][ARM] (C - x) + y -> (y - x) + C fold

[lebedev.ri]

Summary

All changes except ARM look great.
https://rise4fun.com/Alive/R2M

The regression test/CodeGen/ARM/addsubcarry-promotion.ll
is recovered fully by D62392 + D62450.

[efriedma]

So the DAG is simpler, but that particular case is worse for the backend.

It's "simpler", in some sense, but it's more complicated to lower because it's using the carry output of a uaddo in a non-addcarry operation, which is legal, but more expensive than using in an addcarry, particularly on ARM. I think the missing combine is that we should be able to turn the subtraction into some form of addcarry/subcarry.

[lebedev.ri]

In D62266#1512613, @efriedma wrote:

So the DAG is simpler, but that particular case is worse for the backend.

It's "simpler", in some sense, but it's more complicated to lower because it's using the carry output of a uaddo in a non-addcarry operation, which is legal, but more expensive than using in an addcarry, particularly on ARM. I think the missing combine is that we should be able to turn the subtraction into some form of addcarry/subcarry.

Hmm, that does not seem to be all.
I've added

bool HasSUBCARRY = TLI.isOperationLegalOrCustom(ISD::SUBCARRY, VT);
bool HasADDCARRY = TLI.isOperationLegalOrCustom(ISD::ADDCARRY, VT);
if (HasSUBCARRY || HasADDCARRY) {
  unsigned ANYCARRY = HasSUBCARRY ? ISD::SUBCARRY : ISD::ADDCARRY;
  // (sub X, Carry) -> (subcarry/addcarry X, 0, Carry)
  if (SDValue Carry = getAsCarry(TLI, N1)) {
    return DAG.getNode(ANYCARRY, DL, DAG.getVTList(VT, Carry.getValueType()),
                       N0, DAG.getConstant(0, DL, VT), Carry);
  }

  if (HasSUBCARRY) {
    // (sub Carry, X) -> (subcarry 0, X, Carry)
    if (SDValue Carry = getAsCarry(TLI, N0)) {
      return DAG.getNode(ISD::SUBCARRY, DL,
                         DAG.getVTList(VT, Carry.getValueType()),
                         DAG.getConstant(0, DL, VT), N1, Carry);
    }
  }
}

at the end of DAGCombiner::visitSUB(), and get this

SelectionDAG has 17 nodes:
  t0: ch = EntryToken
          t6: i32,ch = CopyFromReg t0, Register:i32 %2
            t4: i32,ch = CopyFromReg t0, Register:i32 %1
            t2: i32,ch = CopyFromReg t0, Register:i32 %0
          t30: i32,i32 = uaddo t4, t2
        t31: i32,i32 = subcarry Constant:i32<0>, t6, t30:1
      t39: i32 = and t31, Constant:i32<65535>
    t40: ch = br_cc t0, seteq:ch, t39, Constant:i32<65535>, BasicBlock:ch<if.end 0x55706fb58828>
  t19: ch = br t40, BasicBlock:ch<for.cond.preheader 0x55706fb58698>

which is then resulting in

SelectionDAG has 22 nodes:
  t0: ch = EntryToken
            t6: i32,ch = CopyFromReg t0, Register:i32 %2
                    t4: i32,ch = CopyFromReg t0, Register:i32 %1
                    t2: i32,ch = CopyFromReg t0, Register:i32 %0
                  t51: i32,i32 = ARMISD::ADDC t4, t2
                t52: i32,i32 = ARMISD::ADDE Constant:i32<0>, Constant:i32<0>, t51:1
              t45: i32 = sub Constant:i32<1>, t52
            t46: i32,i32 = ARMISD::SUBC t45, Constant:i32<1>
          t47: i32,i32 = ARMISD::SUBE Constant:i32<0>, t6, t46:1
        t39: i32 = and t47, Constant:i32<65535>
      t41: glue = ARMISD::CMPZ t39, Constant:i32<65535>
    t43: ch = ARMISD::BRCOND t0, BasicBlock:ch<if.end 0x55706fb58828>, Constant:i32<0>, Register:i32 $cpsr, t41
  t19: ch = br t43, BasicBlock:ch<for.cond.preheader 0x55706fb58698>

So some other ARM-specific combine is missing, too?
Or am i totally misreading this?

[efriedma]

uaddo+subcarry doesn't really work the way you'd want it to on ARM... I think you have to invert the carry bit, and there isn't any convenient way to do that. (Note the extra "SUB" between the ARMISD::ADDE and the ARMISD::SUBC.) So I think to get the optimal code here you have to produce neg+addcarry instead of subcarry.

[lebedev.ri]

In D62266#1512853, @efriedma wrote:

uaddo+subcarry doesn't really work the way you'd want it to on ARM... I think you have to invert the carry bit, and there isn't any convenient way to do that. (Note the extra "SUB" between the ARMISD::ADDE and the ARMISD::SUBC.) So I think to get the optimal code here you have to produce neg+addcarry instead of subcarry.

Hmm. I'm sorry, that did not make the problem any clearer to me.
Sadly, i have literally never encountered 'carry' before, so this makes very little sense to me.
Are you saying that the following transform should be performed: (sub Carry, X) -> (addcarry (sub 0, X), 0, !Carry)?
That regresses the test even further..

[efriedma]

Are you saying that the following transform should be performed: (sub Carry, X) -> (addcarry (sub 0, X), 0, !Carry)?

The correct transform is (sub Carry, X) -> (add Carry, (sub 0, X)) -> (addcarry (sub 0, X), 0, Carry), no "!".

Sorry, I wasn't really clear; the carry bit doesn't need to be inverted in target-independent code. It's just that on ARM, "subs"/"sbcs" produce a carry bit which has the opposite meaning of the "overflow" bit from usubo, and "sbc(s)" consumes a carry bit that's inverted from the carry bit operand of subcarry. If the result of a usubo is consumed by a subcarry, the two inversions cancel; otherwise, we need to transfer the the carry bit to a GPR, invert it, then transfer it back to the flags register. (Maybe this could be optimized a bit in certain cases, but it doesn't really optimize in general.)

[lebedev.ri]

In D62266#1514711, @efriedma wrote:

Are you saying that the following transform should be performed: (sub Carry, X) -> (addcarry (sub 0, X), 0, !Carry)?

The correct transform is (sub Carry, X) -> (add Carry, (sub 0, X)) -> (addcarry (sub 0, X), 0, Carry), no "!".

Oh right, i think i may have mistyped here. D62392

Sorry, I wasn't really clear; the carry bit doesn't need to be inverted in target-independent code. It's just that on ARM, "subs"/"sbcs" produce a carry bit which has the opposite meaning of the "overflow" bit from usubo, and "sbc(s)" consumes a carry bit that's inverted from the carry bit operand of subcarry. If the result of a usubo is consumed by a subcarry, the two inversions cancel; otherwise, we need to transfer the the carry bit to a GPR, invert it, then transfer it back to the flags register. (Maybe this could be optimized a bit in certain cases, but it doesn't really optimize in general.)

Thanks! That helps improves half of the cases (although still regressed as compared to LHS of this diff).

[lebedev.ri]

This comment was removed by lebedev.ri.

[efriedma]

test/CodeGen/ARM/addsubcarry-promotion.ll

10 ↗

(On Diff #200806)

It looks like this test didn't get rebased correctly?

35 ↗

(On Diff #200806)

The big difference that's making the new code worse, now, is that somehow the compare x==0 is getting transformed to something more like x==-1... and I guess something isn't handling that well. That's probably something the ARM backend should be handling in target-specific code, though; feel free to just file a bug for the missed optimization on void a(int s, void f()) { if ((short)s==-1)f(); }

[lebedev.ri]

test/CodeGen/ARM/addsubcarry-promotion.ll

10 ↗

(On Diff #200806)

No, D62392 is a child revision of this.

35 ↗

(On Diff #200806)

I understand how we get here. In IR terms, this is just two folds:
https://rise4fun.com/Alive/fTpN
https://godbolt.org/z/6RLVTQ

And thus we end with:

Optimized type-legalized selection DAG: %bb.0 'fn1:entry'
SelectionDAG has 20 nodes:
  t0: ch = EntryToken
              t6: i32,ch = CopyFromReg t0, Register:i32 %2
            t37: i32 = sub Constant:i32<0>, t6
          t39: i32 = sign_extend_inreg t37, ValueType:ch:i16
            t4: i32,ch = CopyFromReg t0, Register:i32 %1
            t2: i32,ch = CopyFromReg t0, Register:i32 %0
          t36: i32,i32 = uaddo t4, t2
        t47: i32,i32 = addcarry t39, Constant:i32<0>, t36:1
      t46: i32 = and t47, Constant:i32<65535>
    t50: ch = br_cc t0, seteq:ch, t46, Constant:i32<65535>, BasicBlock:ch<if.end 0x5618ef707f18>
  t19: ch = br t50, BasicBlock:ch<for.cond.preheader 0x5618ef707d88>

I wonder, maybe this can still be undone in DAGCombine.
The important bits are:

    t47: i32,i32 = addcarry t39, Constant:i32<0>, t36:1
  t46: i32 = and t47, Constant:i32<65535>
t50: ch = br_cc t0, seteq:ch, t46, Constant:i32<65535>, BasicBlock:ch<if.end 0x5618ef707f18>

I wonder if the problem can be fixed by transforming this back into:

  t47: i32,i32 = addcarry t39, Constant:i32<65535>, t36:1
t50: ch = br_cc t0, seteq:ch, t47, Constant:i32<0>, BasicBlock:ch<if.end 0x5618ef707f18>

[lebedev.ri]

test/CodeGen/ARM/addsubcarry-promotion.ll

35 ↗

(On Diff #200806)

Or more specifically, https://rise4fun.com/Alive/slGX
This would be an ok transform here because the second argument of addcarry is 0 here anyway.

[lebedev.ri]

test/CodeGen/ARM/addsubcarry-promotion.ll

35 ↗

(On Diff #200806)

Or without hardcoded constants: https://rise4fun.com/Alive/B2k
Not sure if it's worthwhile doing this if it requires creating whole new 'add',
but here we should be able to do it.

[lebedev.ri]

test/CodeGen/ARM/addsubcarry-promotion.ll

35 ↗

(On Diff #200806)

Done, D62450.
I think that fixes (and improves upon?) the last regression, feel free to review these three patches :)

[lebedev.ri]

test/CodeGen/ARM/addsubcarry-promotion.ll

35 ↗

(On Diff #200806)

ping @efriedma :)

[dmgreen]

Hello. I found this code that seems to loop forever:

target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "thumbv6m-arm-none-eabi"

@a = dso_local local_unnamed_addr global i32 0, align 4
@b = dso_local local_unnamed_addr global i32 0, align 4

; Function Attrs: minsize nounwind optsize
define dso_local i32 @c() local_unnamed_addr #0 {
entry:
  %0 = load i32, i32* @a, align 4
  %sub = sub nsw i32 2000, %0
  %call = tail call i32 bitcast (i32 (...)* @d to i32 (i32)*)(i32 %sub) #2
  %1 = load i32, i32* @b, align 4
  %cmp = icmp sgt i32 %1, 1999
  br i1 %cmp, label %if.then, label %if.end

if.then:                                          ; preds = %entry
  %call1 = tail call i32 bitcast (i32 (...)* @e to i32 ()*)() #2
  br label %if.end

if.end:                                           ; preds = %if.then, %entry
  ret i32 undef
}

declare dso_local i32 @d(...) local_unnamed_addr #1
declare dso_local i32 @e(...) local_unnamed_addr #1

[lebedev.ri]

In D62266#1521995, @dmgreen wrote:

Hello. I found this code that seems to loop forever:

*This* code? As in *this* patch, correct?

target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "thumbv6m-arm-none-eabi"

@a = dso_local local_unnamed_addr global i32 0, align 4
@b = dso_local local_unnamed_addr global i32 0, align 4

; Function Attrs: minsize nounwind optsize
define dso_local i32 @c() local_unnamed_addr #0 {
entry:
  %0 = load i32, i32* @a, align 4
  %sub = sub nsw i32 2000, %0
  %call = tail call i32 bitcast (i32 (...)* @d to i32 (i32)*)(i32 %sub) #2
  %1 = load i32, i32* @b, align 4
  %cmp = icmp sgt i32 %1, 1999
  br i1 %cmp, label %if.then, label %if.end

if.then:                                          ; preds = %entry
  %call1 = tail call i32 bitcast (i32 (...)* @e to i32 ()*)() #2
  br label %if.end

if.end:                                           ; preds = %if.then, %entry
  ret i32 undef
}

declare dso_local i32 @d(...) local_unnamed_addr #1
declare dso_local i32 @e(...) local_unnamed_addr #1

Thanks for the reproducer.
There may be one more (as in, an unrelated to this one) endless loop in D62257 in test-suite, did not reduce it yet (i reverted that patch)

[dmgreen]

*This* code? As in *this* patch, correct?

Hello. I meant this .ll file seems to loop forever. With this patch or one of the three others (D62257, D62392, D62450). I only tested them together, so it may be the same as the problem from the test-suite.

[lebedev.ri]

In D62266#1522025, @dmgreen wrote:

*This* code? As in *this* patch, correct?

Hello. I meant this .ll file seems to loop forever. With this patch or one of the three others (D62257, D62392, D62450). I only tested them together, so it may be the same as the problem from the test-suite.

Confirmed, thank you for the reproducer.
This is another hang from what i have seen with D62257.

[lebedev.ri]

ping @efriedma.

Rebased, fixed endless combine loop (by not hoisting if constant is opaque, since we don't constant-fold those).

[efriedma]

LGTM. I'm okay with working on adding more test coverage and fixes for the ADDCARRY cases in a followup.

[lebedev.ri]

Rebased, NFC.

llvm/trunk/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

SDValue DAGCombiner::visitADDLikeCommutative(SDValue N0, SDValue N1,
   // Hoist one-use subtraction by non-opaque constant:
   //   (x - C) + y  ->  (x + y) - C
   // This is necessary because SUB(X,C) -> ADD(X,-C) doesn't work for vectors.
   if (N0.hasOneUse() && N0.getOpcode() == ISD::SUB &&
       isConstantOrConstantVector(N0.getOperand(1), /*NoOpaques=*/true)) {
     SDValue Add = DAG.getNode(ISD::ADD, DL, VT, N0.getOperand(0), N1);
     return DAG.getNode(ISD::SUB, DL, VT, Add, N0.getOperand(1));
   }
+  // Hoist one-use subtraction from non-opaque constant:
+  //   (C - x) + y  ->  (y - x) + C
+  if (N0.hasOneUse() && N0.getOpcode() == ISD::SUB &&
+      isConstantOrConstantVector(N0.getOperand(0), /*NoOpaques=*/true)) {
+    SDValue Sub = DAG.getNode(ISD::SUB, DL, VT, N1, N0.getOperand(1));
+    return DAG.getNode(ISD::ADD, DL, VT, Sub, N0.getOperand(0));
+  }
 
   // If the target's bool is represented as 0/1, prefer to make this 'sub 0/1'
   // rather than 'add 0/-1' (the zext should get folded).
   // add (sext i1 Y), X --> sub X, (zext i1 Y)
   if (N0.getOpcode() == ISD::SIGN_EXTEND &&
       N0.getOperand(0).getScalarValueSizeInBits() == 1 &&
       TLI.getBooleanContents(VT) == TargetLowering::ZeroOrOneBooleanContent) {
     SDValue ZExt = DAG.getNode(ISD::ZERO_EXTEND, DL, VT, N0.getOperand(0));

llvm/trunk/test/CodeGen/AArch64/shift-amount-mod.ll

 
 ;==============================================================================;
 ; add to negated shift amount
 ;
 
 define i32 @reg32_lshr_by_add_to_negated(i32 %val, i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: reg32_lshr_by_add_to_negated:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mov w8, #32
-; CHECK-NEXT:    sub w8, w8, w1
-; CHECK-NEXT:    add w8, w8, w2
+; CHECK-NEXT:    sub w8, w2, w1
 ; CHECK-NEXT:    lsr w0, w0, w8
 ; CHECK-NEXT:    ret
   %nega = sub i32 32, %a
   %negasubb = add i32 %nega, %b
   %shifted = lshr i32 %val, %negasubb
   ret i32 %shifted
 }
 define i64 @reg64_lshr_by_add_to_negated(i64 %val, i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: reg64_lshr_by_add_to_negated:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mov w8, #64
-; CHECK-NEXT:    sub x8, x8, x1
-; CHECK-NEXT:    add x8, x8, x2
+; CHECK-NEXT:    sub x8, x2, x1
 ; CHECK-NEXT:    lsr x0, x0, x8
 ; CHECK-NEXT:    ret
   %nega = sub i64 64, %a
   %negasubb = add i64 %nega, %b
   %shifted = lshr i64 %val, %negasubb
   ret i64 %shifted
 }
 

llvm/trunk/test/CodeGen/AArch64/sink-addsub-of-const.ll

 }
 
 ; add (sub C, %x), %y
 ; Outer 'add' is commutative - 2 variants.
 
 define i32 @sink_sub_from_const_to_add0(i32 %a, i32 %b) {
 ; CHECK-LABEL: sink_sub_from_const_to_add0:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mov w8, #32
-; CHECK-NEXT:    sub w8, w8, w0
-; CHECK-NEXT:    add w0, w8, w1
+; CHECK-NEXT:    sub w8, w1, w0
+; CHECK-NEXT:    add w0, w8, #32 // =32
 ; CHECK-NEXT:    ret
   %t0 = sub i32 32, %a
   %r = add i32 %t0, %b
   ret i32 %r
 }
 define i32 @sink_sub_from_const_to_add1(i32 %a, i32 %b) {
 ; CHECK-LABEL: sink_sub_from_const_to_add1:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mov w8, #32
-; CHECK-NEXT:    sub w8, w8, w0
-; CHECK-NEXT:    add w0, w1, w8
+; CHECK-NEXT:    sub w8, w1, w0
+; CHECK-NEXT:    add w0, w8, #32 // =32
 ; CHECK-NEXT:    ret
   %t0 = sub i32 32, %a
   %r = add i32 %b, %t0
   ret i32 %r
 }
 
 ; sub (add %x, C), %y
 ; sub %y, (add %x, C)

 ; CHECK-NEXT:    ret
   %t0 = sub i32 %a, 32
   %r = sub i32 %t0, %b
   ret i32 %r
 }
 define i32 @sink_sub_of_const_to_sub2(i32 %a, i32 %b) {
 ; CHECK-LABEL: sink_sub_of_const_to_sub2:
 ; CHECK:       // %bb.0:
-; CHECK-NEXT:    mov w8, #32
-; CHECK-NEXT:    sub w8, w8, w0
-; CHECK-NEXT:    add w0, w1, w8
+; CHECK-NEXT:    sub w8, w1, w0
+; CHECK-NEXT:    add w0, w8, #32 // =32
 ; CHECK-NEXT:    ret
   %t0 = sub i32 %a, 32
   %r = sub i32 %b, %t0
   ret i32 %r
 }
 
 ; sub (sub C, %x), %y
 ; sub %y, (sub C, %x)

 ; add (sub C, %x), %y
 ; Outer 'add' is commutative - 2 variants.
 
 define <4 x i32> @vec_sink_sub_from_const_to_add0(<4 x i32> %a, <4 x i32> %b) {
 ; CHECK-LABEL: vec_sink_sub_from_const_to_add0:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x8, .LCPI16_0
 ; CHECK-NEXT:    ldr q2, [x8, :lo12:.LCPI16_0]
-; CHECK-NEXT:    sub v0.4s, v2.4s, v0.4s
-; CHECK-NEXT:    add v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    sub v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    add v0.4s, v0.4s, v2.4s
 ; CHECK-NEXT:    ret
   %t0 = sub <4 x i32> <i32 42, i32 24, i32 undef, i32 46>, %a
   %r = add <4 x i32> %t0, %b
   ret <4 x i32> %r
 }
 define <4 x i32> @vec_sink_sub_from_const_to_add1(<4 x i32> %a, <4 x i32> %b) {
 ; CHECK-LABEL: vec_sink_sub_from_const_to_add1:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x8, .LCPI17_0
 ; CHECK-NEXT:    ldr q2, [x8, :lo12:.LCPI17_0]
-; CHECK-NEXT:    sub v0.4s, v2.4s, v0.4s
-; CHECK-NEXT:    add v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    sub v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    add v0.4s, v0.4s, v2.4s
 ; CHECK-NEXT:    ret
   %t0 = sub <4 x i32> <i32 42, i32 24, i32 undef, i32 46>, %a
   %r = add <4 x i32> %b, %t0
   ret <4 x i32> %r
 }
 
 ; sub (add %x, C), %y
 ; sub %y, (add %x, C)
; CHECK-NEXT:    ret
   %r = sub <4 x i32> %t0, %b
   ret <4 x i32> %r
 }
 define <4 x i32> @vec_sink_sub_of_const_to_sub2(<4 x i32> %a, <4 x i32> %b) {
 ; CHECK-LABEL: vec_sink_sub_of_const_to_sub2:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    adrp x8, .LCPI21_0
 ; CHECK-NEXT:    ldr q2, [x8, :lo12:.LCPI21_0]
-; CHECK-NEXT:    sub v0.4s, v2.4s, v0.4s
-; CHECK-NEXT:    add v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    sub v0.4s, v1.4s, v0.4s
+; CHECK-NEXT:    add v0.4s, v0.4s, v2.4s
 ; CHECK-NEXT:    ret
   %t0 = sub <4 x i32> %a, <i32 42, i32 24, i32 undef, i32 46>
   %r = sub <4 x i32> %b, %t0
   ret <4 x i32> %r
 }
 
 ; sub (sub C, %x), %y
 ; sub %y, (sub C, %x)

llvm/trunk/test/CodeGen/ARM/addsubcarry-promotion.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O2 -mtriple armv7a < %s | FileCheck --check-prefixes=ARM,ARMV7A %s
 
 ; RUN: llc -O2 -mtriple thumbv6m < %s | FileCheck --check-prefixes=THUMB1,THUMBV6M %s
 ; RUN: llc -O2 -mtriple thumbv8m.base < %s | FileCheck --check-prefixes=THUMB1,THUMBV8M-BASE %s
 
 ; RUN: llc -O2 -mtriple thumbv7a < %s | FileCheck --check-prefixes=THUMB,THUMBV7A %s
 ; RUN: llc -O2 -mtriple thumbv8m.main < %s | FileCheck --check-prefixes=THUMB,THUMBV8M-MAIN %s
 
 define void @fn1(i32 %a, i32 %b, i32 %c) local_unnamed_addr #0 {
 ; ARM-LABEL: fn1:
 ; ARM:       @ %bb.0: @ %entry
-; ARM-NEXT:    rsb r2, r2, #1
 ; ARM-NEXT:    adds r0, r1, r0
+; ARM-NEXT:    mov r3, #0
+; ARM-NEXT:    adc r0, r3, #0
 ; ARM-NEXT:    movw r1, #65535
-; ARM-NEXT:    sxth r2, r2
-; ARM-NEXT:    adc r0, r2, #0
-; ARM-NEXT:    tst r0, r1
+; ARM-NEXT:    sub r0, r0, r2
+; ARM-NEXT:    uxth r0, r0
+; ARM-NEXT:    cmp r0, r1
 ; ARM-NEXT:    bxeq lr
 ; ARM-NEXT:  .LBB0_1: @ %for.cond
 ; ARM-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; ARM-NEXT:    b .LBB0_1
 ;
-; THUMB1-LABEL: fn1:
-; THUMB1:       @ %bb.0: @ %entry
-; THUMB1-NEXT:    movs r3, #1
-; THUMB1-NEXT:    subs r2, r3, r2
-; THUMB1-NEXT:    sxth r2, r2
-; THUMB1-NEXT:    movs r3, #0
-; THUMB1-NEXT:    adds r0, r1, r0
-; THUMB1-NEXT:    adcs r3, r2
-; THUMB1-NEXT:    lsls r0, r3, #16
-; THUMB1-NEXT:    beq .LBB0_2
-; THUMB1-NEXT:  .LBB0_1: @ %for.cond
-; THUMB1-NEXT:    @ =>This Inner Loop Header: Depth=1
-; THUMB1-NEXT:    b .LBB0_1
-; THUMB1-NEXT:  .LBB0_2: @ %if.end
-; THUMB1-NEXT:    bx lr
+; THUMBV6M-LABEL: fn1:
+; THUMBV6M:       @ %bb.0: @ %entry
+; THUMBV6M-NEXT:    movs r3, #0
+; THUMBV6M-NEXT:    adds r0, r1, r0
+; THUMBV6M-NEXT:    adcs r3, r3
+; THUMBV6M-NEXT:    subs r0, r3, r2
+; THUMBV6M-NEXT:    uxth r0, r0
+; THUMBV6M-NEXT:    ldr r1, .LCPI0_0
+; THUMBV6M-NEXT:    cmp r0, r1
+; THUMBV6M-NEXT:    beq .LBB0_2
+; THUMBV6M-NEXT:  .LBB0_1: @ %for.cond
+; THUMBV6M-NEXT:    @ =>This Inner Loop Header: Depth=1
+; THUMBV6M-NEXT:    b .LBB0_1
+; THUMBV6M-NEXT:  .LBB0_2: @ %if.end
+; THUMBV6M-NEXT:    bx lr
+; THUMBV6M-NEXT:    .p2align 2
+; THUMBV6M-NEXT:  @ %bb.3:
+; THUMBV6M-NEXT:  .LCPI0_0:
+; THUMBV6M-NEXT:    .long 65535 @ 0xffff
+;
+; THUMBV8M-BASE-LABEL: fn1:
+; THUMBV8M-BASE:       @ %bb.0: @ %entry
+; THUMBV8M-BASE-NEXT:    movs r3, #0
+; THUMBV8M-BASE-NEXT:    adds r0, r1, r0
+; THUMBV8M-BASE-NEXT:    adcs r3, r3
+; THUMBV8M-BASE-NEXT:    subs r0, r3, r2
+; THUMBV8M-BASE-NEXT:    uxth r0, r0
+; THUMBV8M-BASE-NEXT:    movw r1, #65535
+; THUMBV8M-BASE-NEXT:    cmp r0, r1
+; THUMBV8M-BASE-NEXT:    beq .LBB0_2
+; THUMBV8M-BASE-NEXT:  .LBB0_1: @ %for.cond
+; THUMBV8M-BASE-NEXT:    @ =>This Inner Loop Header: Depth=1
+; THUMBV8M-BASE-NEXT:    b .LBB0_1
+; THUMBV8M-BASE-NEXT:  .LBB0_2: @ %if.end
+; THUMBV8M-BASE-NEXT:    bx lr
 ;
 ; THUMB-LABEL: fn1:
 ; THUMB:       @ %bb.0: @ %entry
-; THUMB-NEXT:    rsb.w r2, r2, #1
 ; THUMB-NEXT:    adds r0, r0, r1
-; THUMB-NEXT:    sxth r2, r2
-; THUMB-NEXT:    adc r0, r2, #0
-; THUMB-NEXT:    lsls r0, r0, #16
+; THUMB-NEXT:    mov.w r3, #0
+; THUMB-NEXT:    adc r0, r3, #0
+; THUMB-NEXT:    movw r1, #65535
+; THUMB-NEXT:    subs r0, r0, r2
+; THUMB-NEXT:    uxth r0, r0
+; THUMB-NEXT:    cmp r0, r1
 ; THUMB-NEXT:    it eq
 ; THUMB-NEXT:    bxeq lr
 ; THUMB-NEXT:  .LBB0_1: @ %for.cond
 ; THUMB-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; THUMB-NEXT:    b .LBB0_1
 entry:
   %add = add i32 %b, %a
   %cmp = icmp ult i32 %add, %b

llvm/trunk/test/CodeGen/X86/shift-amount-mod.ll

 ;==============================================================================;
 ; add to negated shift amount
 ;
 
 define i32 @reg32_lshr_by_add_to_negated(i32 %val, i32 %a, i32 %b) nounwind {
 ; X32-LABEL: reg32_lshr_by_add_to_negated:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X32-NEXT:    movl $32, %ecx
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %ecx
-; X32-NEXT:    addl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    # kill: def $cl killed $cl killed $ecx
 ; X32-NEXT:    shrl %cl, %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: reg32_lshr_by_add_to_negated:
 ; X64:       # %bb.0:
+; X64-NEXT:    movl %edx, %ecx
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    movl $32, %ecx
 ; X64-NEXT:    subl %esi, %ecx
-; X64-NEXT:    addl %edx, %ecx
 ; X64-NEXT:    # kill: def $cl killed $cl killed $ecx
 ; X64-NEXT:    shrl %cl, %eax
 ; X64-NEXT:    retq
   %nega = sub i32 32, %a
   %negasubb = add i32 %nega, %b
   %shifted = lshr i32 %val, %negasubb
   ret i32 %shifted
 }
 define i64 @reg64_lshr_by_add_to_negated(i64 %val, i64 %a, i64 %b) nounwind {
 ; X32-LABEL: reg64_lshr_by_add_to_negated:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushl %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X32-NEXT:    movl $64, %ecx
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %ecx
-; X32-NEXT:    addl {{[0-9]+}}(%esp), %ecx
+; X32-NEXT:    addb $64, %cl
 ; X32-NEXT:    movl %esi, %edx
 ; X32-NEXT:    shrl %cl, %edx
 ; X32-NEXT:    shrdl %cl, %esi, %eax
 ; X32-NEXT:    testb $32, %cl
 ; X32-NEXT:    je .LBB29_2
 ; X32-NEXT:  # %bb.1:
 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:  .LBB29_2:
 ; X32-NEXT:    popl %esi
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: reg64_lshr_by_add_to_negated:
 ; X64:       # %bb.0:
+; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    movl $64, %ecx
 ; X64-NEXT:    subl %esi, %ecx
-; X64-NEXT:    addl %edx, %ecx
-; X64-NEXT:    # kill: def $cl killed $cl killed $ecx
+; X64-NEXT:    # kill: def $cl killed $cl killed $rcx
 ; X64-NEXT:    shrq %cl, %rax
 ; X64-NEXT:    retq
   %nega = sub i64 64, %a
   %negasubb = add i64 %nega, %b
   %shifted = lshr i64 %val, %negasubb
   ret i64 %shifted
 }
 

 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:  .LBB41_2:
 ; X32-NEXT:    popl %esi
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: reg64_lshr_by_negated_unfolded_add_b:
 ; X64:       # %bb.0:
+; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    movl $64, %ecx
 ; X64-NEXT:    subl %esi, %ecx
-; X64-NEXT:    addl %edx, %ecx
-; X64-NEXT:    # kill: def $cl killed $cl killed $ecx
+; X64-NEXT:    # kill: def $cl killed $cl killed $rcx
 ; X64-NEXT:    shrq %cl, %rax
 ; X64-NEXT:    retq
   %nega = sub i64 0, %a
   %negaaddbitwidth = add i64 %nega, 64
   %negaaddbitwidthaddb = add i64 %negaaddbitwidth, %b
   %shifted = lshr i64 %val, %negaaddbitwidthaddb
   ret i64 %shifted
 }

llvm/trunk/test/CodeGen/X86/sink-addsub-of-const.ll

 }
 
 ; add (sub C, %x), %y
 ; Outer 'add' is commutative - 2 variants.
 
 define i32 @sink_sub_from_const_to_add0(i32 %a, i32 %b) {
 ; X32-LABEL: sink_sub_from_const_to_add0:
 ; X32:       # %bb.0:
-; X32-NEXT:    movl $32, %eax
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %eax
-; X32-NEXT:    addl {{[0-9]+}}(%esp), %eax
+; X32-NEXT:    addl $32, %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: sink_sub_from_const_to_add0:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl $32, %eax
-; X64-NEXT:    subl %edi, %eax
-; X64-NEXT:    addl %esi, %eax
+; X64-NEXT:    # kill: def $esi killed $esi def $rsi
+; X64-NEXT:    subl %edi, %esi
+; X64-NEXT:    leal 32(%rsi), %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 32, %a
   %r = add i32 %t0, %b
   ret i32 %r
 }
 define i32 @sink_sub_from_const_to_add1(i32 %a, i32 %b) {
 ; X32-LABEL: sink_sub_from_const_to_add1:
 ; X32:       # %bb.0:
-; X32-NEXT:    movl $32, %eax
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %eax
-; X32-NEXT:    addl {{[0-9]+}}(%esp), %eax
+; X32-NEXT:    addl $32, %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: sink_sub_from_const_to_add1:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl $32, %eax
-; X64-NEXT:    subl %edi, %eax
-; X64-NEXT:    addl %esi, %eax
+; X64-NEXT:    # kill: def $esi killed $esi def $rsi
+; X64-NEXT:    subl %edi, %esi
+; X64-NEXT:    leal 32(%rsi), %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 32, %a
   %r = add i32 %b, %t0
   ret i32 %r
 }
 
 ; sub (add %x, C), %y
 ; sub %y, (add %x, C)

 ; X64-NEXT:    retq
   %t0 = sub i32 %a, 32
   %r = sub i32 %t0, %b
   ret i32 %r
 }
 define i32 @sink_sub_of_const_to_sub2(i32 %a, i32 %b) {
 ; X32-LABEL: sink_sub_of_const_to_sub2:
 ; X32:       # %bb.0:
-; X32-NEXT:    movl $32, %eax
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %eax
-; X32-NEXT:    addl {{[0-9]+}}(%esp), %eax
+; X32-NEXT:    addl $32, %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: sink_sub_of_const_to_sub2:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl $32, %eax
-; X64-NEXT:    subl %edi, %eax
-; X64-NEXT:    addl %esi, %eax
+; X64-NEXT:    # kill: def $esi killed $esi def $rsi
+; X64-NEXT:    subl %edi, %esi
+; X64-NEXT:    leal 32(%rsi), %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 %a, 32
   %r = sub i32 %b, %t0
   ret i32 %r
 }
 
 ; sub (sub C, %x), %y
 ; sub %y, (sub C, %x)
; X64-NEXT:    retq
   %r = add <4 x i32> %b, %t0
   ret <4 x i32> %r
 }
 
 ; add (sub C, %x), %y
 ; Outer 'add' is commutative - 2 variants.
 
 define <4 x i32> @vec_sink_sub_from_const_to_add0(<4 x i32> %a, <4 x i32> %b) {
-; ALL-LABEL: vec_sink_sub_from_const_to_add0:
-; ALL:       # %bb.0:
-; ALL-NEXT:    movdqa {{.*#+}} xmm2 = <42,24,u,46>
-; ALL-NEXT:    psubd %xmm0, %xmm2
-; ALL-NEXT:    paddd %xmm1, %xmm2
-; ALL-NEXT:    movdqa %xmm2, %xmm0
-; ALL-NEXT:    ret{{[l|q]}}
+; X32-LABEL: vec_sink_sub_from_const_to_add0:
+; X32:       # %bb.0:
+; X32-NEXT:    psubd %xmm0, %xmm1
+; X32-NEXT:    paddd {{\.LCPI.*}}, %xmm1
+; X32-NEXT:    movdqa %xmm1, %xmm0
+; X32-NEXT:    retl
+;
+; X64-LABEL: vec_sink_sub_from_const_to_add0:
+; X64:       # %bb.0:
+; X64-NEXT:    psubd %xmm0, %xmm1
+; X64-NEXT:    paddd {{.*}}(%rip), %xmm1
+; X64-NEXT:    movdqa %xmm1, %xmm0
+; X64-NEXT:    retq
   %t0 = sub <4 x i32> <i32 42, i32 24, i32 undef, i32 46>, %a
   %r = add <4 x i32> %t0, %b
   ret <4 x i32> %r
 }
 define <4 x i32> @vec_sink_sub_from_const_to_add1(<4 x i32> %a, <4 x i32> %b) {
-; ALL-LABEL: vec_sink_sub_from_const_to_add1:
-; ALL:       # %bb.0:
-; ALL-NEXT:    movdqa {{.*#+}} xmm2 = <42,24,u,46>
-; ALL-NEXT:    psubd %xmm0, %xmm2
-; ALL-NEXT:    paddd %xmm1, %xmm2
-; ALL-NEXT:    movdqa %xmm2, %xmm0
-; ALL-NEXT:    ret{{[l|q]}}
+; X32-LABEL: vec_sink_sub_from_const_to_add1:
+; X32:       # %bb.0:
+; X32-NEXT:    psubd %xmm0, %xmm1
+; X32-NEXT:    paddd {{\.LCPI.*}}, %xmm1
+; X32-NEXT:    movdqa %xmm1, %xmm0
+; X32-NEXT:    retl
+;
+; X64-LABEL: vec_sink_sub_from_const_to_add1:
+; X64:       # %bb.0:
+; X64-NEXT:    psubd %xmm0, %xmm1
+; X64-NEXT:    paddd {{.*}}(%rip), %xmm1
+; X64-NEXT:    movdqa %xmm1, %xmm0
+; X64-NEXT:    retq
   %t0 = sub <4 x i32> <i32 42, i32 24, i32 undef, i32 46>, %a
   %r = add <4 x i32> %b, %t0
   ret <4 x i32> %r
 }
 
 ; sub (add %x, C), %y
 ; sub %y, (add %x, C)
 

 ; X64-NEXT:    psubd %xmm1, %xmm0
 ; X64-NEXT:    psubd {{.*}}(%rip), %xmm0
 ; X64-NEXT:    retq
   %t0 = sub <4 x i32> %a, <i32 42, i32 24, i32 undef, i32 46>
   %r = sub <4 x i32> %t0, %b
   ret <4 x i32> %r
 }
 define <4 x i32> @vec_sink_sub_of_const_to_sub2(<4 x i32> %a, <4 x i32> %b) {
-; ALL-LABEL: vec_sink_sub_of_const_to_sub2:
-; ALL:       # %bb.0:
-; ALL-NEXT:    movdqa {{.*#+}} xmm2 = <42,24,u,46>
-; ALL-NEXT:    psubd %xmm0, %xmm2
-; ALL-NEXT:    paddd %xmm1, %xmm2
-; ALL-NEXT:    movdqa %xmm2, %xmm0
-; ALL-NEXT:    ret{{[l|q]}}
+; X32-LABEL: vec_sink_sub_of_const_to_sub2:
+; X32:       # %bb.0:
+; X32-NEXT:    psubd %xmm0, %xmm1
+; X32-NEXT:    paddd {{\.LCPI.*}}, %xmm1
+; X32-NEXT:    movdqa %xmm1, %xmm0
+; X32-NEXT:    retl
+;
+; X64-LABEL: vec_sink_sub_of_const_to_sub2:
+; X64:       # %bb.0:
+; X64-NEXT:    psubd %xmm0, %xmm1
+; X64-NEXT:    paddd {{.*}}(%rip), %xmm1
+; X64-NEXT:    movdqa %xmm1, %xmm0
+; X64-NEXT:    retq
   %t0 = sub <4 x i32> %a, <i32 42, i32 24, i32 undef, i32 46>
   %r = sub <4 x i32> %b, %t0
   ret <4 x i32> %r
 }
 
 ; sub (sub C, %x), %y
 ; sub %y, (sub C, %x)