[DAGCombiner] Add X / X -> 1 & X % X -> 0 folds.

[RKSimon]

Summary

Adds more divrem folds to try and get in sync with InstructionSimplify

This hits a number of reduced tests so I've had to tweak them; through a mixture of visual scrutiny and debugging they seem to be hitting the same points as the original test failed at.

[jonpa]

Not sure if I'm doing something wrong (I use git and tried first patch -p0, and then also with dos2unix, but then that didn't work so I had to write the patch manually although just a few lines...)

This did not give any change at all on benchmarks on SystemZ (NFC), and the SystemZ test succeeds with or without the patch...?

[RKSimon]

In D50636#1197251, @jonpa wrote:

Not sure if I'm doing something wrong (I use git and tried first patch -p0, and then also with dos2unix, but then that didn't work so I had to write the patch manually although just a few lines...)

Sorry - I've never worked out why the upload always screws up my line endings.....

This did not give any change at all on benchmarks on SystemZ (NFC), and the SystemZ test succeeds with or without the patch...?

Are the x86 tests changing? They contain explicit tests for this fold

[jonpa]

In D50636#1197255, @RKSimon wrote:

In D50636#1197251, @jonpa wrote:

Not sure if I'm doing something wrong (I use git and tried first patch -p0, and then also with dos2unix, but then that didn't work so I had to write the patch manually although just a few lines...)

Sorry - I've never worked out why the upload always screws up my line endings.....

This did not give any change at all on benchmarks on SystemZ (NFC), and the SystemZ test succeeds with or without the patch...?

Are the x86 tests changing? They contain explicit tests for this fold

Yes, change in x86 test (at least the one I tried)...

[RKSimon]

test/CodeGen/SystemZ/pr32372.ll

14

Tried again against bleeding edge trunk and I still see this change and without the volatile (which prevents the loads being combined) the checks reduce to:

define void @pr32372(i8*) {
; CHECK-LABEL: pr32372:
; CHECK:       # %bb.0: # %BB
; CHECK-NEXT:    mvhhi 0(%r1), -3825
; CHECK-NEXT:  .LBB0_1: # %CF251
; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
; CHECK-NEXT:    j .LBB0_1

[craig.topper]

test/CodeGen/X86/combine-srem.ll

455–456

Should these test IR changes be committed separately?

[jonpa]

test/CodeGen/SystemZ/pr32372.ll

14

Yes, I see the change after updating the test, but also note that this updated test passes also without the patch... So I am not sure why the test is changed if the patch does not affect it...

[craig.topper]

test/CodeGen/SystemZ/pr32372.ll

14

I think this patch affects the original version of the test. Without the volatile the two loads get CSEd to the same SDNode. This causes the same SDNode to be both operands of the urem. This patch would simplify that to 0. Simon added the volatile to prevent the loads from being CSEd. This way the urem won't get affected by this patch. So the new version should pass on trunk as well.

[jonpa]

test/CodeGen/SystemZ/pr32372.ll

14

Ah, I see. Thank you for the explanation.

[RKSimon]

I'm happy to apply the test changes beforehand once everybody is happy with the patch - but I'll keep them in the diff for now to make its effects clear.

test/CodeGen/SystemZ/pr32372.ll

14

Sorry - I missed that you were applying the test changes before the code changes!

[craig.topper]

The DAGCombiner changes and the X86 test changes all look good to me.

[RKSimon]

@jonpa @pftbest Any more comments?

[jonpa]

The origin for that test case I believe is that I had found a crash during testing and I reported this as PR32372.

The test changes here look OK to me, but when I reverted the fix for that original bug it seems the new test does no longer fail, while the unmodified one still does... :-/

Unfortunately, I don't seem to have the unreduced test failure available, so I can't make a new reduced version or even try if the original test case still fails...

Not sure if we should remove this test then, or keep it in case it may trigger somehow in the future...? Of course, the assert that the test used to trigger is still there, so removing it isn't all that unthinkable to me...

I tried some other experiments to the test, such as having two i8* arguments - one for each load - but that did not help...

[RKSimon]

Rebased patch

[craig.topper]

@jonpa, it looks like changing the urem to this will trigger the original bug and avoid the optimization Simon is adding here.

%B8 = call { i8, i1 } @llvm.umul.with.overflow.i8(i8 %L5, i8 %L)
%B9 = extractvalue {i8, i1} %B8, 0

[jonpa]

it looks like changing the urem to this will trigger the original bug and avoid the optimization Simon is adding here.

Ah, nice :-) I tried it, and just like you found that it triggers the original bug. I updated the test, patch is:

pr32372.ll.patch1 KBDownload

[RKSimon]

Applied @jonpa's test fix.

@craig.topper I also made a fix to vector-idiv-v2i32.ll which I think was a typo?

[craig.topper]

LGTM

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

 }
 
 static SDValue simplifyDivRem(SDNode *N, SelectionDAG &DAG) {
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
   EVT VT = N->getValueType(0);
   SDLoc DL(N);
 
+  unsigned Opc = N->getOpcode();
+  bool IsDiv = (ISD::SDIV == Opc) || (ISD::UDIV == Opc);
+
   // X / undef -> undef
   // X % undef -> undef
   // X / 0 -> undef
   // X % 0 -> undef
   // NOTE: This includes vectors where any divisor element is zero/undef.
-  if (DAG.isUndef(N->getOpcode(), {N0, N1}))
+  if (DAG.isUndef(Opc, {N0, N1}))
     return DAG.getUNDEF(VT);
 
   // undef / X -> 0
   // undef % X -> 0
   if (N0.isUndef())
     return DAG.getConstant(0, DL, VT);
 
+  // TODO: 0 / X -> 0
+  // TODO: 0 % X -> 0
+
+  // X / X -> 1
+  // X % X -> 0
+  if (N0 == N1)
+    return DAG.getConstant(IsDiv ? 1 : 0, DL, VT);
+
+  // TODO: X / 1 -> X
+  // TODO: X % 1 -> 0
+  // If this is a boolean op (single-bit element type), we can't have
+  // division-by-zero or remainder-by-zero, so assume the divisor is 1.
+  // Similarly, if we're zero-extending a boolean divisor, then assume it's a 1.
+
   return SDValue();
 }
 
 SDValue DAGCombiner::visitSDIV(SDNode *N) {
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
   EVT VT = N->getValueType(0);
   EVT CCVT = getSetCCResultType(VT);

test/CodeGen/MSP430/libcalls.ll

 }
 
 define i16 @divi() #0 {
 entry:
 ; CHECK: divi:
 
 ; CHECK: call #__mspabi_divi
   %0 = load volatile i16, i16* @g_i16, align 8
-  %1 = sdiv i16 %0, %0
+  %1 = load volatile i16, i16* @g_i16, align 8
+  %2 = sdiv i16 %0, %1
 
-  ret i16 %1
+  ret i16 %2
 }
 
 define i32 @divli() #0 {
 entry:
 ; CHECK: divli:
 
 ; CHECK: call #__mspabi_divli
   %0 = load volatile i32, i32* @g_i32, align 8
-  %1 = sdiv i32 %0, %0
+  %1 = load volatile i32, i32* @g_i32, align 8
+  %2 = sdiv i32 %0, %1
 
-  ret i32 %1
+  ret i32 %2
 }
 
 define i64 @divlli() #0 {
 entry:
 ; CHECK: divlli:
 
 ; CHECK: call #__mspabi_divlli
   %0 = load volatile i64, i64* @g_i64, align 8
-  %1 = sdiv i64 %0, %0
+  %1 = load volatile i64, i64* @g_i64, align 8
+  %2 = sdiv i64 %0, %1
 
-  ret i64 %1
+  ret i64 %2
 }
 
 define i16 @divu() #0 {
 entry:
 ; CHECK: divu:
 
 ; CHECK: call #__mspabi_divu
   %0 = load volatile i16, i16* @g_i16, align 8
-  %1 = udiv i16 %0, %0
+  %1 = load volatile i16, i16* @g_i16, align 8
+  %2 = udiv i16 %0, %1
 
-  ret i16 %1
+  ret i16 %2
 }
 
 define i32 @divul() #0 {
 entry:
 ; CHECK: divul:
 
 ; CHECK: call #__mspabi_divul
   %0 = load volatile i32, i32* @g_i32, align 8
-  %1 = udiv i32 %0, %0
+  %1 = load volatile i32, i32* @g_i32, align 8
+  %2 = udiv i32 %0, %1
 
-  ret i32 %1
+  ret i32 %2
 }
 
 define i64 @divull() #0 {
 entry:
 ; CHECK: divull:
 
 ; CHECK: call #__mspabi_divull
   %0 = load volatile i64, i64* @g_i64, align 8
-  %1 = udiv i64 %0, %0
+  %1 = load volatile i64, i64* @g_i64, align 8
+  %2 = udiv i64 %0, %1
 
-  ret i64 %1
+  ret i64 %2
 }
 
 define i16 @remi() #0 {
 entry:
 ; CHECK: remi:
 
 ; CHECK: call #__mspabi_remi
   %0 = load volatile i16, i16* @g_i16, align 8
-  %1 = srem i16 %0, %0
+  %1 = load volatile i16, i16* @g_i16, align 8
+  %2 = srem i16 %0, %1
 
-  ret i16 %1
+  ret i16 %2
 }
 
 define i32 @remli() #0 {
 entry:
 ; CHECK: remli:
 
 ; CHECK: call #__mspabi_remli
   %0 = load volatile i32, i32* @g_i32, align 8
-  %1 = srem i32 %0, %0
+  %1 = load volatile i32, i32* @g_i32, align 8
+  %2 = srem i32 %0, %1
 
-  ret i32 %1
+  ret i32 %2
 }
 
 define i64 @remlli() #0 {
 entry:
 ; CHECK: remlli:
 
 ; CHECK: call #__mspabi_remlli
   %0 = load volatile i64, i64* @g_i64, align 8
-  %1 = srem i64 %0, %0
+  %1 = load volatile i64, i64* @g_i64, align 8
+  %2 = srem i64 %0, %1
 
-  ret i64 %1
+  ret i64 %2
 }
 
 define i16 @remu() #0 {
 entry:
 ; CHECK: remu:
 
 ; CHECK: call #__mspabi_remu
   %0 = load volatile i16, i16* @g_i16, align 8
-  %1 = urem i16 %0, %0
+  %1 = load volatile i16, i16* @g_i16, align 8
+  %2 = urem i16 %0, %1
 
-  ret i16 %1
+  ret i16 %2
 }
 
 define i32 @remul() #0 {
 entry:
 ; CHECK: remul:
 
 ; CHECK: call #__mspabi_remul
   %0 = load volatile i32, i32* @g_i32, align 8
-  %1 = urem i32 %0, %0
+  %1 = load volatile i32, i32* @g_i32, align 8
+  %2 = urem i32 %0, %1
 
-  ret i32 %1
+  ret i32 %2
 }
 
 define i64 @remull() #0 {
 entry:
 ; CHECK: remull:
 
 ; CHECK: call #__mspabi_remull
   %0 = load volatile i64, i64* @g_i64, align 8
-  %1 = urem i64 %0, %0
+  %1 = load volatile i64, i64* @g_i64, align 8
+  %2 = urem i64 %0, %1
 
-  ret i64 %1
+  ret i64 %2
 }
 
 define i16 @mpyi() #0 {
 entry:
 ; CHECK: mpyi:
 
 ; CHECK: call #__mspabi_mpyi
   %0 = load volatile i16, i16* @g_i16, align 8

test/CodeGen/SystemZ/pr32372.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc %s -o - -mtriple=s390x-linux-gnu | FileCheck %s
 
 define void @pr32372(i8*) {
 ; CHECK-LABEL: pr32372:
 ; CHECK:       # %bb.0: # %BB
-; CHECK-NEXT:    llc %r1, 0(%r2)
 ; CHECK-NEXT:    mvhhi 0(%r1), -3825
-; CHECK-NEXT:    llill %r0, 0
-; CHECK-NEXT:    dlr %r0, %r1
 ; CHECK-NEXT:  .LBB0_1: # %CF251
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    j .LBB0_1
 BB:
   %L = load i8, i8* %0
   store i16 -3825, i16* undef
   %L5 = load i8, i8* %0
-  %B9 = urem i8 %L5, %L
+  %B8 = call { i8, i1 } @llvm.umul.with.overflow.i8(i8 %L5, i8 %L)
+  %B9 = extractvalue {i8, i1} %B8, 0
   %I107 = insertelement <8 x i8> zeroinitializer, i8 %B9, i32 7
   %ZE141 = zext i8 %L5 to i16
   br label %CF251
 
 CF251:                                            ; preds = %CF258, %CF251, %BB
   %Shuff217 = shufflevector <8 x i8> zeroinitializer, <8 x i8> %I107, <8 x i32> <i32 0, i32 2, i32 undef, i32 6, i32 8, i32 undef, i32 12, i32 14>
   %Cmp227 = icmp sge i16 %ZE141, 0
   br i1 %Cmp227, label %CF251, label %CF258
 
 CF258:                                            ; preds = %CF251
   %Shuff230 = shufflevector <2 x i16> undef, <2 x i16> undef, <2 x i32> <i32 3, i32 1>
   br label %CF251
 }
+
+declare {i8, i1} @llvm.umul.with.overflow.i8(i8, i8) nounwind readnone

test/CodeGen/X86/combine-sdiv.ll

 ; AVX-NEXT:    xorl %edx, %edx
 ; AVX-NEXT:    idivl %ecx
 ; AVX-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm0
 ; AVX-NEXT:    retq
   %1 = sdiv <4 x i32> zeroinitializer, %x
   ret <4 x i32> %1
 }
 
-; TODO fold (sdiv x, x) -> 1
+; fold (sdiv x, x) -> 1
 define i32 @combine_sdiv_dupe(i32 %x) {
 ; CHECK-LABEL: combine_sdiv_dupe:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    cltd
-; CHECK-NEXT:    idivl %edi
+; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retq
   %1 = sdiv i32 %x, %x
   ret i32 %1
 }
 
 define <4 x i32> @combine_vec_sdiv_dupe(<4 x i32> %x) {
-; SSE2-LABEL: combine_vec_sdiv_dupe:
-; SSE2:       # %bb.0:
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[3,1,2,3]
-; SSE2-NEXT:    movd %xmm1, %ecx
-; SSE2-NEXT:    movl %ecx, %eax
-; SSE2-NEXT:    cltd
-; SSE2-NEXT:    idivl %ecx
-; SSE2-NEXT:    movd %eax, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
-; SSE2-NEXT:    movd %xmm2, %ecx
-; SSE2-NEXT:    movl %ecx, %eax
-; SSE2-NEXT:    cltd
-; SSE2-NEXT:    idivl %ecx
-; SSE2-NEXT:    movd %eax, %xmm2
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; SSE2-NEXT:    movd %xmm0, %ecx
-; SSE2-NEXT:    movl %ecx, %eax
-; SSE2-NEXT:    cltd
-; SSE2-NEXT:    idivl %ecx
-; SSE2-NEXT:    movd %eax, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
-; SSE2-NEXT:    movd %xmm0, %ecx
-; SSE2-NEXT:    movl %ecx, %eax
-; SSE2-NEXT:    cltd
-; SSE2-NEXT:    idivl %ecx
-; SSE2-NEXT:    movd %eax, %xmm0
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; SSE2-NEXT:    movdqa %xmm1, %xmm0
-; SSE2-NEXT:    retq
+; SSE-LABEL: combine_vec_sdiv_dupe:
+; SSE:       # %bb.0:
+; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1,1,1,1]
+; SSE-NEXT:    retq
 ;
-; SSE41-LABEL: combine_vec_sdiv_dupe:
-; SSE41:       # %bb.0:
-; SSE41-NEXT:    pextrd $1, %xmm0, %ecx
-; SSE41-NEXT:    movl %ecx, %eax
-; SSE41-NEXT:    cltd
-; SSE41-NEXT:    idivl %ecx
-; SSE41-NEXT:    movl %eax, %ecx
-; SSE41-NEXT:    movd %xmm0, %esi
-; SSE41-NEXT:    movl %esi, %eax
-; SSE41-NEXT:    cltd
-; SSE41-NEXT:    idivl %esi
-; SSE41-NEXT:    movd %eax, %xmm1
-; SSE41-NEXT:    pinsrd $1, %ecx, %xmm1
-; SSE41-NEXT:    pextrd $2, %xmm0, %ecx
-; SSE41-NEXT:    movl %ecx, %eax
-; SSE41-NEXT:    cltd
-; SSE41-NEXT:    idivl %ecx
-; SSE41-NEXT:    pinsrd $2, %eax, %xmm1
-; SSE41-NEXT:    pextrd $3, %xmm0, %ecx
-; SSE41-NEXT:    movl %ecx, %eax
-; SSE41-NEXT:    cltd
-; SSE41-NEXT:    idivl %ecx
-; SSE41-NEXT:    pinsrd $3, %eax, %xmm1
-; SSE41-NEXT:    movdqa %xmm1, %xmm0
-; SSE41-NEXT:    retq
+; AVX1-LABEL: combine_vec_sdiv_dupe:
+; AVX1:       # %bb.0:
+; AVX1-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
+; AVX1-NEXT:    retq
 ;
-; AVX-LABEL: combine_vec_sdiv_dupe:
-; AVX:       # %bb.0:
-; AVX-NEXT:    vpextrd $1, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    movl %eax, %ecx
-; AVX-NEXT:    vmovd %xmm0, %esi
-; AVX-NEXT:    movl %esi, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %esi
-; AVX-NEXT:    vmovd %eax, %xmm1
-; AVX-NEXT:    vpinsrd $1, %ecx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $2, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $3, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm0
-; AVX-NEXT:    retq
+; AVX2ORLATER-LABEL: combine_vec_sdiv_dupe:
+; AVX2ORLATER:       # %bb.0:
+; AVX2ORLATER-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; AVX2ORLATER-NEXT:    retq
+;
+; XOP-LABEL: combine_vec_sdiv_dupe:
+; XOP:       # %bb.0:
+; XOP-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
+; XOP-NEXT:    retq
   %1 = sdiv <4 x i32> %x, %x
   ret <4 x i32> %1
 }
 
 ; fold (sdiv x, y) -> (udiv x, y) iff x and y are positive
 define <4 x i32> @combine_vec_sdiv_by_pos0(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_sdiv_by_pos0:
 ; SSE:       # %bb.0:

test/CodeGen/X86/combine-srem.ll

 ; AVX-NEXT:    xorl %edx, %edx
 ; AVX-NEXT:    idivl %ecx
 ; AVX-NEXT:    vpinsrd $3, %edx, %xmm1, %xmm0
 ; AVX-NEXT:    retq
   %1 = srem <4 x i32> zeroinitializer, %x
   ret <4 x i32> %1
 }
 
-; TODO fold (srem x, x) -> 0
+; fold (srem x, x) -> 0
 define i32 @combine_srem_dupe(i32 %x) {
 ; CHECK-LABEL: combine_srem_dupe:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    cltd
-; CHECK-NEXT:    idivl %edi
-; CHECK-NEXT:    movl %edx, %eax
+; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    retq
   %1 = srem i32 %x, %x
   ret i32 %1
 }
 
 define <4 x i32> @combine_vec_srem_dupe(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_srem_dupe:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    pextrd $1, %xmm0, %ecx
-; SSE-NEXT:    movl %ecx, %eax
-; SSE-NEXT:    cltd
-; SSE-NEXT:    idivl %ecx
-; SSE-NEXT:    movl %edx, %ecx
-; SSE-NEXT:    movd %xmm0, %esi
-; SSE-NEXT:    movl %esi, %eax
-; SSE-NEXT:    cltd
-; SSE-NEXT:    idivl %esi
-; SSE-NEXT:    movd %edx, %xmm1
-; SSE-NEXT:    pinsrd $1, %ecx, %xmm1
-; SSE-NEXT:    pextrd $2, %xmm0, %ecx
-; SSE-NEXT:    movl %ecx, %eax
-; SSE-NEXT:    cltd
-; SSE-NEXT:    idivl %ecx
-; SSE-NEXT:    pinsrd $2, %edx, %xmm1
-; SSE-NEXT:    pextrd $3, %xmm0, %ecx
-; SSE-NEXT:    movl %ecx, %eax
-; SSE-NEXT:    cltd
-; SSE-NEXT:    idivl %ecx
-; SSE-NEXT:    pinsrd $3, %edx, %xmm1
-; SSE-NEXT:    movdqa %xmm1, %xmm0
+; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: combine_vec_srem_dupe:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vpextrd $1, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    movl %edx, %ecx
-; AVX-NEXT:    vmovd %xmm0, %esi
-; AVX-NEXT:    movl %esi, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %esi
-; AVX-NEXT:    vmovd %edx, %xmm1
-; AVX-NEXT:    vpinsrd $1, %ecx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $2, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    vpinsrd $2, %edx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $3, %xmm0, %ecx
-; AVX-NEXT:    movl %ecx, %eax
-; AVX-NEXT:    cltd
-; AVX-NEXT:    idivl %ecx
-; AVX-NEXT:    vpinsrd $3, %edx, %xmm1, %xmm0
+; AVX-NEXT:    vxorps %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %1 = srem <4 x i32> %x, %x
   ret <4 x i32> %1
 }
 
 ; fold (srem x, y) -> (urem x, y) iff x and y are positive
 define <4 x i32> @combine_vec_srem_by_pos0(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_srem_by_pos0:

 ; CHECK-NEXT:    movl %edx, %edi
 ; CHECK-NEXT:    xorl %edx, %edx
 ; CHECK-NEXT:    movl %ecx, %eax
 ; CHECK-NEXT:    divl %esi
 ; CHECK-NEXT:    andl %edi, %eax
 ; CHECK-NEXT:    retq
   %B17 = or i32 0, 2147483647
   %L6 = load i32, i32* undef
   %B11 = sdiv i32 %B17, %L6
   %B13 = udiv i32 %B17, %L6
   %B14 = srem i32 %B11, %B13
   %B16 = srem i32 %L6, %B14
   %B10 = udiv i32 %L6, %B14
   %B6 = and i32 %B16, %B10
   ret i32 %B6
 }

test/CodeGen/X86/combine-udiv.ll

 ; XOP-NEXT:    xorl %edx, %edx
 ; XOP-NEXT:    divl %ecx
 ; XOP-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm0
 ; XOP-NEXT:    retq
   %1 = udiv <4 x i32> zeroinitializer, %x
   ret <4 x i32> %1
 }
 
-; TODO fold (udiv x, x) -> 1
+; fold (udiv x, x) -> 1
 define i32 @combine_udiv_dupe(i32 %x) {
 ; CHECK-LABEL: combine_udiv_dupe:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    xorl %edx, %edx
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    divl %edi
+; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    retq
   %1 = udiv i32 %x, %x
   ret i32 %1
 }
 
 define <4 x i32> @combine_vec_udiv_dupe(<4 x i32> %x) {
-; SSE2-LABEL: combine_vec_udiv_dupe:
-; SSE2:       # %bb.0:
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[3,1,2,3]
-; SSE2-NEXT:    movd %xmm1, %eax
-; SSE2-NEXT:    xorl %edx, %edx
-; SSE2-NEXT:    divl %eax
-; SSE2-NEXT:    movd %eax, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
-; SSE2-NEXT:    movd %xmm2, %eax
-; SSE2-NEXT:    xorl %edx, %edx
-; SSE2-NEXT:    divl %eax
-; SSE2-NEXT:    movd %eax, %xmm2
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; SSE2-NEXT:    movd %xmm0, %eax
-; SSE2-NEXT:    xorl %edx, %edx
-; SSE2-NEXT:    divl %eax
-; SSE2-NEXT:    movd %eax, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
-; SSE2-NEXT:    movd %xmm0, %eax
-; SSE2-NEXT:    xorl %edx, %edx
-; SSE2-NEXT:    divl %eax
-; SSE2-NEXT:    movd %eax, %xmm0
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; SSE2-NEXT:    movdqa %xmm1, %xmm0
-; SSE2-NEXT:    retq
+; SSE-LABEL: combine_vec_udiv_dupe:
+; SSE:       # %bb.0:
+; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1,1,1,1]
+; SSE-NEXT:    retq
 ;
-; SSE41-LABEL: combine_vec_udiv_dupe:
-; SSE41:       # %bb.0:
-; SSE41-NEXT:    pextrd $1, %xmm0, %eax
-; SSE41-NEXT:    xorl %edx, %edx
-; SSE41-NEXT:    divl %eax
-; SSE41-NEXT:    movl %eax, %ecx
-; SSE41-NEXT:    movd %xmm0, %eax
-; SSE41-NEXT:    xorl %edx, %edx
-; SSE41-NEXT:    divl %eax
-; SSE41-NEXT:    movd %eax, %xmm1
-; SSE41-NEXT:    pinsrd $1, %ecx, %xmm1
-; SSE41-NEXT:    pextrd $2, %xmm0, %eax
-; SSE41-NEXT:    xorl %edx, %edx
-; SSE41-NEXT:    divl %eax
-; SSE41-NEXT:    pinsrd $2, %eax, %xmm1
-; SSE41-NEXT:    pextrd $3, %xmm0, %eax
-; SSE41-NEXT:    xorl %edx, %edx
-; SSE41-NEXT:    divl %eax
-; SSE41-NEXT:    pinsrd $3, %eax, %xmm1
-; SSE41-NEXT:    movdqa %xmm1, %xmm0
-; SSE41-NEXT:    retq
+; AVX1-LABEL: combine_vec_udiv_dupe:
+; AVX1:       # %bb.0:
+; AVX1-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
+; AVX1-NEXT:    retq
 ;
-; AVX-LABEL: combine_vec_udiv_dupe:
-; AVX:       # %bb.0:
-; AVX-NEXT:    vpextrd $1, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    movl %eax, %ecx
-; AVX-NEXT:    vmovd %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vmovd %eax, %xmm1
-; AVX-NEXT:    vpinsrd $1, %ecx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $2, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $3, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm0
-; AVX-NEXT:    retq
+; AVX2-LABEL: combine_vec_udiv_dupe:
+; AVX2:       # %bb.0:
+; AVX2-NEXT:    vbroadcastss {{.*#+}} xmm0 = [1,1,1,1]
+; AVX2-NEXT:    retq
 ;
 ; XOP-LABEL: combine_vec_udiv_dupe:
 ; XOP:       # %bb.0:
-; XOP-NEXT:    vpextrd $1, %xmm0, %eax
-; XOP-NEXT:    xorl %edx, %edx
-; XOP-NEXT:    divl %eax
-; XOP-NEXT:    movl %eax, %ecx
-; XOP-NEXT:    vmovd %xmm0, %eax
-; XOP-NEXT:    xorl %edx, %edx
-; XOP-NEXT:    divl %eax
-; XOP-NEXT:    vmovd %eax, %xmm1
-; XOP-NEXT:    vpinsrd $1, %ecx, %xmm1, %xmm1
-; XOP-NEXT:    vpextrd $2, %xmm0, %eax
-; XOP-NEXT:    xorl %edx, %edx
-; XOP-NEXT:    divl %eax
-; XOP-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
-; XOP-NEXT:    vpextrd $3, %xmm0, %eax
-; XOP-NEXT:    xorl %edx, %edx
-; XOP-NEXT:    divl %eax
-; XOP-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm0
+; XOP-NEXT:    vmovaps {{.*#+}} xmm0 = [1,1,1,1]
 ; XOP-NEXT:    retq
   %1 = udiv <4 x i32> %x, %x
   ret <4 x i32> %1
 }
 
 ; fold (udiv x, (1 << c)) -> x >>u c
 define <4 x i32> @combine_vec_udiv_by_pow2a(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_udiv_by_pow2a:

test/CodeGen/X86/combine-urem.ll

 ; AVX-NEXT:    xorl %edx, %edx
 ; AVX-NEXT:    divl %ecx
 ; AVX-NEXT:    vpinsrd $3, %edx, %xmm1, %xmm0
 ; AVX-NEXT:    retq
   %1 = urem <4 x i32> zeroinitializer, %x
   ret <4 x i32> %1
 }
 
-; TODO fold (urem x, x) -> 0
+; fold (urem x, x) -> 0
 define i32 @combine_urem_dupe(i32 %x) {
 ; CHECK-LABEL: combine_urem_dupe:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    xorl %edx, %edx
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    divl %edi
-; CHECK-NEXT:    movl %edx, %eax
+; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    retq
   %1 = urem i32 %x, %x
   ret i32 %1
 }
 
 define <4 x i32> @combine_vec_urem_dupe(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_urem_dupe:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    pextrd $1, %xmm0, %eax
-; SSE-NEXT:    xorl %edx, %edx
-; SSE-NEXT:    divl %eax
-; SSE-NEXT:    movl %edx, %ecx
-; SSE-NEXT:    movd %xmm0, %eax
-; SSE-NEXT:    xorl %edx, %edx
-; SSE-NEXT:    divl %eax
-; SSE-NEXT:    movd %edx, %xmm1
-; SSE-NEXT:    pinsrd $1, %ecx, %xmm1
-; SSE-NEXT:    pextrd $2, %xmm0, %eax
-; SSE-NEXT:    xorl %edx, %edx
-; SSE-NEXT:    divl %eax
-; SSE-NEXT:    pinsrd $2, %edx, %xmm1
-; SSE-NEXT:    pextrd $3, %xmm0, %eax
-; SSE-NEXT:    xorl %edx, %edx
-; SSE-NEXT:    divl %eax
-; SSE-NEXT:    pinsrd $3, %edx, %xmm1
-; SSE-NEXT:    movdqa %xmm1, %xmm0
+; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: combine_vec_urem_dupe:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vpextrd $1, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    movl %edx, %ecx
-; AVX-NEXT:    vmovd %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vmovd %edx, %xmm1
-; AVX-NEXT:    vpinsrd $1, %ecx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $2, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vpinsrd $2, %edx, %xmm1, %xmm1
-; AVX-NEXT:    vpextrd $3, %xmm0, %eax
-; AVX-NEXT:    xorl %edx, %edx
-; AVX-NEXT:    divl %eax
-; AVX-NEXT:    vpinsrd $3, %edx, %xmm1, %xmm0
+; AVX-NEXT:    vxorps %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %1 = urem <4 x i32> %x, %x
   ret <4 x i32> %1
 }
 
 ; fold (urem x, pow2) -> (and x, (pow2-1))
 define <4 x i32> @combine_vec_urem_by_pow2a(<4 x i32> %x) {
 ; SSE-LABEL: combine_vec_urem_by_pow2a:

test/CodeGen/X86/pr38539.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown -verify-machineinstrs | FileCheck %s --check-prefix=X64
 ; RUN: llc < %s -mtriple=i686-unknown -verify-machineinstrs | FileCheck %s --check-prefix=X86
 
 ; This test is targeted at 64-bit mode. It used to crash due to the creation of an EXTRACT_SUBREG after the peephole pass had ran.
 define void @f() {
 ; X64-LABEL: f:
 ; X64:       # %bb.0: # %BB
-; X64-NEXT:    pushq %rbp
-; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    pushq %r14
-; X64-NEXT:    .cfi_def_cfa_offset 24
+; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    pushq %rbx
-; X64-NEXT:    .cfi_def_cfa_offset 32
-; X64-NEXT:    subq $16, %rsp
+; X64-NEXT:    .cfi_def_cfa_offset 24
+; X64-NEXT:    subq $24, %rsp
 ; X64-NEXT:    .cfi_def_cfa_offset 48
-; X64-NEXT:    .cfi_offset %rbx, -32
-; X64-NEXT:    .cfi_offset %r14, -24
-; X64-NEXT:    .cfi_offset %rbp, -16
+; X64-NEXT:    .cfi_offset %rbx, -24
+; X64-NEXT:    .cfi_offset %r14, -16
 ; X64-NEXT:    movzbl {{[0-9]+}}(%rsp), %ebx
 ; X64-NEXT:    movq %rbx, %rcx
 ; X64-NEXT:    shlq $62, %rcx
 ; X64-NEXT:    sarq $62, %rcx
-; X64-NEXT:    movq (%rsp), %r14
-; X64-NEXT:    movb (%rax), %bpl
+; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r14
 ; X64-NEXT:    xorl %edi, %edi
 ; X64-NEXT:    xorl %esi, %esi
 ; X64-NEXT:    movq %r14, %rdx
 ; X64-NEXT:    callq __modti3
 ; X64-NEXT:    andl $3, %edx
+; X64-NEXT:    testb %al, %al
+; X64-NEXT:    setne (%rax)
 ; X64-NEXT:    cmpq %rax, %r14
 ; X64-NEXT:    sbbq %rdx, %rbx
-; X64-NEXT:    setb %sil
-; X64-NEXT:    setae %bl
+; X64-NEXT:    setae %dl
+; X64-NEXT:    sbbb %cl, %cl
 ; X64-NEXT:    testb %al, %al
-; X64-NEXT:    setne %dl
-; X64-NEXT:    setne (%rax)
-; X64-NEXT:    movzbl %bpl, %eax
-; X64-NEXT:    xorl %ecx, %ecx
-; X64-NEXT:    subb %sil, %cl
-; X64-NEXT:    # kill: def $eax killed $eax def $ax
-; X64-NEXT:    divb %al
-; X64-NEXT:    negb %bl
-; X64-NEXT:    cmpb %al, %al
-; X64-NEXT:    setle %al
+; X64-NEXT:    setne %bl
+; X64-NEXT:    negb %dl
+; X64-NEXT:    cmpb $2, %al
+; X64-NEXT:    setl %al
 ; X64-NEXT:    negb %al
 ; X64-NEXT:    cbtw
-; X64-NEXT:    idivb %bl
+; X64-NEXT:    idivb %dl
 ; X64-NEXT:    movsbl %ah, %eax
 ; X64-NEXT:    movzbl %al, %eax
 ; X64-NEXT:    andl $1, %eax
 ; X64-NEXT:    shlq $4, %rax
 ; X64-NEXT:    negq %rax
-; X64-NEXT:    negb %dl
-; X64-NEXT:    leaq -16(%rsp,%rax), %rax
+; X64-NEXT:    negb %bl
+; X64-NEXT:    leaq -8(%rsp,%rax), %rax
 ; X64-NEXT:    movq %rax, (%rax)
 ; X64-NEXT:    movl %ecx, %eax
 ; X64-NEXT:    cbtw
-; X64-NEXT:    idivb %dl
+; X64-NEXT:    idivb %bl
 ; X64-NEXT:    movsbl %ah, %eax
 ; X64-NEXT:    andb $1, %al
 ; X64-NEXT:    movb %al, (%rax)
-; X64-NEXT:    addq $16, %rsp
-; X64-NEXT:    .cfi_def_cfa_offset 32
-; X64-NEXT:    popq %rbx
+; X64-NEXT:    addq $24, %rsp
 ; X64-NEXT:    .cfi_def_cfa_offset 24
-; X64-NEXT:    popq %r14
+; X64-NEXT:    popq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 16
-; X64-NEXT:    popq %rbp
+; X64-NEXT:    popq %r14
 ; X64-NEXT:    .cfi_def_cfa_offset 8
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: f:
 ; X86:       # %bb.0: # %BB
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    .cfi_offset %ebp, -8
 ; X86-NEXT:    movl %esp, %ebp
 ; X86-NEXT:    .cfi_def_cfa_register %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    andl $-8, %esp
-; X86-NEXT:    subl $48, %esp
+; X86-NEXT:    subl $40, %esp
 ; X86-NEXT:    .cfi_offset %esi, -20
 ; X86-NEXT:    .cfi_offset %edi, -16
 ; X86-NEXT:    .cfi_offset %ebx, -12
 ; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    shll $30, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    sarl $30, %ecx
 ; X86-NEXT:    sarl $31, %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    movl (%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movb (%eax), %dl
-; X86-NEXT:    movb %dl, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
 ; X86-NEXT:    leal {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    pushl %eax
 ; X86-NEXT:    pushl %ecx
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl %edx
 ; X86-NEXT:    calll __modti3
 ; X86-NEXT:    addl $32, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    andl $3, %eax
 ; X86-NEXT:    xorl %ecx, %ecx
 ; X86-NEXT:    cmpl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    sbbl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    sbbl %eax, %esi
 ; X86-NEXT:    sbbl $0, %ecx
 ; X86-NEXT:    setae %dl
 ; X86-NEXT:    sbbb %cl, %cl
 ; X86-NEXT:    testb %al, %al
 ; X86-NEXT:    setne %ch
 ; X86-NEXT:    setne (%eax)
-; X86-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %dh # 1-byte Reload
-; X86-NEXT:    movzbl %dh, %eax
-; X86-NEXT:    # kill: def $eax killed $eax def $ax
-; X86-NEXT:    divb %dh
 ; X86-NEXT:    negb %ch
 ; X86-NEXT:    negb %dl
-; X86-NEXT:    cmpb %al, %al
-; X86-NEXT:    setle %al
+; X86-NEXT:    cmpb $2, %al
+; X86-NEXT:    setl %al
 ; X86-NEXT:    negb %al
 ; X86-NEXT:    cbtw
 ; X86-NEXT:    idivb %dl
 ; X86-NEXT:    movsbl %ah, %eax
 ; X86-NEXT:    movzbl %al, %eax
 ; X86-NEXT:    andl $1, %eax
 ; X86-NEXT:    negl %eax
 ; X86-NEXT:    leal (%eax,%eax,2), %eax
-; X86-NEXT:    leal -4(%esp,%eax,4), %eax
+; X86-NEXT:    leal -12(%esp,%eax,4), %eax
 ; X86-NEXT:    movl %eax, (%eax)
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    cbtw
 ; X86-NEXT:    idivb %ch
 ; X86-NEXT:    movsbl %ah, %eax
 ; X86-NEXT:    andb $1, %al
 ; X86-NEXT:    movb %al, (%eax)
 ; X86-NEXT:    leal -12(%ebp), %esp

 ; X64-NEXT:    shlq $32, %rsi
 ; X64-NEXT:    orq %rax, %rsi
 ; X64-NEXT:    movq %rsi, %rdi
 ; X64-NEXT:    shlq $30, %rdi
 ; X64-NEXT:    sarq $30, %rdi
 ; X64-NEXT:    movb (%rax), %al
 ; X64-NEXT:    movzbl %al, %eax
 ; X64-NEXT:    # kill: def $eax killed $eax def $ax
-; X64-NEXT:    divb %al
+; X64-NEXT:    divb (%rax)
 ; X64-NEXT:    movl %eax, %r8d
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    xorl %edx, %edx
 ; X64-NEXT:    idivq %rdi
 ; X64-NEXT:    movabsq $17179869183, %rax # imm = 0x3FFFFFFFF
 ; X64-NEXT:    andq %rdx, %rax
 ; X64-NEXT:    testb %al, %al
 ; X64-NEXT:    setne %dil

 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    andl $-8, %esp
 ; X86-NEXT:    subl $16, %esp
 ; X86-NEXT:    .cfi_offset %esi, -20
 ; X86-NEXT:    .cfi_offset %edi, -16
 ; X86-NEXT:    .cfi_offset %ebx, -12
 ; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    movl %esi, %eax
-; X86-NEXT:    shll $30, %eax
-; X86-NEXT:    sarl $30, %eax
+; X86-NEXT:    movl %esi, %ecx
+; X86-NEXT:    shll $30, %ecx
+; X86-NEXT:    sarl $30, %ecx
 ; X86-NEXT:    movl (%esp), %edi
-; X86-NEXT:    movb (%eax), %bl
-; X86-NEXT:    pushl %eax
+; X86-NEXT:    movb (%eax), %al
+; X86-NEXT:    movzbl %al, %eax
+; X86-NEXT:    # kill: def $eax killed $eax def $ax
+; X86-NEXT:    divb (%eax)
+; X86-NEXT:    movl %eax, %ebx
+; X86-NEXT:    pushl %ecx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    calll __moddi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    andl $3, %edx
+; X86-NEXT:    testb %al, %al
+; X86-NEXT:    setne (%eax)
 ; X86-NEXT:    cmpl %eax, %edi
 ; X86-NEXT:    sbbl %edx, %esi
-; X86-NEXT:    setb %dl
-; X86-NEXT:    setae %dh
+; X86-NEXT:    setae %dl
+; X86-NEXT:    sbbb %cl, %cl
 ; X86-NEXT:    testb %al, %al
-; X86-NEXT:    setne %bh
-; X86-NEXT:    setne (%eax)
-; X86-NEXT:    movzbl %bl, %eax
-; X86-NEXT:    xorl %ecx, %ecx
-; X86-NEXT:    subb %dl, %cl
-; X86-NEXT:    # kill: def $eax killed $eax def $ax
-; X86-NEXT:    divb %bl
-; X86-NEXT:    negb %dh
-; X86-NEXT:    cmpb %al, %al
+; X86-NEXT:    setne %ch
+; X86-NEXT:    negb %dl
+; X86-NEXT:    cmpb %bl, %al
 ; X86-NEXT:    setle %al
 ; X86-NEXT:    negb %al
 ; X86-NEXT:    cbtw
-; X86-NEXT:    idivb %dh
+; X86-NEXT:    idivb %dl
 ; X86-NEXT:    movsbl %ah, %eax
 ; X86-NEXT:    movzbl %al, %eax
 ; X86-NEXT:    andl $1, %eax
 ; X86-NEXT:    shll $3, %eax
 ; X86-NEXT:    negl %eax
-; X86-NEXT:    negb %bh
+; X86-NEXT:    negb %ch
 ; X86-NEXT:    leal -8(%esp,%eax), %eax
 ; X86-NEXT:    movl %eax, (%eax)
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    cbtw
-; X86-NEXT:    idivb %bh
+; X86-NEXT:    idivb %ch
 ; X86-NEXT:    movsbl %ah, %eax
 ; X86-NEXT:    andb $1, %al
 ; X86-NEXT:    movb %al, (%eax)
 ; X86-NEXT:    leal -12(%ebp), %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    .cfi_def_cfa %esp, 4
 ; X86-NEXT:    retl
 BB:
   %A30 = alloca i34
   %L17 = load i34, i34* %A30
   %B20 = and i34 %L17, -1
   %G2 = getelementptr i34, i34* %A30, i1 true
-  %L10 = load i8, i8* undef
-  %B6 = udiv i8 %L10, %L10
+  %L10 = load volatile i8, i8* undef
+  %L11 = load volatile i8, i8* undef
+  %B6 = udiv i8 %L10, %L11
   %C15 = icmp eq i8 undef, 0
   %B8 = srem i34 0, %B20
   %C2 = icmp ule i34 %B8, %B20
   %B5 = or i8 0, %B6
   %C19 = icmp uge i1 false, %C2
   %C1 = icmp sle i8 undef, %B5
   %B37 = srem i1 %C1, %C2
   %C7 = icmp uge i1 false, %C15
   store i1 %C7, i1* undef
   %G6 = getelementptr i34, i34* %G2, i1 %B37
   store i34* %G6, i34** undef
   %B30 = srem i1 %C19, %C7
   store i1 %B30, i1* undef
   ret void
 }

test/CodeGen/X86/vector-idiv-v2i32.ll

 
 define void @test_urem_v2i32(<2 x i32>* %x, <2 x i32>* %y, <2 x i32>* %z) nounwind {
 ; X64-LABEL: test_urem_v2i32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; X64-NEXT:    pxor %xmm1, %xmm1
 ; X64-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
+; X64-NEXT:    movq {{.*#+}} xmm2 = mem[0],zero
+; X64-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
 ; X64-NEXT:    movq %xmm0, %rax
+; X64-NEXT:    movq %xmm2, %rsi
 ; X64-NEXT:    xorl %edx, %edx
-; X64-NEXT:    divq %rax
+; X64-NEXT:    divq %rsi
 ; X64-NEXT:    movq %rdx, %xmm1
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,0,1]
 ; X64-NEXT:    movq %xmm0, %rax
+; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[2,3,0,1]
+; X64-NEXT:    movq %xmm0, %rsi
 ; X64-NEXT:    xorl %edx, %edx
-; X64-NEXT:    divq %rax
+; X64-NEXT:    divq %rsi
 ; X64-NEXT:    movq %rdx, %xmm0
 ; X64-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
 ; X64-NEXT:    movq %xmm0, (%rcx)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: test_urem_v2i32:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    subl $40, %esp
+; X86-NEXT:    subl $56, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; X86-NEXT:    movups %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
-; X86-NEXT:    movss %xmm0, {{[0-9]+}}(%esp)
-; X86-NEXT:    movss %xmm0, (%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
+; X86-NEXT:    pxor %xmm1, %xmm1
+; X86-NEXT:    movdqa %xmm0, %xmm2
+; X86-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
+; X86-NEXT:    movdqu %xmm2, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
+; X86-NEXT:    movq {{.*#+}} xmm2 = mem[0],zero
+; X86-NEXT:    movdqa %xmm2, %xmm3
+; X86-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
+; X86-NEXT:    movd %xmm2, {{[0-9]+}}(%esp)
+; X86-NEXT:    movd %xmm0, (%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; X86-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[2,3,0,1]
+; X86-NEXT:    movdqu %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X86-NEXT:    calll __umoddi3
 ; X86-NEXT:    movups {{[-0-9]+}}(%e{{[sb]}}p), %xmm0 # 16-byte Reload
-; X86-NEXT:    shufps {{.*#+}} xmm0 = xmm0[1,1,0,1]
 ; X86-NEXT:    movss %xmm0, {{[0-9]+}}(%esp)
-; X86-NEXT:    movss %xmm0, (%esp)
+; X86-NEXT:    movdqu {{[-0-9]+}}(%e{{[sb]}}p), %xmm0 # 16-byte Reload
+; X86-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,0,1]
+; X86-NEXT:    movd %xmm0, (%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movd %eax, %xmm0
 ; X86-NEXT:    movdqu %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X86-NEXT:    calll __umoddi3
 ; X86-NEXT:    movd %eax, %xmm0
 ; X86-NEXT:    movdqu {{[-0-9]+}}(%e{{[sb]}}p), %xmm1 # 16-byte Reload
 ; X86-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X86-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
 ; X86-NEXT:    movq %xmm0, (%esi)
-; X86-NEXT:    addl $40, %esp
+; X86-NEXT:    addl $56, %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    retl
   %a = load <2 x i32>, <2 x i32>* %x
-  %b = load <2 x i32>, <2 x i32>* %x
+  %b = load <2 x i32>, <2 x i32>* %y
   %c = urem <2 x i32> %a, %b
   store <2 x i32> %c, <2 x i32>* %z
   ret void
 }
 
 define void @test_sdiv_v2i32(<2 x i32>* %x, <2 x i32>* %y, <2 x i32>* %z) nounwind {
 ; X64-LABEL: test_sdiv_v2i32:
 ; X64:       # %bb.0:
-; X64-NEXT:    movq %rdx, %rcx
-; X64-NEXT:    movslq (%rdi), %rsi
-; X64-NEXT:    movslq 4(%rdi), %rdi
-; X64-NEXT:    movq %rdi, %rax
+; X64-NEXT:    movq %rdx, %r8
+; X64-NEXT:    movslq (%rdi), %rcx
+; X64-NEXT:    movslq 4(%rdi), %rax
+; X64-NEXT:    movslq (%rsi), %rdi
+; X64-NEXT:    movslq 4(%rsi), %rsi
 ; X64-NEXT:    cqto
-; X64-NEXT:    idivq %rdi
+; X64-NEXT:    idivq %rsi
 ; X64-NEXT:    movq %rax, %xmm0
-; X64-NEXT:    movq %rsi, %rax
+; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    cqto
-; X64-NEXT:    idivq %rsi
+; X64-NEXT:    idivq %rdi
 ; X64-NEXT:    movq %rax, %xmm1
 ; X64-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
-; X64-NEXT:    movq %xmm0, (%rcx)
+; X64-NEXT:    movq %xmm0, (%r8)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: test_sdiv_v2i32:
 ; X86:       # %bb.0:
+; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    subl $16, %esp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    subl $44, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl (%eax), %edi
-; X86-NEXT:    movl 4(%eax), %eax
-; X86-NEXT:    movl %edi, %ebx
-; X86-NEXT:    sarl $31, %ebx
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    sarl $31, %ecx
-; X86-NEXT:    pushl %ecx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl (%ecx), %edi
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl 4(%ecx), %esi
+; X86-NEXT:    sarl $31, %edi
+; X86-NEXT:    movl %esi, %edx
+; X86-NEXT:    sarl $31, %edx
+; X86-NEXT:    movl (%eax), %ebx
+; X86-NEXT:    movl 4(%eax), %ecx
+; X86-NEXT:    movl %ebx, %ebp
+; X86-NEXT:    sarl $31, %ebp
+; X86-NEXT:    movl %ecx, %eax
+; X86-NEXT:    sarl $31, %eax
 ; X86-NEXT:    pushl %eax
 ; X86-NEXT:    pushl %ecx
-; X86-NEXT:    pushl %eax
+; X86-NEXT:    pushl %edx
+; X86-NEXT:    pushl %esi
 ; X86-NEXT:    calll __divdi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    movd %eax, %xmm0
-; X86-NEXT:    movdqu %xmm0, (%esp) # 16-byte Spill
-; X86-NEXT:    pushl %ebx
-; X86-NEXT:    pushl %edi
+; X86-NEXT:    movdqu %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
+; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
+; X86-NEXT:    pushl {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    calll __divdi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    movd %eax, %xmm0
-; X86-NEXT:    movdqu (%esp), %xmm1 # 16-byte Reload
+; X86-NEXT:    movdqu {{[-0-9]+}}(%e{{[sb]}}p), %xmm1 # 16-byte Reload
 ; X86-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; X86-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
-; X86-NEXT:    movq %xmm0, (%esi)
-; X86-NEXT:    addl $16, %esp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movq %xmm0, (%eax)
+; X86-NEXT:    addl $44, %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
+; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
   %a = load <2 x i32>, <2 x i32>* %x
-  %b = load <2 x i32>, <2 x i32>* %x
+  %b = load <2 x i32>, <2 x i32>* %y
   %c = sdiv <2 x i32> %a, %b
   store <2 x i32> %c, <2 x i32>* %z
   ret void
 }
 
 define void @test_srem_v2i32(<2 x i32>* %x, <2 x i32>* %y, <2 x i32>* %z) nounwind {
 ; X64-LABEL: test_srem_v2i32:
 ; X64:       # %bb.0:
-; X64-NEXT:    movq %rdx, %rcx
-; X64-NEXT:    movslq (%rdi), %rsi
-; X64-NEXT:    movslq 4(%rdi), %rdi
-; X64-NEXT:    movq %rdi, %rax
+; X64-NEXT:    movq %rdx, %r8
+; X64-NEXT:    movslq (%rdi), %rcx
+; X64-NEXT:    movslq 4(%rdi), %rax
+; X64-NEXT:    movslq (%rsi), %rdi
+; X64-NEXT:    movslq 4(%rsi), %rsi
 ; X64-NEXT:    cqto
-; X64-NEXT:    idivq %rdi
+; X64-NEXT:    idivq %rsi
 ; X64-NEXT:    movq %rax, %xmm0
-; X64-NEXT:    movq %rsi, %rax
+; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    cqto
-; X64-NEXT:    idivq %rsi
+; X64-NEXT:    idivq %rdi
 ; X64-NEXT:    movq %rax, %xmm1
 ; X64-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X64-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
-; X64-NEXT:    movq %xmm0, (%rcx)
+; X64-NEXT:    movq %xmm0, (%r8)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: test_srem_v2i32:
 ; X86:       # %bb.0:
+; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    subl $16, %esp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; X86-NEXT:    subl $44, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl (%eax), %edi
-; X86-NEXT:    movl 4(%eax), %eax
-; X86-NEXT:    movl %edi, %ebx
-; X86-NEXT:    sarl $31, %ebx
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    sarl $31, %ecx
-; X86-NEXT:    pushl %ecx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl (%ecx), %edi
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl 4(%ecx), %esi
+; X86-NEXT:    sarl $31, %edi
+; X86-NEXT:    movl %esi, %edx
+; X86-NEXT:    sarl $31, %edx
+; X86-NEXT:    movl (%eax), %ebx
+; X86-NEXT:    movl 4(%eax), %ecx
+; X86-NEXT:    movl %ebx, %ebp
+; X86-NEXT:    sarl $31, %ebp
+; X86-NEXT:    movl %ecx, %eax
+; X86-NEXT:    sarl $31, %eax
 ; X86-NEXT:    pushl %eax
 ; X86-NEXT:    pushl %ecx
-; X86-NEXT:    pushl %eax
+; X86-NEXT:    pushl %edx
+; X86-NEXT:    pushl %esi
 ; X86-NEXT:    calll __divdi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    movd %eax, %xmm0
-; X86-NEXT:    movdqu %xmm0, (%esp) # 16-byte Spill
-; X86-NEXT:    pushl %ebx
-; X86-NEXT:    pushl %edi
+; X86-NEXT:    movdqu %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
+; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
+; X86-NEXT:    pushl {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    calll __divdi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    movd %eax, %xmm0
-; X86-NEXT:    movdqu (%esp), %xmm1 # 16-byte Reload
+; X86-NEXT:    movdqu {{[-0-9]+}}(%e{{[sb]}}p), %xmm1 # 16-byte Reload
 ; X86-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; X86-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
-; X86-NEXT:    movq %xmm0, (%esi)
-; X86-NEXT:    addl $16, %esp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movq %xmm0, (%eax)
+; X86-NEXT:    addl $44, %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    popl %ebx
+; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
   %a = load <2 x i32>, <2 x i32>* %x
-  %b = load <2 x i32>, <2 x i32>* %x
+  %b = load <2 x i32>, <2 x i32>* %y
   %c = sdiv <2 x i32> %a, %b
   store <2 x i32> %c, <2 x i32>* %z
   ret void
 }