[X86] Enable the post-RA-scheduler for 32-bit cpus

[mbodart]

Summary

This change set enables the post RA scheduler for pentium4 and SSE3 class cpus.

The intent is that a vanilla "clang -m32 -O2" compilation, with no specific
-march setting, will get post scheduled. This has demonstrated significant
performance improvements when the code is run on Silvermont, with
essentially neutral performance on avx2 systems.

Some silvermont highlights include over 20% improvements
in 456.hmmer and several EEMBC benchmarks, and 4 to 15%
improvements in over a dozen other industry benchmarks.
There were only a few drops, in the 2 to 4% range.

On an AVX2 system, performance was generally flat, with
a balance of smallish gains and losses (in the 2 to 4% range).

The key scheduling improvement is that loads get separated
from their users.

As coded, the change should not affect "clang -m64" compilations.

With -m32, clang currently defaults to -mcpu=pentium4.
So I changed the scheduling model for pentium4 from the
GenericModel, to a new GenericPostRAModel (same properties,
but also enables the PostRAScheduler).

As clang's default CPU could theoretically change, I also
made the same change for "nearby" cpus pentium-m, pentium4m,
prescott and nocona. Arguably yonah should be in this set
as well, for consistency. But I'd like to get feedback on
whether this is an OK approach overall, as changing yonah will
affect many lit tests. I suspect most of these older cpus
are no longer used in practice, but don't really know.

[aaboud]

lib/Target/X86/X86.td

290

Can you create a class for GenereicPostRAModel, similar to class Proc, something like this:

class ProcPostRA<string Name, list<SubtargetFeature> Features>
: ProcessorModel<Name, GenereicPostRAModel, Features>;

test/CodeGen/X86/pr16360.ll

2

Why did not you simply add the "-post-RA-scheduler=false" flag like in other tests?

[mbodart]

lib/Target/X86/X86.td

290

I think that might add more confusion than clarity.

There are currently two base classes used to define the cpus, Proc and ProcessorModel.
ProcessorModel is used in cases where we want to override the scheduling model.
Post-RA scheduling is just one property of the scheduling model.

In the cases where we do create a new class derived from ProcessorModel, it is done to encapsulate the processor "features" in one place.

So my preference is to keep the current approach, though it's a minor enough detail
that I can change it if others feel strongly.

test/CodeGen/X86/pr16360.ll

2

As a general rule I would think we want to avoid adding special options to tests,
as that can sometimes mask issues. But there are some exceptions.

I disabled the post-RA-scheduler in misched-ilp.ll because that test is looking for
a specific behavior from an earlier scheduling phase.

As for machine-cp.ll, I could go either way (updating the test or disabling the post scheduler).
The test is built with the x86_64 triple and nocona cpu. If built by clang with an x86_64 triple,
the post scheduler would not be enabled as the cpu would be set to "x86_64", not nocona.
So I arbitrarily chose to match this behavior. But I could simply update the test as well.
Is there any precedence or BKM here?

[spatel]

test/CodeGen/X86/pr16360.ll

2

I don't think this is actually written anywhere as an official guideline, but yes, I think we should prefer to only use special options when they are supposed to affect the outcome of the test. That gives us better test coverage for the common case.

There's a huge pile of x86 tests that excessively specify a CPU model when they really should specify an attribute (for example as seen in this patch, -mcpu=pentium4 rather than -mattr=sse2). We should try to fix those as we encounter them.

So my vote would be to update all of the affected test files with the flags that they are really testing (these would be test-file-only commits ahead of this one). It will make this patch smaller and the intended effect of this patch will be made clear rather than confused by a bunch of unintentional changes because test files were poorly specified.

[mbodart]

test/CodeGen/X86/pr16360.ll

2

Thanks for the suggestions Sanjay.

I created D19568 for the test changes, and will remove them from this review.

But note that the new test still needs use of -mcpu. The scheduler behavior
is tied to the cpu, not -mattr, AFAICT. If you know of a better mechanism for
verifying the post-scheduling behavior for a vanilla "clang -m32 -O2" compilation,
I'd be happy to use it.

[mbodart]

Rebased, and removed several lit tests from this change set as I updated them separately.

[spatel]

LGTM.

A couple of options to make sure the behavior is as expected:

1. To confirm the codegen difference with -post-RA-scheduler, you could add a RUN (and corresponding CHECKs) that disables -post-RA-scheduler for one of the tested CPUs. Or add a run line for a newer CPU that does not have -post-RA-scheduler turned on by default.

2. To confirm that the default i386 CPU model has -post-RA-scheduler enabled, you could add a RUN line that doesn't explicitly specify the CPU. Then if someone comes along and changes the default i386 target to something not in the current list, it should cause this test to fail.

[mbodart]

Thanks for the testing suggestions Sanjay, but I think I'll stick with the current test for now.

Regarding 1), that would only test that some other scheduling paradigm did not separate the loads, which may or may not be a desired behavior, and may change from time to time. So I prefer to limit the test to getting the expected behavior in the known settings where it is desired.

As for 2), the default cpu varies by tool. With clang -m32 it is pentium4 and with clang -m64 it is x86_64,
but with llc it is generic. So the test would indeed fail if I added an llc RUN case with no explicit cpu.

lib/Target/X86/X86.td

 def : Proc<"i686",            [FeatureX87, FeatureSlowUAMem16]>;
 def : Proc<"pentiumpro",      [FeatureX87, FeatureSlowUAMem16, FeatureCMOV]>;
 def : Proc<"pentium2",        [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
                                FeatureCMOV, FeatureFXSR]>;
 def : Proc<"pentium3",        [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
                                FeatureSSE1, FeatureFXSR]>;
 def : Proc<"pentium3m",       [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
                                FeatureSSE1, FeatureFXSR, FeatureSlowBTMem]>;
-def : Proc<"pentium-m",       [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
-                               FeatureSSE2, FeatureFXSR, FeatureSlowBTMem]>;
-def : Proc<"pentium4",        [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
-                               FeatureSSE2, FeatureFXSR]>;
-def : Proc<"pentium4m",       [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
-                               FeatureSSE2, FeatureFXSR, FeatureSlowBTMem]>;
+
+// Enable the PostRAScheduler for SSE2 and SSE3 class cpus.
+// The intent is to enable it for pentium4 which is the current default
+// processor in a vanilla 32-bit clang compilation when no specific
+// architecture is specified.  This generally gives a nice performance
+// increase on silvermont, with largely neutral behavior on other
+// contemporary large core processors.
+// pentium-m, pentium4m, prescott and nocona are included as a preventative
+// measure to avoid performance surprises, in case clang's default cpu
+// changes slightly.
+
+def : ProcessorModel<"pentium-m", GenericPostRAModel,
+                     [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
+
+def : ProcessorModel<"pentium4", GenericPostRAModel,
+                     [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
+
+def : ProcessorModel<"pentium4m", GenericPostRAModel,
+                     [FeatureX87, FeatureSlowUAMem16, FeatureMMX,
 
 // Intel Quark.
 def : Proc<"lakemont",        []>;
 
 // Intel Core Duo.
 def : ProcessorModel<"yonah", SandyBridgeModel,
                      [FeatureX87, FeatureSlowUAMem16, FeatureMMX, FeatureSSE3,
                       FeatureFXSR, FeatureSlowBTMem]>;
 
 // NetBurst.
-def : Proc<"prescott",
+def : ProcessorModel<"prescott", GenericPostRAModel,
                      [FeatureX87, FeatureSlowUAMem16, FeatureMMX, FeatureSSE3,
                       FeatureFXSR, FeatureSlowBTMem]>;
-def : Proc<"nocona", [
+def : ProcessorModel<"nocona", GenericPostRAModel, [
   FeatureX87,
   FeatureSlowUAMem16,
   FeatureMMX,
   FeatureSSE3,
   FeatureFXSR,
   FeatureCMPXCHG16B,
   FeatureSlowBTMem
 ]>;

lib/Target/X86/X86Schedule.td

 // number of in-flight instructions.
 //
 // HighLatency=10 is optimistic. X86InstrInfo::isHighLatencyDef
 // indicates high latency opcodes. Alternatively, InstrItinData
 // entries may be included here to define specific operand
 // latencies. Since these latencies are not used for pipeline hazards,
 // they do not need to be exact.
 //
-// The GenericModel contains no instruction itineraries.
-def GenericModel : SchedMachineModel {
+// The GenericX86Model contains no instruction itineraries
+// and disables PostRAScheduler.
+class GenericX86Model : SchedMachineModel {
   let IssueWidth = 4;
   let MicroOpBufferSize = 32;
   let LoadLatency = 4;
   let HighLatency = 10;
   let PostRAScheduler = 0;
   let CompleteModel = 0;
 }
 
+def GenericModel : GenericX86Model;
+
+// Define a model with the PostRAScheduler enabled.
+def GenericPostRAModel : GenericX86Model {
+  let PostRAScheduler = 1;
+}
+
 include "X86ScheduleAtom.td"
 include "X86SchedSandyBridge.td"
 include "X86SchedHaswell.td"
 include "X86ScheduleSLM.td"
 include "X86ScheduleBtVer2.td"
 

test/CodeGen/X86/machine-cp.ll

-; RUN: llc -mtriple=x86_64-apple-macosx -mcpu=nocona -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-apple-macosx -mcpu=nocona -verify-machineinstrs -post-RA-scheduler=false < %s | FileCheck %s
 
 ; After tail duplication, two copies in an early exit BB can be cancelled out.
 ; rdar://10640363
 define i32 @t1(i32 %a, i32 %b) nounwind  {
 entry:
 ; CHECK-LABEL: t1:
 ; CHECK: je [[LABEL:.*BB.*]]
   %cmp1 = icmp eq i32 %b, 0

test/CodeGen/X86/misched-ilp.ll

-; RUN: llc < %s -mtriple=x86_64-apple-macosx -mcpu=nocona -enable-misched -misched=ilpmax | FileCheck -check-prefix=MAX %s
-; RUN: llc < %s -mtriple=x86_64-apple-macosx -mcpu=nocona -enable-misched -misched=ilpmin | FileCheck -check-prefix=MIN %s
+; RUN: llc < %s -mtriple=x86_64-apple-macosx -mcpu=nocona -enable-misched -misched=ilpmax -post-RA-scheduler=false | FileCheck -check-prefix=MAX %s
+; RUN: llc < %s -mtriple=x86_64-apple-macosx -mcpu=nocona -enable-misched -misched=ilpmin -post-RA-scheduler=false | FileCheck -check-prefix=MIN %s
 ;
 ; Basic verification of the ScheduleDAGILP metric.
 ;
 ; MAX: addss
 ; MAX: addss
 ; MAX: addss
 ; MAX: subss
 ; MAX: addss

test/CodeGen/X86/post-ra-sched.ll

+; RUN: llc < %s -mtriple=i386 -mcpu=pentium4 | FileCheck %s
+; RUN: llc < %s -mtriple=i386 -mcpu=pentium4m | FileCheck %s
+; RUN: llc < %s -mtriple=i386 -mcpu=pentium-m | FileCheck %s
+; RUN: llc < %s -mtriple=i386 -mcpu=prescott | FileCheck %s
+; RUN: llc < %s -mtriple=i386 -mcpu=nocona | FileCheck %s
+;
+; Verify that scheduling puts some distance between a load feeding into
+; the address of another load, and that second load.  This currently
+; happens during the post-RA-scheduler, which should be enabled by
+; default with the above specified cpus.
+
+@ptrs = external global [0 x i32*], align 4
+@idxa = common global i32 0, align 4
+@idxb = common global i32 0, align 4
+@res = common global i32 0, align 4
+
+define void @addindirect() {
+; CHECK-LABEL: addindirect:
+; CHECK:       # BB#0: # %entry
+; CHECK-NEXT:    movl idxb, %ecx
+; CHECK-NEXT:    movl idxa, %eax
+; CHECK-NEXT:    movl ptrs(,%ecx,4), %ecx
+; CHECK-NEXT:    movl ptrs(,%eax,4), %eax
+; CHECK-NEXT:    movl (%ecx), %ecx
+; CHECK-NEXT:    addl (%eax), %ecx
+; CHECK-NEXT:    movl %ecx, res
+; CHECK-NEXT:    retl
+entry:
+  %0 = load i32, i32* @idxa, align 4
+  %arrayidx = getelementptr inbounds [0 x i32*], [0 x i32*]* @ptrs, i32 0, i32 %0
+  %1 = load i32*, i32** %arrayidx, align 4
+  %2 = load i32, i32* %1, align 4
+  %3 = load i32, i32* @idxb, align 4
+  %arrayidx1 = getelementptr inbounds [0 x i32*], [0 x i32*]* @ptrs, i32 0, i32 %3
+  %4 = load i32*, i32** %arrayidx1, align 4
+  %5 = load i32, i32* %4, align 4
+  %add = add i32 %5, %2
+  store i32 %add, i32* @res, align 4
+  ret void
+}

test/CodeGen/X86/pr16360.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mcpu=pentium4 -mtriple=i686-pc-linux | FileCheck %s
 
 define i64 @foo(i32 %sum) {
 ; CHECK-LABEL: foo:
 ; CHECK:       # BB#0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl $1073741823, %edx # imm = 0x3FFFFFFF
 ; CHECK-NEXT:    shrl $2, %eax
 ; CHECK-NEXT:    orl $-67108864, %eax # imm = 0xFFFFFFFFFC000000
-; CHECK-NEXT:    movl $1073741823, %edx # imm = 0x3FFFFFFF
 ; CHECK-NEXT:    retl
 entry:
   %conv = sext i32 %sum to i64
   %shr = lshr i64 %conv, 2
   %or = or i64 4611686018360279040, %shr
   ret i64 %or
 }

test/CodeGen/X86/sse2.ll

+; NOTE: Assertions have been autogenerated by update_llc_test_checks.py
 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; Tests for SSE2 and below, without SSE3+.
 ; RUN: llc < %s -mtriple=i386-apple-darwin10 -mcpu=pentium4 -O3 | FileCheck %s
 
 define void @test1(<2 x double>* %r, <2 x double>* %A, double %B) nounwind  {
 ; CHECK-LABEL: test1:
 ; CHECK:       ## BB#0:
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movapd (%ecx), %xmm0
 ; CHECK-NEXT:    movlpd {{.*#+}} xmm0 = mem[0],xmm0[1]
 ; CHECK-NEXT:    movapd %xmm0, (%eax)
 ; CHECK-NEXT:    retl
   %tmp3 = load <2 x double>, <2 x double>* %A, align 16
   %tmp7 = insertelement <2 x double> undef, double %B, i32 0
   %tmp9 = shufflevector <2 x double> %tmp3, <2 x double> %tmp7, <2 x i32> < i32 2, i32 1 >
   store <2 x double> %tmp9, <2 x double>* %r, align 16
   ret void
 }
 
 define void @test2(<2 x double>* %r, <2 x double>* %A, double %B) nounwind  {
 ; CHECK-LABEL: test2:
 ; CHECK:       ## BB#0:
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movapd (%ecx), %xmm0
 ; CHECK-NEXT:    movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
 ; CHECK-NEXT:    movapd %xmm0, (%eax)
 ; CHECK-NEXT:    retl
   %tmp3 = load <2 x double>, <2 x double>* %A, align 16
   %tmp7 = insertelement <2 x double> undef, double %B, i32 0
   %tmp9 = shufflevector <2 x double> %tmp3, <2 x double> %tmp7, <2 x i32> < i32 0, i32 2 >
   store <2 x double> %tmp9, <2 x double>* %r, align 16
   ret void
 }
 
 
 define void @test3(<4 x float>* %res, <4 x float>* %A, <4 x float>* %B) nounwind {
 ; CHECK-LABEL: test3:
 ; CHECK:       ## BB#0:
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movaps (%edx), %xmm0
 ; CHECK-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[1],mem[1]
 ; CHECK-NEXT:    movaps %xmm0, (%eax)
 ; CHECK-NEXT:    retl
   %tmp = load <4 x float>, <4 x float>* %B    ; <<4 x float>> [#uses=2]
   %tmp3 = load <4 x float>, <4 x float>* %A    ; <<4 x float>> [#uses=2]
   %tmp.upgrd.1 = extractelement <4 x float> %tmp3, i32 0    ; <float> [#uses=1]
   %tmp7 = extractelement <4 x float> %tmp, i32 0    ; <float> [#uses=1]
; CHECK-NEXT:    retl
   store <4 x float> %tmp5, <4 x float>* %res
   ret void
 }
 
 define <4 x i32> @test5(i8** %ptr) nounwind {
 ; CHECK-LABEL: test5:
 ; CHECK:       ## BB#0:
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    pxor %xmm0, %xmm0
 ; CHECK-NEXT:    movl (%eax), %eax
 ; CHECK-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; CHECK-NEXT:    pxor %xmm0, %xmm0
 ; CHECK-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
 ; CHECK-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
 ; CHECK-NEXT:    retl
   %tmp = load i8*, i8** %ptr    ; <i8*> [#uses=1]
   %tmp.upgrd.1 = bitcast i8* %tmp to float*    ; <float*> [#uses=1]
   %tmp.upgrd.2 = load float, float* %tmp.upgrd.1    ; <float> [#uses=1]
   %tmp.upgrd.3 = insertelement <4 x float> undef, float %tmp.upgrd.2, i32 0    ; <<4 x float>> [#uses=1]
   %tmp9 = insertelement <4 x float> %tmp.upgrd.3, float 0.000000e+00, i32 1    ; <<4 x float>> [#uses=1]
   %tmp10 = insertelement <4 x float> %tmp9, float 0.000000e+00, i32 2    ; <<4 x float>> [#uses=1]
   %tmp11 = insertelement <4 x float> %tmp10, float 0.000000e+00, i32 3    ; <<4 x float>> [#uses=1]
   %tmp21 = bitcast <4 x float> %tmp11 to <16 x i8>    ; <<16 x i8>> [#uses=1]
   %tmp22 = shufflevector <16 x i8> %tmp21, <16 x i8> zeroinitializer, <16 x i32> < i32 0, i32 16, i32 1, i32 17, i32 2, i32 18, i32 3, i32 19, i32 4, i32 20, i32 5, i32 21, i32 6, i32 22, i32 7, i32 23 >    ; <<16 x i8>> [#uses=1]
   %tmp31 = bitcast <16 x i8> %tmp22 to <8 x i16>    ; <<8 x i16>> [#uses=1]
   %tmp.upgrd.4 = shufflevector <8 x i16> zeroinitializer, <8 x i16> %tmp31, <8 x i32> < i32 0, i32 8, i32 1, i32 9, i32 2, i32 10, i32 3, i32 11 >    ; <<8 x i16>> [#uses=1]
   %tmp36 = bitcast <8 x i16> %tmp.upgrd.4 to <4 x i32>    ; <<4 x i32>> [#uses=1]
   ret <4 x i32> %tmp36
 }
 
 define void @test6(<4 x float>* %res, <4 x float>* %A) nounwind {
 ; CHECK-LABEL: test6:
 ; CHECK:       ## BB#0:
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movaps (%ecx), %xmm0
 ; CHECK-NEXT:    movaps %xmm0, (%eax)
 ; CHECK-NEXT:    retl
   %tmp1 = load <4 x float>, <4 x float>* %A            ; <<4 x float>> [#uses=1]
   %tmp2 = shufflevector <4 x float> %tmp1, <4 x float> undef, <4 x i32> < i32 0, i32 5, i32 6, i32 7 >          ; <<4 x float>> [#uses=1]
   store <4 x float> %tmp2, <4 x float>* %res
   ret void
 }
; CHECK-NEXT:    retl
   ret <2 x double> %tmp7
 }
 
 define void @test12() nounwind {
 ; CHECK-LABEL: test12:
 ; CHECK:       ## BB#0:
 ; CHECK-NEXT:    movapd 0, %xmm0
 ; CHECK-NEXT:    movapd {{.*#+}} xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; CHECK-NEXT:    movsd {{.*#+}} xmm1 = xmm0[0],xmm1[1]
 ; CHECK-NEXT:    xorpd %xmm2, %xmm2
+; CHECK-NEXT:    movsd {{.*#+}} xmm1 = xmm0[0],xmm1[1]
 ; CHECK-NEXT:    unpckhpd {{.*#+}} xmm0 = xmm0[1],xmm2[1]
 ; CHECK-NEXT:    addps %xmm1, %xmm0
 ; CHECK-NEXT:    movaps %xmm0, 0
 ; CHECK-NEXT:    retl
   %tmp1 = load <4 x float>, <4 x float>* null          ; <<4 x float>> [#uses=2]
   %tmp2 = shufflevector <4 x float> %tmp1, <4 x float> < float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00 >, <4 x i32> < i32 0, i32 1, i32 6, i32 7 >             ; <<4 x float>> [#uses=1]
   %tmp3 = shufflevector <4 x float> %tmp1, <4 x float> zeroinitializer, <4 x i32> < i32 2, i32 3, i32 6, i32 7 >                ; <<4 x float>> [#uses=1]
   %tmp4 = fadd <4 x float> %tmp2, %tmp3            ; <<4 x float>> [#uses=1]
   store <4 x float> %tmp4, <4 x float>* null
   ret void
 }
 
 define void @test13(<4 x float>* %res, <4 x float>* %A, <4 x float>* %B, <4 x float>* %C) nounwind {
 ; CHECK-LABEL: test13:
 ; CHECK:       ## BB#0:
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movaps (%edx), %xmm0
 ; CHECK-NEXT:    shufps {{.*#+}} xmm0 = xmm0[1,1],mem[0,1]
 ; CHECK-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2,1,3]
 ; CHECK-NEXT:    movaps %xmm0, (%eax)
 ; CHECK-NEXT:    retl
   %tmp3 = load <4 x float>, <4 x float>* %B            ; <<4 x float>> [#uses=1]
   %tmp5 = load <4 x float>, <4 x float>* %C            ; <<4 x float>> [#uses=1]
   %tmp11 = shufflevector <4 x float> %tmp3, <4 x float> %tmp5, <4 x i32> < i32 1, i32 4, i32 1, i32 5 >         ; <<4 x float>> [#uses=1]
   store <4 x float> %tmp11, <4 x float>* %res
   ret void
 }
 
 define <4 x float> @test14(<4 x float>* %x, <4 x float>* %y) nounwind {
 ; CHECK-LABEL: test14:
 ; CHECK:       ## BB#0:
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movaps (%ecx), %xmm1
 ; CHECK-NEXT:    movaps (%eax), %xmm2
+; CHECK-NEXT:    movaps (%ecx), %xmm1
 ; CHECK-NEXT:    movaps %xmm2, %xmm0
-; CHECK-NEXT:    addps %xmm1, %xmm0
 ; CHECK-NEXT:    subps %xmm1, %xmm2
+; CHECK-NEXT:    addps %xmm1, %xmm0
 ; CHECK-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; CHECK-NEXT:    retl
   %tmp = load <4 x float>, <4 x float>* %y             ; <<4 x float>> [#uses=2]
   %tmp5 = load <4 x float>, <4 x float>* %x            ; <<4 x float>> [#uses=2]
   %tmp9 = fadd <4 x float> %tmp5, %tmp             ; <<4 x float>> [#uses=1]
   %tmp21 = fsub <4 x float> %tmp5, %tmp            ; <<4 x float>> [#uses=1]
   %tmp27 = shufflevector <4 x float> %tmp9, <4 x float> %tmp21, <4 x i32> < i32 0, i32 1, i32 4, i32 5 >                ; <<4 x float>> [#uses=1]
   ret <4 x float> %tmp27
 }
 
 define <4 x float> @test15(<4 x float>* %x, <4 x float>* %y) nounwind {
 ; CHECK-LABEL: test15:
 ; CHECK:       ## BB#0: ## %entry
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movapd (%ecx), %xmm0
 ; CHECK-NEXT:    unpckhpd {{.*#+}} xmm0 = xmm0[1],mem[1]
 ; CHECK-NEXT:    retl
 entry:
   %tmp = load <4 x float>, <4 x float>* %y             ; <<4 x float>> [#uses=1]
   %tmp3 = load <4 x float>, <4 x float>* %x            ; <<4 x float>> [#uses=1]
   %tmp4 = shufflevector <4 x float> %tmp3, <4 x float> %tmp, <4 x i32> < i32 2, i32 3, i32 6, i32 7 >           ; <<4 x float>> [#uses=1]
   ret <4 x float> %tmp4
; CHECK-NEXT:    retl
   ret <4 x i32> %bc2
 }
 
 define <4 x i32> @test_mul(<4 x i32> %x, <4 x i32> %y) {
 ; CHECK-LABEL: test_mul:
 ; CHECK:       ## BB#0:
 ; CHECK-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[1,1,3,3]
 ; CHECK-NEXT:    pmuludq %xmm1, %xmm0
-; CHECK-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
 ; CHECK-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,3,3]
 ; CHECK-NEXT:    pmuludq %xmm2, %xmm1
+; CHECK-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
 ; CHECK-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[0,2,2,3]
 ; CHECK-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; CHECK-NEXT:    retl
   %m = mul <4 x i32> %x, %y
   ret <4 x i32> %m
 }