[x86] try harder to scalarize a vector load with extracted integer op uses

[spatel]

Summary

extract_vec_elt (load X), C --> scalar load (X+C)

As noted in the comment, DAGCombiner has this fold -- and the code in this patch is adapted from DAGCombiner::scalarizeExtractedVectorLoad() -- but x86 should benefit even if the loaded vector has other uses as long as we apply some other x86-specific conditions. The motivating example from #50310 is shown in vec_int_to_fp.ll.

I'm still looking over the diffs, but they all seem like wins so far.

[RKSimon]

Thanks for taking this on!

The loss of the shuffles is my main concern - I don't think EltsFromConsecutiveLoads can help us recover from this yet :(

llvm/test/CodeGen/X86/avx512-shuffles/partial_permute.ll

1725 ↗

(On Diff #403661)

yuck

llvm/test/CodeGen/X86/pr45378.ll

80–81

We might be able to get this as well if you can move the fold inside combineExtractWithShuffle ?

[spatel]

llvm/test/CodeGen/X86/avx512-shuffles/partial_permute.ll

1725 ↗

(On Diff #403661)

Yes, this diff suggests we should limit the fold a bit more. In this case, the extracts are immediately converted back to vector via scalar_to_vector or insert_vector_elt, and the sequence doesn't become a shuffle until later.

I suspect there's no 1 right answer about where to draw the line, but this is easy to avoid - we can just make a list of bailout user opcodes (in this 1st draft, it was only ISD::STORE).

[spatel]

Patch updated:
Try to avoid a missed shuffle opportunity by bailing out on more user opcodes.

[spatel]

llvm/test/CodeGen/X86/pr45378.ll

80–81

With only SSE2 (but not the other RUNs), the extract of element 1 is not legal, so it becomes a shuffle before we see it.

We might be able to adjust the isAfterLegalizeDAG predicate and get this, but that could also lead to unintended diffs. I'd prefer to go with the more conservative approach at first, so we don't jeopardize fixing the motivating bug.

[pengfei]

Thanks for the improvement! LGTM.

llvm/lib/Target/X86/X86ISelLowering.cpp

43238–43240

Add a regression test for it?

[RKSimon]

LGTM - please can you fix the clang-format warnings? Cheers.

[spatel]

llvm/lib/Target/X86/X86ISelLowering.cpp

43238–43240

There are examples of this in the existing tests diffs - we have both a scalar load and vector load from the same address.
See:
vec_cast.ll -> define <3 x i16> @h(<3 x i32> %a)
vec_int_to_fp.ll -> define <2 x double> @sitofp_load_2i64_to_2f64(<2 x i64> *%a)

We might still be able to do better on those tests, but that demonstrates what I suggested in this comment. Does that make sense? I can try to come up with another case if the existing tests are not clear.

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

43238–43240

I see your point, it's fine. Thank you.

[spatel]

Patch updated:

1. Fixed formatting (must have an expired flavor of clang-format on my system).
2. Added dedicated/minimal test to show the extra load vs. register transfer trade-off: @multi_use_load_scalarization

[spatel]

llvm/lib/Target/X86/X86ISelLowering.cpp

43238–43240

You're right, and it's not difficult to add a dedicated test. :)
e9768a2a44a1
The other tests may not show the minimum difference if this transform changes in the future.

[haowei]

Hi, we found out that this patch breaks following tests in the Fuchsia's stage-2 mac clang builders:

Clang :: Refactor/Extract/ExtractExprIntoFunction.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.m
Clang :: Refactor/Extract/FromMethodToFunction.cpp
Clang :: Refactor/Extract/ObjCProperty.m

An example of the failure: https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/overview . And test error messages can be found at https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/test-results . These failures only happens in stage 2 build so it took us some time to bisect and confirm it. Could you take a look please? And if it takes a long time to fix, could you revert this change first please? Thanks.

[spatel]

In D118376#3292090, @haowei wrote:

Hi, we found out that this patch breaks following tests in the Fuchsia's stage-2 mac clang builders:

Clang :: Refactor/Extract/ExtractExprIntoFunction.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.m
Clang :: Refactor/Extract/FromMethodToFunction.cpp
Clang :: Refactor/Extract/ObjCProperty.m

An example of the failure: https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/overview . And test error messages can be found at https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/test-results . These failures only happens in stage 2 build so it took us some time to bisect and confirm it. Could you take a look please? And if it takes a long time to fix, could you revert this change first please? Thanks.

Thanks for letting me know. I don't have a system set up to debug a stage 2 failure, so this is going to take a while unless someone else has a way to diagnose/reduce the errors more quickly. I will revert within a day if there is no update/progress.

[haowei]

In D118376#3293280, @spatel wrote:

In D118376#3292090, @haowei wrote:

Hi, we found out that this patch breaks following tests in the Fuchsia's stage-2 mac clang builders:

Clang :: Refactor/Extract/ExtractExprIntoFunction.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.cpp
Clang :: Refactor/Extract/ExtractionSemicolonPolicy.m
Clang :: Refactor/Extract/FromMethodToFunction.cpp
Clang :: Refactor/Extract/ObjCProperty.m

An example of the failure: https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/overview . And test error messages can be found at https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-x64/b8823769353085181809/test-results . These failures only happens in stage 2 build so it took us some time to bisect and confirm it. Could you take a look please? And if it takes a long time to fix, could you revert this change first please? Thanks.

Thanks for letting me know. I don't have a system set up to debug a stage 2 failure, so this is going to take a while unless someone else has a way to diagnose/reduce the errors more quickly. I will revert within a day if there is no update/progress.

I am not an expert on Mac. I did some experiments. Using the clang built from this revision (b4b97ec813a02585000f30ac7d532dda74e8bfda) to build stage 2 clang, regardless of the revision of the second stage clang, will cause clang-refactor related test to fail. So I think this patch introduced a bug that changed clang's code gen behavior and cause clang to generate buggy code on the Mac. I still don't quite understand why the clang-refactor tool is the only one affected so far. If you need any binaries from stage 1 and 2 build to debug this issue, please let me know. I can help build them for you.

[spatel]

In D118376#3295685, @haowei wrote:

I am not an expert on Mac. I did some experiments. Using the clang built from this revision (b4b97ec813a02585000f30ac7d532dda74e8bfda) to build stage 2 clang, regardless of the revision of the second stage clang, will cause clang-refactor related test to fail. So I think this patch introduced a bug that changed clang's code gen behavior and cause clang to generate buggy code on the Mac. I still don't quite understand why the clang-refactor tool is the only one affected so far. If you need any binaries from stage 1 and 2 build to debug this issue, please let me know. I can help build them for you.

Thanks - I'll take any help I can get to reduce the bug. :)
I have reverted the patch on main, and I think I should also revert it on the 14 branch (if you can confirm that the bug is present resolved with that revert).

Is it correct that the bug is only visible on the Mac build (the same tests are not failing on other platforms)?
It's not clear to me from the logs what the failure is: all 5 of the tests in "clang/test/Refactor/Extract" cause clang-refactor to crash?

[dyung]

Just a heads up that we potentially saw an issue with this commit on some of our internal tests. I've bisected it down to your commit, but I need to make sure it reproduces with an opensource compiler (without our private changes), and if so I will post a bug with the information.

[spatel]

In D118376#3297610, @dyung wrote:

Just a heads up that we potentially saw an issue with this commit on some of our internal tests. I've bisected it down to your commit, but I need to make sure it reproduces with an opensource compiler (without our private changes), and if so I will post a bug with the information.

Thanks! That would be great. I'm guessing this failed to account for some combination of load/store ordering, but I can't see the bug yet.

[lebedev.ri]

llvm/lib/Target/X86/X86ISelLowering.cpp

43260

I think i don't quite understand what is going on here,
but i really don't understand why we'd want to
replace SDValue(LoadVec, 1) (which is, i guess, LoadVec->getChain()?)
with the new Chain?
We only simply replace the extract with a new load,
why could we possibly invalidate the other memory chains?

This *reeks*. (Yes, this is copied from DAGCombiner::scalarizeExtractedVectorLoad())

[haowei]

In D118376#3296441, @spatel wrote:

In D118376#3295685, @haowei wrote:

I am not an expert on Mac. I did some experiments. Using the clang built from this revision (b4b97ec813a02585000f30ac7d532dda74e8bfda) to build stage 2 clang, regardless of the revision of the second stage clang, will cause clang-refactor related test to fail. So I think this patch introduced a bug that changed clang's code gen behavior and cause clang to generate buggy code on the Mac. I still don't quite understand why the clang-refactor tool is the only one affected so far. If you need any binaries from stage 1 and 2 build to debug this issue, please let me know. I can help build them for you.

Thanks - I'll take any help I can get to reduce the bug. :)
I have reverted the patch on main, and I think I should also revert it on the 14 branch (if you can confirm that the bug is present resolved with that revert).

Is it correct that the bug is only visible on the Mac build (the same tests are not failing on other platforms)?
It's not clear to me from the logs what the failure is: all 5 of the tests in "clang/test/Refactor/Extract" cause clang-refactor to crash?

I can confirm the stage 2 builds are green after the revert. I am doing some local builds to see if I can generate stage 2 clang-refactor binaries with and without your patch so I can hand them to you for comparison. I am also working on simpler steps to reproduce the issue without Fuchsia related stuff to it can be easier to debug. Will file a bug in github once I get there.

[efriedma]

llvm/lib/Target/X86/X86ISelLowering.cpp

43260

LoadVec->getChain() is the chain operand; SDValue(LoadVec, 1) is the chain result.

I suspect the issue here is that the chains aren't connected correctly. The current code RAUW's the chain output of LoadVec with the chain output of Load... and leaves the chain of LoadVec dangling. If LoadVec isn't dead, this is wrong; the chain result of LoadVec needs to stay connected.

[dyung]

I was able to reproduce the issue with a small testcase that I have filed as issue 53695. Hope that helps.

[spatel]

In D118376#3310203, @dyung wrote:

I was able to reproduce the issue with a small testcase that I have filed as issue 53695. Hope that helps.

That was very helpful - thanks!
If I'm seeing it correctly, that example shows that the bug fix for this patch requires using 'DAG.makeEquivalentMemoryOrdering(OriginalLoad, Load)'.
But the test example can also be modified slightly to expose a bug in the DAGCombiner code that this patch copied from. Ie, there's a visible miscompile even with this patch theoretically fixed.
I haven't been able to come up with a test that exposes the bug independently of applying this patch yet though.

[spatel]

Patch updated:

1. Use makeEquivalentMemoryOrdering (see D119549 for a similar change to the DAGCombiner code)
2. Added a regression test based on https://github.com/llvm/llvm-project/issues/53695 (with the old version of this patch, this test will fail).

[haowei]

The latest patch no longer causes clang-refactor test failures on stage2 mac build on Fuchsia builders.

[spatel]

In D118376#3316151, @haowei wrote:

The latest patch no longer causes clang-refactor test failures on stage2 mac build on Fuchsia builders.

Great - thank you for checking!

llvm/test/CodeGen/X86/extractelement-load.ll

341–345

This is the new test based on @dyung 's reduction. In the old version of this patch, a movl load from @zero was placed after the vmovaps %ymm0, zero(%rip)

[RKSimon]

LGTM - cheers

[manojgupta]

We are noticing some failures in internal unit tests with this change.

Tests pass at 588f121ada6d541 but start failing at b4b97ec813a0258.

[manojgupta]

In D118376#3360001, @manojgupta wrote:

We are noticing some failures in internal unit tests with this change.

Tests pass at 588f121ada6d541 but start failing at b4b97ec813a0258.

Oh, the patch was already reverted at 7b037250978. I'll test it again and report back if there are issues.

[RKSimon]

In D118376#3360031, @manojgupta wrote:

In D118376#3360001, @manojgupta wrote:

We are noticing some failures in internal unit tests with this change.

Tests pass at 588f121ada6d541 but start failing at b4b97ec813a0258.

Oh, the patch was already reverted at 7b037250978. I'll test it again and report back if there are issues.

@manojgupta Please can you confirm that everything is OK after the patch was recommitted at rGc486b82cfbe59929a80e5f29bab82112555c8bf4

llvm/lib/Target/X86/X86ISelLowering.cpp

if (all_of(InputVector->uses(), IsBoolExtract) &&
           Res = DAG.getSetCC(dl, MVT::i1, Res, Mask, ISD::SETEQ);
           DCI.CombineTo(Use, Res);
         }
         return SDValue(N, 0);
       }
     }
   }
 
+  // If this extract is from a loaded vector value and will be used as an
+  // integer, that requires a potentially expensive XMM -> GPR transfer.
+  // Additionally, if we can convert to a scalar integer load, that will likely
+  // be folded into a subsequent integer op.
+  // Note: Unlike the related fold for this in DAGCombiner, this is not limited
+  //       to a single-use of the loaded vector. For the reasons above, we
+  //       expect this to be profitable even if it creates an extra load.
+  bool LikelyUsedAsVector = any_of(N->uses(), [](SDNode *Use) {
+    return Use->getOpcode() == ISD::STORE ||
+           Use->getOpcode() == ISD::INSERT_VECTOR_ELT ||
+           Use->getOpcode() == ISD::SCALAR_TO_VECTOR;
+  });
+  auto *LoadVec = dyn_cast<LoadSDNode>(InputVector);
+  if (LoadVec && CIdx && ISD::isNormalLoad(LoadVec) && VT.isInteger() &&
+      SrcVT.getVectorElementType() == VT && DCI.isAfterLegalizeDAG() &&
+      !LikelyUsedAsVector) {
+    const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+    SDValue NewPtr =
+        TLI.getVectorElementPointer(DAG, LoadVec->getBasePtr(), SrcVT, EltIdx);
+    unsigned PtrOff = VT.getSizeInBits() * CIdx->getZExtValue() / 8;
+    MachinePointerInfo MPI = LoadVec->getPointerInfo().getWithOffset(PtrOff);
+    Align Alignment = commonAlignment(LoadVec->getAlign(), PtrOff);
+    SDValue Load =
+        DAG.getLoad(VT, dl, LoadVec->getChain(), NewPtr, MPI, Alignment,
+                    LoadVec->getMemOperand()->getFlags(), LoadVec->getAAInfo());
+    DAG.makeEquivalentMemoryOrdering(LoadVec, Load);
+    return Load;
+  }
+
   return SDValue();
 }
 
 // Convert (vXiY *ext(vXi1 bitcast(iX))) to extend_in_reg(broadcast(iX)).
 // This is more or less the reverse of combineBitcastvxi1.
 static SDValue combineToExtendBoolVectorInReg(
     unsigned Opcode, const SDLoc &DL, EVT VT, SDValue N0, SelectionDAG &DAG,
     TargetLowering::DAGCombinerInfo &DCI, const X86Subtarget &Subtarget) {

llvm/test/CodeGen/X86/2011-12-26-extractelement-duplicate-load.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-unknown -mattr=-sse4.2,+sse4.1 < %s | FileCheck %s
 
 ; Make sure we don't load from the location pointed to by %p
 ; twice: it has non-obvious performance implications, and
 ; the relevant transformation doesn't know how to update
 ; the chains correctly.
 ; PR10747
 
 define <4 x i32> @test(<4 x i32>* %p) {
 ; CHECK-LABEL: test:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movaps (%rdi), %xmm0
-; CHECK-NEXT:    extractps $2, %xmm0, %eax
-; CHECK-NEXT:    cmpl $3, %eax
-; CHECK-NEXT:    je .LBB0_2
-; CHECK-NEXT:  # %bb.1:
+; CHECK-NEXT:    cmpl $3, 8(%rdi)
+; CHECK-NEXT:    je .LBB0_1
+; CHECK-NEXT:  # %bb.2:
 ; CHECK-NEXT:    xorps %xmm0, %xmm0
-; CHECK-NEXT:  .LBB0_2:
+; CHECK-NEXT:    retq
+; CHECK-NEXT:  .LBB0_1:
+; CHECK-NEXT:    movaps (%rdi), %xmm0
 ; CHECK-NEXT:    retq
   %v = load <4 x i32>, <4 x i32>* %p
   %e = extractelement <4 x i32> %v, i32 2
   %cmp = icmp eq i32 %e, 3
   %sel = select i1 %cmp, <4 x i32> %v, <4 x i32> zeroinitializer
   ret <4 x i32> %sel
 }

llvm/test/CodeGen/X86/avx512-cvt.ll

 ; DQNOVL-NEXT:    retq
   %b = sitofp <2 x i64> %a to <2 x float>
   ret <2 x float>%b
 }
 
 define <4 x float> @slto4f32_mem(<4 x i64>* %a) {
 ; NODQ-LABEL: slto4f32_mem:
 ; NODQ:       # %bb.0:
-; NODQ-NEXT:    vmovdqu (%rdi), %xmm0
-; NODQ-NEXT:    vmovdqu 16(%rdi), %xmm1
-; NODQ-NEXT:    vpextrq $1, %xmm0, %rax
-; NODQ-NEXT:    vcvtsi2ss %rax, %xmm2, %xmm2
-; NODQ-NEXT:    vmovq %xmm0, %rax
-; NODQ-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm0
-; NODQ-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; NODQ-NEXT:    vmovq %xmm1, %rax
-; NODQ-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm2
-; NODQ-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; NODQ-NEXT:    vpextrq $1, %xmm1, %rax
-; NODQ-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm1
+; NODQ-NEXT:    vcvtsi2ssq 8(%rdi), %xmm0, %xmm0
+; NODQ-NEXT:    vcvtsi2ssq (%rdi), %xmm1, %xmm1
+; NODQ-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; NODQ-NEXT:    vcvtsi2ssq 16(%rdi), %xmm2, %xmm1
+; NODQ-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; NODQ-NEXT:    vcvtsi2ssq 24(%rdi), %xmm2, %xmm1
 ; NODQ-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; NODQ-NEXT:    retq
 ;
 ; VLDQ-LABEL: slto4f32_mem:
 ; VLDQ:       # %bb.0:
 ; VLDQ-NEXT:    vcvtqq2psy (%rdi), %xmm0
 ; VLDQ-NEXT:    retq
 ;

llvm/test/CodeGen/X86/bitcast-vector-bool.ll

 ; AVX2-NEXT:    addl %ecx, %eax
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq
 ;
 ; AVX512-LABEL: bitcast_v64i8_to_v2i32:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    vpmovb2m %zmm0, %k0
 ; AVX512-NEXT:    kmovq %k0, -{{[0-9]+}}(%rsp)
-; AVX512-NEXT:    vmovdqa -{{[0-9]+}}(%rsp), %xmm0
-; AVX512-NEXT:    vmovd %xmm0, %ecx
-; AVX512-NEXT:    vpextrd $1, %xmm0, %eax
-; AVX512-NEXT:    addl %ecx, %eax
+; AVX512-NEXT:    movl -{{[0-9]+}}(%rsp), %eax
+; AVX512-NEXT:    addl -{{[0-9]+}}(%rsp), %eax
 ; AVX512-NEXT:    vzeroupper
 ; AVX512-NEXT:    retq
   %1 = icmp slt <64 x i8> %a0, zeroinitializer
   %2 = bitcast <64 x i1> %1 to <2 x i32>
   %3 = extractelement <2 x i32> %2, i32 0
   %4 = extractelement <2 x i32> %2, i32 1
   %5 = add i32 %3, %4
   ret i32 %5

llvm/test/CodeGen/X86/extractelement-load.ll

; X64-NEXT:    retq
   %8 = load <2 x i16>, <2 x i16>* %0, align 4
   %9 = extractelement <2 x i16> %8, i32 0
   store i16 %9, i16* %1, align 2
   %10 = extractelement <2 x i16> %8, i32 1
   store i16 %10, i16* %2, align 2
   ret void
 }
 
+; A scalar load is favored over a XMM->GPR register transfer in this example.
+
 define i32 @multi_use_load_scalarization(<4 x i32>* %p) nounwind {
 ; X32-SSE2-LABEL: multi_use_load_scalarization:
 ; X32-SSE2:       # %bb.0:
 ; X32-SSE2-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X32-SSE2-NEXT:    movl (%ecx), %eax
 ; X32-SSE2-NEXT:    movdqu (%ecx), %xmm0
 ; X32-SSE2-NEXT:    pcmpeqd %xmm1, %xmm1
-; X32-SSE2-NEXT:    movd %xmm0, %eax
 ; X32-SSE2-NEXT:    psubd %xmm1, %xmm0
 ; X32-SSE2-NEXT:    movdqa %xmm0, (%ecx)
 ; X32-SSE2-NEXT:    retl
 ;
 ; X64-SSSE3-LABEL: multi_use_load_scalarization:
 ; X64-SSSE3:       # %bb.0:
+; X64-SSSE3-NEXT:    movl (%rdi), %eax
 ; X64-SSSE3-NEXT:    movdqu (%rdi), %xmm0
 ; X64-SSSE3-NEXT:    pcmpeqd %xmm1, %xmm1
-; X64-SSSE3-NEXT:    movd %xmm0, %eax
 ; X64-SSSE3-NEXT:    psubd %xmm1, %xmm0
 ; X64-SSSE3-NEXT:    movdqa %xmm0, (%rdi)
 ; X64-SSSE3-NEXT:    retq
 ;
 ; X64-AVX-LABEL: multi_use_load_scalarization:
 ; X64-AVX:       # %bb.0:
+; X64-AVX-NEXT:    movl (%rdi), %eax
 ; X64-AVX-NEXT:    vmovdqu (%rdi), %xmm0
 ; X64-AVX-NEXT:    vpcmpeqd %xmm1, %xmm1, %xmm1
-; X64-AVX-NEXT:    vpsubd %xmm1, %xmm0, %xmm1
-; X64-AVX-NEXT:    vmovdqa %xmm1, (%rdi)
-; X64-AVX-NEXT:    vmovd %xmm0, %eax
+; X64-AVX-NEXT:    vpsubd %xmm1, %xmm0, %xmm0
+; X64-AVX-NEXT:    vmovdqa %xmm0, (%rdi)
 ; X64-AVX-NEXT:    retq
   %v = load <4 x i32>, <4 x i32>* %p, align 1
   %v1 = add <4 x i32> %v, <i32 1, i32 1, i32 1, i32 1>
   store <4 x i32> %v1, <4 x i32>* %p
   %r = extractelement <4 x i32> %v, i64 0
   ret i32 %r
 }
 
+; This test is reduced from a C source example that showed a miscompile:
+; https://github.com/llvm/llvm-project/issues/53695
+; The scalarized loads from 'zero' in the AVX asm must occur before
+; the vector store to 'zero' overwrites the values.
+; If compiled to a binary, this test should return 0 if correct.
+
 @n1 = local_unnamed_addr global <8 x i32> <i32 0, i32 42, i32 6, i32 0, i32 0, i32 0, i32 0, i32 0>, align 32
 @zero = internal unnamed_addr global <8 x i32> zeroinitializer, align 32
 
 define i32 @main() nounwind {
 ; X32-SSE2-LABEL: main:
 ; X32-SSE2:       # %bb.0:
 ; X32-SSE2-NEXT:    pushl %ebp
 ; X32-SSE2-NEXT:    movl %esp, %ebp

 ; X64-AVX1-LABEL: main:
 ; X64-AVX1:       # %bb.0:
 ; X64-AVX1-NEXT:    pushq %rbp
 ; X64-AVX1-NEXT:    movq %rsp, %rbp
 ; X64-AVX1-NEXT:    andq $-32, %rsp
 ; X64-AVX1-NEXT:    subq $64, %rsp
 ; X64-AVX1-NEXT:    movq n1@GOTPCREL(%rip), %rax
 ; X64-AVX1-NEXT:    vmovaps (%rax), %ymm0
-; X64-AVX1-NEXT:    vmovaps zero(%rip), %xmm1
+; X64-AVX1-NEXT:    movl zero+4(%rip), %ecx
+; X64-AVX1-NEXT:    movl zero+8(%rip), %eax
 ; X64-AVX1-NEXT:    vmovaps %ymm0, zero(%rip)
 ; X64-AVX1-NEXT:    vmovaps {{.*#+}} ymm0 = [2,2,2,2,2,2,2,2]
 ; X64-AVX1-NEXT:    vmovaps %ymm0, (%rsp)
 ; X64-AVX1-NEXT:    vmovaps (%rsp), %ymm0
-; X64-AVX1-NEXT:    vextractps $2, %xmm1, %eax
-; X64-AVX1-NEXT:    vextractps $2, %xmm0, %ecx
-; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
-; X64-AVX1-NEXT:    movl %eax, %ecx
-; X64-AVX1-NEXT:    vextractps $1, %xmm1, %eax
-; X64-AVX1-NEXT:    vextractps $1, %xmm0, %esi
+; X64-AVX1-NEXT:    vextractps $2, %xmm0, %esi
 ; X64-AVX1-NEXT:    xorl %edx, %edx
 ; X64-AVX1-NEXT:    divl %esi
-; X64-AVX1-NEXT:    addl %ecx, %eax
+; X64-AVX1-NEXT:    movl %eax, %esi
+; X64-AVX1-NEXT:    vextractps $1, %xmm0, %edi
+; X64-AVX1-NEXT:    movl %ecx, %eax
+; X64-AVX1-NEXT:    xorl %edx, %edx
+; X64-AVX1-NEXT:    divl %edi
+; X64-AVX1-NEXT:    addl %esi, %eax
 ; X64-AVX1-NEXT:    movq %rbp, %rsp
 ; X64-AVX1-NEXT:    popq %rbp
 ; X64-AVX1-NEXT:    vzeroupper
 ; X64-AVX1-NEXT:    retq
 ;
 ; X64-AVX2-LABEL: main:
 ; X64-AVX2:       # %bb.0:
 ; X64-AVX2-NEXT:    pushq %rbp
 ; X64-AVX2-NEXT:    movq %rsp, %rbp
 ; X64-AVX2-NEXT:    andq $-32, %rsp
 ; X64-AVX2-NEXT:    subq $64, %rsp
 ; X64-AVX2-NEXT:    movq n1@GOTPCREL(%rip), %rax
 ; X64-AVX2-NEXT:    vmovaps (%rax), %ymm0
-; X64-AVX2-NEXT:    vmovaps zero(%rip), %xmm1
+; X64-AVX2-NEXT:    movl zero+4(%rip), %ecx
+; X64-AVX2-NEXT:    movl zero+8(%rip), %eax
 ; X64-AVX2-NEXT:    vmovaps %ymm0, zero(%rip)
 ; X64-AVX2-NEXT:    vbroadcastss {{.*#+}} ymm0 = [2,2,2,2,2,2,2,2]
 ; X64-AVX2-NEXT:    vmovaps %ymm0, (%rsp)
 ; X64-AVX2-NEXT:    vmovaps (%rsp), %ymm0
-; X64-AVX2-NEXT:    vextractps $2, %xmm1, %eax
-; X64-AVX2-NEXT:    vextractps $2, %xmm0, %ecx
-; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
-; X64-AVX2-NEXT:    movl %eax, %ecx
-; X64-AVX2-NEXT:    vextractps $1, %xmm1, %eax
-; X64-AVX2-NEXT:    vextractps $1, %xmm0, %esi
+; X64-AVX2-NEXT:    vextractps $2, %xmm0, %esi
 ; X64-AVX2-NEXT:    xorl %edx, %edx
 ; X64-AVX2-NEXT:    divl %esi
-; X64-AVX2-NEXT:    addl %ecx, %eax
+; X64-AVX2-NEXT:    movl %eax, %esi
+; X64-AVX2-NEXT:    vextractps $1, %xmm0, %edi
+; X64-AVX2-NEXT:    movl %ecx, %eax
+; X64-AVX2-NEXT:    xorl %edx, %edx
+; X64-AVX2-NEXT:    divl %edi
+; X64-AVX2-NEXT:    addl %esi, %eax
 ; X64-AVX2-NEXT:    movq %rbp, %rsp
 ; X64-AVX2-NEXT:    popq %rbp
 ; X64-AVX2-NEXT:    vzeroupper
 ; X64-AVX2-NEXT:    retq
   %stackptr = alloca <8 x i32>, align 32
   %z = load <8 x i32>, <8 x i32>* @zero, align 32
   %t1 = load <8 x i32>, <8 x i32>* @n1, align 32
   store <8 x i32> %t1, <8 x i32>* @zero, align 32
   store volatile <8 x i32> <i32 2, i32 2, i32 2, i32 2, i32 2, i32 2, i32 2, i32 2>, <8 x i32>* %stackptr, align 32
   %stackload = load volatile <8 x i32>, <8 x i32>* %stackptr, align 32
   %div = udiv <8 x i32> %z, %stackload
   %e1 = extractelement <8 x i32> %div, i64 1
   %e2 = extractelement <8 x i32> %div, i64 2
   %r = add i32 %e1, %e2
   ret i32 %r
 }

llvm/test/CodeGen/X86/oddsubvector.ll

 
 @b = dso_local local_unnamed_addr global i32 0, align 4
 @c = dso_local local_unnamed_addr global [49 x i32] zeroinitializer, align 16
 @d = dso_local local_unnamed_addr global [49 x i32] zeroinitializer, align 16
 
 define void @PR42833() {
 ; SSE2-LABEL: PR42833:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    movdqa c+144(%rip), %xmm1
-; SSE2-NEXT:    movdqa c+128(%rip), %xmm0
-; SSE2-NEXT:    movd %xmm0, %eax
-; SSE2-NEXT:    addl b(%rip), %eax
+; SSE2-NEXT:    movl b(%rip), %eax
+; SSE2-NEXT:    movdqa c+144(%rip), %xmm0
+; SSE2-NEXT:    movdqa c+128(%rip), %xmm1
+; SSE2-NEXT:    addl c+128(%rip), %eax
 ; SSE2-NEXT:    movd %eax, %xmm2
 ; SSE2-NEXT:    movd %eax, %xmm3
-; SSE2-NEXT:    paddd %xmm0, %xmm3
+; SSE2-NEXT:    paddd %xmm1, %xmm3
 ; SSE2-NEXT:    movdqa d+144(%rip), %xmm4
-; SSE2-NEXT:    psubd %xmm1, %xmm4
-; SSE2-NEXT:    paddd %xmm1, %xmm1
-; SSE2-NEXT:    movdqa %xmm0, %xmm5
-; SSE2-NEXT:    paddd %xmm0, %xmm5
+; SSE2-NEXT:    psubd %xmm0, %xmm4
+; SSE2-NEXT:    paddd %xmm0, %xmm0
+; SSE2-NEXT:    movdqa %xmm1, %xmm5
+; SSE2-NEXT:    paddd %xmm1, %xmm5
 ; SSE2-NEXT:    movss {{.*#+}} xmm5 = xmm3[0],xmm5[1,2,3]
-; SSE2-NEXT:    movdqa %xmm1, c+144(%rip)
+; SSE2-NEXT:    movdqa %xmm0, c+144(%rip)
 ; SSE2-NEXT:    movaps %xmm5, c+128(%rip)
-; SSE2-NEXT:    movdqa c+160(%rip), %xmm1
+; SSE2-NEXT:    movdqa c+160(%rip), %xmm0
 ; SSE2-NEXT:    movdqa c+176(%rip), %xmm3
 ; SSE2-NEXT:    movdqa d+160(%rip), %xmm5
 ; SSE2-NEXT:    movdqa d+176(%rip), %xmm6
 ; SSE2-NEXT:    movdqa d+128(%rip), %xmm7
-; SSE2-NEXT:    movss {{.*#+}} xmm0 = xmm2[0],xmm0[1,2,3]
-; SSE2-NEXT:    psubd %xmm0, %xmm7
+; SSE2-NEXT:    movss {{.*#+}} xmm1 = xmm2[0],xmm1[1,2,3]
+; SSE2-NEXT:    psubd %xmm1, %xmm7
 ; SSE2-NEXT:    psubd %xmm3, %xmm6
-; SSE2-NEXT:    psubd %xmm1, %xmm5
+; SSE2-NEXT:    psubd %xmm0, %xmm5
 ; SSE2-NEXT:    movdqa %xmm5, d+160(%rip)
 ; SSE2-NEXT:    movdqa %xmm6, d+176(%rip)
 ; SSE2-NEXT:    movdqa %xmm4, d+144(%rip)
 ; SSE2-NEXT:    movdqa %xmm7, d+128(%rip)
 ; SSE2-NEXT:    paddd %xmm3, %xmm3
-; SSE2-NEXT:    paddd %xmm1, %xmm1
-; SSE2-NEXT:    movdqa %xmm1, c+160(%rip)
+; SSE2-NEXT:    paddd %xmm0, %xmm0
+; SSE2-NEXT:    movdqa %xmm0, c+160(%rip)
 ; SSE2-NEXT:    movdqa %xmm3, c+176(%rip)
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: PR42833:
 ; SSE42:       # %bb.0:
+; SSE42-NEXT:    movl b(%rip), %eax
 ; SSE42-NEXT:    movdqa c+144(%rip), %xmm0
 ; SSE42-NEXT:    movdqa c+128(%rip), %xmm1
-; SSE42-NEXT:    movd %xmm1, %eax
-; SSE42-NEXT:    addl b(%rip), %eax
+; SSE42-NEXT:    addl c+128(%rip), %eax
 ; SSE42-NEXT:    movd %eax, %xmm2
 ; SSE42-NEXT:    paddd %xmm1, %xmm2
 ; SSE42-NEXT:    movdqa d+144(%rip), %xmm3
 ; SSE42-NEXT:    psubd %xmm0, %xmm3
 ; SSE42-NEXT:    paddd %xmm0, %xmm0
 ; SSE42-NEXT:    movdqa %xmm1, %xmm4
 ; SSE42-NEXT:    paddd %xmm1, %xmm4
 ; SSE42-NEXT:    pblendw {{.*#+}} xmm4 = xmm2[0,1],xmm4[2,3,4,5,6,7]

 ; SSE42-NEXT:    paddd %xmm2, %xmm2
 ; SSE42-NEXT:    paddd %xmm0, %xmm0
 ; SSE42-NEXT:    movdqa %xmm0, c+160(%rip)
 ; SSE42-NEXT:    movdqa %xmm2, c+176(%rip)
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: PR42833:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vmovdqa c+128(%rip), %xmm0
-; AVX1-NEXT:    vmovd %xmm0, %eax
-; AVX1-NEXT:    addl b(%rip), %eax
-; AVX1-NEXT:    vmovd %eax, %xmm1
-; AVX1-NEXT:    vpaddd %xmm1, %xmm0, %xmm1
-; AVX1-NEXT:    vpaddd %xmm0, %xmm0, %xmm2
+; AVX1-NEXT:    movl b(%rip), %eax
+; AVX1-NEXT:    addl c+128(%rip), %eax
+; AVX1-NEXT:    vmovd %eax, %xmm0
+; AVX1-NEXT:    vmovdqa c+128(%rip), %xmm1
+; AVX1-NEXT:    vpaddd %xmm0, %xmm1, %xmm0
+; AVX1-NEXT:    vpaddd %xmm1, %xmm1, %xmm2
 ; AVX1-NEXT:    vmovdqa c+144(%rip), %xmm3
 ; AVX1-NEXT:    vpaddd %xmm3, %xmm3, %xmm3
 ; AVX1-NEXT:    vinsertf128 $1, %xmm3, %ymm2, %ymm2
-; AVX1-NEXT:    vblendps {{.*#+}} ymm1 = ymm1[0],ymm2[1,2,3,4,5,6,7]
+; AVX1-NEXT:    vblendps {{.*#+}} ymm0 = ymm0[0],ymm2[1,2,3,4,5,6,7]
 ; AVX1-NEXT:    vmovdqa d+144(%rip), %xmm2
 ; AVX1-NEXT:    vpsubd c+144(%rip), %xmm2, %xmm2
-; AVX1-NEXT:    vmovups %ymm1, c+128(%rip)
-; AVX1-NEXT:    vpinsrd $0, %eax, %xmm0, %xmm0
+; AVX1-NEXT:    vmovups %ymm0, c+128(%rip)
+; AVX1-NEXT:    vpinsrd $0, %eax, %xmm1, %xmm0
 ; AVX1-NEXT:    vmovdqa d+128(%rip), %xmm1
 ; AVX1-NEXT:    vpsubd %xmm0, %xmm1, %xmm0
 ; AVX1-NEXT:    vmovdqa d+176(%rip), %xmm1
 ; AVX1-NEXT:    vmovdqa c+176(%rip), %xmm3
 ; AVX1-NEXT:    vpsubd %xmm3, %xmm1, %xmm1
 ; AVX1-NEXT:    vmovdqa d+160(%rip), %xmm4
 ; AVX1-NEXT:    vmovdqa c+160(%rip), %xmm5
 ; AVX1-NEXT:    vpsubd %xmm5, %xmm4, %xmm4

 ; AVX512-NEXT:    vmovdqu64 %zmm1, d+128(%rip)
 ; AVX512-NEXT:    vpaddd %ymm0, %ymm0, %ymm0
 ; AVX512-NEXT:    vmovdqu %ymm0, c+160(%rip)
 ; AVX512-NEXT:    vzeroupper
 ; AVX512-NEXT:    retq
 ;
 ; XOP-LABEL: PR42833:
 ; XOP:       # %bb.0:
-; XOP-NEXT:    vmovdqa c+128(%rip), %xmm0
-; XOP-NEXT:    vmovd %xmm0, %eax
-; XOP-NEXT:    addl b(%rip), %eax
-; XOP-NEXT:    vmovd %eax, %xmm1
-; XOP-NEXT:    vpaddd %xmm1, %xmm0, %xmm1
-; XOP-NEXT:    vpaddd %xmm0, %xmm0, %xmm2
+; XOP-NEXT:    movl b(%rip), %eax
+; XOP-NEXT:    addl c+128(%rip), %eax
+; XOP-NEXT:    vmovd %eax, %xmm0
+; XOP-NEXT:    vmovdqa c+128(%rip), %xmm1
+; XOP-NEXT:    vpaddd %xmm0, %xmm1, %xmm0
+; XOP-NEXT:    vpaddd %xmm1, %xmm1, %xmm2
 ; XOP-NEXT:    vmovdqa c+144(%rip), %xmm3
 ; XOP-NEXT:    vpaddd %xmm3, %xmm3, %xmm3
 ; XOP-NEXT:    vinsertf128 $1, %xmm3, %ymm2, %ymm2
-; XOP-NEXT:    vblendps {{.*#+}} ymm1 = ymm1[0],ymm2[1,2,3,4,5,6,7]
+; XOP-NEXT:    vblendps {{.*#+}} ymm0 = ymm0[0],ymm2[1,2,3,4,5,6,7]
 ; XOP-NEXT:    vmovdqa d+144(%rip), %xmm2
 ; XOP-NEXT:    vpsubd c+144(%rip), %xmm2, %xmm2
-; XOP-NEXT:    vmovups %ymm1, c+128(%rip)
-; XOP-NEXT:    vpinsrd $0, %eax, %xmm0, %xmm0
+; XOP-NEXT:    vmovups %ymm0, c+128(%rip)
+; XOP-NEXT:    vpinsrd $0, %eax, %xmm1, %xmm0
 ; XOP-NEXT:    vmovdqa d+128(%rip), %xmm1
 ; XOP-NEXT:    vpsubd %xmm0, %xmm1, %xmm0
 ; XOP-NEXT:    vmovdqa d+176(%rip), %xmm1
 ; XOP-NEXT:    vmovdqa c+176(%rip), %xmm3
 ; XOP-NEXT:    vpsubd %xmm3, %xmm1, %xmm1
 ; XOP-NEXT:    vmovdqa d+160(%rip), %xmm4
 ; XOP-NEXT:    vmovdqa c+160(%rip), %xmm5
 ; XOP-NEXT:    vpsubd %xmm5, %xmm4, %xmm4

llvm/test/CodeGen/X86/pr45378.ll

; AVX-NEXT:    retq
   %vcheck = icmp eq i64 %vreduce, 0
   ret i1 %vcheck
 }
 
 define i1 @parseHeaders2_scalar_and(i64 * %ptr) nounwind {
 ; SSE2-LABEL: parseHeaders2_scalar_and:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqu (%rdi), %xmm0
-; SSE2-NEXT:    movq %xmm0, %rax
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
-; SSE2-NEXT:    movq %xmm0, %rcx
-; SSE2-NEXT:    testq %rcx, %rax
+; SSE2-NEXT:    movq %xmm0, %rax
+; SSE2-NEXT:    testq %rax, (%rdi)
 ; SSE2-NEXT:    sete %al
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: parseHeaders2_scalar_and:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqu (%rdi), %xmm0
-; SSE41-NEXT:    movq %xmm0, %rax
-; SSE41-NEXT:    pextrq $1, %xmm0, %rcx
-; SSE41-NEXT:    testq %rcx, %rax
+; SSE41-NEXT:    movq (%rdi), %rax
+; SSE41-NEXT:    testq %rax, 8(%rdi)
 ; SSE41-NEXT:    sete %al
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: parseHeaders2_scalar_and:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vmovdqu (%rdi), %xmm0
-; AVX-NEXT:    vmovq %xmm0, %rax
-; AVX-NEXT:    vpextrq $1, %xmm0, %rcx
-; AVX-NEXT:    testq %rcx, %rax
+; AVX-NEXT:    movq (%rdi), %rax
+; AVX-NEXT:    testq %rax, 8(%rdi)
 ; AVX-NEXT:    sete %al
 ; AVX-NEXT:    retq
   %vptr = bitcast i64 * %ptr to <2 x i64> *
   %vload = load <2 x i64>, <2 x i64> * %vptr, align 8
   %v1 = extractelement <2 x i64> %vload, i32 0
   %v2 = extractelement <2 x i64> %vload, i32 1
   %vreduce = and i64 %v1, %v2
   %vcheck = icmp eq i64 %vreduce, 0
   ret i1 %vcheck
 }

llvm/test/CodeGen/X86/scalar_widen_div.ll

 
 ; CHECK: test_int_div
 define void @test_int_div(<3 x i32>* %dest, <3 x i32>* %old, i32 %n) {
 ; CHECK-LABEL: test_int_div:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    testl %edx, %edx
 ; CHECK-NEXT:    jle .LBB12_3
 ; CHECK-NEXT:  # %bb.1: # %bb.nph
-; CHECK-NEXT:    movl %edx, %r9d
+; CHECK-NEXT:    movl %edx, %r10d
 ; CHECK-NEXT:    xorl %ecx, %ecx
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB12_2: # %for.body
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    movdqa (%rdi,%rcx), %xmm0
-; CHECK-NEXT:    movdqa (%rsi,%rcx), %xmm1
-; CHECK-NEXT:    pextrd $1, %xmm0, %eax
-; CHECK-NEXT:    pextrd $1, %xmm1, %r8d
+; CHECK-NEXT:    movl (%rdi,%rcx), %r8d
+; CHECK-NEXT:    movl 4(%rdi,%rcx), %eax
 ; CHECK-NEXT:    cltd
-; CHECK-NEXT:    idivl %r8d
-; CHECK-NEXT:    movl %eax, %r8d
-; CHECK-NEXT:    movd %xmm0, %eax
-; CHECK-NEXT:    movd %xmm1, %r10d
+; CHECK-NEXT:    idivl 4(%rsi,%rcx)
+; CHECK-NEXT:    movl %eax, %r9d
+; CHECK-NEXT:    movl %r8d, %eax
 ; CHECK-NEXT:    cltd
-; CHECK-NEXT:    idivl %r10d
-; CHECK-NEXT:    movd %eax, %xmm2
-; CHECK-NEXT:    pinsrd $1, %r8d, %xmm2
-; CHECK-NEXT:    pextrd $2, %xmm0, %eax
-; CHECK-NEXT:    pextrd $2, %xmm1, %r8d
+; CHECK-NEXT:    idivl (%rsi,%rcx)
+; CHECK-NEXT:    movd %eax, %xmm0
+; CHECK-NEXT:    pinsrd $1, %r9d, %xmm0
+; CHECK-NEXT:    movl 8(%rdi,%rcx), %eax
 ; CHECK-NEXT:    cltd
-; CHECK-NEXT:    idivl %r8d
+; CHECK-NEXT:    idivl 8(%rsi,%rcx)
 ; CHECK-NEXT:    movl %eax, 8(%rdi,%rcx)
-; CHECK-NEXT:    movq %xmm2, (%rdi,%rcx)
+; CHECK-NEXT:    movq %xmm0, (%rdi,%rcx)
 ; CHECK-NEXT:    addq $16, %rcx
-; CHECK-NEXT:    decl %r9d
+; CHECK-NEXT:    decl %r10d
 ; CHECK-NEXT:    jne .LBB12_2
 ; CHECK-NEXT:  .LBB12_3: # %for.end
 ; CHECK-NEXT:    retq
 entry:
   %cmp13 = icmp sgt i32 %n, 0
   br i1 %cmp13, label %bb.nph, label %for.end
 
 bb.nph:

llvm/test/CodeGen/X86/shrink_vmul.ll

 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm2[2,3,2,3]
 ; X86-SSE-NEXT:    movd %xmm4, %eax
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm1[2,3,2,3]
 ; X86-SSE-NEXT:    movd %xmm4, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm4
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm4 = xmm4[0],xmm3[0],xmm4[1],xmm3[1]
-; X86-SSE-NEXT:    movd %xmm1, %ecx
 ; X86-SSE-NEXT:    movl %esi, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
-; X86-SSE-NEXT:    divl %ecx
+; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; X86-SSE-NEXT:    divl 16(%esi)
 ; X86-SSE-NEXT:    movd %edx, %xmm3
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm2, %eax
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm1, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm1
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
 ; X86-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
-; X86-SSE-NEXT:    movd %xmm0, %ecx
 ; X86-SSE-NEXT:    movl %edi, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
-; X86-SSE-NEXT:    divl %ecx
+; X86-SSE-NEXT:    divl (%esi)
 ; X86-SSE-NEXT:    movd %edx, %xmm1
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm2, %ecx
 ; X86-SSE-NEXT:    movl %ebx, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm2
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]

 ; X86-SSE-NEXT:    movd %xmm0, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm0
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; X86-SSE-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X86-SSE-NEXT:    movl (%esp), %eax # 4-byte Reload
 ; X86-SSE-NEXT:    xorl %edx, %edx
-; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-SSE-NEXT:    divl 32(%ecx)
+; X86-SSE-NEXT:    divl 32(%esi)
 ; X86-SSE-NEXT:    movdqa {{.*#+}} xmm0 = [8199,8199,8199,8199]
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[1,1,3,3]
 ; X86-SSE-NEXT:    pmuludq %xmm0, %xmm1
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[0,2,2,3]
 ; X86-SSE-NEXT:    pmuludq %xmm0, %xmm2
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[0,2,2,3]
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm3[1,1,3,3]

 ; X86-AVX1-NEXT:    pushl %ebp
 ; X86-AVX1-NEXT:    pushl %ebx
 ; X86-AVX1-NEXT:    pushl %edi
 ; X86-AVX1-NEXT:    pushl %esi
 ; X86-AVX1-NEXT:    subl $16, %esp
 ; X86-AVX1-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-AVX1-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm0 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
-; X86-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm2 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
 ; X86-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm1 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
-; X86-AVX1-NEXT:    vmovd %xmm1, %eax
+; X86-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm2 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
+; X86-AVX1-NEXT:    vmovd %xmm2, %eax
 ; X86-AVX1-NEXT:    xorl %edx, %edx
 ; X86-AVX1-NEXT:    divl 32(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-AVX1-NEXT:    vpextrd $3, %xmm2, %eax
-; X86-AVX1-NEXT:    vmovdqa (%ecx), %xmm1
-; X86-AVX1-NEXT:    vmovdqa 16(%ecx), %xmm3
-; X86-AVX1-NEXT:    vpextrd $3, %xmm3, %ecx
+; X86-AVX1-NEXT:    vpextrd $3, %xmm1, %eax
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl 28(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-AVX1-NEXT:    vpextrd $2, %xmm2, %eax
-; X86-AVX1-NEXT:    vpextrd $2, %xmm3, %ecx
+; X86-AVX1-NEXT:    vpextrd $2, %xmm1, %eax
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl 24(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-AVX1-NEXT:    vpextrd $1, %xmm2, %eax
-; X86-AVX1-NEXT:    vpextrd $1, %xmm3, %ecx
+; X86-AVX1-NEXT:    vpextrd $1, %xmm1, %eax
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl 20(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, (%esp) # 4-byte Spill
-; X86-AVX1-NEXT:    vmovd %xmm2, %eax
-; X86-AVX1-NEXT:    vmovd %xmm3, %ecx
+; X86-AVX1-NEXT:    vmovd %xmm1, %eax
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl 16(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, %ebp
 ; X86-AVX1-NEXT:    vpextrd $3, %xmm0, %eax
-; X86-AVX1-NEXT:    vpextrd $3, %xmm1, %ecx
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl 12(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, %ebx
 ; X86-AVX1-NEXT:    vpextrd $2, %xmm0, %eax
-; X86-AVX1-NEXT:    vpextrd $2, %xmm1, %esi
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %esi
+; X86-AVX1-NEXT:    divl 8(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, %esi
 ; X86-AVX1-NEXT:    vpextrd $1, %xmm0, %eax
-; X86-AVX1-NEXT:    vpextrd $1, %xmm1, %edi
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %edi
+; X86-AVX1-NEXT:    divl 4(%ecx)
 ; X86-AVX1-NEXT:    movl %edx, %edi
 ; X86-AVX1-NEXT:    vmovd %xmm0, %eax
-; X86-AVX1-NEXT:    vmovd %xmm1, %ecx
 ; X86-AVX1-NEXT:    xorl %edx, %edx
-; X86-AVX1-NEXT:    divl %ecx
+; X86-AVX1-NEXT:    divl (%ecx)
 ; X86-AVX1-NEXT:    vmovd %edx, %xmm0
 ; X86-AVX1-NEXT:    vpinsrd $1, %edi, %xmm0, %xmm0
 ; X86-AVX1-NEXT:    vpinsrd $2, %esi, %xmm0, %xmm0
 ; X86-AVX1-NEXT:    vpinsrd $3, %ebx, %xmm0, %xmm0
 ; X86-AVX1-NEXT:    vmovd %ebp, %xmm1
 ; X86-AVX1-NEXT:    vpinsrd $1, (%esp), %xmm1, %xmm1 # 4-byte Folded Reload
 ; X86-AVX1-NEXT:    vpinsrd $2, {{[-0-9]+}}(%e{{[sb]}}p), %xmm1, %xmm1 # 4-byte Folded Reload
 ; X86-AVX1-NEXT:    vpinsrd $3, {{[-0-9]+}}(%e{{[sb]}}p), %xmm1, %xmm1 # 4-byte Folded Reload

 ; X86-AVX1-NEXT:    popl %esi
 ; X86-AVX1-NEXT:    popl %edi
 ; X86-AVX1-NEXT:    popl %ebx
 ; X86-AVX1-NEXT:    popl %ebp
 ; X86-AVX1-NEXT:    retl
 ;
 ; X86-AVX2-LABEL: PR34947:
 ; X86-AVX2:       # %bb.0:
-; X86-AVX2-NEXT:    pushl %edi
 ; X86-AVX2-NEXT:    pushl %esi
 ; X86-AVX2-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-AVX2-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-AVX2-NEXT:    vpmovzxwd {{.*#+}} ymm0 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
 ; X86-AVX2-NEXT:    vpmovzxwd {{.*#+}} ymm1 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
-; X86-AVX2-NEXT:    vmovdqa (%esi), %xmm2
-; X86-AVX2-NEXT:    vmovdqa 16(%esi), %xmm3
-; X86-AVX2-NEXT:    vpextrd $1, %xmm3, %ecx
-; X86-AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm4
-; X86-AVX2-NEXT:    vpextrd $1, %xmm4, %eax
+; X86-AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm2
+; X86-AVX2-NEXT:    vpextrd $1, %xmm2, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
+; X86-AVX2-NEXT:    divl 20(%esi)
 ; X86-AVX2-NEXT:    movl %edx, %ecx
-; X86-AVX2-NEXT:    vmovd %xmm3, %edi
-; X86-AVX2-NEXT:    vmovd %xmm4, %eax
+; X86-AVX2-NEXT:    vmovd %xmm2, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %edi
-; X86-AVX2-NEXT:    vmovd %edx, %xmm5
-; X86-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm5, %xmm5
-; X86-AVX2-NEXT:    vpextrd $2, %xmm3, %ecx
-; X86-AVX2-NEXT:    vpextrd $2, %xmm4, %eax
+; X86-AVX2-NEXT:    divl 16(%esi)
+; X86-AVX2-NEXT:    vmovd %edx, %xmm3
+; X86-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm3, %xmm3
+; X86-AVX2-NEXT:    vpextrd $2, %xmm2, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
-; X86-AVX2-NEXT:    vpinsrd $2, %edx, %xmm5, %xmm5
-; X86-AVX2-NEXT:    vpextrd $3, %xmm3, %ecx
-; X86-AVX2-NEXT:    vpextrd $3, %xmm4, %eax
+; X86-AVX2-NEXT:    divl 24(%esi)
+; X86-AVX2-NEXT:    vpinsrd $2, %edx, %xmm3, %xmm3
+; X86-AVX2-NEXT:    vpextrd $3, %xmm2, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
-; X86-AVX2-NEXT:    vpinsrd $3, %edx, %xmm5, %xmm3
-; X86-AVX2-NEXT:    vpextrd $1, %xmm2, %ecx
+; X86-AVX2-NEXT:    divl 28(%esi)
+; X86-AVX2-NEXT:    vpinsrd $3, %edx, %xmm3, %xmm2
 ; X86-AVX2-NEXT:    vpextrd $1, %xmm1, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
+; X86-AVX2-NEXT:    divl 4(%esi)
 ; X86-AVX2-NEXT:    movl %edx, %ecx
-; X86-AVX2-NEXT:    vmovd %xmm2, %edi
 ; X86-AVX2-NEXT:    vmovd %xmm1, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %edi
-; X86-AVX2-NEXT:    vmovd %edx, %xmm4
-; X86-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm4, %xmm4
-; X86-AVX2-NEXT:    vpextrd $2, %xmm2, %ecx
+; X86-AVX2-NEXT:    divl (%esi)
+; X86-AVX2-NEXT:    vmovd %edx, %xmm3
+; X86-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm3, %xmm3
 ; X86-AVX2-NEXT:    vpextrd $2, %xmm1, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
-; X86-AVX2-NEXT:    vpinsrd $2, %edx, %xmm4, %xmm4
-; X86-AVX2-NEXT:    vpextrd $3, %xmm2, %ecx
+; X86-AVX2-NEXT:    divl 8(%esi)
+; X86-AVX2-NEXT:    vpinsrd $2, %edx, %xmm3, %xmm3
 ; X86-AVX2-NEXT:    vpextrd $3, %xmm1, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
-; X86-AVX2-NEXT:    divl %ecx
-; X86-AVX2-NEXT:    vpinsrd $3, %edx, %xmm4, %xmm1
-; X86-AVX2-NEXT:    vinserti128 $1, %xmm3, %ymm1, %ymm1
+; X86-AVX2-NEXT:    divl 12(%esi)
+; X86-AVX2-NEXT:    vpinsrd $3, %edx, %xmm3, %xmm1
+; X86-AVX2-NEXT:    vinserti128 $1, %xmm2, %ymm1, %ymm1
 ; X86-AVX2-NEXT:    vmovd %xmm0, %eax
 ; X86-AVX2-NEXT:    xorl %edx, %edx
 ; X86-AVX2-NEXT:    divl 32(%esi)
 ; X86-AVX2-NEXT:    vpbroadcastd {{.*#+}} ymm0 = [8199,8199,8199,8199,8199,8199,8199,8199]
 ; X86-AVX2-NEXT:    vpmulld %ymm0, %ymm1, %ymm0
 ; X86-AVX2-NEXT:    imull $8199, %edx, %eax # imm = 0x2007
 ; X86-AVX2-NEXT:    movl %eax, (%eax)
 ; X86-AVX2-NEXT:    vmovdqa %ymm0, (%eax)
 ; X86-AVX2-NEXT:    popl %esi
-; X86-AVX2-NEXT:    popl %edi
 ; X86-AVX2-NEXT:    vzeroupper
 ; X86-AVX2-NEXT:    retl
 ;
 ; X64-SSE-LABEL: PR34947:
 ; X64-SSE:       # %bb.0:
 ; X64-SSE-NEXT:    movzwl 16(%rdi), %r8d
 ; X64-SSE-NEXT:    movdqa (%rdi), %xmm3
 ; X64-SSE-NEXT:    movdqa (%rsi), %xmm0

 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm2[2,3,2,3]
 ; X64-SSE-NEXT:    movd %xmm4, %eax
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm1[2,3,2,3]
 ; X64-SSE-NEXT:    movd %xmm4, %edi
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %edi
 ; X64-SSE-NEXT:    movd %edx, %xmm4
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm4 = xmm4[0],xmm3[0],xmm4[1],xmm3[1]
-; X64-SSE-NEXT:    movd %xmm1, %edi
 ; X64-SSE-NEXT:    movl %r9d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
-; X64-SSE-NEXT:    divl %edi
+; X64-SSE-NEXT:    divl 16(%rsi)
 ; X64-SSE-NEXT:    movd %edx, %xmm3
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm2, %eax
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm1, %edi
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %edi
 ; X64-SSE-NEXT:    movd %edx, %xmm1
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
 ; X64-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
-; X64-SSE-NEXT:    movd %xmm0, %edi
 ; X64-SSE-NEXT:    movl %r10d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
-; X64-SSE-NEXT:    divl %edi
+; X64-SSE-NEXT:    divl (%rsi)
 ; X64-SSE-NEXT:    movd %edx, %xmm1
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm2, %edi
 ; X64-SSE-NEXT:    movl %r11d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %edi
 ; X64-SSE-NEXT:    movd %edx, %xmm2
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]

 ; X64-SSE-NEXT:    movdqa %xmm1, (%rax)
 ; X64-SSE-NEXT:    retq
 ;
 ; X64-AVX1-LABEL: PR34947:
 ; X64-AVX1:       # %bb.0:
 ; X64-AVX1-NEXT:    pushq %rbp
 ; X64-AVX1-NEXT:    pushq %rbx
 ; X64-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm0 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
-; X64-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm2 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
 ; X64-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm1 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
-; X64-AVX1-NEXT:    vmovd %xmm1, %eax
+; X64-AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm2 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero
+; X64-AVX1-NEXT:    vmovd %xmm2, %eax
 ; X64-AVX1-NEXT:    xorl %edx, %edx
 ; X64-AVX1-NEXT:    divl 32(%rsi)
 ; X64-AVX1-NEXT:    movl %edx, %r8d
-; X64-AVX1-NEXT:    vpextrd $3, %xmm2, %eax
-; X64-AVX1-NEXT:    vmovdqa (%rsi), %xmm1
-; X64-AVX1-NEXT:    vmovdqa 16(%rsi), %xmm3
-; X64-AVX1-NEXT:    vpextrd $3, %xmm3, %ecx
+; X64-AVX1-NEXT:    vpextrd $3, %xmm1, %eax
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
+; X64-AVX1-NEXT:    divl 28(%rsi)
 ; X64-AVX1-NEXT:    movl %edx, %r9d
-; X64-AVX1-NEXT:    vpextrd $2, %xmm2, %eax
-; X64-AVX1-NEXT:    vpextrd $2, %xmm3, %ecx
+; X64-AVX1-NEXT:    vpextrd $2, %xmm1, %eax
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
+; X64-AVX1-NEXT:    divl 24(%rsi)
 ; X64-AVX1-NEXT:    movl %edx, %r10d
-; X64-AVX1-NEXT:    vpextrd $1, %xmm2, %eax
-; X64-AVX1-NEXT:    vpextrd $1, %xmm3, %ecx
+; X64-AVX1-NEXT:    vpextrd $1, %xmm1, %eax
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
+; X64-AVX1-NEXT:    divl 20(%rsi)
 ; X64-AVX1-NEXT:    movl %edx, %r11d
-; X64-AVX1-NEXT:    vmovd %xmm2, %eax
-; X64-AVX1-NEXT:    vmovd %xmm3, %ecx
+; X64-AVX1-NEXT:    vmovd %xmm1, %eax
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
-; X64-AVX1-NEXT:    movl %edx, %esi
+; X64-AVX1-NEXT:    divl 16(%rsi)
+; X64-AVX1-NEXT:    movl %edx, %ecx
 ; X64-AVX1-NEXT:    vpextrd $3, %xmm0, %eax
-; X64-AVX1-NEXT:    vpextrd $3, %xmm1, %ecx
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
+; X64-AVX1-NEXT:    divl 12(%rsi)
 ; X64-AVX1-NEXT:    movl %edx, %edi
 ; X64-AVX1-NEXT:    vpextrd $2, %xmm0, %eax
-; X64-AVX1-NEXT:    vpextrd $2, %xmm1, %ecx
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ecx
-; X64-AVX1-NEXT:    movl %edx, %ecx
+; X64-AVX1-NEXT:    divl 8(%rsi)
+; X64-AVX1-NEXT:    movl %edx, %ebx
 ; X64-AVX1-NEXT:    vpextrd $1, %xmm0, %eax
-; X64-AVX1-NEXT:    vpextrd $1, %xmm1, %ebx
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ebx
-; X64-AVX1-NEXT:    movl %edx, %ebx
+; X64-AVX1-NEXT:    divl 4(%rsi)
+; X64-AVX1-NEXT:    movl %edx, %ebp
 ; X64-AVX1-NEXT:    vmovd %xmm0, %eax
-; X64-AVX1-NEXT:    vmovd %xmm1, %ebp
 ; X64-AVX1-NEXT:    xorl %edx, %edx
-; X64-AVX1-NEXT:    divl %ebp
+; X64-AVX1-NEXT:    divl (%rsi)
 ; X64-AVX1-NEXT:    vmovd %edx, %xmm0
-; X64-AVX1-NEXT:    vpinsrd $1, %ebx, %xmm0, %xmm0
-; X64-AVX1-NEXT:    vpinsrd $2, %ecx, %xmm0, %xmm0
+; X64-AVX1-NEXT:    vpinsrd $1, %ebp, %xmm0, %xmm0
+; X64-AVX1-NEXT:    vpinsrd $2, %ebx, %xmm0, %xmm0
 ; X64-AVX1-NEXT:    vpinsrd $3, %edi, %xmm0, %xmm0
 ; X64-AVX1-NEXT:    vmovdqa {{.*#+}} xmm1 = [8199,8199,8199,8199]
 ; X64-AVX1-NEXT:    vpmulld %xmm1, %xmm0, %xmm0
-; X64-AVX1-NEXT:    vmovd %esi, %xmm2
+; X64-AVX1-NEXT:    vmovd %ecx, %xmm2
 ; X64-AVX1-NEXT:    vpinsrd $1, %r11d, %xmm2, %xmm2
 ; X64-AVX1-NEXT:    vpinsrd $2, %r10d, %xmm2, %xmm2
 ; X64-AVX1-NEXT:    vpinsrd $3, %r9d, %xmm2, %xmm2
 ; X64-AVX1-NEXT:    vpmulld %xmm1, %xmm2, %xmm1
 ; X64-AVX1-NEXT:    imull $8199, %r8d, %eax # imm = 0x2007
 ; X64-AVX1-NEXT:    movl %eax, (%rax)
 ; X64-AVX1-NEXT:    vmovdqa %xmm1, (%rax)
 ; X64-AVX1-NEXT:    vmovdqa %xmm0, (%rax)
 ; X64-AVX1-NEXT:    popq %rbx
 ; X64-AVX1-NEXT:    popq %rbp
 ; X64-AVX1-NEXT:    retq
 ;
 ; X64-AVX2-LABEL: PR34947:
 ; X64-AVX2:       # %bb.0:
 ; X64-AVX2-NEXT:    vpmovzxwd {{.*#+}} ymm0 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
 ; X64-AVX2-NEXT:    vpmovzxwd {{.*#+}} ymm1 = mem[0],zero,mem[1],zero,mem[2],zero,mem[3],zero,mem[4],zero,mem[5],zero,mem[6],zero,mem[7],zero
-; X64-AVX2-NEXT:    vmovdqa (%rsi), %xmm2
-; X64-AVX2-NEXT:    vmovdqa 16(%rsi), %xmm3
-; X64-AVX2-NEXT:    vpextrd $1, %xmm3, %ecx
-; X64-AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm4
-; X64-AVX2-NEXT:    vpextrd $1, %xmm4, %eax
+; X64-AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm2
+; X64-AVX2-NEXT:    vpextrd $1, %xmm2, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
+; X64-AVX2-NEXT:    divl 20(%rsi)
 ; X64-AVX2-NEXT:    movl %edx, %ecx
-; X64-AVX2-NEXT:    vmovd %xmm3, %edi
-; X64-AVX2-NEXT:    vmovd %xmm4, %eax
+; X64-AVX2-NEXT:    vmovd %xmm2, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %edi
-; X64-AVX2-NEXT:    vmovd %edx, %xmm5
-; X64-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm5, %xmm5
-; X64-AVX2-NEXT:    vpextrd $2, %xmm3, %ecx
-; X64-AVX2-NEXT:    vpextrd $2, %xmm4, %eax
+; X64-AVX2-NEXT:    divl 16(%rsi)
+; X64-AVX2-NEXT:    vmovd %edx, %xmm3
+; X64-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm3, %xmm3
+; X64-AVX2-NEXT:    vpextrd $2, %xmm2, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
-; X64-AVX2-NEXT:    vpinsrd $2, %edx, %xmm5, %xmm5
-; X64-AVX2-NEXT:    vpextrd $3, %xmm3, %ecx
-; X64-AVX2-NEXT:    vpextrd $3, %xmm4, %eax
+; X64-AVX2-NEXT:    divl 24(%rsi)
+; X64-AVX2-NEXT:    vpinsrd $2, %edx, %xmm3, %xmm3
+; X64-AVX2-NEXT:    vpextrd $3, %xmm2, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
-; X64-AVX2-NEXT:    vpinsrd $3, %edx, %xmm5, %xmm3
-; X64-AVX2-NEXT:    vpextrd $1, %xmm2, %ecx
+; X64-AVX2-NEXT:    divl 28(%rsi)
+; X64-AVX2-NEXT:    vpinsrd $3, %edx, %xmm3, %xmm2
 ; X64-AVX2-NEXT:    vpextrd $1, %xmm1, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
+; X64-AVX2-NEXT:    divl 4(%rsi)
 ; X64-AVX2-NEXT:    movl %edx, %ecx
-; X64-AVX2-NEXT:    vmovd %xmm2, %edi
 ; X64-AVX2-NEXT:    vmovd %xmm1, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %edi
-; X64-AVX2-NEXT:    vmovd %edx, %xmm4
-; X64-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm4, %xmm4
-; X64-AVX2-NEXT:    vpextrd $2, %xmm2, %ecx
+; X64-AVX2-NEXT:    divl (%rsi)
+; X64-AVX2-NEXT:    vmovd %edx, %xmm3
+; X64-AVX2-NEXT:    vpinsrd $1, %ecx, %xmm3, %xmm3
 ; X64-AVX2-NEXT:    vpextrd $2, %xmm1, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
-; X64-AVX2-NEXT:    vpinsrd $2, %edx, %xmm4, %xmm4
-; X64-AVX2-NEXT:    vpextrd $3, %xmm2, %ecx
+; X64-AVX2-NEXT:    divl 8(%rsi)
+; X64-AVX2-NEXT:    vpinsrd $2, %edx, %xmm3, %xmm3
 ; X64-AVX2-NEXT:    vpextrd $3, %xmm1, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
-; X64-AVX2-NEXT:    divl %ecx
-; X64-AVX2-NEXT:    vpinsrd $3, %edx, %xmm4, %xmm1
-; X64-AVX2-NEXT:    vinserti128 $1, %xmm3, %ymm1, %ymm1
+; X64-AVX2-NEXT:    divl 12(%rsi)
+; X64-AVX2-NEXT:    vpinsrd $3, %edx, %xmm3, %xmm1
+; X64-AVX2-NEXT:    vinserti128 $1, %xmm2, %ymm1, %ymm1
 ; X64-AVX2-NEXT:    vmovd %xmm0, %eax
 ; X64-AVX2-NEXT:    xorl %edx, %edx
 ; X64-AVX2-NEXT:    divl 32(%rsi)
 ; X64-AVX2-NEXT:    vpbroadcastd {{.*#+}} ymm0 = [8199,8199,8199,8199,8199,8199,8199,8199]
 ; X64-AVX2-NEXT:    vpmulld %ymm0, %ymm1, %ymm0
 ; X64-AVX2-NEXT:    imull $8199, %edx, %eax # imm = 0x2007
 ; X64-AVX2-NEXT:    movl %eax, (%rax)
 ; X64-AVX2-NEXT:    vmovdqa %ymm0, (%rax)

llvm/test/CodeGen/X86/vec_cast.ll

 ; CHECK-LIN-NEXT:    # kill: def $ax killed $ax killed $eax
 ; CHECK-LIN-NEXT:    # kill: def $dx killed $dx killed $edx
 ; CHECK-LIN-NEXT:    # kill: def $cx killed $cx killed $ecx
 ; CHECK-LIN-NEXT:    retq
 ;
 ; CHECK-WIN-LABEL: h:
 ; CHECK-WIN:       # %bb.0:
 ; CHECK-WIN-NEXT:    movdqa (%rcx), %xmm0
-; CHECK-WIN-NEXT:    movd %xmm0, %eax
+; CHECK-WIN-NEXT:    movl (%rcx), %eax
 ; CHECK-WIN-NEXT:    pextrw $2, %xmm0, %edx
 ; CHECK-WIN-NEXT:    pextrw $4, %xmm0, %ecx
 ; CHECK-WIN-NEXT:    # kill: def $ax killed $ax killed $eax
 ; CHECK-WIN-NEXT:    # kill: def $dx killed $dx killed $edx
 ; CHECK-WIN-NEXT:    # kill: def $cx killed $cx killed $ecx
 ; CHECK-WIN-NEXT:    retq
   %c = trunc <3 x i32> %a to <3 x i16>
   ret <3 x i16> %c

llvm/test/CodeGen/X86/vec_int_to_fp.ll

 ;
 ; Load Signed Integer to Double
 ;
 
 define <2 x double> @sitofp_load_2i64_to_2f64(<2 x i64> *%a) {
 ; SSE2-LABEL: sitofp_load_2i64_to_2f64:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa (%rdi), %xmm1
-; SSE2-NEXT:    movq %xmm1, %rax
-; SSE2-NEXT:    cvtsi2sd %rax, %xmm0
+; SSE2-NEXT:    cvtsi2sdq (%rdi), %xmm0
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2sd %rax, %xmm1
 ; SSE2-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: sitofp_load_2i64_to_2f64:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa (%rdi), %xmm0
-; SSE41-NEXT:    pextrq $1, %xmm0, %rax
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm1
-; SSE41-NEXT:    movq %xmm0, %rax
-; SSE41-NEXT:    xorps %xmm0, %xmm0
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm0
+; SSE41-NEXT:    cvtsi2sdq 8(%rdi), %xmm1
+; SSE41-NEXT:    cvtsi2sdq (%rdi), %xmm0
 ; SSE41-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: sitofp_load_2i64_to_2f64:
 ; VEX:       # %bb.0:
-; VEX-NEXT:    vmovdqa (%rdi), %xmm0
-; VEX-NEXT:    vpextrq $1, %xmm0, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm1, %xmm1
-; VEX-NEXT:    vmovq %xmm0, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm0
-; VEX-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; VEX-NEXT:    vcvtsi2sdq 8(%rdi), %xmm0, %xmm0
+; VEX-NEXT:    vcvtsi2sdq (%rdi), %xmm1, %xmm1
+; VEX-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: sitofp_load_2i64_to_2f64:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm1, %xmm1
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm0
-; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; AVX512F-NEXT:    vcvtsi2sdq 8(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtsi2sdq (%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: sitofp_load_2i64_to_2f64:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm1, %xmm1
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm0
-; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; AVX512VL-NEXT:    vcvtsi2sdq 8(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtsi2sdq (%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: sitofp_load_2i64_to_2f64:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vmovaps (%rdi), %xmm0
 ; AVX512DQ-NEXT:    vcvtqq2pd %zmm0, %zmm0
 ; AVX512DQ-NEXT:    # kill: def $xmm0 killed $xmm0 killed $zmm0
 ; AVX512DQ-NEXT:    vzeroupper
; AVX-NEXT:    retq
   ret <2 x double> %cvt
 }
 
 define <4 x double> @sitofp_load_4i64_to_4f64(<4 x i64> *%a) {
 ; SSE2-LABEL: sitofp_load_4i64_to_4f64:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa (%rdi), %xmm1
 ; SSE2-NEXT:    movdqa 16(%rdi), %xmm2
-; SSE2-NEXT:    movq %xmm1, %rax
-; SSE2-NEXT:    cvtsi2sd %rax, %xmm0
+; SSE2-NEXT:    cvtsi2sdq (%rdi), %xmm0
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2sd %rax, %xmm1
 ; SSE2-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
-; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
-; SSE2-NEXT:    cvtsi2sd %rax, %xmm1
+; SSE2-NEXT:    cvtsi2sdq 16(%rdi), %xmm1
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    xorps %xmm2, %xmm2
 ; SSE2-NEXT:    cvtsi2sd %rax, %xmm2
 ; SSE2-NEXT:    unpcklpd {{.*#+}} xmm1 = xmm1[0],xmm2[0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: sitofp_load_4i64_to_4f64:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa (%rdi), %xmm0
-; SSE41-NEXT:    movdqa 16(%rdi), %xmm1
-; SSE41-NEXT:    pextrq $1, %xmm0, %rax
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm2
-; SSE41-NEXT:    movq %xmm0, %rax
-; SSE41-NEXT:    xorps %xmm0, %xmm0
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm0
-; SSE41-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; SSE41-NEXT:    pextrq $1, %xmm1, %rax
-; SSE41-NEXT:    xorps %xmm2, %xmm2
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm2
-; SSE41-NEXT:    movq %xmm1, %rax
+; SSE41-NEXT:    cvtsi2sdq 8(%rdi), %xmm1
+; SSE41-NEXT:    cvtsi2sdq (%rdi), %xmm0
+; SSE41-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; SSE41-NEXT:    cvtsi2sdq 24(%rdi), %xmm2
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    cvtsi2sd %rax, %xmm1
+; SSE41-NEXT:    cvtsi2sdq 16(%rdi), %xmm1
 ; SSE41-NEXT:    unpcklpd {{.*#+}} xmm1 = xmm1[0],xmm2[0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: sitofp_load_4i64_to_4f64:
 ; VEX:       # %bb.0:
-; VEX-NEXT:    vmovapd (%rdi), %xmm0
-; VEX-NEXT:    vmovdqa 16(%rdi), %xmm1
-; VEX-NEXT:    vpextrq $1, %xmm1, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm2
-; VEX-NEXT:    vmovq %xmm1, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm1
-; VEX-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; VEX-NEXT:    vpextrq $1, %xmm0, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm2
-; VEX-NEXT:    vmovq %xmm0, %rax
-; VEX-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm0
-; VEX-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; VEX-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; VEX-NEXT:    vcvtsi2sdq 24(%rdi), %xmm0, %xmm0
+; VEX-NEXT:    vcvtsi2sdq 16(%rdi), %xmm1, %xmm1
+; VEX-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
+; VEX-NEXT:    vcvtsi2sdq 8(%rdi), %xmm2, %xmm1
+; VEX-NEXT:    vcvtsi2sdq (%rdi), %xmm2, %xmm2
+; VEX-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm2[0],xmm1[0]
+; VEX-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: sitofp_load_4i64_to_4f64:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovapd (%rdi), %xmm0
-; AVX512F-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512F-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm2
-; AVX512F-NEXT:    vmovq %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm1
-; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm2
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm0
-; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; AVX512F-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; AVX512F-NEXT:    vcvtsi2sdq 24(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtsi2sdq 16(%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
+; AVX512F-NEXT:    vcvtsi2sdq 8(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vcvtsi2sdq (%rdi), %xmm2, %xmm2
+; AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm2[0],xmm1[0]
+; AVX512F-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: sitofp_load_4i64_to_4f64:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovapd (%rdi), %xmm0
-; AVX512VL-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512VL-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm2, %xmm2
-; AVX512VL-NEXT:    vmovq %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm1
-; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm2
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2sd %rax, %xmm3, %xmm0
-; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; AVX512VL-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; AVX512VL-NEXT:    vcvtsi2sdq 24(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtsi2sdq 16(%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
+; AVX512VL-NEXT:    vcvtsi2sdq 8(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vcvtsi2sdq (%rdi), %xmm2, %xmm2
+; AVX512VL-NEXT:    vunpcklpd {{.*#+}} xmm1 = xmm2[0],xmm1[0]
+; AVX512VL-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: sitofp_load_4i64_to_4f64:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vmovaps (%rdi), %ymm0
 ; AVX512DQ-NEXT:    vcvtqq2pd %zmm0, %zmm0
 ; AVX512DQ-NEXT:    # kill: def $ymm0 killed $ymm0 killed $zmm0
 ; AVX512DQ-NEXT:    retq

 ; Load Signed Integer to Float
 ;
 
 define <4 x float> @sitofp_load_4i64_to_4f32(<4 x i64> *%a) {
 ; SSE2-LABEL: sitofp_load_4i64_to_4f32:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa (%rdi), %xmm1
 ; SSE2-NEXT:    movdqa 16(%rdi), %xmm0
-; SSE2-NEXT:    movq %xmm0, %rax
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
+; SSE2-NEXT:    cvtsi2ssq 16(%rdi), %xmm2
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm0, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
-; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
+; SSE2-NEXT:    cvtsi2ssq (%rdi), %xmm0
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: sitofp_load_4i64_to_4f32:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa (%rdi), %xmm0
-; SSE41-NEXT:    movdqa 16(%rdi), %xmm1
-; SSE41-NEXT:    pextrq $1, %xmm0, %rax
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm2
-; SSE41-NEXT:    movq %xmm0, %rax
-; SSE41-NEXT:    xorps %xmm0, %xmm0
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm0
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; SSE41-NEXT:    movq %xmm1, %rax
-; SSE41-NEXT:    xorps %xmm2, %xmm2
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm2
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; SSE41-NEXT:    pextrq $1, %xmm1, %rax
+; SSE41-NEXT:    cvtsi2ssq 8(%rdi), %xmm1
+; SSE41-NEXT:    cvtsi2ssq (%rdi), %xmm0
+; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm1
+; SSE41-NEXT:    cvtsi2ssq 16(%rdi), %xmm1
+; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; SSE41-NEXT:    xorps %xmm1, %xmm1
+; SSE41-NEXT:    cvtsi2ssq 24(%rdi), %xmm1
 ; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: sitofp_load_4i64_to_4f32:
 ; VEX:       # %bb.0:
-; VEX-NEXT:    vmovdqa (%rdi), %xmm0
-; VEX-NEXT:    vmovdqa 16(%rdi), %xmm1
-; VEX-NEXT:    vpextrq $1, %xmm0, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm2, %xmm2
-; VEX-NEXT:    vmovq %xmm0, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm0
-; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; VEX-NEXT:    vmovq %xmm1, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm2
-; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; VEX-NEXT:    vpextrq $1, %xmm1, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm1
+; VEX-NEXT:    vcvtsi2ssq 8(%rdi), %xmm0, %xmm0
+; VEX-NEXT:    vcvtsi2ssq (%rdi), %xmm1, %xmm1
+; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; VEX-NEXT:    vcvtsi2ssq 16(%rdi), %xmm2, %xmm1
+; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; VEX-NEXT:    vcvtsi2ssq 24(%rdi), %xmm2, %xmm1
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: sitofp_load_4i64_to_4f32:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512F-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm2, %xmm2
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm0
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; AVX512F-NEXT:    vmovq %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm2
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm1
+; AVX512F-NEXT:    vcvtsi2ssq 8(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtsi2ssq (%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512F-NEXT:    vcvtsi2ssq 16(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512F-NEXT:    vcvtsi2ssq 24(%rdi), %xmm2, %xmm1
 ; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: sitofp_load_4i64_to_4f32:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512VL-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm2, %xmm2
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm0
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; AVX512VL-NEXT:    vmovq %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm2
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm3, %xmm1
+; AVX512VL-NEXT:    vcvtsi2ssq 8(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtsi2ssq (%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512VL-NEXT:    vcvtsi2ssq 16(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512VL-NEXT:    vcvtsi2ssq 24(%rdi), %xmm2, %xmm1
 ; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: sitofp_load_4i64_to_4f32:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vmovaps (%rdi), %ymm0
 ; AVX512DQ-NEXT:    vcvtqq2ps %zmm0, %ymm0
 ; AVX512DQ-NEXT:    # kill: def $xmm0 killed $xmm0 killed $ymm0

 
 define <8 x float> @sitofp_load_8i64_to_8f32(<8 x i64> *%a) {
 ; SSE2-LABEL: sitofp_load_8i64_to_8f32:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa (%rdi), %xmm1
 ; SSE2-NEXT:    movdqa 16(%rdi), %xmm0
 ; SSE2-NEXT:    movdqa 32(%rdi), %xmm2
 ; SSE2-NEXT:    movdqa 48(%rdi), %xmm3
-; SSE2-NEXT:    movq %xmm0, %rax
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm4
+; SSE2-NEXT:    cvtsi2ssq 16(%rdi), %xmm4
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm0, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm4 = xmm4[0],xmm0[0],xmm4[1],xmm0[1]
-; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
+; SSE2-NEXT:    cvtsi2ssq (%rdi), %xmm0
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm4[0]
-; SSE2-NEXT:    movq %xmm3, %rax
 ; SSE2-NEXT:    xorps %xmm4, %xmm4
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm4
+; SSE2-NEXT:    cvtsi2ssq 48(%rdi), %xmm4
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm3[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm1, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm4 = xmm4[0],xmm1[0],xmm4[1],xmm1[1]
-; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
+; SSE2-NEXT:    cvtsi2ssq 32(%rdi), %xmm1
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    xorps %xmm2, %xmm2
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
 ; SSE2-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm4[0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: sitofp_load_8i64_to_8f32:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movdqa (%rdi), %xmm0
-; SSE41-NEXT:    movdqa 16(%rdi), %xmm1
-; SSE41-NEXT:    movdqa 32(%rdi), %xmm2
-; SSE41-NEXT:    movdqa 48(%rdi), %xmm3
-; SSE41-NEXT:    pextrq $1, %xmm0, %rax
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm4
-; SSE41-NEXT:    movq %xmm0, %rax
-; SSE41-NEXT:    xorps %xmm0, %xmm0
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm0
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[2,3]
-; SSE41-NEXT:    movq %xmm1, %rax
-; SSE41-NEXT:    xorps %xmm4, %xmm4
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm4
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],xmm4[0],xmm0[3]
-; SSE41-NEXT:    pextrq $1, %xmm1, %rax
+; SSE41-NEXT:    cvtsi2ssq 8(%rdi), %xmm1
+; SSE41-NEXT:    cvtsi2ssq (%rdi), %xmm0
+; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[2,3]
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm1
+; SSE41-NEXT:    cvtsi2ssq 16(%rdi), %xmm1
+; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; SSE41-NEXT:    xorps %xmm1, %xmm1
+; SSE41-NEXT:    cvtsi2ssq 24(%rdi), %xmm1
 ; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; SSE41-NEXT:    pextrq $1, %xmm2, %rax
-; SSE41-NEXT:    xorps %xmm4, %xmm4
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm4
-; SSE41-NEXT:    movq %xmm2, %rax
+; SSE41-NEXT:    cvtsi2ssq 40(%rdi), %xmm2
 ; SSE41-NEXT:    xorps %xmm1, %xmm1
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm1
-; SSE41-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[2,3]
-; SSE41-NEXT:    movq %xmm3, %rax
+; SSE41-NEXT:    cvtsi2ssq 32(%rdi), %xmm1
+; SSE41-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[2,3]
 ; SSE41-NEXT:    xorps %xmm2, %xmm2
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm2
+; SSE41-NEXT:    cvtsi2ssq 48(%rdi), %xmm2
 ; SSE41-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
-; SSE41-NEXT:    pextrq $1, %xmm3, %rax
 ; SSE41-NEXT:    xorps %xmm2, %xmm2
-; SSE41-NEXT:    cvtsi2ss %rax, %xmm2
+; SSE41-NEXT:    cvtsi2ssq 56(%rdi), %xmm2
 ; SSE41-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
 ; SSE41-NEXT:    retq
 ;
 ; VEX-LABEL: sitofp_load_8i64_to_8f32:
 ; VEX:       # %bb.0:
-; VEX-NEXT:    vmovaps (%rdi), %xmm0
-; VEX-NEXT:    vmovdqa 16(%rdi), %xmm1
-; VEX-NEXT:    vmovdqa 32(%rdi), %xmm2
-; VEX-NEXT:    vmovdqa 48(%rdi), %xmm3
-; VEX-NEXT:    vpextrq $1, %xmm2, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm4, %xmm4
-; VEX-NEXT:    vmovq %xmm2, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm2
-; VEX-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[2,3]
-; VEX-NEXT:    vmovq %xmm3, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm4
-; VEX-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0],xmm2[3]
-; VEX-NEXT:    vpextrq $1, %xmm3, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; VEX-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[0]
-; VEX-NEXT:    vpextrq $1, %xmm0, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; VEX-NEXT:    vmovq %xmm0, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm0
-; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[2,3]
-; VEX-NEXT:    vmovq %xmm1, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0],xmm0[3]
-; VEX-NEXT:    vpextrq $1, %xmm1, %rax
-; VEX-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm1
+; VEX-NEXT:    vcvtsi2ssq 40(%rdi), %xmm0, %xmm0
+; VEX-NEXT:    vcvtsi2ssq 32(%rdi), %xmm1, %xmm1
+; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; VEX-NEXT:    vcvtsi2ssq 48(%rdi), %xmm2, %xmm1
+; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; VEX-NEXT:    vcvtsi2ssq 56(%rdi), %xmm2, %xmm1
 ; VEX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; VEX-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
+; VEX-NEXT:    vcvtsi2ssq 8(%rdi), %xmm2, %xmm1
+; VEX-NEXT:    vcvtsi2ssq (%rdi), %xmm2, %xmm2
+; VEX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[2,3]
+; VEX-NEXT:    vcvtsi2ssq 16(%rdi), %xmm3, %xmm2
+; VEX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
+; VEX-NEXT:    vcvtsi2ssq 24(%rdi), %xmm3, %xmm2
+; VEX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
+; VEX-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; VEX-NEXT:    retq
 ;
 ; AVX512F-LABEL: sitofp_load_8i64_to_8f32:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovaps (%rdi), %xmm0
-; AVX512F-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512F-NEXT:    vmovdqa 32(%rdi), %xmm2
-; AVX512F-NEXT:    vmovdqa 48(%rdi), %xmm3
-; AVX512F-NEXT:    vpextrq $1, %xmm2, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm4, %xmm4
-; AVX512F-NEXT:    vmovq %xmm2, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm2
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[2,3]
-; AVX512F-NEXT:    vmovq %xmm3, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm4
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0],xmm2[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm3, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[0]
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm0
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[2,3]
-; AVX512F-NEXT:    vmovq %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0],xmm0[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512F-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm1
+; AVX512F-NEXT:    vcvtsi2ssq 40(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtsi2ssq 32(%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512F-NEXT:    vcvtsi2ssq 48(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512F-NEXT:    vcvtsi2ssq 56(%rdi), %xmm2, %xmm1
 ; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; AVX512F-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
+; AVX512F-NEXT:    vcvtsi2ssq 8(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vcvtsi2ssq (%rdi), %xmm2, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[2,3]
+; AVX512F-NEXT:    vcvtsi2ssq 16(%rdi), %xmm3, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
+; AVX512F-NEXT:    vcvtsi2ssq 24(%rdi), %xmm3, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
+; AVX512F-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: sitofp_load_8i64_to_8f32:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovaps (%rdi), %xmm0
-; AVX512VL-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512VL-NEXT:    vmovdqa 32(%rdi), %xmm2
-; AVX512VL-NEXT:    vmovdqa 48(%rdi), %xmm3
-; AVX512VL-NEXT:    vpextrq $1, %xmm2, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm4, %xmm4
-; AVX512VL-NEXT:    vmovq %xmm2, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm2
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[2,3]
-; AVX512VL-NEXT:    vmovq %xmm3, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm4
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0],xmm2[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm3, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[0]
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm0
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[2,3]
-; AVX512VL-NEXT:    vmovq %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0],xmm0[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512VL-NEXT:    vcvtsi2ss %rax, %xmm5, %xmm1
+; AVX512VL-NEXT:    vcvtsi2ssq 40(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtsi2ssq 32(%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512VL-NEXT:    vcvtsi2ssq 48(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512VL-NEXT:    vcvtsi2ssq 56(%rdi), %xmm2, %xmm1
 ; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; AVX512VL-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
+; AVX512VL-NEXT:    vcvtsi2ssq 8(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vcvtsi2ssq (%rdi), %xmm2, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[2,3]
+; AVX512VL-NEXT:    vcvtsi2ssq 16(%rdi), %xmm3, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
+; AVX512VL-NEXT:    vcvtsi2ssq 24(%rdi), %xmm3, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
+; AVX512VL-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: sitofp_load_8i64_to_8f32:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vcvtqq2ps (%rdi), %ymm0
 ; AVX512DQ-NEXT:    retq
 ;
 ; AVX512VLDQ-LABEL: sitofp_load_8i64_to_8f32:

 ;
 ; Load Unsigned Integer to Float
 ;
 
 define <4 x float> @uitofp_load_4i64_to_4f32(<4 x i64> *%a) {
 ; SSE2-LABEL: uitofp_load_4i64_to_4f32:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa 16(%rdi), %xmm0
-; SSE2-NEXT:    movq %xmm0, %rax
+; SSE2-NEXT:    movq 16(%rdi), %rax
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB83_1
 ; SSE2-NEXT:  # %bb.2:
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    jmp .LBB83_3
 ; SSE2-NEXT:  .LBB83_1:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    addss %xmm1, %xmm1
 ; SSE2-NEXT:  .LBB83_3:
-; SSE2-NEXT:    movdqa (%rdi), %xmm2
+; SSE2-NEXT:    movq (%rdi), %rax
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
-; SSE2-NEXT:    movq %xmm0, %rax
-; SSE2-NEXT:    testq %rax, %rax
+; SSE2-NEXT:    movq %xmm0, %rcx
+; SSE2-NEXT:    testq %rcx, %rcx
 ; SSE2-NEXT:    js .LBB83_4
 ; SSE2-NEXT:  # %bb.5:
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm3
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm2
 ; SSE2-NEXT:    jmp .LBB83_6
 ; SSE2-NEXT:  .LBB83_4:
-; SSE2-NEXT:    movq %rax, %rcx
-; SSE2-NEXT:    shrq %rcx
-; SSE2-NEXT:    andl $1, %eax
-; SSE2-NEXT:    orq %rcx, %rax
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm3
-; SSE2-NEXT:    addss %xmm3, %xmm3
+; SSE2-NEXT:    movq %rcx, %rdx
+; SSE2-NEXT:    shrq %rdx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm2
+; SSE2-NEXT:    addss %xmm2, %xmm2
 ; SSE2-NEXT:  .LBB83_6:
-; SSE2-NEXT:    movq %xmm2, %rax
+; SSE2-NEXT:    movdqa (%rdi), %xmm3
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB83_7
 ; SSE2-NEXT:  # %bb.8:
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    jmp .LBB83_9
 ; SSE2-NEXT:  .LBB83_7:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    addss %xmm0, %xmm0
 ; SSE2-NEXT:  .LBB83_9:
-; SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[2,3,2,3]
+; SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm3[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB83_10
 ; SSE2-NEXT:  # %bb.11:
 ; SSE2-NEXT:    xorps %xmm2, %xmm2
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
 ; SSE2-NEXT:    jmp .LBB83_12
 ; SSE2-NEXT:  .LBB83_10:

 ; AVX2-NEXT:    vmovdqa (%rdi), %xmm2
 ; AVX2-NEXT:    vpackssdw 16(%rdi), %xmm2, %xmm2
 ; AVX2-NEXT:    vblendvps %xmm2, %xmm1, %xmm0, %xmm0
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_load_4i64_to_4f32:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512F-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm2, %xmm2
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm0
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; AVX512F-NEXT:    vmovq %xmm1, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm2
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm1
+; AVX512F-NEXT:    vcvtusi2ssq 8(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtusi2ssq (%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512F-NEXT:    vcvtusi2ssq 16(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512F-NEXT:    vcvtusi2ssq 24(%rdi), %xmm2, %xmm1
 ; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: uitofp_load_4i64_to_4f32:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovdqa (%rdi), %xmm0
-; AVX512VL-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm2, %xmm2
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm0
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
-; AVX512VL-NEXT:    vmovq %xmm1, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm2
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm2[0],xmm0[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm3, %xmm1
+; AVX512VL-NEXT:    vcvtusi2ssq 8(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtusi2ssq (%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512VL-NEXT:    vcvtusi2ssq 16(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512VL-NEXT:    vcvtusi2ssq 24(%rdi), %xmm2, %xmm1
 ; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: uitofp_load_4i64_to_4f32:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vmovaps (%rdi), %ymm0
 ; AVX512DQ-NEXT:    vcvtuqq2ps %zmm0, %ymm0
 ; AVX512DQ-NEXT:    # kill: def $xmm0 killed $xmm0 killed $ymm0
; AVX-NEXT:    retq
   %cvt = uitofp <4 x i8> %ld to <4 x float>
   ret <4 x float> %cvt
 }
 
 define <8 x float> @uitofp_load_8i64_to_8f32(<8 x i64> *%a) {
 ; SSE2-LABEL: uitofp_load_8i64_to_8f32:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa 16(%rdi), %xmm0
-; SSE2-NEXT:    movq %xmm0, %rax
+; SSE2-NEXT:    movq 16(%rdi), %rax
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB87_1
 ; SSE2-NEXT:  # %bb.2:
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
 ; SSE2-NEXT:    jmp .LBB87_3
 ; SSE2-NEXT:  .LBB87_1:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
 ; SSE2-NEXT:    addss %xmm2, %xmm2
 ; SSE2-NEXT:  .LBB87_3:
-; SSE2-NEXT:    movdqa (%rdi), %xmm3
+; SSE2-NEXT:    movq (%rdi), %rax
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
-; SSE2-NEXT:    movq %xmm0, %rax
-; SSE2-NEXT:    testq %rax, %rax
+; SSE2-NEXT:    movq %xmm0, %rcx
+; SSE2-NEXT:    testq %rcx, %rcx
 ; SSE2-NEXT:    js .LBB87_4
 ; SSE2-NEXT:  # %bb.5:
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm1
 ; SSE2-NEXT:    jmp .LBB87_6
 ; SSE2-NEXT:  .LBB87_4:
-; SSE2-NEXT:    movq %rax, %rcx
-; SSE2-NEXT:    shrq %rcx
-; SSE2-NEXT:    andl $1, %eax
-; SSE2-NEXT:    orq %rcx, %rax
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
+; SSE2-NEXT:    movq %rcx, %rdx
+; SSE2-NEXT:    shrq %rdx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm1
 ; SSE2-NEXT:    addss %xmm1, %xmm1
 ; SSE2-NEXT:  .LBB87_6:
-; SSE2-NEXT:    movq %xmm3, %rax
+; SSE2-NEXT:    movdqa (%rdi), %xmm3
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB87_7
 ; SSE2-NEXT:  # %bb.8:
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    jmp .LBB87_9
 ; SSE2-NEXT:  .LBB87_7:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm0, %xmm0
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm0
 ; SSE2-NEXT:    addss %xmm0, %xmm0
 ; SSE2-NEXT:  .LBB87_9:
-; SSE2-NEXT:    movdqa 48(%rdi), %xmm6
+; SSE2-NEXT:    movq 48(%rdi), %rax
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm3[2,3,2,3]
-; SSE2-NEXT:    movq %xmm3, %rax
-; SSE2-NEXT:    testq %rax, %rax
+; SSE2-NEXT:    movq %xmm3, %rcx
+; SSE2-NEXT:    testq %rcx, %rcx
 ; SSE2-NEXT:    js .LBB87_10
 ; SSE2-NEXT:  # %bb.11:
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm4
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm4
 ; SSE2-NEXT:    jmp .LBB87_12
 ; SSE2-NEXT:  .LBB87_10:
-; SSE2-NEXT:    movq %rax, %rcx
-; SSE2-NEXT:    shrq %rcx
-; SSE2-NEXT:    andl $1, %eax
-; SSE2-NEXT:    orq %rcx, %rax
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm4
+; SSE2-NEXT:    movq %rcx, %rdx
+; SSE2-NEXT:    shrq %rdx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm4
 ; SSE2-NEXT:    addss %xmm4, %xmm4
 ; SSE2-NEXT:  .LBB87_12:
-; SSE2-NEXT:    movq %xmm6, %rax
+; SSE2-NEXT:    movdqa 48(%rdi), %xmm5
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB87_13
 ; SSE2-NEXT:  # %bb.14:
 ; SSE2-NEXT:    xorps %xmm3, %xmm3
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm3
 ; SSE2-NEXT:    jmp .LBB87_15
 ; SSE2-NEXT:  .LBB87_13:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm3, %xmm3
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm3
 ; SSE2-NEXT:    addss %xmm3, %xmm3
 ; SSE2-NEXT:  .LBB87_15:
-; SSE2-NEXT:    movdqa 32(%rdi), %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[2,3,2,3]
-; SSE2-NEXT:    movq %xmm6, %rax
-; SSE2-NEXT:    testq %rax, %rax
+; SSE2-NEXT:    movq 32(%rdi), %rax
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[2,3,2,3]
+; SSE2-NEXT:    movq %xmm5, %rcx
+; SSE2-NEXT:    testq %rcx, %rcx
 ; SSE2-NEXT:    js .LBB87_16
 ; SSE2-NEXT:  # %bb.17:
-; SSE2-NEXT:    xorps %xmm6, %xmm6
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm6
+; SSE2-NEXT:    xorps %xmm5, %xmm5
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm5
 ; SSE2-NEXT:    jmp .LBB87_18
 ; SSE2-NEXT:  .LBB87_16:
-; SSE2-NEXT:    movq %rax, %rcx
-; SSE2-NEXT:    shrq %rcx
-; SSE2-NEXT:    andl $1, %eax
-; SSE2-NEXT:    orq %rcx, %rax
-; SSE2-NEXT:    xorps %xmm6, %xmm6
-; SSE2-NEXT:    cvtsi2ss %rax, %xmm6
-; SSE2-NEXT:    addss %xmm6, %xmm6
+; SSE2-NEXT:    movq %rcx, %rdx
+; SSE2-NEXT:    shrq %rdx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    xorps %xmm5, %xmm5
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm5
+; SSE2-NEXT:    addss %xmm5, %xmm5
 ; SSE2-NEXT:  .LBB87_18:
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1]
-; SSE2-NEXT:    movq %xmm5, %rax
+; SSE2-NEXT:    movdqa 32(%rdi), %xmm4
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB87_19
 ; SSE2-NEXT:  # %bb.20:
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    jmp .LBB87_21
 ; SSE2-NEXT:  .LBB87_19:
 ; SSE2-NEXT:    movq %rax, %rcx
 ; SSE2-NEXT:    shrq %rcx
 ; SSE2-NEXT:    andl $1, %eax
 ; SSE2-NEXT:    orq %rcx, %rax
 ; SSE2-NEXT:    xorps %xmm1, %xmm1
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm1
 ; SSE2-NEXT:    addss %xmm1, %xmm1
 ; SSE2-NEXT:  .LBB87_21:
 ; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; SSE2-NEXT:    unpcklps {{.*#+}} xmm3 = xmm3[0],xmm6[0],xmm3[1],xmm6[1]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm5[2,3,2,3]
+; SSE2-NEXT:    unpcklps {{.*#+}} xmm3 = xmm3[0],xmm5[0],xmm3[1],xmm5[1]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm4[2,3,2,3]
 ; SSE2-NEXT:    movq %xmm2, %rax
 ; SSE2-NEXT:    testq %rax, %rax
 ; SSE2-NEXT:    js .LBB87_22
 ; SSE2-NEXT:  # %bb.23:
 ; SSE2-NEXT:    xorps %xmm2, %xmm2
 ; SSE2-NEXT:    cvtsi2ss %rax, %xmm2
 ; SSE2-NEXT:    jmp .LBB87_24
 ; SSE2-NEXT:  .LBB87_22:

 ; AVX2-NEXT:    vaddps %xmm0, %xmm0, %xmm2
 ; AVX2-NEXT:    vpackssdw 16(%rdi), %xmm4, %xmm3
 ; AVX2-NEXT:    vblendvps %xmm3, %xmm2, %xmm0, %xmm0
 ; AVX2-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
 ; AVX2-NEXT:    retq
 ;
 ; AVX512F-LABEL: uitofp_load_8i64_to_8f32:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vmovaps (%rdi), %xmm0
-; AVX512F-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512F-NEXT:    vmovdqa 32(%rdi), %xmm2
-; AVX512F-NEXT:    vmovdqa 48(%rdi), %xmm3
-; AVX512F-NEXT:    vpextrq $1, %xmm2, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm4, %xmm4
-; AVX512F-NEXT:    vmovq %xmm2, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm2
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[2,3]
-; AVX512F-NEXT:    vmovq %xmm3, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm4
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0],xmm2[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm3, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[0]
-; AVX512F-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vmovq %xmm0, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm0
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[2,3]
-; AVX512F-NEXT:    vmovq %xmm1, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0],xmm0[3]
-; AVX512F-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512F-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm1
+; AVX512F-NEXT:    vcvtusi2ssq 40(%rdi), %xmm0, %xmm0
+; AVX512F-NEXT:    vcvtusi2ssq 32(%rdi), %xmm1, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512F-NEXT:    vcvtusi2ssq 48(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512F-NEXT:    vcvtusi2ssq 56(%rdi), %xmm2, %xmm1
 ; AVX512F-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; AVX512F-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
+; AVX512F-NEXT:    vcvtusi2ssq 8(%rdi), %xmm2, %xmm1
+; AVX512F-NEXT:    vcvtusi2ssq (%rdi), %xmm2, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[2,3]
+; AVX512F-NEXT:    vcvtusi2ssq 16(%rdi), %xmm3, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
+; AVX512F-NEXT:    vcvtusi2ssq 24(%rdi), %xmm3, %xmm2
+; AVX512F-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
+; AVX512F-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512VL-LABEL: uitofp_load_8i64_to_8f32:
 ; AVX512VL:       # %bb.0:
-; AVX512VL-NEXT:    vmovaps (%rdi), %xmm0
-; AVX512VL-NEXT:    vmovdqa 16(%rdi), %xmm1
-; AVX512VL-NEXT:    vmovdqa 32(%rdi), %xmm2
-; AVX512VL-NEXT:    vmovdqa 48(%rdi), %xmm3
-; AVX512VL-NEXT:    vpextrq $1, %xmm2, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm4, %xmm4
-; AVX512VL-NEXT:    vmovq %xmm2, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm2
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[2,3]
-; AVX512VL-NEXT:    vmovq %xmm3, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm4
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0],xmm2[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm3, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[0]
-; AVX512VL-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vmovq %xmm0, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm0
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[2,3]
-; AVX512VL-NEXT:    vmovq %xmm1, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm3
-; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0],xmm0[3]
-; AVX512VL-NEXT:    vpextrq $1, %xmm1, %rax
-; AVX512VL-NEXT:    vcvtusi2ss %rax, %xmm5, %xmm1
+; AVX512VL-NEXT:    vcvtusi2ssq 40(%rdi), %xmm0, %xmm0
+; AVX512VL-NEXT:    vcvtusi2ssq 32(%rdi), %xmm1, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
+; AVX512VL-NEXT:    vcvtusi2ssq 48(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0],xmm0[3]
+; AVX512VL-NEXT:    vcvtusi2ssq 56(%rdi), %xmm2, %xmm1
 ; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[0]
-; AVX512VL-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
+; AVX512VL-NEXT:    vcvtusi2ssq 8(%rdi), %xmm2, %xmm1
+; AVX512VL-NEXT:    vcvtusi2ssq (%rdi), %xmm2, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[2,3]
+; AVX512VL-NEXT:    vcvtusi2ssq 16(%rdi), %xmm3, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0],xmm1[3]
+; AVX512VL-NEXT:    vcvtusi2ssq 24(%rdi), %xmm3, %xmm2
+; AVX512VL-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],xmm2[0]
+; AVX512VL-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; AVX512VL-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: uitofp_load_8i64_to_8f32:
 ; AVX512DQ:       # %bb.0:
 ; AVX512DQ-NEXT:    vcvtuqq2ps (%rdi), %ymm0
 ; AVX512DQ-NEXT:    retq
 ;
 ; AVX512VLDQ-LABEL: uitofp_load_8i64_to_8f32: