[x86] use an insert op to put one variable element into a constant of vectors

[spatel]

Summary

Instead of loading (a potential ton of) scalar constants, load those as a vector and then insert into it.

I don't know how to solve the FP cases, so I'm posting this as-is to see if anyone else has ideas or if I've just approached this in the wrong way. I stepped through the FP examples, and we show something like this:

Legalizing: t6: f32 = ConstantFP<3.000000e+00>
Trying to expand node
Succesfully expanded node
 ... replacing: t6: f32 = ConstantFP<3.000000e+00>
     with:      t22: f32,ch = load<LD4[ConstantPool]> t0, ConstantPool:i64<float 3.000000e+00> 0, undef:i64

This doesn't happen with int constants...because int constants are legal but FP constants are not? I looked at creating target FP constants instead, but that can cause crashing for reasons I haven't tracked down yet.

[RKSimon]

test/CodeGen/X86/insert-into-constant-vector.ll

9 ↗

(On Diff #118384)

You're using X32SSE/X32AVX on 64-bit targets

[spatel]

Patch updated - no code changes, but tried to fix the bogus test assertions:

1. Recommitted the baseline checks with the check-prefix fixes for X64 (rL315368).
2. Rebased this patch.

[efriedma]

This doesn't happen with int constants...because int constants are legal but FP constants are not?

Yes, exactly.

We call LowerBUILD_VECTOR before we legalize the operands of a BUILD_VECTOR, so that shouldn't block this optimization. But we don't recompute the topological ordering when you create new nodes, so legalization can happen in an unexpected order if you create illegal nodes (by, for example, calling getBuildVector()).

[spatel]

In D38756#893854, @efriedma wrote:

This doesn't happen with int constants...because int constants are legal but FP constants are not?

Yes, exactly.

We call LowerBUILD_VECTOR before we legalize the operands of a BUILD_VECTOR, so that shouldn't block this optimization. But we don't recompute the topological ordering when you create new nodes, so legalization can happen in an unexpected order if you create illegal nodes (by, for example, calling getBuildVector()).

Ah, so I should be checking legality of the BV and the insert to be safe?
I tried to work-around the FP issue by creating a constant pool load, but that got a bit ugly - I had to call LowerConstantPool() to create a legal vector of constants. Am I just waiting too long by trying to do this in lowering rather than combining? I did try this as a combine too, but that caused an inf-loop with the opposite DAGCombiner transform that sticks the insertelement back into the BV.

[efriedma]

I tried to work-around the FP issue by creating a constant pool load, but that got a bit ugly - I had to call LowerConstantPool() to create a legal vector of constants.

Yes, it's a little ugly, but it's self-contained. I don't have a better suggestion.

but that caused an inf-loop with the opposite DAGCombiner transform that sticks the insertelement back into the BV.

DAGCombine prefers to simplify the other way to eliminate chains of inserts... I think you would run into serious problems trying to change that.

[spatel]

In D38756#893892, @efriedma wrote:

I tried to work-around the FP issue by creating a constant pool load, but that got a bit ugly - I had to call LowerConstantPool() to create a legal vector of constants.

Yes, it's a little ugly, but it's self-contained. I don't have a better suggestion.

Ok - thanks! I'd managed to mostly avoid lowering after all this time by transforming things sooner. :)

[spatel]

Patch updated:
Instead of creating a potentially illegal build vector of constants, create a legal vector of constants and explicitly load it for use in the insertelement.

I don't think we care about the potential regression in 'test_negative_zero_2' because that pattern would be simplified by instcombine; we're just making sure the negative zero constant is handled correctly.

Also, I think we're alright without checking for any particular vector ISA version. When we create the insertelement, if we don't have a particular insert op, that node will get expanded to something else, but that thing is still better than the chain of inserts that we currently produce. Example with only SSE2:

Legalizing: t21: v4i32 = insert_vector_elt t20, t2, Constant:i64<1>
Trying custom legalization
Could not custom legalize node
Trying to expand node
Creating new node: t22: v4i32 = scalar_to_vector t2
Succesfully expanded node
... replacing: t21: v4i32 = insert_vector_elt t20, t2, Constant:i64<1>

with:      t23: v4i32 = vector_shuffle<0,4,2,3> t20, t22

[spatel]

Patch updated:
IIUC, it's not safe to assume that we can always expand the insertelement node properly (even though I haven't found a case where that's a problem). So I've added an additional check before we try this:

(isOperationLegalOrCustom(ISD::INSERT_VECTOR_ELT, VT) ||
 isOperationLegalOrCustom(ISD::VECTOR_SHUFFLE, VT))) {

The assumption is that if we're going to expand the insertelement, then it's going to be turned into a shuffle, so we must be able to lower that shuffle in order to do this transform.

This enables all of the same test diffs, so no changes there.

[spatel]

Ping.

[RKSimon]

lib/Target/X86/X86ISelLowering.cpp

7682 ↗

(On Diff #119020)

Worth splatting this with UndefValue::get(EltType) ?

7685 ↗

(On Diff #119020)

We have other uses of DAG.GetContext - maybe worth pulling this out?

7708 ↗

(On Diff #119020)

This is causing a lot of shorter loads (movsd etc.) to become full width loads - either add a TODO or create a helper with a TODO that we can use to create shorter loads (broadcast/VZEXT_LOAD etc) with a future commit.

[spatel]

Patch updated:

1. Pull LLVMContext local variable out, so it can be used later in LowerBUILD_VECTOR().
2. Initialize constant vector with undefs, so we don't have to explicitly set that in the loop.
3. Add TODO comment about reducing the size of the loads.

[RKSimon]

LGTM

lib/Target/X86/X86ISelLowering.cpp

7725 ↗

(On Diff #120433)

assert that VarElts + InsIndex are null?

test/CodeGen/X86/buildvec-insertvec.ll

74 ↗

(On Diff #120433)

Add a FIXME comment to this test

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG) const {
 
   unsigned EVTBits = ExtVT.getSizeInBits();
 
   unsigned NumZero  = 0;
   unsigned NumNonZero = 0;
   uint64_t NonZeros = 0;
   bool IsAllConstants = true;
   SmallSet<SDValue, 8> Values;
+  unsigned NumConstants = NumElems;
   for (unsigned i = 0; i < NumElems; ++i) {
     SDValue Elt = Op.getOperand(i);
     if (Elt.isUndef())
       continue;
     Values.insert(Elt);
-    if (Elt.getOpcode() != ISD::Constant &&
-        Elt.getOpcode() != ISD::ConstantFP)
+    if (!isa<ConstantSDNode>(Elt) && !isa<ConstantFPSDNode>(Elt)) {
       IsAllConstants = false;
+      NumConstants--;
+    }
     if (X86::isZeroNode(Elt))
       NumZero++;
     else {
       assert(i < sizeof(NonZeros) * 8); // Make sure the shift is within range.
       NonZeros |= ((uint64_t)1 << i);
       NumNonZero++;
     }
   }
 
   // All undef vector. Return an UNDEF.  All zero vectors were handled above.
   if (NumNonZero == 0)
     return DAG.getUNDEF(VT);
 
+  // If we are inserting one variable into a vector of non-zero constants, try
+  // to avoid loading each constant element as a scalar. Load the constants as a
+  // vector and then insert the variable scalar element. If insertion is not
+  // supported, we assume that we will fall back to a shuffle to get the scalar
+  // blended with the constants. Insertion into a zero vector is handled as a
+  // special-case somewhere below here.
+  LLVMContext &Context = *DAG.getContext();
+  if (NumConstants == NumElems - 1 && NumNonZero != 1 &&
+      (isOperationLegalOrCustom(ISD::INSERT_VECTOR_ELT, VT) ||
+       isOperationLegalOrCustom(ISD::VECTOR_SHUFFLE, VT))) {
+    // Create an all-constant vector. The variable element in the old
+    // build vector is replaced by undef in the constant vector. Save the
+    // variable scalar element and its index for use in the insertelement.
+    Type *EltType = Op.getValueType().getScalarType().getTypeForEVT(Context);
+    SmallVector<Constant *, 16> ConstVecOps(NumElems, UndefValue::get(EltType));
+    SDValue VarElt;
+    SDValue InsIndex;
+    for (unsigned i = 0; i != NumElems; ++i) {
+      SDValue Elt = Op.getOperand(i);
+      if (auto *C = dyn_cast<ConstantSDNode>(Elt))
+        ConstVecOps[i] = ConstantInt::get(Context, C->getAPIntValue());
+      else if (auto *C = dyn_cast<ConstantFPSDNode>(Elt))
+        ConstVecOps[i] = ConstantFP::get(Context, C->getValueAPF());
+      else if (!Elt.isUndef()) {
+        assert(!VarElt.getNode() && !InsIndex.getNode() &&
+               "Expected one variable element in this vector");
+        VarElt = Elt;
+        InsIndex = DAG.getConstant(i, dl, getVectorIdxTy(DAG.getDataLayout()));
+      }
+    }
+    Constant *CV = ConstantVector::get(ConstVecOps);
+    SDValue DAGConstVec = DAG.getConstantPool(CV, VT);
+
+    // The constants we just created may not be legal (eg, floating point). We
+    // must lower the vector right here because we can not guarantee that we'll
+    // legalize it before loading it. This is also why we could not just create
+    // a new build vector here. If the build vector contains illegal constants,
+    // it could get split back up into a series of insert elements.
+    // TODO: Improve this by using shorter loads with broadcast/VZEXT_LOAD.
+    SDValue LegalDAGConstVec = LowerConstantPool(DAGConstVec, DAG);
+    MachineFunction &MF = DAG.getMachineFunction();
+    MachinePointerInfo MPI = MachinePointerInfo::getConstantPool(MF);
+    SDValue Ld = DAG.getLoad(VT, dl, DAG.getEntryNode(), LegalDAGConstVec, MPI);
+    return DAG.getNode(ISD::INSERT_VECTOR_ELT, dl, VT, Ld, VarElt, InsIndex);
+  }
+
   // Special case for single non-zero, non-undef, element.
   if (NumNonZero == 1) {
     unsigned Idx = countTrailingZeros(NonZeros);
     SDValue Item = Op.getOperand(Idx);
 
     // If this is an insertion of an i64 value on x86-32, and if the top bits of
     // the value are obviously zero, truncate the value to i32 and do the
     // insertion that way.  Only do this if the value is non-constant or if the
if (VT.is128BitVector() || VT.is256BitVector() || VT.is512BitVector()) {
     if (SDValue LD =
             EltsFromConsecutiveLoads(VT, Ops, dl, DAG, Subtarget, false))
       return LD;
   }
 
   // For AVX-length vectors, build the individual 128-bit pieces and use
   // shuffles to put them in place.
   if (VT.is256BitVector() || VT.is512BitVector()) {
-    EVT HVT = EVT::getVectorVT(*DAG.getContext(), ExtVT, NumElems/2);
+    EVT HVT = EVT::getVectorVT(Context, ExtVT, NumElems/2);
 
     // Build both the lower and upper subvector.
     SDValue Lower =
         DAG.getBuildVector(HVT, dl, Op->ops().slice(0, NumElems / 2));
     SDValue Upper = DAG.getBuildVector(
         HVT, dl, Op->ops().slice(NumElems / 2, NumElems /2));
 
     // Recreate the wider vector with the lower and upper part.

llvm/trunk/test/CodeGen/X86/buildvec-insertvec.ll

entry:
   %1 = insertelement <4 x float> undef, float %0, i32 0
   %2 = insertelement <4 x float> %1, float -0.0, i32 1
   %3 = extractelement <4 x float> %A, i32 2
   %4 = insertelement <4 x float> %2, float %3, i32 2
   %5 = insertelement <4 x float> %4, float 0.0, i32 3
   ret <4 x float> %5
 }
 
+; FIXME: This could be 'movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]'.
+
 define <2 x double> @test_negative_zero_2(<2 x double> %A) {
-; CHECK-LABEL: test_negative_zero_2:
-; CHECK:       # BB#0: # %entry
-; CHECK-NEXT:    movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
-; CHECK-NEXT:    retq
+; SSE2-LABEL: test_negative_zero_2:
+; SSE2:       # BB#0: # %entry
+; SSE2-NEXT:    movapd {{.*#+}} xmm1 = <u,-0>
+; SSE2-NEXT:    movsd {{.*#+}} xmm1 = xmm0[0],xmm1[1]
+; SSE2-NEXT:    movapd %xmm1, %xmm0
+; SSE2-NEXT:    retq
+;
+; SSE41-LABEL: test_negative_zero_2:
+; SSE41:       # BB#0: # %entry
+; SSE41-NEXT:    blendpd {{.*#+}} xmm0 = xmm0[0],mem[1]
+; SSE41-NEXT:    retq
 entry:
   %0 = extractelement <2 x double> %A, i32 0
   %1 = insertelement <2 x double> undef, double %0, i32 0
   %2 = insertelement <2 x double> %1, double -0.0, i32 1
   ret <2 x double> %2
 }
 
 define <4 x float> @test_buildvector_v4f32_register(float %f0, float %f1, float %f2, float %f3) {

llvm/trunk/test/CodeGen/X86/insert-into-constant-vector.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-unknown   -mattr=+sse2     | FileCheck %s --check-prefix=X32SSE --check-prefix=X32SSE2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse2     | FileCheck %s --check-prefix=X64SSE --check-prefix=X64SSE2
 ; RUN: llc < %s -mtriple=i686-unknown-unknown   -mattr=+sse4.1   | FileCheck %s --check-prefix=X32SSE --check-prefix=X32SSE4
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse4.1   | FileCheck %s --check-prefix=X64SSE --check-prefix=X64SSE4
 ; RUN: llc < %s -mtriple=i686-unknown-unknown   -mattr=+avx2     | FileCheck %s --check-prefix=X32AVX --check-prefix=X32AVX2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2     | FileCheck %s --check-prefix=X64AVX --check-prefix=X64AVX2
 ; RUN: llc < %s -mtriple=i686-unknown-unknown   -mattr=+avx512f  | FileCheck %s --check-prefix=X32AVX --check-prefix=X32AVX512F
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f  | FileCheck %s --check-prefix=X64AVX --check-prefix=X64AVX512F
 
 define <16 x i8> @elt0_v16i8(i8 %x) {
 ; X32SSE2-LABEL: elt0_v16i8:
 ; X32SSE2:       # BB#0:
-; X32SSE2-NEXT:    movl $15, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $14, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
-; X32SSE2-NEXT:    movl $13, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $12, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1],xmm2[2],xmm1[2],xmm2[3],xmm1[3]
-; X32SSE2-NEXT:    movl $11, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $10, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm3
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1],xmm3[2],xmm0[2],xmm3[3],xmm0[3],xmm3[4],xmm0[4],xmm3[5],xmm0[5],xmm3[6],xmm0[6],xmm3[7],xmm0[7]
-; X32SSE2-NEXT:    movl $9, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $8, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1],xmm1[2],xmm3[2],xmm1[3],xmm3[3]
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
-; X32SSE2-NEXT:    movl $7, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $6, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X32SSE2-NEXT:    movl $5, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $4, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm3
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1],xmm3[2],xmm0[2],xmm3[3],xmm0[3],xmm3[4],xmm0[4],xmm3[5],xmm0[5],xmm3[6],xmm0[6],xmm3[7],xmm0[7]
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm3 = xmm3[0],xmm2[0],xmm3[1],xmm2[1],xmm3[2],xmm2[2],xmm3[3],xmm2[3]
-; X32SSE2-NEXT:    movl $3, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $2, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X32SSE2-NEXT:    movl $1, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm4
-; X32SSE2-NEXT:    movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3],xmm0[4],xmm4[4],xmm0[5],xmm4[5],xmm0[6],xmm4[6],xmm0[7],xmm4[7]
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1],xmm0[2],xmm2[2],xmm0[3],xmm2[3]
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1]
-; X32SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X32SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; X32SSE2-NEXT:    movaps {{.*#+}} xmm0 = [0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
+; X32SSE2-NEXT:    andnps %xmm1, %xmm0
+; X32SSE2-NEXT:    orps {{\.LCPI.*}}, %xmm0
 ; X32SSE2-NEXT:    retl
 ;
 ; X64SSE2-LABEL: elt0_v16i8:
 ; X64SSE2:       # BB#0:
-; X64SSE2-NEXT:    movl $15, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $14, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
-; X64SSE2-NEXT:    movl $13, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $12, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1],xmm2[2],xmm1[2],xmm2[3],xmm1[3]
-; X64SSE2-NEXT:    movl $11, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $10, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm3
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1],xmm3[2],xmm0[2],xmm3[3],xmm0[3],xmm3[4],xmm0[4],xmm3[5],xmm0[5],xmm3[6],xmm0[6],xmm3[7],xmm0[7]
-; X64SSE2-NEXT:    movl $9, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $8, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1],xmm1[2],xmm3[2],xmm1[3],xmm3[3]
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
-; X64SSE2-NEXT:    movl $7, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $6, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X64SSE2-NEXT:    movl $5, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $4, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm3
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1],xmm3[2],xmm0[2],xmm3[3],xmm0[3],xmm3[4],xmm0[4],xmm3[5],xmm0[5],xmm3[6],xmm0[6],xmm3[7],xmm0[7]
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm3 = xmm3[0],xmm2[0],xmm3[1],xmm2[1],xmm3[2],xmm2[2],xmm3[3],xmm2[3]
-; X64SSE2-NEXT:    movl $3, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $2, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3],xmm2[4],xmm0[4],xmm2[5],xmm0[5],xmm2[6],xmm0[6],xmm2[7],xmm0[7]
-; X64SSE2-NEXT:    movl $1, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm4
-; X64SSE2-NEXT:    movd %edi, %xmm0
-; X64SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3],xmm0[4],xmm4[4],xmm0[5],xmm4[5],xmm0[6],xmm4[6],xmm0[7],xmm4[7]
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1],xmm0[2],xmm2[2],xmm0[3],xmm2[3]
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1]
-; X64SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X64SSE2-NEXT:    movd %edi, %xmm1
+; X64SSE2-NEXT:    movdqa {{.*#+}} xmm0 = [0,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255]
+; X64SSE2-NEXT:    pandn %xmm1, %xmm0
+; X64SSE2-NEXT:    por {{.*}}(%rip), %xmm0
 ; X64SSE2-NEXT:    retq
 ;
 ; X32SSE4-LABEL: elt0_v16i8:
 ; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE4-NEXT:    movl $1, %eax
-; X32SSE4-NEXT:    pinsrb $1, %eax, %xmm0
-; X32SSE4-NEXT:    movl $2, %eax
-; X32SSE4-NEXT:    pinsrb $2, %eax, %xmm0
-; X32SSE4-NEXT:    movl $3, %eax
-; X32SSE4-NEXT:    pinsrb $3, %eax, %xmm0
-; X32SSE4-NEXT:    movl $4, %eax
-; X32SSE4-NEXT:    pinsrb $4, %eax, %xmm0
-; X32SSE4-NEXT:    movl $5, %eax
-; X32SSE4-NEXT:    pinsrb $5, %eax, %xmm0
-; X32SSE4-NEXT:    movl $6, %eax
-; X32SSE4-NEXT:    pinsrb $6, %eax, %xmm0
-; X32SSE4-NEXT:    movl $7, %eax
-; X32SSE4-NEXT:    pinsrb $7, %eax, %xmm0
-; X32SSE4-NEXT:    movl $8, %eax
-; X32SSE4-NEXT:    pinsrb $8, %eax, %xmm0
-; X32SSE4-NEXT:    movl $9, %eax
-; X32SSE4-NEXT:    pinsrb $9, %eax, %xmm0
-; X32SSE4-NEXT:    movl $10, %eax
-; X32SSE4-NEXT:    pinsrb $10, %eax, %xmm0
-; X32SSE4-NEXT:    movl $11, %eax
-; X32SSE4-NEXT:    pinsrb $11, %eax, %xmm0
-; X32SSE4-NEXT:    movl $12, %eax
-; X32SSE4-NEXT:    pinsrb $12, %eax, %xmm0
-; X32SSE4-NEXT:    movl $13, %eax
-; X32SSE4-NEXT:    pinsrb $13, %eax, %xmm0
-; X32SSE4-NEXT:    movl $14, %eax
-; X32SSE4-NEXT:    pinsrb $14, %eax, %xmm0
-; X32SSE4-NEXT:    movl $15, %eax
-; X32SSE4-NEXT:    pinsrb $15, %eax, %xmm0
+; X32SSE4-NEXT:    movdqa {{.*#+}} xmm0 = <u,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15>
+; X32SSE4-NEXT:    pinsrb $0, {{[0-9]+}}(%esp), %xmm0
 ; X32SSE4-NEXT:    retl
 ;
 ; X64SSE4-LABEL: elt0_v16i8:
 ; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movd %edi, %xmm0
-; X64SSE4-NEXT:    movl $1, %eax
-; X64SSE4-NEXT:    pinsrb $1, %eax, %xmm0
-; X64SSE4-NEXT:    movl $2, %eax
-; X64SSE4-NEXT:    pinsrb $2, %eax, %xmm0
-; X64SSE4-NEXT:    movl $3, %eax
-; X64SSE4-NEXT:    pinsrb $3, %eax, %xmm0
-; X64SSE4-NEXT:    movl $4, %eax
-; X64SSE4-NEXT:    pinsrb $4, %eax, %xmm0
-; X64SSE4-NEXT:    movl $5, %eax
-; X64SSE4-NEXT:    pinsrb $5, %eax, %xmm0
-; X64SSE4-NEXT:    movl $6, %eax
-; X64SSE4-NEXT:    pinsrb $6, %eax, %xmm0
-; X64SSE4-NEXT:    movl $7, %eax
-; X64SSE4-NEXT:    pinsrb $7, %eax, %xmm0
-; X64SSE4-NEXT:    movl $8, %eax
-; X64SSE4-NEXT:    pinsrb $8, %eax, %xmm0
-; X64SSE4-NEXT:    movl $9, %eax
-; X64SSE4-NEXT:    pinsrb $9, %eax, %xmm0
-; X64SSE4-NEXT:    movl $10, %eax
-; X64SSE4-NEXT:    pinsrb $10, %eax, %xmm0
-; X64SSE4-NEXT:    movl $11, %eax
-; X64SSE4-NEXT:    pinsrb $11, %eax, %xmm0
-; X64SSE4-NEXT:    movl $12, %eax
-; X64SSE4-NEXT:    pinsrb $12, %eax, %xmm0
-; X64SSE4-NEXT:    movl $13, %eax
-; X64SSE4-NEXT:    pinsrb $13, %eax, %xmm0
-; X64SSE4-NEXT:    movl $14, %eax
-; X64SSE4-NEXT:    pinsrb $14, %eax, %xmm0
-; X64SSE4-NEXT:    movl $15, %eax
-; X64SSE4-NEXT:    pinsrb $15, %eax, %xmm0
+; X64SSE4-NEXT:    movdqa {{.*#+}} xmm0 = <u,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15>
+; X64SSE4-NEXT:    pinsrb $0, %edi, %xmm0
 ; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt0_v16i8:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    vmovd {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32AVX-NEXT:    movl $1, %eax
-; X32AVX-NEXT:    vpinsrb $1, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $2, %eax
-; X32AVX-NEXT:    vpinsrb $2, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $3, %eax
-; X32AVX-NEXT:    vpinsrb $3, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $4, %eax
-; X32AVX-NEXT:    vpinsrb $4, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $5, %eax
-; X32AVX-NEXT:    vpinsrb $5, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $6, %eax
-; X32AVX-NEXT:    vpinsrb $6, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $7, %eax
-; X32AVX-NEXT:    vpinsrb $7, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $8, %eax
-; X32AVX-NEXT:    vpinsrb $8, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $9, %eax
-; X32AVX-NEXT:    vpinsrb $9, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $10, %eax
-; X32AVX-NEXT:    vpinsrb $10, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $11, %eax
-; X32AVX-NEXT:    vpinsrb $11, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $12, %eax
-; X32AVX-NEXT:    vpinsrb $12, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $13, %eax
-; X32AVX-NEXT:    vpinsrb $13, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $14, %eax
-; X32AVX-NEXT:    vpinsrb $14, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $15, %eax
-; X32AVX-NEXT:    vpinsrb $15, %eax, %xmm0, %xmm0
+; X32AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <u,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15>
+; X32AVX-NEXT:    vpinsrb $0, {{[0-9]+}}(%esp), %xmm0, %xmm0
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt0_v16i8:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovd %edi, %xmm0
-; X64AVX-NEXT:    movl $1, %eax
-; X64AVX-NEXT:    vpinsrb $1, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $2, %eax
-; X64AVX-NEXT:    vpinsrb $2, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $3, %eax
-; X64AVX-NEXT:    vpinsrb $3, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $4, %eax
-; X64AVX-NEXT:    vpinsrb $4, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $5, %eax
-; X64AVX-NEXT:    vpinsrb $5, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $6, %eax
-; X64AVX-NEXT:    vpinsrb $6, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $7, %eax
-; X64AVX-NEXT:    vpinsrb $7, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $8, %eax
-; X64AVX-NEXT:    vpinsrb $8, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $9, %eax
-; X64AVX-NEXT:    vpinsrb $9, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $10, %eax
-; X64AVX-NEXT:    vpinsrb $10, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $11, %eax
-; X64AVX-NEXT:    vpinsrb $11, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $12, %eax
-; X64AVX-NEXT:    vpinsrb $12, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $13, %eax
-; X64AVX-NEXT:    vpinsrb $13, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $14, %eax
-; X64AVX-NEXT:    vpinsrb $14, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $15, %eax
-; X64AVX-NEXT:    vpinsrb $15, %eax, %xmm0, %xmm0
+; X64AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <u,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15>
+; X64AVX-NEXT:    vpinsrb $0, %edi, %xmm0, %xmm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <16 x i8> <i8 42, i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 8, i8 9, i8 10, i8 11, i8 12, i8 13, i8 14, i8 15>, i8 %x, i32 0
    ret <16 x i8> %ins
 }
 
 define <8 x i16> @elt5_v8i16(i16 %x) {
-; X32SSE2-LABEL: elt5_v8i16:
-; X32SSE2:       # BB#0:
-; X32SSE2-NEXT:    movl $7, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $6, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3]
-; X32SSE2-NEXT:    movl $4, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3]
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; X32SSE2-NEXT:    movl $3, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movl $2, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3]
-; X32SSE2-NEXT:    movl $1, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm3
-; X32SSE2-NEXT:    movl $42, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3]
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; X32SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; X32SSE2-NEXT:    retl
-;
-; X64SSE2-LABEL: elt5_v8i16:
-; X64SSE2:       # BB#0:
-; X64SSE2-NEXT:    movl $7, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $6, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3]
-; X64SSE2-NEXT:    movd %edi, %xmm0
-; X64SSE2-NEXT:    movl $4, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1],xmm2[2],xmm0[2],xmm2[3],xmm0[3]
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; X64SSE2-NEXT:    movl $3, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $2, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3]
-; X64SSE2-NEXT:    movl $1, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm3
-; X64SSE2-NEXT:    movl $42, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3]
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; X64SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
-; X64SSE2-NEXT:    retq
-;
-; X32SSE4-LABEL: elt5_v8i16:
-; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movl $42, %eax
-; X32SSE4-NEXT:    movd %eax, %xmm0
-; X32SSE4-NEXT:    movl $1, %eax
-; X32SSE4-NEXT:    pinsrw $1, %eax, %xmm0
-; X32SSE4-NEXT:    movl $2, %eax
-; X32SSE4-NEXT:    pinsrw $2, %eax, %xmm0
-; X32SSE4-NEXT:    movl $3, %eax
-; X32SSE4-NEXT:    pinsrw $3, %eax, %xmm0
-; X32SSE4-NEXT:    movl $4, %eax
-; X32SSE4-NEXT:    pinsrw $4, %eax, %xmm0
-; X32SSE4-NEXT:    pinsrw $5, {{[0-9]+}}(%esp), %xmm0
-; X32SSE4-NEXT:    movl $6, %eax
-; X32SSE4-NEXT:    pinsrw $6, %eax, %xmm0
-; X32SSE4-NEXT:    movl $7, %eax
-; X32SSE4-NEXT:    pinsrw $7, %eax, %xmm0
-; X32SSE4-NEXT:    retl
+; X32SSE-LABEL: elt5_v8i16:
+; X32SSE:       # BB#0:
+; X32SSE-NEXT:    movdqa {{.*#+}} xmm0 = <42,1,2,3,4,u,6,7>
+; X32SSE-NEXT:    pinsrw $5, {{[0-9]+}}(%esp), %xmm0
+; X32SSE-NEXT:    retl
 ;
-; X64SSE4-LABEL: elt5_v8i16:
-; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movl $42, %eax
-; X64SSE4-NEXT:    movd %eax, %xmm0
-; X64SSE4-NEXT:    movl $1, %eax
-; X64SSE4-NEXT:    pinsrw $1, %eax, %xmm0
-; X64SSE4-NEXT:    movl $2, %eax
-; X64SSE4-NEXT:    pinsrw $2, %eax, %xmm0
-; X64SSE4-NEXT:    movl $3, %eax
-; X64SSE4-NEXT:    pinsrw $3, %eax, %xmm0
-; X64SSE4-NEXT:    movl $4, %eax
-; X64SSE4-NEXT:    pinsrw $4, %eax, %xmm0
-; X64SSE4-NEXT:    pinsrw $5, %edi, %xmm0
-; X64SSE4-NEXT:    movl $6, %eax
-; X64SSE4-NEXT:    pinsrw $6, %eax, %xmm0
-; X64SSE4-NEXT:    movl $7, %eax
-; X64SSE4-NEXT:    pinsrw $7, %eax, %xmm0
-; X64SSE4-NEXT:    retq
+; X64SSE-LABEL: elt5_v8i16:
+; X64SSE:       # BB#0:
+; X64SSE-NEXT:    movdqa {{.*#+}} xmm0 = <42,1,2,3,4,u,6,7>
+; X64SSE-NEXT:    pinsrw $5, %edi, %xmm0
+; X64SSE-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt5_v8i16:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    movl $42, %eax
-; X32AVX-NEXT:    vmovd %eax, %xmm0
-; X32AVX-NEXT:    movl $1, %eax
-; X32AVX-NEXT:    vpinsrw $1, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $2, %eax
-; X32AVX-NEXT:    vpinsrw $2, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $3, %eax
-; X32AVX-NEXT:    vpinsrw $3, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $4, %eax
-; X32AVX-NEXT:    vpinsrw $4, %eax, %xmm0, %xmm0
+; X32AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <42,1,2,3,4,u,6,7>
 ; X32AVX-NEXT:    vpinsrw $5, {{[0-9]+}}(%esp), %xmm0, %xmm0
-; X32AVX-NEXT:    movl $6, %eax
-; X32AVX-NEXT:    vpinsrw $6, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $7, %eax
-; X32AVX-NEXT:    vpinsrw $7, %eax, %xmm0, %xmm0
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt5_v8i16:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    movl $42, %eax
-; X64AVX-NEXT:    vmovd %eax, %xmm0
-; X64AVX-NEXT:    movl $1, %eax
-; X64AVX-NEXT:    vpinsrw $1, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $2, %eax
-; X64AVX-NEXT:    vpinsrw $2, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $3, %eax
-; X64AVX-NEXT:    vpinsrw $3, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $4, %eax
-; X64AVX-NEXT:    vpinsrw $4, %eax, %xmm0, %xmm0
+; X64AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <42,1,2,3,4,u,6,7>
 ; X64AVX-NEXT:    vpinsrw $5, %edi, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $6, %eax
-; X64AVX-NEXT:    vpinsrw $6, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $7, %eax
-; X64AVX-NEXT:    vpinsrw $7, %eax, %xmm0, %xmm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <8 x i16> <i16 42, i16 1, i16 2, i16 3, i16 4, i16 5, i16 6, i16 7>, i16 %x, i32 5
    ret <8 x i16> %ins
 }
 
 define <4 x i32> @elt3_v4i32(i32 %x) {
 ; X32SSE2-LABEL: elt3_v4i32:
 ; X32SSE2:       # BB#0:
-; X32SSE2-NEXT:    movl $2, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X32SSE2-NEXT:    movl $1, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    movl $42, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; X32SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X32SSE2-NEXT:    movaps {{.*#+}} xmm0 = <42,1,2,u>
+; X32SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,0],xmm0[2,0]
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[2,0]
 ; X32SSE2-NEXT:    retl
 ;
 ; X64SSE2-LABEL: elt3_v4i32:
 ; X64SSE2:       # BB#0:
-; X64SSE2-NEXT:    movd %edi, %xmm0
-; X64SSE2-NEXT:    movl $2, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X64SSE2-NEXT:    movl $1, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    movl $42, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; X64SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X64SSE2-NEXT:    movd %edi, %xmm1
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm0 = <42,1,2,u>
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,0],xmm0[2,0]
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[2,0]
 ; X64SSE2-NEXT:    retq
 ;
 ; X32SSE4-LABEL: elt3_v4i32:
 ; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movl $42, %eax
-; X32SSE4-NEXT:    movd %eax, %xmm0
-; X32SSE4-NEXT:    movl $1, %eax
-; X32SSE4-NEXT:    pinsrd $1, %eax, %xmm0
-; X32SSE4-NEXT:    movl $2, %eax
-; X32SSE4-NEXT:    pinsrd $2, %eax, %xmm0
+; X32SSE4-NEXT:    movdqa {{.*#+}} xmm0 = <42,1,2,u>
 ; X32SSE4-NEXT:    pinsrd $3, {{[0-9]+}}(%esp), %xmm0
 ; X32SSE4-NEXT:    retl
 ;
 ; X64SSE4-LABEL: elt3_v4i32:
 ; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movl $42, %eax
-; X64SSE4-NEXT:    movd %eax, %xmm0
-; X64SSE4-NEXT:    movl $1, %eax
-; X64SSE4-NEXT:    pinsrd $1, %eax, %xmm0
-; X64SSE4-NEXT:    movl $2, %eax
-; X64SSE4-NEXT:    pinsrd $2, %eax, %xmm0
+; X64SSE4-NEXT:    movdqa {{.*#+}} xmm0 = <42,1,2,u>
 ; X64SSE4-NEXT:    pinsrd $3, %edi, %xmm0
 ; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt3_v4i32:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    movl $42, %eax
-; X32AVX-NEXT:    vmovd %eax, %xmm0
-; X32AVX-NEXT:    movl $1, %eax
-; X32AVX-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
-; X32AVX-NEXT:    movl $2, %eax
-; X32AVX-NEXT:    vpinsrd $2, %eax, %xmm0, %xmm0
+; X32AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <42,1,2,u>
 ; X32AVX-NEXT:    vpinsrd $3, {{[0-9]+}}(%esp), %xmm0, %xmm0
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt3_v4i32:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    movl $42, %eax
-; X64AVX-NEXT:    vmovd %eax, %xmm0
-; X64AVX-NEXT:    movl $1, %eax
-; X64AVX-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
-; X64AVX-NEXT:    movl $2, %eax
-; X64AVX-NEXT:    vpinsrd $2, %eax, %xmm0, %xmm0
+; X64AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <42,1,2,u>
 ; X64AVX-NEXT:    vpinsrd $3, %edi, %xmm0, %xmm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <4 x i32> <i32 42, i32 1, i32 2, i32 3>, i32 %x, i32 3
    ret <4 x i32> %ins
 }
 
 define <2 x i64> @elt0_v2i64(i64 %x) {
 ; X32SSE-LABEL: elt0_v2i64:
 ; X32SSE:       # BB#0:
 ; X32SSE-NEXT:    movl $1, %eax
 ; X32SSE-NEXT:    movd %eax, %xmm1
 ; X32SSE-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; X32SSE-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; X32SSE-NEXT:    retl
 ;
-; X64SSE-LABEL: elt0_v2i64:
-; X64SSE:       # BB#0:
-; X64SSE-NEXT:    movq %rdi, %xmm0
-; X64SSE-NEXT:    movl $1, %eax
-; X64SSE-NEXT:    movq %rax, %xmm1
-; X64SSE-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
-; X64SSE-NEXT:    retq
+; X64SSE2-LABEL: elt0_v2i64:
+; X64SSE2:       # BB#0:
+; X64SSE2-NEXT:    movq %rdi, %xmm1
+; X64SSE2-NEXT:    movapd {{.*#+}} xmm0 = <u,1>
+; X64SSE2-NEXT:    movsd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
+; X64SSE2-NEXT:    retq
+;
+; X64SSE4-LABEL: elt0_v2i64:
+; X64SSE4:       # BB#0:
+; X64SSE4-NEXT:    movdqa {{.*#+}} xmm0 = <u,1>
+; X64SSE4-NEXT:    pinsrq $0, %rdi, %xmm0
+; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt0_v2i64:
 ; X32AVX:       # BB#0:
 ; X32AVX-NEXT:    movl $1, %eax
 ; X32AVX-NEXT:    vmovd %eax, %xmm0
 ; X32AVX-NEXT:    vmovq {{.*#+}} xmm1 = mem[0],zero
 ; X32AVX-NEXT:    vpunpcklqdq {{.*#+}} xmm0 = xmm1[0],xmm0[0]
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt0_v2i64:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovq %rdi, %xmm0
-; X64AVX-NEXT:    movl $1, %eax
-; X64AVX-NEXT:    vmovq %rax, %xmm1
-; X64AVX-NEXT:    vpunpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X64AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = <u,1>
+; X64AVX-NEXT:    vpinsrq $0, %rdi, %xmm0, %xmm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <2 x i64> <i64 42, i64 1>, i64 %x, i32 0
    ret <2 x i64> %ins
 }
 
 define <4 x float> @elt1_v4f32(float %x) {
 ; X32SSE2-LABEL: elt1_v4f32:
 ; X32SSE2:       # BB#0:
 ; X32SSE2-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X32SSE2-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; X32SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; X32SSE2-NEXT:    movaps {{.*#+}} xmm1 = <42,u,2,3>
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[0,0]
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[2,0],xmm1[2,3]
 ; X32SSE2-NEXT:    retl
 ;
 ; X64SSE2-LABEL: elt1_v4f32:
 ; X64SSE2:       # BB#0:
-; X64SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; X64SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X64SSE2-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; X64SSE2-NEXT:    movaps %xmm1, %xmm0
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm1 = <42,u,2,3>
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[0,0]
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[2,0],xmm1[2,3]
 ; X64SSE2-NEXT:    retq
 ;
 ; X32SSE4-LABEL: elt1_v4f32:
 ; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; X32SSE4-NEXT:    movaps {{.*#+}} xmm0 = <42,u,2,3>
 ; X32SSE4-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
-; X32SSE4-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; X32SSE4-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
 ; X32SSE4-NEXT:    retl
 ;
 ; X64SSE4-LABEL: elt1_v4f32:
 ; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; X64SSE4-NEXT:    movaps {{.*#+}} xmm1 = <42,u,2,3>
 ; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[2,3]
-; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
-; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
 ; X64SSE4-NEXT:    movaps %xmm1, %xmm0
 ; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt1_v4f32:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; X32AVX-NEXT:    vmovaps {{.*#+}} xmm0 = <42,u,2,3>
 ; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt1_v4f32:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; X64AVX-NEXT:    vmovaps {{.*#+}} xmm1 = <42,u,2,3>
 ; X64AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[2,3]
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
 ; X64AVX-NEXT:    retq
    %ins = insertelement <4 x float> <float 42.0, float 1.0, float 2.0, float 3.0>, float %x, i32 1
    ret <4 x float> %ins
 }
 
 define <2 x double> @elt1_v2f64(double %x) {
 ; X32SSE-LABEL: elt1_v2f64:
 ; X32SSE:       # BB#0:
-; X32SSE-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; X32SSE-NEXT:    movapd {{.*#+}} xmm0 = <42,u>
 ; X32SSE-NEXT:    movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
 ; X32SSE-NEXT:    retl
 ;
 ; X64SSE-LABEL: elt1_v2f64:
 ; X64SSE:       # BB#0:
-; X64SSE-NEXT:    movsd {{.*#+}} xmm1 = mem[0],zero
+; X64SSE-NEXT:    movaps {{.*#+}} xmm1 = <42,u>
 ; X64SSE-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; X64SSE-NEXT:    movaps %xmm1, %xmm0
 ; X64SSE-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt1_v2f64:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; X32AVX-NEXT:    vmovapd {{.*#+}} xmm0 = <42,u>
 ; X32AVX-NEXT:    vmovhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt1_v2f64:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
+; X64AVX-NEXT:    vmovaps {{.*#+}} xmm1 = <42,u>
 ; X64AVX-NEXT:    vmovlhps {{.*#+}} xmm0 = xmm1[0],xmm0[0]
 ; X64AVX-NEXT:    retq
    %ins = insertelement <2 x double> <double 42.0, double 1.0>, double %x, i32 1
    ret <2 x double> %ins
 }
 
 define <8 x i32> @elt7_v8i32(i32 %x) {
 ; X32SSE2-LABEL: elt7_v8i32:
 ; X32SSE2:       # BB#0:
-; X32SSE2-NEXT:    movl $6, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm0
-; X32SSE2-NEXT:    movd {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; X32SSE2-NEXT:    movl $5, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm2
-; X32SSE2-NEXT:    movl $4, %eax
-; X32SSE2-NEXT:    movd %eax, %xmm1
-; X32SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
-; X32SSE2-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]
+; X32SSE2-NEXT:    movaps {{.*#+}} xmm1 = <4,5,6,u>
+; X32SSE2-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[2,0]
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,0]
 ; X32SSE2-NEXT:    movaps {{.*#+}} xmm0 = [42,1,2,3]
 ; X32SSE2-NEXT:    retl
 ;
 ; X64SSE2-LABEL: elt7_v8i32:
 ; X64SSE2:       # BB#0:
 ; X64SSE2-NEXT:    movd %edi, %xmm0
-; X64SSE2-NEXT:    movl $6, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm2
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
-; X64SSE2-NEXT:    movl $5, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm0
-; X64SSE2-NEXT:    movl $4, %eax
-; X64SSE2-NEXT:    movd %eax, %xmm1
-; X64SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X64SSE2-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm1 = <4,5,6,u>
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[2,0]
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,0]
 ; X64SSE2-NEXT:    movaps {{.*#+}} xmm0 = [42,1,2,3]
 ; X64SSE2-NEXT:    retq
 ;
 ; X32SSE4-LABEL: elt7_v8i32:
 ; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movl $4, %eax
-; X32SSE4-NEXT:    movd %eax, %xmm1
-; X32SSE4-NEXT:    movl $5, %eax
-; X32SSE4-NEXT:    pinsrd $1, %eax, %xmm1
-; X32SSE4-NEXT:    movl $6, %eax
-; X32SSE4-NEXT:    pinsrd $2, %eax, %xmm1
+; X32SSE4-NEXT:    movdqa {{.*#+}} xmm1 = <4,5,6,u>
 ; X32SSE4-NEXT:    pinsrd $3, {{[0-9]+}}(%esp), %xmm1
 ; X32SSE4-NEXT:    movaps {{.*#+}} xmm0 = [42,1,2,3]
 ; X32SSE4-NEXT:    retl
 ;
 ; X64SSE4-LABEL: elt7_v8i32:
 ; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movl $4, %eax
-; X64SSE4-NEXT:    movd %eax, %xmm1
-; X64SSE4-NEXT:    movl $5, %eax
-; X64SSE4-NEXT:    pinsrd $1, %eax, %xmm1
-; X64SSE4-NEXT:    movl $6, %eax
-; X64SSE4-NEXT:    pinsrd $2, %eax, %xmm1
+; X64SSE4-NEXT:    movdqa {{.*#+}} xmm1 = <4,5,6,u>
 ; X64SSE4-NEXT:    pinsrd $3, %edi, %xmm1
 ; X64SSE4-NEXT:    movaps {{.*#+}} xmm0 = [42,1,2,3]
 ; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt7_v8i32:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = [42,1,2,3]
-; X32AVX-NEXT:    movl $4, %eax
-; X32AVX-NEXT:    vmovd %eax, %xmm1
-; X32AVX-NEXT:    movl $5, %eax
-; X32AVX-NEXT:    vpinsrd $1, %eax, %xmm1, %xmm1
-; X32AVX-NEXT:    movl $6, %eax
-; X32AVX-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
+; X32AVX-NEXT:    vmovdqa {{.*#+}} ymm0 = <42,1,2,3,4,5,6,u>
+; X32AVX-NEXT:    vextracti128 $1, %ymm0, %xmm1
 ; X32AVX-NEXT:    vpinsrd $3, {{[0-9]+}}(%esp), %xmm1, %xmm1
 ; X32AVX-NEXT:    vinserti128 $1, %xmm1, %ymm0, %ymm0
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt7_v8i32:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovdqa {{.*#+}} xmm0 = [42,1,2,3]
-; X64AVX-NEXT:    movl $4, %eax
-; X64AVX-NEXT:    vmovd %eax, %xmm1
-; X64AVX-NEXT:    movl $5, %eax
-; X64AVX-NEXT:    vpinsrd $1, %eax, %xmm1, %xmm1
-; X64AVX-NEXT:    movl $6, %eax
-; X64AVX-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
+; X64AVX-NEXT:    vmovdqa {{.*#+}} ymm0 = <42,1,2,3,4,5,6,u>
+; X64AVX-NEXT:    vextracti128 $1, %ymm0, %xmm1
 ; X64AVX-NEXT:    vpinsrd $3, %edi, %xmm1, %xmm1
 ; X64AVX-NEXT:    vinserti128 $1, %xmm1, %ymm0, %ymm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <8 x i32> <i32 42, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7>, i32 %x, i32 7
    ret <8 x i32> %ins
 }
 
 define <8 x float> @elt6_v8f32(float %x) {
 ; X32SSE2-LABEL: elt6_v8f32:
 ; X32SSE2:       # BB#0:
 ; X32SSE2-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm0[0],xmm2[1],xmm0[1]
-; X32SSE2-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X32SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; X32SSE2-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm2[0]
+; X32SSE2-NEXT:    movaps {{.*#+}} xmm1 = <4,5,u,7>
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[3,0]
+; X32SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[0,2]
 ; X32SSE2-NEXT:    movaps {{.*#+}} xmm0 = [4.200000e+01,1.000000e+00,2.000000e+00,3.000000e+00]
 ; X32SSE2-NEXT:    retl
 ;
 ; X64SSE2-LABEL: elt6_v8f32:
 ; X64SSE2:       # BB#0:
-; X64SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; X64SSE2-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64SSE2-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
-; X64SSE2-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm0[0]
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm1 = <4,5,u,7>
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[3,0]
+; X64SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[0,2]
 ; X64SSE2-NEXT:    movaps {{.*#+}} xmm0 = [4.200000e+01,1.000000e+00,2.000000e+00,3.000000e+00]
 ; X64SSE2-NEXT:    retq
 ;
 ; X32SSE4-LABEL: elt6_v8f32:
 ; X32SSE4:       # BB#0:
-; X32SSE4-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X32SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
+; X32SSE4-NEXT:    movaps {{.*#+}} xmm1 = <4,5,u,7>
 ; X32SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
-; X32SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
 ; X32SSE4-NEXT:    movaps {{.*#+}} xmm0 = [4.200000e+01,1.000000e+00,2.000000e+00,3.000000e+00]
 ; X32SSE4-NEXT:    retl
 ;
 ; X64SSE4-LABEL: elt6_v8f32:
 ; X64SSE4:       # BB#0:
-; X64SSE4-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
+; X64SSE4-NEXT:    movaps {{.*#+}} xmm1 = <4,5,u,7>
 ; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1],xmm0[0],xmm1[3]
-; X64SSE4-NEXT:    insertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
 ; X64SSE4-NEXT:    movaps {{.*#+}} xmm0 = [4.200000e+01,1.000000e+00,2.000000e+00,3.000000e+00]
 ; X64SSE4-NEXT:    retq
 ;
 ; X32AVX-LABEL: elt6_v8f32:
 ; X32AVX:       # BB#0:
-; X32AVX-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
-; X32AVX-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
+; X32AVX-NEXT:    vmovaps {{.*#+}} ymm0 = <42,1,2,3,4,5,u,7>
+; X32AVX-NEXT:    vextractf128 $1, %ymm0, %xmm1
 ; X32AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
-; X32AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
 ; X32AVX-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
 ; X32AVX-NEXT:    retl
 ;
 ; X64AVX-LABEL: elt6_v8f32:
 ; X64AVX:       # BB#0:
-; X64AVX-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
-; X64AVX-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0],mem[0],xmm2[2,3]
+; X64AVX-NEXT:    vmovaps {{.*#+}} ymm1 = <42,1,2,3,4,5,u,7>
+; X64AVX-NEXT:    vextractf128 $1, %ymm1, %xmm2
 ; X64AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm2[0,1],xmm0[0],xmm2[3]
-; X64AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
 ; X64AVX-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
 ; X64AVX-NEXT:    retq
    %ins = insertelement <8 x float> <float 42.0, float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0>, float %x, i32 6
    ret <8 x float> %ins
 }
 
 define <8 x i64> @elt5_v8i64(i64 %x) {
 ; X32SSE-LABEL: elt5_v8i64:
 ; X32SSE:       # BB#0:
 ; X32SSE-NEXT:    movl $4, %eax
 ; X32SSE-NEXT:    movd %eax, %xmm2
 ; X32SSE-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; X32SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm0[0]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm0 = [42,0,1,0]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm1 = [2,0,3,0]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm3 = [6,0,7,0]
 ; X32SSE-NEXT:    retl
 ;
-; X64SSE-LABEL: elt5_v8i64:
-; X64SSE:       # BB#0:
-; X64SSE-NEXT:    movq %rdi, %xmm0
-; X64SSE-NEXT:    movl $4, %eax
-; X64SSE-NEXT:    movq %rax, %xmm2
-; X64SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm0[0]
-; X64SSE-NEXT:    movaps {{.*#+}} xmm0 = [42,1]
-; X64SSE-NEXT:    movaps {{.*#+}} xmm1 = [2,3]
-; X64SSE-NEXT:    movaps {{.*#+}} xmm3 = [6,7]
-; X64SSE-NEXT:    retq
+; X64SSE2-LABEL: elt5_v8i64:
+; X64SSE2:       # BB#0:
+; X64SSE2-NEXT:    movq %rdi, %xmm0
+; X64SSE2-NEXT:    movdqa {{.*#+}} xmm2 = <4,u>
+; X64SSE2-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm0[0]
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm0 = [42,1]
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm1 = [2,3]
+; X64SSE2-NEXT:    movaps {{.*#+}} xmm3 = [6,7]
+; X64SSE2-NEXT:    retq
+;
+; X64SSE4-LABEL: elt5_v8i64:
+; X64SSE4:       # BB#0:
+; X64SSE4-NEXT:    movdqa {{.*#+}} xmm2 = <4,u>
+; X64SSE4-NEXT:    pinsrq $1, %rdi, %xmm2
+; X64SSE4-NEXT:    movaps {{.*#+}} xmm0 = [42,1]
+; X64SSE4-NEXT:    movaps {{.*#+}} xmm1 = [2,3]
+; X64SSE4-NEXT:    movaps {{.*#+}} xmm3 = [6,7]
+; X64SSE4-NEXT:    retq
 ;
 ; X32AVX2-LABEL: elt5_v8i64:
 ; X32AVX2:       # BB#0:
 ; X32AVX2-NEXT:    movl $4, %eax
 ; X32AVX2-NEXT:    vmovd %eax, %xmm0
 ; X32AVX2-NEXT:    vmovq {{.*#+}} xmm1 = mem[0],zero
 ; X32AVX2-NEXT:    vpunpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; X32AVX2-NEXT:    vinserti128 $1, {{\.LCPI.*}}, %ymm0, %ymm1
 ; X32AVX2-NEXT:    vmovaps {{.*#+}} ymm0 = [42,0,1,0,2,0,3,0]
 ; X32AVX2-NEXT:    retl
 ;
 ; X64AVX2-LABEL: elt5_v8i64:
 ; X64AVX2:       # BB#0:
-; X64AVX2-NEXT:    vmovq %rdi, %xmm0
-; X64AVX2-NEXT:    movl $4, %eax
-; X64AVX2-NEXT:    vmovq %rax, %xmm1
-; X64AVX2-NEXT:    vpunpcklqdq {{.*#+}} xmm0 = xmm1[0],xmm0[0]
-; X64AVX2-NEXT:    vinserti128 $1, {{.*}}(%rip), %ymm0, %ymm1
+; X64AVX2-NEXT:    vmovdqa {{.*#+}} ymm0 = <4,u,6,7>
+; X64AVX2-NEXT:    vpinsrq $1, %rdi, %xmm0, %xmm1
+; X64AVX2-NEXT:    vpblendd {{.*#+}} ymm1 = ymm1[0,1,2,3],ymm0[4,5,6,7]
 ; X64AVX2-NEXT:    vmovaps {{.*#+}} ymm0 = [42,1,2,3]
 ; X64AVX2-NEXT:    retq
 ;
 ; X32AVX512F-LABEL: elt5_v8i64:
 ; X32AVX512F:       # BB#0:
 ; X32AVX512F-NEXT:    vmovdqa {{.*#+}} ymm0 = [42,0,1,0,2,0,3,0]
 ; X32AVX512F-NEXT:    movl $4, %eax
 ; X32AVX512F-NEXT:    vmovd %eax, %xmm1
 ; X32AVX512F-NEXT:    vmovq {{.*#+}} xmm2 = mem[0],zero
 ; X32AVX512F-NEXT:    vpunpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
 ; X32AVX512F-NEXT:    vinserti128 $1, {{\.LCPI.*}}, %ymm1, %ymm1
 ; X32AVX512F-NEXT:    vinserti64x4 $1, %ymm1, %zmm0, %zmm0
 ; X32AVX512F-NEXT:    retl
 ;
 ; X64AVX512F-LABEL: elt5_v8i64:
 ; X64AVX512F:       # BB#0:
-; X64AVX512F-NEXT:    vmovdqa {{.*#+}} ymm0 = [42,1,2,3]
-; X64AVX512F-NEXT:    vmovq %rdi, %xmm1
-; X64AVX512F-NEXT:    movl $4, %eax
-; X64AVX512F-NEXT:    vmovq %rax, %xmm2
-; X64AVX512F-NEXT:    vpunpcklqdq {{.*#+}} xmm1 = xmm2[0],xmm1[0]
-; X64AVX512F-NEXT:    vinserti128 $1, {{.*}}(%rip), %ymm1, %ymm1
-; X64AVX512F-NEXT:    vinserti64x4 $1, %ymm1, %zmm0, %zmm0
+; X64AVX512F-NEXT:    vmovdqa64 {{.*#+}} zmm0 = <42,1,2,3,4,u,6,7>
+; X64AVX512F-NEXT:    vextracti32x4 $2, %zmm0, %xmm1
+; X64AVX512F-NEXT:    vpinsrq $1, %rdi, %xmm1, %xmm1
+; X64AVX512F-NEXT:    vinserti32x4 $2, %xmm1, %zmm0, %zmm0
 ; X64AVX512F-NEXT:    retq
    %ins = insertelement <8 x i64> <i64 42, i64 1, i64 2, i64 3, i64 4, i64 5, i64 6, i64 7>, i64 %x, i32 5
    ret <8 x i64> %ins
 }
 
 define <8 x double> @elt1_v8f64(double %x) {
 ; X32SSE-LABEL: elt1_v8f64:
 ; X32SSE:       # BB#0:
-; X32SSE-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; X32SSE-NEXT:    movapd {{.*#+}} xmm0 = <42,u>
 ; X32SSE-NEXT:    movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm1 = [2.000000e+00,3.000000e+00]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm2 = [4.000000e+00,5.000000e+00]
 ; X32SSE-NEXT:    movaps {{.*#+}} xmm3 = [6.000000e+00,7.000000e+00]
 ; X32SSE-NEXT:    retl
 ;
 ; X64SSE-LABEL: elt1_v8f64:
 ; X64SSE:       # BB#0:
-; X64SSE-NEXT:    movsd {{.*#+}} xmm4 = mem[0],zero
+; X64SSE-NEXT:    movaps {{.*#+}} xmm4 = <42,u>
 ; X64SSE-NEXT:    movlhps {{.*#+}} xmm4 = xmm4[0],xmm0[0]
 ; X64SSE-NEXT:    movaps {{.*#+}} xmm1 = [2.000000e+00,3.000000e+00]
 ; X64SSE-NEXT:    movaps {{.*#+}} xmm2 = [4.000000e+00,5.000000e+00]
 ; X64SSE-NEXT:    movaps {{.*#+}} xmm3 = [6.000000e+00,7.000000e+00]
 ; X64SSE-NEXT:    movaps %xmm4, %xmm0
 ; X64SSE-NEXT:    retq
 ;
 ; X32AVX2-LABEL: elt1_v8f64:
 ; X32AVX2:       # BB#0:
-; X32AVX2-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; X32AVX2-NEXT:    vmovhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
-; X32AVX2-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; X32AVX2-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
-; X32AVX2-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; X32AVX2-NEXT:    vmovapd {{.*#+}} ymm0 = <42,u,2,3>
+; X32AVX2-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm0[0],mem[0]
+; X32AVX2-NEXT:    vblendpd {{.*#+}} ymm0 = ymm1[0,1],ymm0[2,3]
 ; X32AVX2-NEXT:    vmovaps {{.*#+}} ymm1 = [4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00]
 ; X32AVX2-NEXT:    retl
 ;
 ; X64AVX2-LABEL: elt1_v8f64:
 ; X64AVX2:       # BB#0:
-; X64AVX2-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
+; X64AVX2-NEXT:    vmovapd {{.*#+}} ymm1 = <42,u,2,3>
 ; X64AVX2-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm1[0],xmm0[0]
-; X64AVX2-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; X64AVX2-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
-; X64AVX2-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; X64AVX2-NEXT:    vblendpd {{.*#+}} ymm0 = ymm0[0,1],ymm1[2,3]
 ; X64AVX2-NEXT:    vmovaps {{.*#+}} ymm1 = [4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00]
 ; X64AVX2-NEXT:    retq
 ;
 ; X32AVX512F-LABEL: elt1_v8f64:
 ; X32AVX512F:       # BB#0:
-; X32AVX512F-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; X32AVX512F-NEXT:    vmovhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
-; X32AVX512F-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; X32AVX512F-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
-; X32AVX512F-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
-; X32AVX512F-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; X32AVX512F-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
-; X32AVX512F-NEXT:    vmovsd {{.*#+}} xmm2 = mem[0],zero
-; X32AVX512F-NEXT:    vmovhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
-; X32AVX512F-NEXT:    vinsertf128 $1, %xmm2, %ymm1, %ymm1
-; X32AVX512F-NEXT:    vinsertf64x4 $1, %ymm0, %zmm1, %zmm0
+; X32AVX512F-NEXT:    vmovapd {{.*#+}} zmm0 = <42,u,2,3,4,5,6,7>
+; X32AVX512F-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm0[0],mem[0]
+; X32AVX512F-NEXT:    vinsertf32x4 $0, %xmm1, %zmm0, %zmm0
 ; X32AVX512F-NEXT:    retl
 ;
 ; X64AVX512F-LABEL: elt1_v8f64:
 ; X64AVX512F:       # BB#0:
-; X64AVX512F-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; X64AVX512F-NEXT:    vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
-; X64AVX512F-NEXT:    vmovsd {{.*#+}} xmm2 = mem[0],zero
-; X64AVX512F-NEXT:    vmovhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
-; X64AVX512F-NEXT:    vinsertf128 $1, %xmm2, %ymm1, %ymm1
-; X64AVX512F-NEXT:    vmovsd {{.*#+}} xmm2 = mem[0],zero
-; X64AVX512F-NEXT:    vunpcklpd {{.*#+}} xmm0 = xmm2[0],xmm0[0]
-; X64AVX512F-NEXT:    vmovsd {{.*#+}} xmm2 = mem[0],zero
-; X64AVX512F-NEXT:    vmovhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
-; X64AVX512F-NEXT:    vinsertf128 $1, %xmm2, %ymm0, %ymm0
-; X64AVX512F-NEXT:    vinsertf64x4 $1, %ymm1, %zmm0, %zmm0
+; X64AVX512F-NEXT:    vmovaps {{.*#+}} zmm1 = <42,u,2,3,4,5,6,7>
+; X64AVX512F-NEXT:    vmovlhps {{.*#+}} xmm0 = xmm1[0],xmm0[0]
+; X64AVX512F-NEXT:    vinsertf32x4 $0, %xmm0, %zmm1, %zmm0
 ; X64AVX512F-NEXT:    retq
    %ins = insertelement <8 x double> <double 42.0, double 1.0, double 2.0, double 3.0, double 4.0, double 5.0, double 6.0, double 7.0>, double %x, i32 1
    ret <8 x double> %ins
 }