[DAGCombine] Preserve shuffles when one of the vector operands is constant

[zvi]

Summary

Do *not* perform combines such as:

vector_shuffle<4,1,2,3>(build_vector(Ud, C0, C1 C2), scalar_to_vector(X))
->
build_vector(X, C0, C1, C2)

Keeping the shuffle allows lowering the constant build_vector to a materialized
constant vector (such as a vector-load from the constant-pool or some other idiom).

[RKSimon]

My concern is whether we should be adding more shuffle specific code to the build vector lowering or whether we should be focussing on having only basic support and merging as much as possible into target shuffle combines.

[mkuper]

I tend to agree with Simon.

Perhaps extend the build_vector -> vector_shuffle code to catch this case (right now it only performs the combine if the constants are zero), and then make sure x86 lowers the shuffle to a blend?

[zvi]

In D25524#568764, @mkuper wrote:

I tend to agree with Simon.

Perhaps extend the build_vector -> vector_shuffle code to catch this case (right now it only performs the combine if the constants are zero), and then make sure x86 lowers the shuffle to a blend?

Will improve patch to follow your suggestion. Thanks Simon and Michael!

[zvi]

This is an improvement over the previous revision, but still not there yet.

Following Simon and Michael's suggestion, this is what i want to achieve:

build_vector(X, C0, C1, C2)
->
vector_shuffle<4,1,2,3>(
  build_vector(Undef, C0, C1, C2),
  scalar_to_vector(X)
)

The new build_vector should be expanded to a constant pool vector-load, and the vector_shuffle should be lowered as a BLEND in shuffle lowering.
The problem i am facing with the above approach is that the constants in the above pattern (C0, C1, C2) are visited by the legalizer before the new build_vector node is visited, and as a result each constant is legalized as a separate constant-pool load which prevents the expansion of the build_vector to a constant-pool load of a vector-load.
In this revision, the constant-pool vector-load is still being created 'manually'.

Would appreciate suggestions on how to avoid the 'manual' creation of the constant-pool load, and, of course, any other comments.

Thanks, Zvi

[mkuper]

You may want to make sure you hit this pre-legalization, then. :-)

Also, what I meant was extending the existing build_vector -> vector_shuffle combine, not creating an x86-specific combine to do this. (It would be nice to be able to hit the case where X is an extract by blending with the original vector.)

If you're worried about that having an unexpected effect on other targets (because they don't have the appropriate blends or can't match them) I think a TTI hook would be better than an x86-specific combine.

[RKSimon]

As @mkuper suggested, we should be able to do more of this in DAGCombiner::reduceBuildVecToShuffle (there is a TODO to add better constant support).

Another area would be to investigate improving isShuffleEquivalent to recognise if a one-use BUILD_VECTOR (not sure if it just needs to be constants) to be regenerated so that it could correctly match with the expected mask - at the moment this barely works for zero vectors only. General constant support in lowerVectorShuffleAsBlend wouldn't go amiss either.

With those in place, LowerBuildVectorAsBLEND might not be needed or could be refactored to generate any subtarget shuffle (not just SSE41 style blends).

[zvi]

In D25524#571683, @mkuper wrote:

You may want to make sure you hit this pre-legalization, then. :-)

Also, what I meant was extending the existing build_vector -> vector_shuffle combine, not creating an x86-specific combine to do this. (It would be nice to be able to hit the case where X is an extract by blending with the original vector.)

If you're worried about that having an unexpected effect on other targets (because they don't have the appropriate blends or can't match them) I think a TTI hook would be better than an x86-specific combine.

Sorry i misunderstood you about doing this as a target-independent combine.
I actually did consider doing a X86-specific combine, but i got discouraged after seeing that DAGCombiner::visitVECTOR_SHUFFLE will perform the reverse-transformation of what i have in mind (see code starting with comment: "// Attempt to combine a shuffle of 2 inputs of 'scalar sources'"). I will need to work out these supposed conflicting transformations.

[zvi]

In D25524#571777, @RKSimon wrote:

As @mkuper suggested, we should be able to do more of this in DAGCombiner::reduceBuildVecToShuffle (there is a TODO to add better constant support).

Another area would be to investigate improving isShuffleEquivalent to recognise if a one-use BUILD_VECTOR (not sure if it just needs to be constants) to be regenerated so that it could correctly match with the expected mask - at the moment this barely works for zero vectors only. General constant support in lowerVectorShuffleAsBlend wouldn't go amiss either.

With those in place, LowerBuildVectorAsBLEND might not be needed or could be refactored to generate any subtarget shuffle (not just SSE41 style blends).

Thanks for the suggestion. Will definitely look into it.

[andreadb]

In D25524#571943, @zvi wrote:

In D25524#571683, @mkuper wrote:

You may want to make sure you hit this pre-legalization, then. :-)

Also, what I meant was extending the existing build_vector -> vector_shuffle combine, not creating an x86-specific combine to do this. (It would be nice to be able to hit the case where X is an extract by blending with the original vector.)

If you're worried about that having an unexpected effect on other targets (because they don't have the appropriate blends or can't match them) I think a TTI hook would be better than an x86-specific combine.

Sorry i misunderstood you about doing this as a target-independent combine.
I actually did consider doing a X86-specific combine, but i got discouraged after seeing that DAGCombiner::visitVECTOR_SHUFFLE will perform the reverse-transformation of what i have in mind (see code starting with comment: "// Attempt to combine a shuffle of 2 inputs of 'scalar sources'"). I will need to work out these supposed conflicting transformations.

That logic was introduced at revision 234004 (http://llvm.org/viewvc/llvm-project?view=revision&revision=234004).
I didn't look carefully at the code, however it looks like we need a special case for when the build_vector in input to the shuffle is a build_vector of all constants.

As a side note: I noticed that all the emails generated by phabricator are not goint to llvm-commits. I guess it was not added as "subscriber".

[zvi]

Updated the patch to a target-independent combine following @andreadb's suggestion.

[andreadb]

Hi Zvi,

Nice patch.
I have a couple of suggestions to improve it (see comment below).

Thanks
-Andrea

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13870–13876 ↗

(On Diff #75016)

One thing I noticed is that we don't perform this transformation if any of the inputs to the shuffle has more than one use. We can check if either N0 and N1 has more than one use, and bail out earlier before we even get to the loop. For simplicity, you may want to move the computation to a helper function (it is up to you).

If we reach this code, then the shuffle has been already canonicalized with rules defined at lines 13754:13788. That means, if one of the operands is 'undef', then this shuffle is effectively a permute (or a splat), and the undef operand is always moved to the "rhs".

By construction, if we have an undef, that undef vector can only be N1. Also, (still by construction), no shuffle indices can reference the undef vector. As a consequence, if one of the operands is Undef, then you don't need to check if N0 is a "suitable" constant. Note that you cannot have both operands undef (that case would be canonicalized before we even reach this code).

If N1 is not undef, then you can check in advance if either N0 or N1 is "unsuitable". If at least one of the operands is not a constant vector (i.e. BothN0N1Const is false), and the other operand is "unsuitable", then we should not combine the shuffle.

The bottom line is: I think that you can probably hoist the 'hasOneUse' checks outside of the loop. You can also avoid to check if a operand is suitable and hoist that check outside of the loop. If we get into the loop, then we know that it is "safe" to attempt to fold the computation. That would let you remove the 'IsConstNotAllZeroes' smallvector.

[zvi]

Andrea, that's a very good analysis. Will move the conditions out of the loop to early-exits. Thanks!

[zvi]

• Refactored the combine to a helper furnction
• Hoisted checks on vector contents to outside of loops. Checks in loop are only on mask indices.

[andreadb]

Hi Zvi,

see my inline comments.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13764 ↗

(On Diff #75083)

Anywhere else in this file we return an SDValue() when the combine is not successful. It would be nicer if you do the same in this function. So, I suggest that you replace all the occurrences of return {} with return SDValue().

13767–13773 ↗

(On Diff #75083)

I don't think that this check is correct. For example, one of the operands in input can be a SCALAR_TO_VECTOR. The extra checks for the opcode should not be there.

Basically, I was expecting something like this:

if (N0AnyConst && !N1AnyConst && !ISD::isBuildVectorAllZeros(N0.getNode()))
  return SDValue();
if (!N0AnyConst && N1AnyConst && !ISD::isBuildVectorAllZeros(N1.getNode()))
  return SDValue();

13792–13793 ↗

(On Diff #75083)

This can never happen because we return at line 13787 if one of the operands cannot be combined.
This if-stmt should be removed.

[andreadb]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13765–13773 ↗

(On Diff #75083)

If N1 is Undef, then we should never early exit. As I wrote in one of my previous comments, by construction N0 can't be Undef; only N1 can be Undef.

Therefore, you can skip the computation of N0AnyConst and N1AnyConst if N1 is known to be Undef. What I am suggesting is to change that code into something like this:

if (!N1.isUndef()) {
  bool N0AnyConst = isAnyConstantBuildVector(N0.getNode());
  bool N1AnyConst = isAnyConstantBuildVector(N1.getNode());
  if (N0AnyConst && !N1AnyConst && !ISD::isBuildVectorAllZeros(N0.getNode()))
    return SDValue();
  if (!N0AnyConst && N1AnyConst && !ISD::isBuildVectorAllZeros(N1.getNode()))
    return SDValue();
}

The extra check for N->isUndef() in isAnyConstantBuildVector would become redundant if your code checks in advance if N1 is Undef.

[mkuper]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13737 ↗

(On Diff #75083)

Any chance we can put this in a place that makes more sense?
See discussion on D25685.

[zvi]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13737 ↗

(On Diff #75083)

Sure

13764 ↗

(On Diff #75083)

I must have done this subconsciously. Would never have done it in a sound mind :)

13765–13773 ↗

(On Diff #75083)

I humbly accept your improvement.

The extra check for N->isUndef() in isAnyConstantBuildVector would become redundant if your code checks in advance if N1 is Undef.

Not sure if you are suggesting to change

bool N1AnyConst = isAnyConstantBuildVector(N1.getNode());

to

bool N1AnyConst = ISD::isBuildVectorOfConstantSDNodes(N) ||
        ISD::isBuildVectorOfConstantFPSDNodes(N);

or that is just a side note.

I prefer the former for brevity.

13792–13793 ↗

(On Diff #75083)

Will remove. Thanks.

[zvi]

• Using @andreadb's improved early-exit conditions
• Moved isAnyConstantBuildVector to a more suitable location
• Just to emphasize, combineShuffleOfScalars still needs to be moved to a better location.

[andreadb]

LGTM

Thanks Zvi!

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13765–13773 ↗

(On Diff #75083)

That's okay. Using isAnyConstantBuildVector is fine.
My point was that the check for isUndef() in that function is not longer needed because you check in advance if N1 is undef.

[zvi]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13765–13773 ↗

(On Diff #75083)

You're right and I will improve this.

[zvi]

Removed the check for undef from isAnyConstantBuildVector()

[RKSimon]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

7754 ↗

(On Diff #75533)

If possible, do this change (along with the addition of isAnyConstantBuildVector) as a separate NFC pre-commit?

13936 ↗

(On Diff #75533)

Remove the outer braces to match style guide.

[zvi]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

7754 ↗

(On Diff #75533)

sure

13936 ↗

(On Diff #75533)

ok

[zvi]

Fixes for @RKSimon's comments.

[RKSimon]

LGTM

llvm/trunk/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

 static bool isAllOnesConstantOrAllOnesSplatConstant(SDValue N) {
   unsigned BitWidth = N.getScalarValueSizeInBits();
   if (ConstantSDNode *Splat = isConstOrConstSplat(N))
     return Splat->isAllOnesValue() &&
            Splat->getAPIntValue().getBitWidth() == BitWidth;
   return false;
 }
 
+// Determines if a BUILD_VECTOR is composed of all-constants possibly mixed with
+// undef's.
+static bool isAnyConstantBuildVector(const SDNode *N) {
+  return ISD::isBuildVectorOfConstantSDNodes(N) ||
+         ISD::isBuildVectorOfConstantFPSDNodes(N);
+}
+
 SDValue DAGCombiner::ReassociateOps(unsigned Opc, const SDLoc &DL, SDValue N0,
                                     SDValue N1) {
   EVT VT = N0.getValueType();
   if (N0.getOpcode() == Opc) {
     if (SDNode *L = DAG.isConstantIntBuildVectorOrConstantInt(N0.getOperand(1))) {
       if (SDNode *R = DAG.isConstantIntBuildVectorOrConstantInt(N1)) {
         // reassoc. (op (op x, c1), c2) -> (op x, (op c1, c2))
         if (SDValue OpNode = DAG.FoldConstantArithmetic(Opc, DL, VT, L, R))
for (unsigned I = 0; I != NumConcats; ++I) {
     } else { // Mixed with general masks and undefs, can't do optimization.
       return SDValue();
     }
   }
 
   return DAG.getNode(ISD::CONCAT_VECTORS, SDLoc(N), VT, Ops);
 }
 
+// Attempt to combine a shuffle of 2 inputs of 'scalar sources' -
+// BUILD_VECTOR or SCALAR_TO_VECTOR into a single BUILD_VECTOR.
+// This combine is done in the following cases:
+// 1. Both N0,N1 are BUILD_VECTOR's composed of constants or undefs.
+// 2. Only one of N0,N1 is a BUILD_VECTOR composed of constants or undefs -
+//    Combine iff that node is ALL_ZEROS. We prefer not to combine a
+//    BUILD_VECTOR of all constants to allow efficient materialization of
+//    constant vectors, but the ALL_ZEROS is an exception because
+//    zero-extension matching seems to rely on having BUILD_VECTOR nodes with
+//    zero padding between elements. FIXME: Eliminate this exception for
+//    ALL_ZEROS constant vectors.
+// 3. Neither N0,N1 are composed of only constants.
+static SDValue combineShuffleOfScalars(ShuffleVectorSDNode *SVN,
+                                       SelectionDAG &DAG,
+                                       const TargetLowering &TLI) {
+  EVT VT = SVN->getValueType(0);
+  unsigned NumElts = VT.getVectorNumElements();
+  SDValue N0 = SVN->getOperand(0);
+  SDValue N1 = SVN->getOperand(1);
+
+  if (!N0->hasOneUse() || !N1->hasOneUse())
+    return SDValue();
+  // If only one of N1,N2 is constant, bail out if it is not ALL_ZEROS as
+  // discussed above.
+  if (!N1.isUndef()) {
+    bool N0AnyConst = isAnyConstantBuildVector(N0.getNode());
+    bool N1AnyConst = isAnyConstantBuildVector(N1.getNode());
+    if (N0AnyConst && !N1AnyConst && !ISD::isBuildVectorAllZeros(N0.getNode()))
+      return SDValue();
+    if (!N0AnyConst && N1AnyConst && !ISD::isBuildVectorAllZeros(N1.getNode()))
+      return SDValue();
+  }
+
+  SmallVector<SDValue, 8> Ops;
+  for (int M : SVN->getMask()) {
+    SDValue Op = DAG.getUNDEF(VT.getScalarType());
+    if (M >= 0) {
+      int Idx = M < (int)NumElts ? M : M - NumElts;
+      SDValue &S = (M < (int)NumElts ? N0 : N1);
+      if (S.getOpcode() == ISD::BUILD_VECTOR) {
+        Op = S.getOperand(Idx);
+      } else if (S.getOpcode() == ISD::SCALAR_TO_VECTOR) {
+        if (Idx == 0)
+          Op = S.getOperand(0);
+      } else {
+        // Operand can't be combined - bail out.
+        return SDValue();
+      }
+    }
+    Ops.push_back(Op);
+  }
+  // BUILD_VECTOR requires all inputs to be of the same type, find the
+  // maximum type and extend them all.
+  EVT SVT = VT.getScalarType();
+  if (SVT.isInteger())
+    for (SDValue &Op : Ops)
+      SVT = (SVT.bitsLT(Op.getValueType()) ? Op.getValueType() : SVT);
+  if (SVT != VT.getScalarType())
+    for (SDValue &Op : Ops)
+      Op = TLI.isZExtFree(Op.getValueType(), SVT)
+               ? DAG.getZExtOrTrunc(Op, SDLoc(SVN), SVT)
+               : DAG.getSExtOrTrunc(Op, SDLoc(SVN), SVT);
+  return DAG.getBuildVector(VT, SDLoc(SVN), Ops);
+}
+
 SDValue DAGCombiner::visitVECTOR_SHUFFLE(SDNode *N) {
   EVT VT = N->getValueType(0);
   unsigned NumElts = VT.getVectorNumElements();
 
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
 
   assert(N0.getValueType() == VT && "Vector shuffle must be normalized in DAG");
if (N0.getOpcode() == ISD::CONCAT_VECTORS &&
       (N1.getOpcode() == ISD::CONCAT_VECTORS &&
        N0.getOperand(0).getValueType() == N1.getOperand(0).getValueType()))) {
     if (SDValue V = partitionShuffleOfConcats(N, DAG))
       return V;
   }
 
   // Attempt to combine a shuffle of 2 inputs of 'scalar sources' -
   // BUILD_VECTOR or SCALAR_TO_VECTOR into a single BUILD_VECTOR.
-  if (Level < AfterLegalizeVectorOps && TLI.isTypeLegal(VT)) {
-    SmallVector<SDValue, 8> Ops;
-    for (int M : SVN->getMask()) {
-      SDValue Op = DAG.getUNDEF(VT.getScalarType());
-      if (M >= 0) {
-        int Idx = M % NumElts;
-        SDValue &S = (M < (int)NumElts ? N0 : N1);
-        if (S.getOpcode() == ISD::BUILD_VECTOR && S.hasOneUse()) {
-          Op = S.getOperand(Idx);
-        } else if (S.getOpcode() == ISD::SCALAR_TO_VECTOR && S.hasOneUse()) {
-          if (Idx == 0)
-            Op = S.getOperand(0);
-        } else {
-          // Operand can't be combined - bail out.
-          break;
-        }
-      }
-      Ops.push_back(Op);
-    }
-    if (Ops.size() == VT.getVectorNumElements()) {
-      // BUILD_VECTOR requires all inputs to be of the same type, find the
-      // maximum type and extend them all.
-      EVT SVT = VT.getScalarType();
-      if (SVT.isInteger())
-        for (SDValue &Op : Ops)
-          SVT = (SVT.bitsLT(Op.getValueType()) ? Op.getValueType() : SVT);
-      if (SVT != VT.getScalarType())
-        for (SDValue &Op : Ops)
-          Op = TLI.isZExtFree(Op.getValueType(), SVT)
-                   ? DAG.getZExtOrTrunc(Op, SDLoc(N), SVT)
-                   : DAG.getSExtOrTrunc(Op, SDLoc(N), SVT);
-      return DAG.getBuildVector(VT, SDLoc(N), Ops);
-    }
-  }
+  if (Level < AfterLegalizeVectorOps && TLI.isTypeLegal(VT))
+    if (SDValue Res = combineShuffleOfScalars(SVN, DAG, TLI))
+      return Res;
 
   // If this shuffle only has a single input that is a bitcasted shuffle,
   // attempt to merge the 2 shuffles and suitably bitcast the inputs/output
   // back to their original types.
   if (N0.getOpcode() == ISD::BITCAST && N0.hasOneUse() &&
       N1.isUndef() && Level < AfterLegalizeVectorOps &&
       TLI.isTypeLegal(VT)) {
 

llvm/trunk/test/CodeGen/X86/vector-shuffle-combining.ll

; AVX-NEXT:    retq
   store <4 x float> %5, <4 x float>* %a1, align 16
   ret void
 }
 
 ; PR30371
 define <4 x float> @combine_constant_insertion_v4f32(float %f) {
 ; SSE2-LABEL: combine_constant_insertion_v4f32:
 ; SSE2:       # BB#0:
-; SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSE2-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SSE2-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
+; SSE2-NEXT:    movaps {{.*#+}} xmm1 = <u,4,5,3>
+; SSE2-NEXT:    movss {{.*#+}} xmm1 = xmm0[0],xmm1[1,2,3]
+; SSE2-NEXT:    movaps %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: combine_constant_insertion_v4f32:
 ; SSSE3:       # BB#0:
-; SSSE3-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSSE3-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SSSE3-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
-; SSSE3-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSSE3-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
-; SSSE3-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
+; SSSE3-NEXT:    movaps {{.*#+}} xmm1 = <u,4,5,3>
+; SSSE3-NEXT:    movss {{.*#+}} xmm1 = xmm0[0],xmm1[1,2,3]
+; SSSE3-NEXT:    movaps %xmm1, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: combine_constant_insertion_v4f32:
 ; SSE41:       # BB#0:
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; SSE41-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
+; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm0[0],mem[1,2,3]
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: combine_constant_insertion_v4f32:
 ; AVX:       # BB#0:
-; AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
-; AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
-; AVX-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
+; AVX-NEXT:    vblendps {{.*#+}} xmm0 = xmm0[0],mem[1,2,3]
 ; AVX-NEXT:    retq
   %a0 = insertelement <4 x float> undef, float %f, i32 0
   %ret = shufflevector <4 x float> %a0, <4 x float> <float undef, float 4.0, float 5.0, float 3.0>, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x float> %ret
 }
 
 define <4 x i32> @combine_constant_insertion_v4i32(i32 %f) {
 ; SSE2-LABEL: combine_constant_insertion_v4i32:
 ; SSE2:       # BB#0:
-; SSE2-NEXT:    movl $30, %eax
-; SSE2-NEXT:    movd %eax, %xmm0
-; SSE2-NEXT:    movl $4, %eax
-; SSE2-NEXT:    movd %eax, %xmm1
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; SSE2-NEXT:    movl $5, %eax
-; SSE2-NEXT:    movd %eax, %xmm2
-; SSE2-NEXT:    movd %edi, %xmm0
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
+; SSE2-NEXT:    movd %edi, %xmm1
+; SSE2-NEXT:    movaps {{.*#+}} xmm0 = <u,4,5,30>
+; SSE2-NEXT:    movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: combine_constant_insertion_v4i32:
 ; SSSE3:       # BB#0:
-; SSSE3-NEXT:    movl $30, %eax
-; SSSE3-NEXT:    movd %eax, %xmm0
-; SSSE3-NEXT:    movl $4, %eax
-; SSSE3-NEXT:    movd %eax, %xmm1
-; SSSE3-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
-; SSSE3-NEXT:    movl $5, %eax
-; SSSE3-NEXT:    movd %eax, %xmm2
-; SSSE3-NEXT:    movd %edi, %xmm0
-; SSSE3-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
-; SSSE3-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
+; SSSE3-NEXT:    movd %edi, %xmm1
+; SSSE3-NEXT:    movaps {{.*#+}} xmm0 = <u,4,5,30>
+; SSSE3-NEXT:    movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: combine_constant_insertion_v4i32:
 ; SSE41:       # BB#0:
 ; SSE41-NEXT:    movd %edi, %xmm0
-; SSE41-NEXT:    movl $4, %eax
-; SSE41-NEXT:    pinsrd $1, %eax, %xmm0
-; SSE41-NEXT:    movl $5, %eax
-; SSE41-NEXT:    pinsrd $2, %eax, %xmm0
-; SSE41-NEXT:    movl $30, %eax
-; SSE41-NEXT:    pinsrd $3, %eax, %xmm0
-; SSE41-NEXT:    retq
-;
-; AVX-LABEL: combine_constant_insertion_v4i32:
-; AVX:       # BB#0:
-; AVX-NEXT:    vmovd %edi, %xmm0
-; AVX-NEXT:    movl $4, %eax
-; AVX-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
-; AVX-NEXT:    movl $5, %eax
-; AVX-NEXT:    vpinsrd $2, %eax, %xmm0, %xmm0
-; AVX-NEXT:    movl $30, %eax
-; AVX-NEXT:    vpinsrd $3, %eax, %xmm0, %xmm0
-; AVX-NEXT:    retq
+; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm0[0,1],mem[2,3,4,5,6,7]
+; SSE41-NEXT:    retq
+;
+; AVX1-LABEL: combine_constant_insertion_v4i32:
+; AVX1:       # BB#0:
+; AVX1-NEXT:    vmovd %edi, %xmm0
+; AVX1-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0,1],mem[2,3,4,5,6,7]
+; AVX1-NEXT:    retq
+;
+; AVX2-LABEL: combine_constant_insertion_v4i32:
+; AVX2:       # BB#0:
+; AVX2-NEXT:    vmovd %edi, %xmm0
+; AVX2-NEXT:    vpblendd {{.*#+}} xmm0 = xmm0[0],mem[1,2,3]
+; AVX2-NEXT:    retq
   %a0 = insertelement <4 x i32> undef, i32 %f, i32 0
   %ret = shufflevector <4 x i32> %a0, <4 x i32> <i32 undef, i32 4, i32 5, i32 30>, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x i32> %ret
 }
 
 define <4 x float> @PR22377(<4 x float> %a, <4 x float> %b) {
 ; SSE-LABEL: PR22377:
 ; SSE:       # BB#0: # %entry