[DAGCombiner] (insert_vector_elt (vector_shuffle X, Y), (extract_vector_elt X, N), IdxC) -> (vector_shuffle X, Y)

[deadalnix]

Summary

This is beneficial when the shuffle is only used once and end up being generated in a few places when some node is combined into a shuffle.

[lebedev.ri]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16432–16433

Do you check that both vector_shuffle and extract_vector_elt have the same first argument?

16434–16436

Here we have two instructions, and only require that the first one is single-use.
Observation: we only create a single new instruction.
Do we want to either require both of them be single-use, the first one, either one of them, or none of them?

Another observation - you end up keeping (using) the instruction you just checked for extra uses.
Did you mean to check the other one instead?

16448–16450

So NewMask is simply inited as a copy of SVN->getMask()?
Would something like SmallVector<int, 16> NewMask(SVN->getMask()); just work?

16452

You either need to check that the cast succeeded, or drop dyn_ prefix

16453

This could use a comment

test/CodeGen/X86/madd.ll

1901–1903

If i'm reading this correctly this does showcase the need to handle this in backend - it can be introduced there?

[deadalnix]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16432–16433

This is checked bellow. It is because we can match either the first or the second argument, as vector_shuffle does in fact shuffle two vectors together.

16434–16436

Even if InsertVal is reused, this transform reduces data dependencies, but indeed do not save anything on the instruction count from. Removing data dependencies is beneficial on its own.

16448–16450

I asked mys elf the same question. This doesn't seem to work.

16452

The cast must succeed, as we check for it in the if guarding this code, so I'll remove dyn_

16453

I'll add a few comments.

[craig.topper]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16448–16450

Yeah SmallVector doesn't take an ArrayRef, but you can do

SmallVector<int, 16> NewMask(Mask.begin(), Mask.end());

[lebedev.ri]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16434–16436

So, do we need that one-use restriction here then?

16448–16450

SmallVector<int, 16> NewMask(SVN->getMask().begin(), SVN->getMask().end()); then.

[deadalnix]

• Add comments
• dyn_cast -> cast

[deadalnix]

Build NewMask using iterators

[lebedev.ri]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16448–16450

Please mark comments as resolved as appropriate.

[deadalnix]

Comments amrked as done.

[RKSimon]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16451

You already have this at line 16438:

ArrayRef<int> Mask = SVN->getMask()

16456

You need to check NewMask is legal: if (!TLI.isShuffleMaskLegal(NewMask, Vec.getValueType()))

[deadalnix]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16456

There are a few places where isShuffleMaskLegal is used. Some try to flip operands inc ase it isn't legal. Should that be an idiom?

[lebedev.ri]

Looks like the link was incorrect, this didn't auto-close?

[deadalnix]

In D66718#1645956, @lebedev.ri wrote:

Looks like the link was incorrect, this didn't auto-close?

Why should this auto close?

It wasn't landed yet. I was wondering if the creation of vector shuffles should be normalized. I think it may be worth doing.

[lebedev.ri]

In D66718#1646859, @deadalnix wrote:

In D66718#1645956, @lebedev.ri wrote:

Looks like the link was incorrect, this didn't auto-close?

Why should this auto close?

It wasn't landed yet. I was wondering if the creation of vector shuffles should be normalized. I think it may be worth doing.

Sorry, i got confused by "commits" tab

[RKSimon]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16456

I've no objection to you creating a static helper that tries isShuffleMaskLegal with original and commuted masks, and call DAG.getVectorShuffle on success.

[deadalnix]

Rebase on top of D66804

[lebedev.ri]

This looks good to me but i find naming choice really confusing here.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16429–16467

Hm, please rename this to ExtractVal instead.

16432

Comment is wrong i think, should be
s/IdxC/InsIndex/

16456

ExtrIndex

[lebedev.ri]

Actually, i have one concern.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16430

OldShuffle

16440–16441

Just x/y like in comments?

16446

Maybe XOffset?

16450

Actually, this is not correct.
While both shuffle inputs must have same types,
the shuffle mask can have different dimensions.
So this should be closer to Offset = X.getType().getVectorNumElements().

https://llvm.org/docs/LangRef.html#shufflevector-instruction

[RKSimon]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16430

There's no need to rename - tbh we don't know its a shuffle at this stage!

16450

That is only true for IR - in the DAG, the result must have the same type as the shuffle operands - so the number of mask elts must match the number of result/operand elts - see SelectionDAG::getVectorShuffle

16456

Please can you assert that IndexX's value (and InsIndex come to that) are both in bounds (0 <= idx < Mask.size())?

[deadalnix]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16430

That is true, and this is why the names are what they are. But I have to agree with @lebedev.ri here, the name are very confusing. I'll try to see if I can come up with better names.

[lebedev.ri]

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16450

Oh, forgot about that, sorry. An assert later on will be fine then.

[deadalnix]

IndexC => ExtrIndex

[lebedev.ri]

I still find naming choices to be actively misleading.
LG otherwise.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

16430

There's no need to rename - tbh we don't know its a shuffle at this stage!

But we will only proceed *if* it will be a shuffle. This is a common pattern everywhere else in the code.

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

SDValue DAGCombiner::splitMergedValStore(StoreSDNode *ST) {
   SDValue St1 =
       DAG.getStore(St0, DL, Hi, Ptr,
                    ST->getPointerInfo().getWithOffset(HalfValBitSize / 8),
                    Alignment / 2, MMOFlags, AAInfo);
   return St1;
 }
 
 /// Convert a disguised subvector insertion into a shuffle:
-/// insert_vector_elt V, (bitcast X from vector type), IdxC -->
-/// bitcast(shuffle (bitcast V), (extended X), Mask)
-/// Note: We do not use an insert_subvector node because that requires a legal
-/// subvector type.
 SDValue DAGCombiner::combineInsertEltToShuffle(SDNode *N, unsigned InsIndex) {
   SDValue InsertVal = N->getOperand(1);
+  SDValue Vec = N->getOperand(0);
+
+  // (insert_vector_elt (vector_shuffle X, Y), (extract_vector_elt X, N), IdxC)
+  //   --> (vector_shuffle X, Y)
+  if (Vec.getOpcode() == ISD::VECTOR_SHUFFLE && Vec.hasOneUse() &&
+      InsertVal.getOpcode() == ISD::EXTRACT_VECTOR_ELT &&
+      isa<ConstantSDNode>(InsertVal.getOperand(1))) {
+    ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(Vec.getNode());
+    ArrayRef<int> Mask = SVN->getMask();
+
+    // Vec's operand 0 is using indices from 0 to N-1 and
+    // operand 1 from N to 2N - 1, where N is the number of
+    // elements in the vectors.
+    int Offset = -1;
+    if (InsertVal.getOperand(0) == Vec.getOperand(0)) {
+      Offset = 0;
+    } else if (InsertVal.getOperand(0) == Vec.getOperand(1)) {
+      Offset = Mask.size();
+    }
+
+    if (Offset != -1) {
+      SmallVector<int, 16> NewMask(Mask.size());
+      for (size_t i = 0; i < Mask.size(); i++)
+        NewMask[i] = Mask[i];
+
+      auto *IndexC = cast<ConstantSDNode>(InsertVal.getOperand(1));
+      NewMask[InsIndex] = Offset + IndexC->getZExtValue();
+
+      return DAG.getVectorShuffle(Vec.getValueType(), SDLoc(N),
+                                  Vec.getOperand(0), Vec.getOperand(1),
+                                  NewMask);
+    }
+  }
+
+  // insert_vector_elt V, (bitcast X from vector type), IdxC -->
+  // bitcast(shuffle (bitcast V), (extended X), Mask)
+  // Note: We do not use an insert_subvector node because that requires a
+  // legal subvector type.
   if (InsertVal.getOpcode() != ISD::BITCAST || !InsertVal.hasOneUse() ||
       !InsertVal.getOperand(0).getValueType().isVector())
     return SDValue();
 
   SDValue SubVec = InsertVal.getOperand(0);
   SDValue DestVec = N->getOperand(0);
   EVT SubVecVT = SubVec.getValueType();
   EVT VT = DestVec.getValueType();

test/CodeGen/X86/madd.ll

 ; AVX512-NEXT:    vpmulld %zmm1, %zmm0, %zmm0
 ; AVX512-NEXT:    vpextrd $2, %xmm0, %eax
 ; AVX512-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm1
 ; AVX512-NEXT:    vextracti128 $1, %ymm0, %xmm2
 ; AVX512-NEXT:    vmovd %xmm2, %eax
 ; AVX512-NEXT:    vpinsrd $2, %eax, %xmm1, %xmm1
 ; AVX512-NEXT:    vpextrd $2, %xmm2, %eax
 ; AVX512-NEXT:    vpinsrd $3, %eax, %xmm1, %xmm1
-; AVX512-NEXT:    vpextrd $3, %xmm0, %eax
-; AVX512-NEXT:    vpshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
-; AVX512-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
+; AVX512-NEXT:    vpshufd {{.*#+}} xmm0 = xmm0[1,3,2,3]
 ; AVX512-NEXT:    vpextrd $1, %xmm2, %eax
 ; AVX512-NEXT:    vpinsrd $2, %eax, %xmm0, %xmm0
 ; AVX512-NEXT:    vpextrd $3, %xmm2, %eax
 ; AVX512-NEXT:    vpinsrd $3, %eax, %xmm0, %xmm0
 ; AVX512-NEXT:    vpaddd %xmm0, %xmm1, %xmm0
 ; AVX512-NEXT:    vzeroupper
 ; AVX512-NEXT:    retq
    %a = sext <16 x i16> %A to <16 x i32>

test/CodeGen/X86/vector-shuffle-combining.ll

; AVX-NEXT:    retq
   %12 = or <4 x i32> %5, %9
   %13 = or <4 x i32> %12, %10
   %14 = or <4 x i32> %13, %11
   %15 = add <4 x i32> %a0, %14
   ret <4 x i32> %15
 }
 
 define <8 x i16> @shuffle_extract_insert(<8 x i16> %a) {
-; SSE2-LABEL: shuffle_extract_insert:
-; SSE2:       # %bb.0:
-; SSE2-NEXT:    pextrw $3, %xmm0, %eax
-; SSE2-NEXT:    pextrw $4, %xmm0, %r8d
-; SSE2-NEXT:    pextrw $5, %xmm0, %edx
-; SSE2-NEXT:    pextrw $6, %xmm0, %esi
-; SSE2-NEXT:    movd %xmm0, %edi
-; SSE2-NEXT:    pextrw $7, %xmm0, %ecx
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[2,1,2,3,4,5,6,7]
-; SSE2-NEXT:    pinsrw $2, %edi, %xmm0
-; SSE2-NEXT:    pinsrw $3, %eax, %xmm0
-; SSE2-NEXT:    pinsrw $4, %esi, %xmm0
-; SSE2-NEXT:    pinsrw $5, %edx, %xmm0
-; SSE2-NEXT:    pinsrw $6, %r8d, %xmm0
-; SSE2-NEXT:    pinsrw $7, %ecx, %xmm0
-; SSE2-NEXT:    retq
-;
-; SSSE3-LABEL: shuffle_extract_insert:
-; SSSE3:       # %bb.0:
-; SSSE3-NEXT:    pextrw $3, %xmm0, %eax
-; SSSE3-NEXT:    pextrw $4, %xmm0, %r8d
-; SSSE3-NEXT:    pextrw $5, %xmm0, %edx
-; SSSE3-NEXT:    pextrw $6, %xmm0, %esi
-; SSSE3-NEXT:    movd %xmm0, %edi
-; SSSE3-NEXT:    pextrw $7, %xmm0, %ecx
-; SSSE3-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[2,1,2,3,4,5,6,7]
-; SSSE3-NEXT:    pinsrw $2, %edi, %xmm0
-; SSSE3-NEXT:    pinsrw $3, %eax, %xmm0
-; SSSE3-NEXT:    pinsrw $4, %esi, %xmm0
-; SSSE3-NEXT:    pinsrw $5, %edx, %xmm0
-; SSSE3-NEXT:    pinsrw $6, %r8d, %xmm0
-; SSSE3-NEXT:    pinsrw $7, %ecx, %xmm0
-; SSSE3-NEXT:    retq
-;
-; SSE41-LABEL: shuffle_extract_insert:
-; SSE41:       # %bb.0:
-; SSE41-NEXT:    pextrw $4, %xmm0, %eax
-; SSE41-NEXT:    pextrw $6, %xmm0, %ecx
-; SSE41-NEXT:    movd %xmm0, %edx
-; SSE41-NEXT:    pshuflw {{.*#+}} xmm1 = xmm0[2,1,2,3,4,5,6,7]
-; SSE41-NEXT:    pinsrw $2, %edx, %xmm1
-; SSE41-NEXT:    pblendw {{.*#+}} xmm1 = xmm1[0,1,2],xmm0[3],xmm1[4,5,6,7]
-; SSE41-NEXT:    pinsrw $4, %ecx, %xmm1
-; SSE41-NEXT:    pblendw {{.*#+}} xmm1 = xmm1[0,1,2,3,4],xmm0[5],xmm1[6,7]
-; SSE41-NEXT:    pinsrw $6, %eax, %xmm1
-; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm1[0,1,2,3,4,5,6],xmm0[7]
-; SSE41-NEXT:    retq
-;
-; AVX-LABEL: shuffle_extract_insert:
-; AVX:       # %bb.0:
-; AVX-NEXT:    vpextrw $4, %xmm0, %eax
-; AVX-NEXT:    vpextrw $6, %xmm0, %ecx
-; AVX-NEXT:    vmovd %xmm0, %edx
-; AVX-NEXT:    vpshuflw {{.*#+}} xmm1 = xmm0[2,1,2,3,4,5,6,7]
-; AVX-NEXT:    vpinsrw $2, %edx, %xmm1, %xmm1
-; AVX-NEXT:    vpblendw {{.*#+}} xmm1 = xmm1[0,1,2],xmm0[3],xmm1[4,5,6,7]
-; AVX-NEXT:    vpinsrw $4, %ecx, %xmm1, %xmm1
-; AVX-NEXT:    vpblendw {{.*#+}} xmm1 = xmm1[0,1,2,3,4],xmm0[5],xmm1[6,7]
-; AVX-NEXT:    vpinsrw $6, %eax, %xmm1, %xmm1
-; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm1[0,1,2,3,4,5,6],xmm0[7]
-; AVX-NEXT:    retq
+; SSE-LABEL: shuffle_extract_insert:
+; SSE:       # %bb.0:
+; SSE-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[2,1,0,3,4,5,6,7]
+; SSE-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,6,5,4,7]
+; SSE-NEXT:    retq
+;
+; AVX1-LABEL: shuffle_extract_insert:
+; AVX1:       # %bb.0:
+; AVX1-NEXT:    vpshuflw {{.*#+}} xmm0 = xmm0[2,1,0,3,4,5,6,7]
+; AVX1-NEXT:    vpshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,6,5,4,7]
+; AVX1-NEXT:    retq
+;
+; AVX2-SLOW-LABEL: shuffle_extract_insert:
+; AVX2-SLOW:       # %bb.0:
+; AVX2-SLOW-NEXT:    vpshuflw {{.*#+}} xmm0 = xmm0[2,1,0,3,4,5,6,7]
+; AVX2-SLOW-NEXT:    vpshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,6,5,4,7]
+; AVX2-SLOW-NEXT:    retq
+;
+; AVX2-FAST-LABEL: shuffle_extract_insert:
+; AVX2-FAST:       # %bb.0:
+; AVX2-FAST-NEXT:    vpshufb {{.*#+}} xmm0 = xmm0[4,5,2,3,0,1,6,7,12,13,10,11,8,9,14,15]
+; AVX2-FAST-NEXT:    retq
   %a0 = extractelement <8 x i16> %a, i32 0
   %a1 = extractelement <8 x i16> %a, i32 1
   %a3 = extractelement <8 x i16> %a, i32 3
   %a4 = extractelement <8 x i16> %a, i32 4
   %a5 = extractelement <8 x i16> %a, i32 5
   %a6 = extractelement <8 x i16> %a, i32 6
   %a7 = extractelement <8 x i16> %a, i32 7
   %1 = shufflevector <8 x i16> %a, <8 x i16> %a, <8 x i32> <i32 2, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
   %2 = insertelement <8 x i16> %1, i16 %a1, i32 1
   %3 = insertelement <8 x i16> %2, i16 %a0, i32 2
   %4 = insertelement <8 x i16> %3, i16 %a3, i32 3
   %5 = insertelement <8 x i16> %4, i16 %a6, i32 4
   %6 = insertelement <8 x i16> %5, i16 %a5, i32 5
   %7 = insertelement <8 x i16> %6, i16 %a4, i32 6
   %8 = insertelement <8 x i16> %7, i16 %a7, i32 7
   ret <8 x i16> %8
 }
 
 define <8 x i16> @shuffle_extract_insert_double(<8 x i16> %a, <8 x i16> %b) {
 ; SSE2-LABEL: shuffle_extract_insert_double:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    movd %xmm0, %eax
-; SSE2-NEXT:    pextrw $4, %xmm0, %r8d
-; SSE2-NEXT:    pextrw $6, %xmm0, %edx
-; SSE2-NEXT:    pextrw $3, %xmm1, %esi
-; SSE2-NEXT:    pextrw $5, %xmm1, %edi
-; SSE2-NEXT:    pextrw $7, %xmm1, %ecx
-; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm1 = xmm1[0,3,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm1 = xmm1[0,1,2,3,7,5,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[0,2,2,3]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm1 = xmm1[0,1,3,2,4,5,6,7]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[1,0,3,2,4,5,6,7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
-; SSE2-NEXT:    pinsrw $2, %eax, %xmm0
-; SSE2-NEXT:    pinsrw $3, %esi, %xmm0
-; SSE2-NEXT:    pinsrw $4, %edx, %xmm0
-; SSE2-NEXT:    pinsrw $5, %edi, %xmm0
-; SSE2-NEXT:    pinsrw $6, %r8d, %xmm0
-; SSE2-NEXT:    pinsrw $7, %ecx, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: shuffle_extract_insert_double:
 ; SSSE3:       # %bb.0:
-; SSSE3-NEXT:    movd %xmm0, %eax
-; SSSE3-NEXT:    pextrw $4, %xmm0, %r8d
-; SSSE3-NEXT:    pextrw $6, %xmm0, %edx
-; SSSE3-NEXT:    pextrw $3, %xmm1, %esi
-; SSSE3-NEXT:    pextrw $5, %xmm1, %edi
-; SSSE3-NEXT:    pextrw $7, %xmm1, %ecx
-; SSSE3-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
+; SSSE3-NEXT:    pshufb {{.*#+}} xmm1 = xmm1[0,1,6,7,10,11,14,15,14,15,10,11,12,13,14,15]
+; SSSE3-NEXT:    pshufb {{.*#+}} xmm0 = xmm0[4,5,0,1,12,13,8,9,8,9,12,13,12,13,14,15]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
-; SSSE3-NEXT:    pinsrw $2, %eax, %xmm0
-; SSSE3-NEXT:    pinsrw $3, %esi, %xmm0
-; SSSE3-NEXT:    pinsrw $4, %edx, %xmm0
-; SSSE3-NEXT:    pinsrw $5, %edi, %xmm0
-; SSSE3-NEXT:    pinsrw $6, %r8d, %xmm0
-; SSSE3-NEXT:    pinsrw $7, %ecx, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: shuffle_extract_insert_double:
 ; SSE41:       # %bb.0:
-; SSE41-NEXT:    movd %xmm0, %eax
-; SSE41-NEXT:    pextrw $4, %xmm0, %ecx
-; SSE41-NEXT:    pextrw $6, %xmm0, %edx
-; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
+; SSE41-NEXT:    pshufb {{.*#+}} xmm1 = xmm1[0,1,6,7,10,11,14,15,14,15,10,11,12,13,14,15]
+; SSE41-NEXT:    pshufb {{.*#+}} xmm0 = xmm0[4,5,0,1,12,13,8,9,8,9,12,13,12,13,14,15]
 ; SSE41-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
-; SSE41-NEXT:    pinsrw $2, %eax, %xmm0
-; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[3],xmm0[4,5,6,7]
-; SSE41-NEXT:    pinsrw $4, %edx, %xmm0
-; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm0[0,1,2,3,4],xmm1[5],xmm0[6,7]
-; SSE41-NEXT:    pinsrw $6, %ecx, %xmm0
-; SSE41-NEXT:    pblendw {{.*#+}} xmm0 = xmm0[0,1,2,3,4,5,6],xmm1[7]
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: shuffle_extract_insert_double:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vmovd %xmm0, %eax
-; AVX-NEXT:    vpextrw $4, %xmm0, %ecx
-; AVX-NEXT:    vpextrw $6, %xmm0, %edx
-; AVX-NEXT:    vpshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
+; AVX-NEXT:    vpshufb {{.*#+}} xmm1 = xmm1[0,1,6,7,10,11,14,15,14,15,10,11,12,13,14,15]
+; AVX-NEXT:    vpshufb {{.*#+}} xmm0 = xmm0[4,5,0,1,12,13,8,9,8,9,12,13,12,13,14,15]
 ; AVX-NEXT:    vpunpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
-; AVX-NEXT:    vpinsrw $2, %eax, %xmm0, %xmm0
-; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0,1,2],xmm1[3],xmm0[4,5,6,7]
-; AVX-NEXT:    vpinsrw $4, %edx, %xmm0, %xmm0
-; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3,4],xmm1[5],xmm0[6,7]
-; AVX-NEXT:    vpinsrw $6, %ecx, %xmm0, %xmm0
-; AVX-NEXT:    vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3,4,5,6],xmm1[7]
 ; AVX-NEXT:    retq
   %a0 = extractelement <8 x i16> %a, i32 0
   %a4 = extractelement <8 x i16> %a, i32 4
   %a6 = extractelement <8 x i16> %a, i32 6
   %b11 = extractelement <8 x i16> %b, i32 3
   %b13 = extractelement <8 x i16> %b, i32 5
   %b15 = extractelement <8 x i16> %b, i32 7
   %1 = shufflevector <8 x i16> %a, <8 x i16> %b, <8 x i32> <i32 2, i32 8, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
   %2 = insertelement <8 x i16> %1, i16 %a0, i32 2
   %3 = insertelement <8 x i16> %2, i16 %b11, i32 3
   %4 = insertelement <8 x i16> %3, i16 %a6, i32 4
   %5 = insertelement <8 x i16> %4, i16 %b13, i32 5
   %6 = insertelement <8 x i16> %5, i16 %a4, i32 6
   %7 = insertelement <8 x i16> %6, i16 %b15, i32 7
   ret <8 x i16> %7
 }