Fix codegen of stores of vectors with non byte-sized elements.

[jonpa]

Summary

https://bugs.llvm.org/show_bug.cgi?id=35520

This is completely broken currently.

[jonpa]

First attempt, which seems to work, sort of:

Most failing tests regenerated (as they had already been generated previously), except for two tests:

Failing Tests (2):

LLVM :: CodeGen/SystemZ/vec-move-17.ll
LLVM :: CodeGen/X86/pr32108.ll

During these tests, an assert triggers in SelectionDAG::computeKnownBits(). It seems somewhere there is a mixup of integer width where identical widths are assumed:

(gdb) p Known
$92 = (llvm::KnownBits &) @0x3ffffffb930: {Zero = {static WORD_MAX = 18446744073709551615, U = {
      VAL = 4294967294, pVal = 0xfffffffe}, BitWidth = 32}, One = {
    static WORD_MAX = 18446744073709551615, U = {VAL = 1, pVal = 0x1}, BitWidth = 32}}
(gdb) p Known2
$93 = {Zero = {static WORD_MAX = 18446744073709551615, U = {VAL = 0, pVal = 0x0}, BitWidth = 64},
  One = {static WORD_MAX = 18446744073709551615, U = {VAL = 0, pVal = 0x0}, BitWidth = 64}}

    // Output known-1 bits are only known if set in both the LHS & RHS.                                                                                                                                             
    Known.One &= Known2.One;
->
  APInt &operator&=(const APInt &RHS) {
    assert(BitWidth == RHS.BitWidth && "Bit widths must be the same");

I am guessing that my new code is probably missing some kind of flag, but I am not sure exactly what the fix is...

(SystemZ/vec-move-17.ll was previously just supposed to compile, but now there should probably be some test as well once it actually compiles with patch).

The tests I have gathered so far are right now in CodeGen/Generic/store_nonbytesized_vecs.ll. They seem to be correctly handled by CodeGen, but I am hoping that you will help to verify this, as well as give a suggestion as to where this test should be placed. In SystemZ run just for that target, or in Generic run by multiple targets, perhaps? Currently, I just added the generated output by update_llc_test_checks.py for SystemZ.

[efriedma]

I don't see a change to DAGTypeLegalizer::SplitVecOp_STORE? (I think you're currently miscompiling the <8 x i31> testcase.)

We're generating pretty terrible code for some of the x86 testcases, but you obviously don't need to fix that here.

For the assertion failure, it looks like you're uncovering some existing bug in computeKnownBits; I can't think of any way your changes could cause that issue. Maybe some bug in the bitcast handling code?

[jonpa]

I found that if I extract the element into its native width, and then truncate, everything works. I then had to narrow this handling to integer types, as I saw X86 failing with v4f80 vectors since TRUNCATE is only for integers.

About the problem in computeKnownBits:

computeKnownBits() gets called with Known of 32 bits, to analyze an EXTRACT_VECTOR_ELT, which is an v2i64 vector. Therefore Known becomes 64 bits wide, and the calling computeKnownBits() crashes on

Known.One &= Known2.One;

, as Known is 32 bits.

I fixed this by

diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
index 837173e..9f98bdd 100644
--- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
+++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp
@@ -2805,6 +2805,10 @@ void SelectionDAG::computeKnownBits(SDValue Op, KnownBits &Known,
     }
     if (BitWidth > EltBitWidth)
       Known = Known.zext(BitWidth);
+    // If the extracted element is wider than BitWidth, truncate to match
+    // callers expectations.
+    if (Known.getBitWidth() > BitWidth)
+      Known = Known.trunc(BitWidth);
     break;
   }
   case ISD::INSERT_VECTOR_ELT: {

, which made the DAGCombiner work for the test case (before trying the explit truncate, which worked without this)

Not sure if this patch should still go in, or if there should be an assert that a EXTRACT_VECTOR_ELT should not implicitly truncate?

I don't see a change to DAGTypeLegalizer::SplitVecOp_STORE? (I think you're currently miscompiling the <8 x i31> testcase.)

You're right, I added the same handling there.

Tests updated.

[efriedma]

if there should be an assert that a EXTRACT_VECTOR_ELT should not implicitly truncate?

There should be an assert, yes.

I then had to narrow this handling to integer types, as I saw X86 failing with v4f80 vectors since TRUNCATE is only for integers.

I think all you need for non-integer types is a bitcast to an integer type after the EXTRACT_VECTOR_ELT.

[efriedma]

I think all you need for non-integer types is a bitcast to an integer type after the EXTRACT_VECTOR_ELT.

Err, actually, wait a sec... why are we triggering this path for v4f80? As long as the element size in bits is a multiple of 8, we can use separate stores for each element. (Whether the element size is a power of two isn't really important.)

[jonpa]

In D42100#978951, @efriedma wrote:

if there should be an assert that a EXTRACT_VECTOR_ELT should not implicitly truncate?

There should be an assert, yes.

Where should this assert be located? Should it be asserted at the point of node creation, or just at the end during isel? I don't see any asserts in the getNode() method. (We could make this a separate patch, I guess)

I then had to narrow this handling to integer types, as I saw X86 failing with v4f80 vectors since TRUNCATE is only for integers.

why are we triggering this path for v4f80? As long as the element size in bits is a multiple of 8, we can use separate stores for each element.

OK, I asked this previously since I was worried that if e.g. storing a v3i24, the i24 element stores might be promoted / split / ... somehow, so that it would be safest to just build the integer.

(Whether the element size is a power of two isn't really important.)

So then we trust that the original loop in scalarizeVectorStore() will do the job, which seems to be the case.

I changed the check then to simply call isByteSized() on the vector element.

In case we get a floating point element which is not byte-sized, the assert for ISD::TRUNCATE will trigger. Since there are no such types in the language reference, I skipped that check.

Tests still pass unmodified from before.

[efriedma]

I don't see any asserts in the getNode() method.

Really? I see a bunch. (getNode() is overloaded; make sure you're looking at the version for nodes with two operands.) But anyway, that can be a separate patch.

test/CodeGen/X86/avx512-mask-op.ll

2786

This new code is substantially worse than the old code, without solving any correctness problem.

Specifically, the key here is that you can split without scalarizing if the size of each half in bits is a multiple of 8. So on AVX-512, we can split a <64 x i1> store into four legal <16 x i1> stores.

[jonpa]

In D42100#980794, @efriedma wrote:

I don't see any asserts in the getNode() method.

Really? I see a bunch. (getNode() is overloaded; make sure you're looking at the version for nodes with two operands.) But anyway, that can be a separate patch.

Aah, sorry, now I see it. But actually, it says

if (VT != Elt.getValueType())
   // If the vector element type is not legal, the BUILD_VECTOR operands
   // are promoted and implicitly truncated, and the result implicitly
   // extended. Make that explicit here.
   Elt = getAnyExtOrTrunc(Elt, DL, VT);

So it is not really true that there could be a general assert since this is actually even expected, or? I don't think I have a patch for this, then.

Specifically, the key here is that you can split without scalarizing if the size of each half in bits is a multiple of 8. So on AVX-512, we can split a <64 x i1> store into four legal <16 x i1> stores.

I updated the test in SplitVecOp_STORE to

+ // Scalarize if elements and the split halves are not byte-sized.
+ if (!MemoryVT.getScalarType().isByteSized() &&
+ (!LoMemVT.isByteSized() || !HiMemVT.isByteSized()))
+ return TLI.scalarizeVectorStore(N, DAG);

, which seems to work. Two stores will be produced, and the second store will have the byte-offset of the first store, which works as long as that first store is byte-sized.

Two X86 tests updated (test/CodeGen/X86/avx512-mask-op.ll, test/CodeGen/X86/vector-compare-results.ll)

[efriedma]

LGTM

So it is not really true that there could be a general assert since this is actually even expected, or? I don't think I have a patch for this, then.

That code is specifically dealing with an EXTRACT_VECTOR_ELT from a BUILD_VECTOR; in that case, before type legalization is finished, you could have a mix of types: either the BUILD_VECTOR operand type could be wider than the EXTRACT_VECTOR_ELT result, or the EXTRACT_VECTOR_ELT result type could be wider than the BUILD_VECTOR operand. But they should both be at least as wide as the vector element type.

lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

2021

I think the "MemoryVT.getScalarType().isByteSized()" check is redundant. Not a big deal either way.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3471

Extra space after assert.

[craig.topper]

test/CodeGen/SystemZ/vec-move-17.ll

65

Why did this comment get deleted?

[jonpa]

Thanks for review!

Committed as trunk@323042.

lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

2021

Yes, I think you are right :-) removed.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3471

removed

test/CodeGen/SystemZ/vec-move-17.ll

65

On Jan 16, I wrote: "(SystemZ/vec-move-17.ll was previously just supposed to compile, but now there should probably be some test as well once it actually compiles with patch)."

I have now added some CHECKs and left the comment deleted.

[efriedma]

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3466

I just realized there's a problem here. I guess I've spent too much time on little-endian targets.

Specifically, the problem is that this code will store the elements in the wrong order on big-endian targets. This will lead to inconsistent, and generally messy, results.

I think you can solve this by reversing the loop on big-endian targets, so high element of the vector is in the low bits of the integer.

[uabelho]

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3466

So we have a bug for big-endian targets on trunk now?

Will you deal with this Jonas?

[jonpa]

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3466

sorry for delay (vacation). Yes, I will propose a patch soon.

[uweigand]

I've been talking to Jonas about the big-endian question, and I now think that there may actually be open questions about what the layout should be.

In general, these are the same questions that need to be answered when laying out a struct type including bitfields. However, for bitfields you usually have a platform ABI that specifies the precise layout. But these vector types are LLVM IR types that do not match any high-level language type directly, so really should be defined by LLVM. I was not able to find any such definition in the IR documentation -- am I missing something here?

Starting with the simple case of <8 x i1>, the question that needs to be answered is, if you convert this to an i8 by either storing to memory and loading the byte, or else doing a bitcast, where in the resulting i8 can you find the elements 0 ... 7 of the original <8 x i1>. And specifically, should the answer to that question depend on whether the system is big- or little-endian?

Note that comparing this to a bit field that implements a similar type: struct { char b0 : 1; char b1 : 1; ... char b7 : 1 } ... here, the platform ABI defines where to find the bits b0 .. b7, and this does actually differ. On (most) little-endian machines, b0 is placed in the least-significant bit of the byte in memory, while on (most) big-endian machines, b0 is placed in the most-significant bit. [ GCC in fact also supports a few rare platforms where this doesn't hold. I think those are not supported by LLVM today. ]

Should LLVM define the layout of <8 x i1> the same way? Or should it impose its own layout, which may be defined independently of byte order? (Again my question, maybe this has been defined and I just don't know where to find this information?)

Once we've decided the simple case, we can probably extend the definition to more complex cases like <7 x i1> or <2 x i3> in a straightforward manner ...

[efriedma]

We never really formally defined the encoding of vectors, I think. The DataLayout has computed the size in bits of vector types as element_size_in_bits*num_elements for a long time, though (so there's no padding between elements), and we've generally had the rule that the lowest element has the lowest address, since that's generally how vector units work on both big-endian and little-endian targets.

Given those two constraints, there's only one possible rule for non-power-of-two byte-sized elements: <4 x i24> should have the same layout as <{ i24, i24, i24, i24 }> (a packed struct). And there's only one way to extrapolate that rule to non-byte-size element types: essentially, if you bitcast the vector to an integer, the first element should be in the high bit(s).

[uweigand]

Yes, that makes sense to me. It also agrees with the layout of bitfields on (all currently supported) big-endian platforms, so that's another argument for it.

[jonpa]

This patch follows your suggestion by reversing the order of the vector elements in the built integer for big endian targets.

SystemZ tests updated.

[efriedma]

Could you open a new review request instead of reusing the existing one with a new patch? This is going to get confusing.

[jonpa]

In D42100#993814, @efriedma wrote:

Could you open a new review request instead of reusing the existing one with a new patch? This is going to get confusing.

sure...

include/llvm/CodeGen/TargetLowering.h

public:
   /// \returns MERGE_VALUEs of the scalar loads with their chains.
   SDValue scalarizeVectorLoad(LoadSDNode *LD, SelectionDAG &DAG) const;
 
   // Turn a store of a vector type into stores of the individual elements.
   /// \param ST Store with a vector value type
   /// \returns MERGE_VALUs of the individual store chains.
   SDValue scalarizeVectorStore(StoreSDNode *ST, SelectionDAG &DAG) const;
 
+  bool shouldStoreVectorAsOneInt(EVT StoreVT) const;
+
   /// Expands an unaligned load to 2 half-size loads for an integer, and
   /// possibly more for vectors.
   std::pair<SDValue, SDValue> expandUnalignedLoad(LoadSDNode *LD,
                                                   SelectionDAG &DAG) const;
 
   /// Expands an unaligned store to 2 half-size stores for integer values, and
   /// possibly more for vectors.
   SDValue expandUnalignedStore(StoreSDNode *ST, SelectionDAG &DAG) const;

lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp

SDValue VectorLegalizer::ExpandLoad(SDValue Op) {
   AddLegalizedOperand(Op.getValue(0), Value);
   AddLegalizedOperand(Op.getValue(1), NewChain);
 
   return (Op.getResNo() ? NewChain : Value);
 }
 
 SDValue VectorLegalizer::ExpandStore(SDValue Op) {
   StoreSDNode *ST = cast<StoreSDNode>(Op.getNode());
-
   SDValue TF = TLI.scalarizeVectorStore(ST, DAG);
   AddLegalizedOperand(Op, TF);
   return TF;
 }
 
 SDValue VectorLegalizer::Expand(SDValue Op) {
   switch (Op->getOpcode()) {
   case ISD::SIGN_EXTEND_INREG:

lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

return DAG.getMaskedScatter(DAG.getVTList(MVT::Other), DataHi.getValueType(),
                               DL, OpsHi, MMO);
 }
 
 SDValue DAGTypeLegalizer::SplitVecOp_STORE(StoreSDNode *N, unsigned OpNo) {
   assert(N->isUnindexed() && "Indexed store of vector?");
   assert(OpNo == 1 && "Can only split the stored value");
   SDLoc DL(N);
 
+  if (TLI.shouldStoreVectorAsOneInt(N->getMemoryVT()))
+    return TLI.scalarizeVectorStore(N, DAG);
+
   bool isTruncating = N->isTruncatingStore();
   SDValue Ch  = N->getChain();
   SDValue Ptr = N->getBasePtr();
   EVT MemoryVT = N->getMemoryVT();
   unsigned Alignment = N->getOriginalAlignment();
   MachineMemOperand::Flags MMOFlags = N->getMemOperand()->getFlags();
   AAMDNodes AAInfo = N->getAAInfo();
   SDValue Lo, Hi;
   GetSplitVector(N->getOperand(1), Lo, Hi);
 
   EVT LoMemVT, HiMemVT;
   std::tie(LoMemVT, HiMemVT) = DAG.GetSplitDestVTs(MemoryVT);
 
   unsigned IncrementSize = LoMemVT.getSizeInBits()/8;
 
   if (isTruncating)
     Lo = DAG.getTruncStore(Ch, DL, Lo, Ptr, N->getPointerInfo(), LoMemVT,
                            Alignment, MMOFlags, AAInfo);
   else
     Lo = DAG.getStore(Ch, DL, Lo, Ptr, N->getPointerInfo(), Alignment, MMOFlags,
                       AAInfo);
 
   // Increment the pointer to the other half.
return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, SDLoc(N),
                      N->getValueType(0), InOp, N->getOperand(1));
 }
 
 SDValue DAGTypeLegalizer::WidenVecOp_STORE(SDNode *N) {
   // We have to widen the value, but we want only to store the original
   // vector type.
   StoreSDNode *ST = cast<StoreSDNode>(N);
 
+  if (TLI.shouldStoreVectorAsOneInt(ST->getMemoryVT()))
+    return TLI.scalarizeVectorStore(ST, DAG);
+
   SmallVector<SDValue, 16> StChain;
   if (ST->isTruncatingStore())
     GenWidenVectorTruncStores(StChain, ST);
   else
     GenWidenVectorStores(StChain, ST);
 
   if (StChain.size() == 1)
     return StChain[0];

lib/CodeGen/SelectionDAG/TargetLowering.cpp

SDValue TargetLowering::scalarizeVectorLoad(LoadSDNode *LD,
   }
 
   SDValue NewChain = DAG.getNode(ISD::TokenFactor, SL, MVT::Other, LoadChains);
   SDValue Value = DAG.getBuildVector(LD->getValueType(0), SL, Vals);
 
   return DAG.getMergeValues({ Value, NewChain }, SL);
 }
 
-// FIXME: This relies on each element having a byte size, otherwise the stride
-// is 0 and just overwrites the same location. ExpandStore currently expects
-// this broken behavior.
 SDValue TargetLowering::scalarizeVectorStore(StoreSDNode *ST,
                                              SelectionDAG &DAG) const {
   SDLoc SL(ST);
 
   SDValue Chain = ST->getChain();
   SDValue BasePtr = ST->getBasePtr();
   SDValue Value = ST->getValue();
   EVT StVT = ST->getMemoryVT();
 
   // The type of the data we want to save
   EVT RegVT = Value.getValueType();
   EVT RegSclVT = RegVT.getScalarType();
 
   // The type of data as saved in memory.
   EVT MemSclVT = StVT.getScalarType();
 
-  // Store Stride in bytes
-  unsigned Stride = MemSclVT.getSizeInBits() / 8;
   EVT IdxVT = getVectorIdxTy(DAG.getDataLayout());
   unsigned NumElem = StVT.getVectorNumElements();
 
+  // A vector must always be stored in memory as-is, i.e. without any padding
+  // between the elements, since various code depend on it, e.g. in the
+  // handling of a bitcast of a vector type to int, which may be done with a
+  // vector store followed by an integer load. A vector that does not have
+  // elements that are at least one byte in size which is also a power-of-2
+  // must therefore be stored as an integer built out of inserted vector
+  // elements.
+  if (shouldStoreVectorAsOneInt(ST->getMemoryVT())) {
+    unsigned NumBits = StVT.getSizeInBits();
+    EVT IntVT = EVT::getIntegerVT(*DAG.getContext(), NumBits);
+
+    SDValue CurrVal = DAG.getConstant(0, SL, IntVT);
+
+    for (unsigned Idx = 0; Idx < NumElem; ++Idx) {
+      SDValue Elt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, SL, RegSclVT, Value,
+                                DAG.getConstant(Idx, SL, IdxVT));
+      SDValue Trunc = DAG.getNode(ISD::TRUNCATE, SL, MemSclVT, Elt);
+      SDValue ExtElt = DAG.getNode(ISD::ZERO_EXTEND, SL, IntVT, Trunc);
+      SDValue ShiftAmount =
+        DAG.getConstant(Idx * MemSclVT.getSizeInBits(), SL, IntVT);
+      SDValue ShiftedElt = DAG.getNode(ISD::SHL, SL, IntVT, ExtElt, ShiftAmount);
+      CurrVal = DAG.getNode(ISD::OR, SL, IntVT, CurrVal, ShiftedElt);
+    }
+
+    return DAG.getStore(Chain, SL, CurrVal, BasePtr, ST->getPointerInfo(),
+                        ST->getAlignment(), ST->getMemOperand()->getFlags(),
+                        ST->getAAInfo());
+  }
+
+  // Store Stride in bytes
+  unsigned Stride = MemSclVT.getSizeInBits() / 8;
+  assert (Stride && "Zero stride!");
+
   // Extract each of the elements from the original vector and save them into
   // memory individually.
   SmallVector<SDValue, 8> Stores;
   for (unsigned Idx = 0; Idx < NumElem; ++Idx) {
     SDValue Elt = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, SL, RegSclVT, Value,
                               DAG.getConstant(Idx, SL, IdxVT));
 
     SDValue Ptr = DAG.getObjectPtrOffset(SL, BasePtr, Idx * Stride);
 
     // This scalar TruncStore may be illegal, but we legalize it later.
     SDValue Store = DAG.getTruncStore(
         Chain, SL, Elt, Ptr, ST->getPointerInfo().getWithOffset(Idx * Stride),
         MemSclVT, MinAlign(ST->getAlignment(), Idx * Stride),
         ST->getMemOperand()->getFlags(), ST->getAAInfo());
 
     Stores.push_back(Store);
   }
 
   return DAG.getNode(ISD::TokenFactor, SL, MVT::Other, Stores);
 }
 
+bool TargetLowering::shouldStoreVectorAsOneInt(EVT StoreVT) const {
+  // Trust that any floating point vectors are handled by target.
+  if (!StoreVT.isInteger())
+    return false;
+  unsigned ElemBits = StoreVT.getScalarType().getSizeInBits();
+  return (ElemBits < 8 || !isPowerOf2_32(ElemBits));
+}
+
 std::pair<SDValue, SDValue>
 TargetLowering::expandUnalignedLoad(LoadSDNode *LD, SelectionDAG &DAG) const {
   assert(LD->getAddressingMode() == ISD::UNINDEXED &&
          "unaligned indexed loads not implemented!");
   SDValue Chain = LD->getChain();
   SDValue Ptr = LD->getBasePtr();
   EVT VT = LD->getValueType(0);
   EVT LoadedVT = LD->getMemoryVT();

test/CodeGen/Generic/store_nonbytesized_vecs.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=s390x-linux-gnu -mcpu=z13 < %s  | FileCheck %s
+
+; Store a <4 x i31> vector.
+define void @fun0(<4 x i31> %src, <4 x i31>* %p)
+; CHECK-LABEL: fun0:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    vlgvf %r1, %v24, 3
+; CHECK-NEXT:    vlgvf %r0, %v24, 2
+; CHECK-NEXT:    sllg %r1, %r1, 29
+; CHECK-NEXT:    rosbg %r1, %r0, 35, 63, 62
+; CHECK-NEXT:    nihh %r1, 4095
+; CHECK-NEXT:    vlgvf %r3, %v24, 0
+; CHECK-NEXT:    stg %r1, 0(%r2)
+; CHECK-NEXT:    vlgvf %r1, %v24, 1
+; CHECK-NEXT:    llgtr %r3, %r3
+; CHECK-NEXT:    rosbg %r3, %r1, 2, 32, 31
+; CHECK-NEXT:    rosbg %r3, %r0, 0, 1, 62
+; CHECK-NEXT:    stg %r3, 8(%r2)
+; CHECK-NEXT:    br %r14
+{
+  store <4 x i31> %src, <4 x i31>* %p
+  ret void
+}
+
+; Store a <16 x i1> vector.
+define i16 @fun1(<16 x i1> %src)
+; CHECK-LABEL: fun1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    aghi %r15, -168
+; CHECK-NEXT:    .cfi_def_cfa_offset 328
+; CHECK-NEXT:    vlgvb %r0, %v24, 0
+; CHECK-NEXT:    vlgvb %r1, %v24, 1
+; CHECK-NEXT:    nilf %r0, 1
+; CHECK-NEXT:    rosbg %r0, %r1, 62, 62, 1
+; CHECK-NEXT:    vlgvb %r1, %v24, 2
+; CHECK-NEXT:    rosbg %r0, %r1, 61, 61, 2
+; CHECK-NEXT:    vlgvb %r1, %v24, 3
+; CHECK-NEXT:    rosbg %r0, %r1, 60, 60, 3
+; CHECK-NEXT:    vlgvb %r1, %v24, 4
+; CHECK-NEXT:    rosbg %r0, %r1, 59, 59, 4
+; CHECK-NEXT:    vlgvb %r1, %v24, 5
+; CHECK-NEXT:    rosbg %r0, %r1, 58, 58, 5
+; CHECK-NEXT:    vlgvb %r1, %v24, 6
+; CHECK-NEXT:    rosbg %r0, %r1, 57, 57, 6
+; CHECK-NEXT:    vlgvb %r1, %v24, 7
+; CHECK-NEXT:    rosbg %r0, %r1, 56, 56, 7
+; CHECK-NEXT:    vlgvb %r1, %v24, 8
+; CHECK-NEXT:    rosbg %r0, %r1, 55, 55, 8
+; CHECK-NEXT:    vlgvb %r1, %v24, 9
+; CHECK-NEXT:    rosbg %r0, %r1, 54, 54, 9
+; CHECK-NEXT:    vlgvb %r1, %v24, 10
+; CHECK-NEXT:    rosbg %r0, %r1, 53, 53, 10
+; CHECK-NEXT:    vlgvb %r1, %v24, 11
+; CHECK-NEXT:    rosbg %r0, %r1, 52, 52, 11
+; CHECK-NEXT:    vlgvb %r1, %v24, 12
+; CHECK-NEXT:    rosbg %r0, %r1, 51, 51, 12
+; CHECK-NEXT:    vlgvb %r1, %v24, 13
+; CHECK-NEXT:    rosbg %r0, %r1, 50, 50, 13
+; CHECK-NEXT:    vlgvb %r1, %v24, 14
+; CHECK-NEXT:    rosbg %r0, %r1, 49, 49, 14
+; CHECK-NEXT:    vlgvb %r1, %v24, 15
+; CHECK-NEXT:    rosbg %r0, %r1, 32, 48, 15
+; CHECK-NEXT:    sth %r0, 160(%r15)
+; CHECK-NEXT:    lh %r2, 160(%r15)
+; CHECK-NEXT:    aghi %r15, 168
+; CHECK-NEXT:    br %r14
+{
+  %res = bitcast <16 x i1> %src to i16
+  ret i16 %res
+}
+
+; Truncate a <8 x i32> vector to <8 x i31> and store it (test splitting).
+define void @fun2(<8 x i32> %src, <8 x i31>* %p)
+; CHECK-LABEL: fun2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    stmg %r14, %r15, 112(%r15)
+; CHECK-NEXT:    .cfi_offset %r14, -48
+; CHECK-NEXT:    .cfi_offset %r15, -40
+; CHECK-NEXT:    vlgvf %r1, %v24, 0
+; CHECK-NEXT:    stc %r1, 30(%r2)
+; CHECK-NEXT:    llgtr %r0, %r1
+; CHECK-NEXT:    # kill: def %r1l killed %r1l killed %r1d def %r1d
+; CHECK-NEXT:    srl %r1, 8
+; CHECK-NEXT:    sth %r1, 28(%r2)
+; CHECK-NEXT:    vlgvf %r1, %v24, 1
+; CHECK-NEXT:    rosbg %r0, %r1, 2, 32, 31
+; CHECK-NEXT:    srlg %r1, %r0, 24
+; CHECK-NEXT:    vlgvf %r3, %v26, 0
+; CHECK-NEXT:    st %r1, 24(%r2)
+; CHECK-NEXT:    vlgvf %r1, %v26, 1
+; CHECK-NEXT:    risbgn %r4, %r3, 37, 191, 60
+; CHECK-NEXT:    vlgvf %r5, %v26, 3
+; CHECK-NEXT:    sllg %r5, %r5, 25
+; CHECK-NEXT:    rosbg %r4, %r1, 6, 36, 27
+; CHECK-NEXT:    vlgvf %r1, %v26, 2
+; CHECK-NEXT:    rosbg %r5, %r1, 39, 63, 58
+; CHECK-NEXT:    rosbg %r4, %r1, 0, 5, 58
+; CHECK-NEXT:    sllg %r1, %r5, 8
+; CHECK-NEXT:    rosbg %r1, %r4, 56, 63, 8
+; CHECK-NEXT:    vlgvf %r5, %v24, 2
+; CHECK-NEXT:    rosbg %r0, %r5, 0, 1, 62
+; CHECK-NEXT:    stg %r1, 0(%r2)
+; CHECK-NEXT:    vlgvf %r1, %v24, 3
+; CHECK-NEXT:    risbgn %r14, %r5, 35, 191, 62
+; CHECK-NEXT:    rosbg %r14, %r1, 4, 34, 29
+; CHECK-NEXT:    sllg %r1, %r4, 8
+; CHECK-NEXT:    rosbg %r14, %r3, 0, 3, 60
+; CHECK-NEXT:    rosbg %r1, %r14, 56, 63, 8
+; CHECK-NEXT:    stg %r1, 8(%r2)
+; CHECK-NEXT:    sllg %r1, %r14, 8
+; CHECK-NEXT:    rosbg %r1, %r0, 56, 63, 8
+; CHECK-NEXT:    stg %r1, 16(%r2)
+; CHECK-NEXT:    lmg %r14, %r15, 112(%r15)
+; CHECK-NEXT:    br %r14
+{
+  %tmp = trunc <8 x i32> %src to <8 x i31>
+  store <8 x i31> %tmp, <8 x i31>* %p
+  ret void
+}
+
+; Load and store a <3 x i31> vector (test widening).
+define void @fun3(<3 x i31>* %src, <3 x i31>* %p)
+; CHECK-LABEL: fun3:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    llgf %r0, 3(%r2)
+; CHECK-NEXT:    llgf %r1, 0(%r2)
+; CHECK-NEXT:    rosbg %r1, %r0, 0, 32, 31
+; CHECK-NEXT:    llgf %r0, 6(%r2)
+; CHECK-NEXT:    st %r1, 8(%r3)
+; CHECK-NEXT:    rosbg %r1, %r0, 0, 1, 62
+; CHECK-NEXT:    srlg %r1, %r1, 32
+; CHECK-NEXT:    sllg %r0, %r0, 30
+; CHECK-NEXT:    lr %r0, %r1
+; CHECK-NEXT:    nihh %r0, 8191
+; CHECK-NEXT:    stg %r0, 0(%r3)
+; CHECK-NEXT:    br %r14
+{
+  %tmp = load <3 x i31>, <3 x i31>* %src
+  store <3 x i31> %tmp, <3 x i31>* %p
+  ret void
+}

test/CodeGen/SystemZ/vec-move-17.ll

 ; CHECK: br %r14
   %trunc = trunc <4 x i32> %val to <4 x i16>
   store <4 x i16> %trunc, <4 x i16> *%ptr
   ret void
 }
 
 ; Test a v2i64->v2i1 truncation.
 define void @f7(<2 x i64> %val, <2 x i1> *%ptr) {
-; No expected output, but must compile.
   %trunc = trunc <2 x i64> %val to <2 x i1>
   store <2 x i1> %trunc, <2 x i1> *%ptr
   ret void
 }
 
 ; Test a v2i64->v2i8 truncation.  At the moment we use a VPERM rather than
 ; a chain of packs.
 define void @f8(<2 x i64> %val, <2 x i8> *%ptr) {

test/CodeGen/X86/avx512-mask-op.ll

; AVX512DQ-NEXT:    retq
   ret void
 }
 
 
 define void @store_64i1(<64 x i1>* %a, <64 x i1> %v) {
 ;
 ; KNL-LABEL: store_64i1:
 ; KNL:       ## %bb.0:
-; KNL-NEXT:    vpmovsxbd %xmm3, %zmm3
-; KNL-NEXT:    vpslld $31, %zmm3, %zmm3
-; KNL-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; KNL-NEXT:    kmovw %k0, 6(%rdi)
-; KNL-NEXT:    vpmovsxbd %xmm2, %zmm2
-; KNL-NEXT:    vpslld $31, %zmm2, %zmm2
-; KNL-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; KNL-NEXT:    kmovw %k0, 4(%rdi)
-; KNL-NEXT:    vpmovsxbd %xmm1, %zmm1
-; KNL-NEXT:    vpslld $31, %zmm1, %zmm1
-; KNL-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; KNL-NEXT:    kmovw %k0, 2(%rdi)
 ; KNL-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; KNL-NEXT:    vpslld $31, %zmm0, %zmm0
+; KNL-NEXT:    vptestmd %zmm0, %zmm0, %k5
+; KNL-NEXT:    kshiftrw $15, %k5, %k0
+; KNL-NEXT:    kmovw %k0, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; KNL-NEXT:    kshiftrw $14, %k5, %k4
+; KNL-NEXT:    kshiftrw $13, %k5, %k7
+; KNL-NEXT:    kshiftrw $12, %k5, %k2
+; KNL-NEXT:    kshiftrw $11, %k5, %k3
+; KNL-NEXT:    kshiftrw $10, %k5, %k1
+; KNL-NEXT:    kshiftrw $7, %k5, %k0
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $1, %k5, %k0
+; KNL-NEXT:    kmovw %k0, %eax
+; KNL-NEXT:    kshiftrw $2, %k5, %k0
+; KNL-NEXT:    kmovw %k5, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    andl $1, %eax
+; KNL-NEXT:    leaq (%rdx,%rax,2), %rax
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $3, %k5, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    leaq (%rax,%rdx,4), %rax
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $4, %k5, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    leaq (%rax,%rdx,8), %rax
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $5, %k5, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $4, %rdx
+; KNL-NEXT:    orq %rax, %rdx
+; KNL-NEXT:    kmovw %k0, %eax
+; KNL-NEXT:    kshiftrw $6, %k5, %k0
+; KNL-NEXT:    andl $1, %eax
+; KNL-NEXT:    shlq $5, %rax
+; KNL-NEXT:    orq %rdx, %rax
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $9, %k5, %k0
+; KNL-NEXT:    kshiftrw $8, %k5, %k5
+; KNL-NEXT:    vpmovsxbd %xmm1, %zmm0
+; KNL-NEXT:    vpslld $31, %zmm0, %zmm0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $6, %rdx
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $7, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k5, %edx
+; KNL-NEXT:    vptestmd %zmm0, %zmm0, %k6
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $8, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $15, %k6, %k0
+; KNL-NEXT:    kmovw %k0, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $9, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k1, %edx
+; KNL-NEXT:    kshiftrw $14, %k6, %k1
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $10, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k3, %ecx
+; KNL-NEXT:    kshiftrw $13, %k6, %k3
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $11, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k2, %edx
+; KNL-NEXT:    kshiftrw $12, %k6, %k5
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $12, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $11, %k6, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $13, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k4, %edx
+; KNL-NEXT:    kshiftrw $10, %k6, %k4
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $14, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k0 ## 2-byte Reload
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $1, %k6, %k0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $15, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k6, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $16, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $2, %k6, %k0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $17, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $3, %k6, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $18, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $4, %k6, %k0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $19, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $5, %k6, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $20, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $6, %k6, %k0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $21, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    kshiftrw $7, %k6, %k0
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $22, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    kshiftrw $9, %k6, %k2
+; KNL-NEXT:    kshiftrw $8, %k6, %k0
+; KNL-NEXT:    vpmovsxbd %xmm2, %zmm0
+; KNL-NEXT:    vpslld $31, %zmm0, %zmm0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $23, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
 ; KNL-NEXT:    vptestmd %zmm0, %zmm0, %k0
-; KNL-NEXT:    kmovw %k0, (%rdi)
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $24, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $15, %k0, %k2
+; KNL-NEXT:    kmovw %k2, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $25, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k4, %edx
+; KNL-NEXT:    kshiftrw $14, %k0, %k4
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $26, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $13, %k0, %k6
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $27, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k5, %edx
+; KNL-NEXT:    kshiftrw $12, %k0, %k5
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $28, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k3, %ecx
+; KNL-NEXT:    kshiftrw $11, %k0, %k3
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $29, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k1, %edx
+; KNL-NEXT:    kshiftrw $10, %k0, %k2
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $30, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k1 ## 2-byte Reload
+; KNL-NEXT:    kmovw %k1, %ecx
+; KNL-NEXT:    kshiftrw $1, %k0, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $31, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $32, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $2, %k0, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $33, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k7, %edx
+; KNL-NEXT:    kshiftrw $3, %k0, %k7
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $34, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $4, %k0, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $35, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k7, %edx
+; KNL-NEXT:    kshiftrw $5, %k0, %k7
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $36, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $6, %k0, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $37, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k7, %edx
+; KNL-NEXT:    kshiftrw $7, %k0, %k7
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $38, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k7, %ecx
+; KNL-NEXT:    kshiftrw $9, %k0, %k1
+; KNL-NEXT:    kshiftrw $8, %k0, %k0
+; KNL-NEXT:    vpmovsxbd %xmm3, %zmm0
+; KNL-NEXT:    vpslld $31, %zmm0, %zmm0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $39, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k0, %edx
+; KNL-NEXT:    vptestmd %zmm0, %zmm0, %k7
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $40, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k1, %ecx
+; KNL-NEXT:    kshiftrw $15, %k7, %k0
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $41, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k2, %edx
+; KNL-NEXT:    kshiftrw $14, %k7, %k1
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $42, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k3, %ecx
+; KNL-NEXT:    kshiftrw $13, %k7, %k3
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $43, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k5, %edx
+; KNL-NEXT:    kshiftrw $12, %k7, %k5
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $44, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k6, %ecx
+; KNL-NEXT:    kshiftrw $11, %k7, %k6
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $45, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k4, %edx
+; KNL-NEXT:    kshiftrw $10, %k7, %k4
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $46, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k2 ## 2-byte Reload
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $1, %k7, %k2
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $47, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k7, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $48, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $2, %k7, %k2
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $49, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k2, %edx
+; KNL-NEXT:    kshiftrw $3, %k7, %k2
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $50, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $4, %k7, %k2
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $51, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k2, %edx
+; KNL-NEXT:    kshiftrw $5, %k7, %k2
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $52, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $6, %k7, %k2
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $53, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k2, %edx
+; KNL-NEXT:    kshiftrw $7, %k7, %k2
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $54, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    kshiftrw $9, %k7, %k2
+; KNL-NEXT:    kshiftrw $8, %k7, %k7
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $55, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k7, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $56, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k2, %ecx
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $57, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k4, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $58, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k6, %ecx
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $59, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k5, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $60, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k3, %ecx
+; KNL-NEXT:    andl $1, %ecx
+; KNL-NEXT:    shlq $61, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    kmovw %k1, %edx
+; KNL-NEXT:    andl $1, %edx
+; KNL-NEXT:    shlq $62, %rdx
+; KNL-NEXT:    orq %rcx, %rdx
+; KNL-NEXT:    kmovw %k0, %ecx
+; KNL-NEXT:    shlq $63, %rcx
+; KNL-NEXT:    orq %rdx, %rcx
+; KNL-NEXT:    orq %rax, %rcx
+; KNL-NEXT:    movq %rcx, (%rdi)
 ; KNL-NEXT:    vzeroupper
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: store_64i1:
 ; SKX:       ## %bb.0:
 ; SKX-NEXT:    vpsllw $7, %zmm0, %zmm0
 ; SKX-NEXT:    vpmovb2m %zmm0, %k0
 ; SKX-NEXT:    kmovq %k0, (%rdi)
 ; SKX-NEXT:    vzeroupper
 ; SKX-NEXT:    retq
 ;
 ; AVX512BW-LABEL: store_64i1:
 ; AVX512BW:       ## %bb.0:
 ; AVX512BW-NEXT:    vpsllw $7, %zmm0, %zmm0
 ; AVX512BW-NEXT:    vpmovb2m %zmm0, %k0
 ; AVX512BW-NEXT:    kmovq %k0, (%rdi)
 ; AVX512BW-NEXT:    vzeroupper
 ; AVX512BW-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: store_64i1:
 ; AVX512DQ:       ## %bb.0:
-; AVX512DQ-NEXT:    vpmovsxbd %xmm3, %zmm3
-; AVX512DQ-NEXT:    vpslld $31, %zmm3, %zmm3
-; AVX512DQ-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 6(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm2, %zmm2
-; AVX512DQ-NEXT:    vpslld $31, %zmm2, %zmm2
-; AVX512DQ-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 4(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm1, %zmm1
-; AVX512DQ-NEXT:    vpslld $31, %zmm1, %zmm1
-; AVX512DQ-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 2(%rdi)
 ; AVX512DQ-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; AVX512DQ-NEXT:    vpslld $31, %zmm0, %zmm0
+; AVX512DQ-NEXT:    vptestmd %zmm0, %zmm0, %k5
+; AVX512DQ-NEXT:    kshiftrw $15, %k5, %k0
+; AVX512DQ-NEXT:    kmovw %k0, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; AVX512DQ-NEXT:    kshiftrw $14, %k5, %k4
+; AVX512DQ-NEXT:    kshiftrw $13, %k5, %k7
+; AVX512DQ-NEXT:    kshiftrw $12, %k5, %k2
+; AVX512DQ-NEXT:    kshiftrw $11, %k5, %k3
+; AVX512DQ-NEXT:    kshiftrw $10, %k5, %k1
+; AVX512DQ-NEXT:    kshiftrw $7, %k5, %k0
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $1, %k5, %k0
+; AVX512DQ-NEXT:    kmovw %k0, %eax
+; AVX512DQ-NEXT:    kshiftrw $2, %k5, %k0
+; AVX512DQ-NEXT:    kmovw %k5, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    leaq (%rdx,%rax,2), %rax
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $3, %k5, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    leaq (%rax,%rdx,4), %rax
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $4, %k5, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    leaq (%rax,%rdx,8), %rax
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $5, %k5, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $4, %rdx
+; AVX512DQ-NEXT:    orq %rax, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %eax
+; AVX512DQ-NEXT:    kshiftrw $6, %k5, %k0
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    shlq $5, %rax
+; AVX512DQ-NEXT:    orq %rdx, %rax
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $9, %k5, %k0
+; AVX512DQ-NEXT:    kshiftrw $8, %k5, %k5
+; AVX512DQ-NEXT:    vpmovsxbd %xmm1, %zmm0
+; AVX512DQ-NEXT:    vpslld $31, %zmm0, %zmm0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $6, %rdx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $7, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k5, %edx
+; AVX512DQ-NEXT:    vptestmd %zmm0, %zmm0, %k6
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $8, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $15, %k6, %k0
+; AVX512DQ-NEXT:    kmovw %k0, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $9, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k1, %edx
+; AVX512DQ-NEXT:    kshiftrw $14, %k6, %k1
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $10, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k3, %ecx
+; AVX512DQ-NEXT:    kshiftrw $13, %k6, %k3
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $11, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k2, %edx
+; AVX512DQ-NEXT:    kshiftrw $12, %k6, %k5
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $12, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $11, %k6, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $13, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k4, %edx
+; AVX512DQ-NEXT:    kshiftrw $10, %k6, %k4
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $14, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k0 ## 2-byte Reload
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $1, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $15, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k6, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $16, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $2, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $17, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $3, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $18, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $4, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $19, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $5, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $20, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $6, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $21, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    kshiftrw $7, %k6, %k0
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $22, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    kshiftrw $9, %k6, %k2
+; AVX512DQ-NEXT:    kshiftrw $8, %k6, %k0
+; AVX512DQ-NEXT:    vpmovsxbd %xmm2, %zmm0
+; AVX512DQ-NEXT:    vpslld $31, %zmm0, %zmm0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $23, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
 ; AVX512DQ-NEXT:    vptestmd %zmm0, %zmm0, %k0
-; AVX512DQ-NEXT:    kmovw %k0, (%rdi)
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $24, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $15, %k0, %k2
+; AVX512DQ-NEXT:    kmovw %k2, -{{[0-9]+}}(%rsp) ## 2-byte Spill
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $25, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k4, %edx
+; AVX512DQ-NEXT:    kshiftrw $14, %k0, %k4
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $26, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $13, %k0, %k6
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $27, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k5, %edx
+; AVX512DQ-NEXT:    kshiftrw $12, %k0, %k5
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $28, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k3, %ecx
+; AVX512DQ-NEXT:    kshiftrw $11, %k0, %k3
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $29, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k1, %edx
+; AVX512DQ-NEXT:    kshiftrw $10, %k0, %k2
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $30, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k1 ## 2-byte Reload
+; AVX512DQ-NEXT:    kmovw %k1, %ecx
+; AVX512DQ-NEXT:    kshiftrw $1, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $31, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $32, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $2, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $33, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k7, %edx
+; AVX512DQ-NEXT:    kshiftrw $3, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $34, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $4, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $35, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k7, %edx
+; AVX512DQ-NEXT:    kshiftrw $5, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $36, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $6, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $37, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k7, %edx
+; AVX512DQ-NEXT:    kshiftrw $7, %k0, %k7
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $38, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k7, %ecx
+; AVX512DQ-NEXT:    kshiftrw $9, %k0, %k1
+; AVX512DQ-NEXT:    kshiftrw $8, %k0, %k0
+; AVX512DQ-NEXT:    vpmovsxbd %xmm3, %zmm0
+; AVX512DQ-NEXT:    vpslld $31, %zmm0, %zmm0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $39, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k0, %edx
+; AVX512DQ-NEXT:    vptestmd %zmm0, %zmm0, %k7
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $40, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k1, %ecx
+; AVX512DQ-NEXT:    kshiftrw $15, %k7, %k0
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $41, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k2, %edx
+; AVX512DQ-NEXT:    kshiftrw $14, %k7, %k1
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $42, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k3, %ecx
+; AVX512DQ-NEXT:    kshiftrw $13, %k7, %k3
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $43, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k5, %edx
+; AVX512DQ-NEXT:    kshiftrw $12, %k7, %k5
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $44, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k6, %ecx
+; AVX512DQ-NEXT:    kshiftrw $11, %k7, %k6
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $45, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k4, %edx
+; AVX512DQ-NEXT:    kshiftrw $10, %k7, %k4
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $46, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw -{{[0-9]+}}(%rsp), %k2 ## 2-byte Reload
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $1, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $47, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k7, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $48, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $2, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $49, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k2, %edx
+; AVX512DQ-NEXT:    kshiftrw $3, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $50, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $4, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $51, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k2, %edx
+; AVX512DQ-NEXT:    kshiftrw $5, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $52, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $6, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $53, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k2, %edx
+; AVX512DQ-NEXT:    kshiftrw $7, %k7, %k2
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $54, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    kshiftrw $9, %k7, %k2
+; AVX512DQ-NEXT:    kshiftrw $8, %k7, %k7
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $55, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k7, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $56, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $57, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k4, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $58, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k6, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $59, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k5, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $60, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $61, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    kmovw %k1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $62, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    kmovw %k0, %ecx
+; AVX512DQ-NEXT:    shlq $63, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    orq %rax, %rcx
+; AVX512DQ-NEXT:    movq %rcx, (%rdi)
 ; AVX512DQ-NEXT:    vzeroupper
 ; AVX512DQ-NEXT:    retq
   store <64 x i1> %v, <64 x i1>* %a
   ret void
 }
 
 define i32 @test_bitcast_v8i1_zext(<16 x i32> %a) {
 ; KNL-LABEL: test_bitcast_v8i1_zext:

test/CodeGen/X86/bitcast-and-setcc-512.ll

 ; SSE-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm8
 ; SSE-NEXT:    pand %xmm2, %xmm8
 ; SSE-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm9
 ; SSE-NEXT:    pand %xmm3, %xmm9
 ; SSE-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm10
 ; SSE-NEXT:    pand %xmm0, %xmm10
 ; SSE-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm11
 ; SSE-NEXT:    pand %xmm1, %xmm11
-; SSE-NEXT:    pextrb $15, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm11, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $15, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm11, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm11, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm11, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm11, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm11, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm11, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm11, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm10, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $15, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm10, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm10, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm10, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm10, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm10, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm10, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm10, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm9, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $15, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm9, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm9, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm9, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm9, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm9, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm9, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm9, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm8, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm8, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm8, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm8, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm8, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm8, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm8, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm8, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    movl -{{[0-9]+}}(%rsp), %eax
 ; SSE-NEXT:    shll $16, %eax
 ; SSE-NEXT:    movzwl -{{[0-9]+}}(%rsp), %ecx
 ; SSE-NEXT:    orl %eax, %ecx
 ; SSE-NEXT:    movl -{{[0-9]+}}(%rsp), %edx
 ; SSE-NEXT:    shll $16, %edx
 ; SSE-NEXT:    movzwl -{{[0-9]+}}(%rsp), %eax
 ; SSE-NEXT:    orl %edx, %eax

 ; AVX1-NEXT:    vinsertf128 $1, %xmm0, %ymm2, %ymm0
 ; AVX1-NEXT:    vandps %ymm0, %ymm8, %ymm0
 ; AVX1-NEXT:    vextractf128 $1, %ymm6, %xmm2
 ; AVX1-NEXT:    vextractf128 $1, %ymm4, %xmm3
 ; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm3, %xmm2
 ; AVX1-NEXT:    vpcmpgtb %xmm6, %xmm4, %xmm3
 ; AVX1-NEXT:    vinsertf128 $1, %xmm2, %ymm3, %ymm2
 ; AVX1-NEXT:    vandps %ymm2, %ymm1, %ymm1
-; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm2
-; AVX1-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
 ; AVX1-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm1
-; AVX1-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX1-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX1-NEXT:    vpextrb $3, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX1-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $4, %ecx
+; AVX1-NEXT:    orl %eax, %ecx
 ; AVX1-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shll $5, %eax
+; AVX1-NEXT:    orl %ecx, %eax
+; AVX1-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $6, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $7, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $8, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $9, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $10, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $11, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $12, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $13, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $14, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $15, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm1
+; AVX1-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $16, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $17, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $18, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $19, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $20, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $21, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $22, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $23, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $24, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $25, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $26, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $27, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $28, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $29, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $30, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX1-NEXT:    shll $31, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    orl %eax, %edx
+; AVX1-NEXT:    movl %edx, (%rsp)
 ; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX1-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $4, %ecx
+; AVX1-NEXT:    orl %eax, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shll $5, %eax
+; AVX1-NEXT:    orl %ecx, %eax
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $6, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $7, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $8, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $9, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $10, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $11, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $12, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $13, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $14, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $15, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $16, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $17, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $18, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $19, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $20, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $21, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $22, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $23, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $24, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $25, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $26, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $27, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $28, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $29, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $30, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    shll $31, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    orl %eax, %edx
+; AVX1-NEXT:    movl %edx, {{[0-9]+}}(%rsp)
 ; AVX1-NEXT:    movl (%rsp), %ecx
 ; AVX1-NEXT:    movl {{[0-9]+}}(%rsp), %eax
 ; AVX1-NEXT:    shlq $32, %rax
 ; AVX1-NEXT:    orq %rcx, %rax
 ; AVX1-NEXT:    movq %rbp, %rsp
 ; AVX1-NEXT:    popq %rbp
 ; AVX1-NEXT:    vzeroupper
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: v64i8:
 ; AVX2:       # %bb.0:
 ; AVX2-NEXT:    pushq %rbp
 ; AVX2-NEXT:    .cfi_def_cfa_offset 16
 ; AVX2-NEXT:    .cfi_offset %rbp, -16
 ; AVX2-NEXT:    movq %rsp, %rbp
 ; AVX2-NEXT:    .cfi_def_cfa_register %rbp
 ; AVX2-NEXT:    andq $-32, %rsp
 ; AVX2-NEXT:    subq $64, %rsp
 ; AVX2-NEXT:    vpcmpgtb %ymm3, %ymm1, %ymm1
 ; AVX2-NEXT:    vpcmpgtb %ymm2, %ymm0, %ymm2
 ; AVX2-NEXT:    vpcmpgtb %ymm7, %ymm5, %ymm0
 ; AVX2-NEXT:    vpand %ymm0, %ymm1, %ymm0
 ; AVX2-NEXT:    vpcmpgtb %ymm6, %ymm4, %ymm1
 ; AVX2-NEXT:    vpand %ymm1, %ymm2, %ymm1
-; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm2
-; AVX2-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
 ; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX2-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX2-NEXT:    vpextrb $3, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX2-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $4, %ecx
+; AVX2-NEXT:    orl %eax, %ecx
 ; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shll $5, %eax
+; AVX2-NEXT:    orl %ecx, %eax
+; AVX2-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $6, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $7, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $8, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $9, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $10, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $11, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $12, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $13, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $14, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $15, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm1
+; AVX2-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $16, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $17, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $18, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $19, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $20, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $21, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $22, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $23, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $24, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $25, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $26, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $27, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $28, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $29, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $30, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX2-NEXT:    shll $31, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    orl %eax, %edx
+; AVX2-NEXT:    movl %edx, (%rsp)
 ; AVX2-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX2-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $4, %ecx
+; AVX2-NEXT:    orl %eax, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shll $5, %eax
+; AVX2-NEXT:    orl %ecx, %eax
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $6, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $7, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $8, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $9, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $10, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $11, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $12, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $13, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $14, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $15, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $16, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $17, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $18, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $19, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $20, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $21, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $22, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $23, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $24, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $25, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $26, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $27, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $28, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $29, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $30, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    shll $31, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    orl %eax, %edx
+; AVX2-NEXT:    movl %edx, {{[0-9]+}}(%rsp)
 ; AVX2-NEXT:    movl (%rsp), %ecx
 ; AVX2-NEXT:    movl {{[0-9]+}}(%rsp), %eax
 ; AVX2-NEXT:    shlq $32, %rax
 ; AVX2-NEXT:    orq %rcx, %rax
 ; AVX2-NEXT:    movq %rbp, %rsp
 ; AVX2-NEXT:    popq %rbp
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq

test/CodeGen/X86/bitcast-setcc-512.ll

; AVX512BW-NEXT:    retq
   %res = bitcast <16 x i1> %x to i16
   ret i16 %res
 }
 
 define i64 @v64i8(<64 x i8> %a, <64 x i8> %b) {
 ; SSE-LABEL: v64i8:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    pcmpgtb %xmm5, %xmm1
-; SSE-NEXT:    pextrb $15, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm1, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm1, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm1, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm1, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm1, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm1, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm1, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm1, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pcmpgtb %xmm4, %xmm0
-; SSE-NEXT:    pextrb $15, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm0, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm0, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm0, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm0, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm0, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm0, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm0, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm0, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pcmpgtb %xmm7, %xmm3
-; SSE-NEXT:    pextrb $15, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm3, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm3, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm3, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm3, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm3, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm3, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm3, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm3, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pcmpgtb %xmm6, %xmm2
-; SSE-NEXT:    pextrb $15, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $14, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $13, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $12, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $11, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $10, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $9, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $8, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $7, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $6, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $5, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $4, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $3, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $2, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    pextrb $1, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE-NEXT:    pextrb $0, %xmm2, %eax
-; SSE-NEXT:    andb $1, %al
-; SSE-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE-NEXT:    pextrb $2, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE-NEXT:    pextrb $3, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE-NEXT:    pextrb $4, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $4, %ecx
+; SSE-NEXT:    orl %eax, %ecx
+; SSE-NEXT:    pextrb $5, %xmm2, %eax
+; SSE-NEXT:    andl $1, %eax
+; SSE-NEXT:    shll $5, %eax
+; SSE-NEXT:    orl %ecx, %eax
+; SSE-NEXT:    pextrb $6, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $6, %ecx
+; SSE-NEXT:    pextrb $7, %xmm2, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $7, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $8, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $9, %xmm2, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $9, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $10, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $10, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $11, %xmm2, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $11, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $12, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $12, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $13, %xmm2, %edx
+; SSE-NEXT:    andl $1, %edx
+; SSE-NEXT:    shll $13, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    pextrb $14, %xmm2, %ecx
+; SSE-NEXT:    andl $1, %ecx
+; SSE-NEXT:    shll $14, %ecx
+; SSE-NEXT:    orl %edx, %ecx
+; SSE-NEXT:    pextrb $15, %xmm2, %edx
+; SSE-NEXT:    shll $15, %edx
+; SSE-NEXT:    orl %ecx, %edx
+; SSE-NEXT:    orl %eax, %edx
+; SSE-NEXT:    movw %dx, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    movl -{{[0-9]+}}(%rsp), %eax
 ; SSE-NEXT:    shll $16, %eax
 ; SSE-NEXT:    movzwl -{{[0-9]+}}(%rsp), %ecx
 ; SSE-NEXT:    orl %eax, %ecx
 ; SSE-NEXT:    movl -{{[0-9]+}}(%rsp), %edx
 ; SSE-NEXT:    shll $16, %edx
 ; SSE-NEXT:    movzwl -{{[0-9]+}}(%rsp), %eax
 ; SSE-NEXT:    orl %edx, %eax
 ; SSE-NEXT:    shlq $32, %rax
 ; SSE-NEXT:    orq %rcx, %rax
 ; SSE-NEXT:    retq
 ;
 ; AVX1-LABEL: v64i8:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    pushq %rbp
 ; AVX1-NEXT:    .cfi_def_cfa_offset 16
 ; AVX1-NEXT:    .cfi_offset %rbp, -16
 ; AVX1-NEXT:    movq %rsp, %rbp
 ; AVX1-NEXT:    .cfi_def_cfa_register %rbp
 ; AVX1-NEXT:    andq $-32, %rsp
 ; AVX1-NEXT:    subq $64, %rsp
-; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm5
-; AVX1-NEXT:    vpcmpgtb %xmm4, %xmm5, %xmm4
-; AVX1-NEXT:    vpextrb $15, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
+; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm0, %xmm4
 ; AVX1-NEXT:    vpextrb $1, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX1-NEXT:    vpextrb $2, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX1-NEXT:    vpextrb $3, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX1-NEXT:    vpextrb $4, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $4, %ecx
+; AVX1-NEXT:    orl %eax, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm4, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shll $5, %eax
+; AVX1-NEXT:    orl %ecx, %eax
+; AVX1-NEXT:    vpextrb $6, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $6, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $7, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $8, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $9, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $10, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $11, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $12, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $13, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $14, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $15, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm2
+; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
 ; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm0, %xmm0
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rsp)
-; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm0
-; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm2
-; AVX1-NEXT:    vpcmpgtb %xmm0, %xmm2, %xmm0
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $16, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $17, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $18, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $19, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $20, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $21, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $22, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $23, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $24, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $25, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $26, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $27, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $28, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $29, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $30, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    shll $31, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    orl %eax, %edx
+; AVX1-NEXT:    movl %edx, (%rsp)
 ; AVX1-NEXT:    vpcmpgtb %xmm3, %xmm1, %xmm0
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
 ; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX1-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $4, %ecx
+; AVX1-NEXT:    orl %eax, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shll $5, %eax
+; AVX1-NEXT:    orl %ecx, %eax
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $6, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $7, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $8, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $9, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $10, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $11, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $12, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $13, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $14, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $15, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm0
+; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm1
+; AVX1-NEXT:    vpcmpgtb %xmm0, %xmm1, %xmm0
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $16, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $17, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $18, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $19, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $20, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $21, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $22, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $23, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $24, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $25, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $26, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $27, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $28, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shll $29, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shll $30, %ecx
+; AVX1-NEXT:    orl %edx, %ecx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    shll $31, %edx
+; AVX1-NEXT:    orl %ecx, %edx
+; AVX1-NEXT:    orl %eax, %edx
+; AVX1-NEXT:    movl %edx, {{[0-9]+}}(%rsp)
 ; AVX1-NEXT:    movl (%rsp), %ecx
 ; AVX1-NEXT:    movl {{[0-9]+}}(%rsp), %eax
 ; AVX1-NEXT:    shlq $32, %rax
 ; AVX1-NEXT:    orq %rcx, %rax
 ; AVX1-NEXT:    movq %rbp, %rsp
 ; AVX1-NEXT:    popq %rbp
 ; AVX1-NEXT:    vzeroupper
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: v64i8:
 ; AVX2:       # %bb.0:
 ; AVX2-NEXT:    pushq %rbp
 ; AVX2-NEXT:    .cfi_def_cfa_offset 16
 ; AVX2-NEXT:    .cfi_offset %rbp, -16
 ; AVX2-NEXT:    movq %rsp, %rbp
 ; AVX2-NEXT:    .cfi_def_cfa_register %rbp
 ; AVX2-NEXT:    andq $-32, %rsp
 ; AVX2-NEXT:    subq $64, %rsp
 ; AVX2-NEXT:    vpcmpgtb %ymm2, %ymm0, %ymm0
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm2
-; AVX2-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
 ; AVX2-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rsp)
-; AVX2-NEXT:    vpcmpgtb %ymm3, %ymm1, %ymm0
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX2-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $4, %ecx
+; AVX2-NEXT:    orl %eax, %ecx
 ; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shll $5, %eax
+; AVX2-NEXT:    orl %ecx, %eax
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $6, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $7, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $8, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $9, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $10, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $11, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $12, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $13, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $14, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $15, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $16, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $17, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $18, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $19, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $20, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $21, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $22, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $23, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $24, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $25, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $26, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $27, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $28, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $29, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $30, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    shll $31, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    orl %eax, %edx
+; AVX2-NEXT:    movl %edx, (%rsp)
+; AVX2-NEXT:    vpcmpgtb %ymm3, %ymm1, %ymm0
 ; AVX2-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, {{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rcx,%rax,2), %eax
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,4), %eax
+; AVX2-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leal (%rax,%rcx,8), %eax
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $4, %ecx
+; AVX2-NEXT:    orl %eax, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shll $5, %eax
+; AVX2-NEXT:    orl %ecx, %eax
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $6, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $7, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $8, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $9, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $10, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $11, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $12, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $13, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $14, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $15, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $16, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $17, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $18, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $19, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $20, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $21, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $22, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $23, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $24, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $25, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $26, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $27, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $28, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shll $29, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shll $30, %ecx
+; AVX2-NEXT:    orl %edx, %ecx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    shll $31, %edx
+; AVX2-NEXT:    orl %ecx, %edx
+; AVX2-NEXT:    orl %eax, %edx
+; AVX2-NEXT:    movl %edx, {{[0-9]+}}(%rsp)
 ; AVX2-NEXT:    movl (%rsp), %ecx
 ; AVX2-NEXT:    movl {{[0-9]+}}(%rsp), %eax
 ; AVX2-NEXT:    shlq $32, %rax
 ; AVX2-NEXT:    orq %rcx, %rax
 ; AVX2-NEXT:    movq %rbp, %rsp
 ; AVX2-NEXT:    popq %rbp
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq

test/CodeGen/X86/clear_upper_vector_element_bits.ll

; AVX-NEXT:    retq
   %r15 = insertelement <32 x i8> %r14, i8 zeroinitializer, i32 31
   %r = bitcast <32 x i8> %r15 to <16 x i16>
   ret <16 x i16> %r
 }
 
 define <16 x i8> @_clearupper16xi8b(<16 x i8>) nounwind {
 ; SSE2-LABEL: _clearupper16xi8b:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pushq %r14
 ; SSE2-NEXT:    pushq %rbx
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
-; SSE2-NEXT:    movq %xmm0, %rcx
-; SSE2-NEXT:    movq %rcx, %r8
-; SSE2-NEXT:    movq %rcx, %r9
-; SSE2-NEXT:    movq %rcx, %r10
-; SSE2-NEXT:    movq %rcx, %rax
-; SSE2-NEXT:    movq %rcx, %rdx
-; SSE2-NEXT:    movq %rcx, %rsi
-; SSE2-NEXT:    movq %rcx, %rdi
-; SSE2-NEXT:    andb $15, %cl
-; SSE2-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %xmm1, %rcx
-; SSE2-NEXT:    shrq $56, %rdi
-; SSE2-NEXT:    andb $15, %dil
-; SSE2-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %r11
-; SSE2-NEXT:    shrq $48, %rsi
-; SSE2-NEXT:    andb $15, %sil
-; SSE2-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %r14
-; SSE2-NEXT:    shrq $40, %rdx
-; SSE2-NEXT:    andb $15, %dl
-; SSE2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rdx
-; SSE2-NEXT:    shrq $32, %rax
-; SSE2-NEXT:    andb $15, %al
-; SSE2-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rax
-; SSE2-NEXT:    shrq $24, %r10
-; SSE2-NEXT:    andb $15, %r10b
-; SSE2-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rdi
-; SSE2-NEXT:    shrq $16, %r9
-; SSE2-NEXT:    andb $15, %r9b
-; SSE2-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rsi
-; SSE2-NEXT:    shrq $8, %r8
-; SSE2-NEXT:    andb $15, %r8b
-; SSE2-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rbx
-; SSE2-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    andb $15, %cl
-; SSE2-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $56, %rbx
-; SSE2-NEXT:    andb $15, %bl
-; SSE2-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $48, %rsi
-; SSE2-NEXT:    andb $15, %sil
-; SSE2-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movq %xmm1, %r10
+; SSE2-NEXT:    movq %r10, %r8
+; SSE2-NEXT:    shrq $56, %r8
+; SSE2-NEXT:    andl $15, %r8d
+; SSE2-NEXT:    movq %r10, %r9
+; SSE2-NEXT:    shrq $48, %r9
+; SSE2-NEXT:    andl $15, %r9d
+; SSE2-NEXT:    movq %r10, %rsi
+; SSE2-NEXT:    shrq $40, %rsi
+; SSE2-NEXT:    andl $15, %esi
+; SSE2-NEXT:    movq %r10, %r11
+; SSE2-NEXT:    shrq $32, %r11
+; SSE2-NEXT:    andl $15, %r11d
+; SSE2-NEXT:    movq %xmm0, %rax
+; SSE2-NEXT:    movq %rax, %rdx
+; SSE2-NEXT:    shrq $56, %rdx
+; SSE2-NEXT:    andl $15, %edx
+; SSE2-NEXT:    movq %rax, %rcx
+; SSE2-NEXT:    shrq $48, %rcx
+; SSE2-NEXT:    andl $15, %ecx
+; SSE2-NEXT:    movq %rax, %rdi
 ; SSE2-NEXT:    shrq $40, %rdi
-; SSE2-NEXT:    andb $15, %dil
-; SSE2-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $32, %rax
-; SSE2-NEXT:    andb $15, %al
-; SSE2-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $24, %rdx
-; SSE2-NEXT:    andb $15, %dl
-; SSE2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $16, %r14
-; SSE2-NEXT:    andb $15, %r14b
-; SSE2-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $8, %r11
-; SSE2-NEXT:    andb $15, %r11b
-; SSE2-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; SSE2-NEXT:    movsd {{.*#+}} xmm1 = mem[0],zero
-; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; SSE2-NEXT:    andl $15, %edi
+; SSE2-NEXT:    movq %rax, %rbx
+; SSE2-NEXT:    shrq $32, %rbx
+; SSE2-NEXT:    andl $15, %ebx
+; SSE2-NEXT:    shlq $32, %rbx
+; SSE2-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; SSE2-NEXT:    orq %rbx, %rax
+; SSE2-NEXT:    shlq $40, %rdi
+; SSE2-NEXT:    orq %rax, %rdi
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdi, %rcx
+; SSE2-NEXT:    shlq $56, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    shlq $32, %r11
+; SSE2-NEXT:    andl $252645135, %r10d # imm = 0xF0F0F0F
+; SSE2-NEXT:    orq %r11, %r10
+; SSE2-NEXT:    shlq $40, %rsi
+; SSE2-NEXT:    orq %r10, %rsi
+; SSE2-NEXT:    shlq $48, %r9
+; SSE2-NEXT:    orq %rsi, %r9
+; SSE2-NEXT:    shlq $56, %r8
+; SSE2-NEXT:    orq %r9, %r8
+; SSE2-NEXT:    movq %rdx, %xmm0
+; SSE2-NEXT:    movq %r8, %xmm1
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE2-NEXT:    popq %rbx
-; SSE2-NEXT:    popq %r14
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: _clearupper16xi8b:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pushq %r14
 ; SSE42-NEXT:    pushq %rbx
-; SSE42-NEXT:    movq %xmm0, %rcx
-; SSE42-NEXT:    movq %rcx, %r8
-; SSE42-NEXT:    movq %rcx, %r9
-; SSE42-NEXT:    movq %rcx, %r10
-; SSE42-NEXT:    movq %rcx, %rax
-; SSE42-NEXT:    movq %rcx, %rdx
-; SSE42-NEXT:    movq %rcx, %rsi
-; SSE42-NEXT:    movq %rcx, %rdi
-; SSE42-NEXT:    andb $15, %cl
-; SSE42-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    pextrq $1, %xmm0, %rcx
-; SSE42-NEXT:    shrq $56, %rdi
-; SSE42-NEXT:    andb $15, %dil
-; SSE42-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %r11
-; SSE42-NEXT:    shrq $48, %rsi
-; SSE42-NEXT:    andb $15, %sil
-; SSE42-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %r14
-; SSE42-NEXT:    shrq $40, %rdx
-; SSE42-NEXT:    andb $15, %dl
-; SSE42-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rdx
-; SSE42-NEXT:    shrq $32, %rax
-; SSE42-NEXT:    andb $15, %al
-; SSE42-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rax
-; SSE42-NEXT:    shrq $24, %r10
-; SSE42-NEXT:    andb $15, %r10b
-; SSE42-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rdi
-; SSE42-NEXT:    shrq $16, %r9
-; SSE42-NEXT:    andb $15, %r9b
-; SSE42-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rsi
-; SSE42-NEXT:    shrq $8, %r8
-; SSE42-NEXT:    andb $15, %r8b
-; SSE42-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rbx
-; SSE42-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    andb $15, %cl
-; SSE42-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $56, %rbx
-; SSE42-NEXT:    andb $15, %bl
-; SSE42-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $48, %rsi
-; SSE42-NEXT:    andb $15, %sil
-; SSE42-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; SSE42-NEXT:    pextrq $1, %xmm0, %r10
+; SSE42-NEXT:    movq %r10, %r8
+; SSE42-NEXT:    shrq $56, %r8
+; SSE42-NEXT:    andl $15, %r8d
+; SSE42-NEXT:    movq %r10, %r9
+; SSE42-NEXT:    shrq $48, %r9
+; SSE42-NEXT:    andl $15, %r9d
+; SSE42-NEXT:    movq %r10, %rsi
+; SSE42-NEXT:    shrq $40, %rsi
+; SSE42-NEXT:    andl $15, %esi
+; SSE42-NEXT:    movq %r10, %r11
+; SSE42-NEXT:    shrq $32, %r11
+; SSE42-NEXT:    andl $15, %r11d
+; SSE42-NEXT:    movq %xmm0, %rax
+; SSE42-NEXT:    movq %rax, %rdx
+; SSE42-NEXT:    shrq $56, %rdx
+; SSE42-NEXT:    andl $15, %edx
+; SSE42-NEXT:    movq %rax, %rcx
+; SSE42-NEXT:    shrq $48, %rcx
+; SSE42-NEXT:    andl $15, %ecx
+; SSE42-NEXT:    movq %rax, %rdi
 ; SSE42-NEXT:    shrq $40, %rdi
-; SSE42-NEXT:    andb $15, %dil
-; SSE42-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $32, %rax
-; SSE42-NEXT:    andb $15, %al
-; SSE42-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $24, %rdx
-; SSE42-NEXT:    andb $15, %dl
-; SSE42-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $16, %r14
-; SSE42-NEXT:    andb $15, %r14b
-; SSE42-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $8, %r11
-; SSE42-NEXT:    andb $15, %r11b
-; SSE42-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movsd {{.*#+}} xmm1 = mem[0],zero
-; SSE42-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; SSE42-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; SSE42-NEXT:    andl $15, %edi
+; SSE42-NEXT:    movq %rax, %rbx
+; SSE42-NEXT:    shrq $32, %rbx
+; SSE42-NEXT:    andl $15, %ebx
+; SSE42-NEXT:    shlq $32, %rbx
+; SSE42-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; SSE42-NEXT:    orq %rbx, %rax
+; SSE42-NEXT:    shlq $40, %rdi
+; SSE42-NEXT:    orq %rax, %rdi
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdi, %rcx
+; SSE42-NEXT:    shlq $56, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    shlq $32, %r11
+; SSE42-NEXT:    andl $252645135, %r10d # imm = 0xF0F0F0F
+; SSE42-NEXT:    orq %r11, %r10
+; SSE42-NEXT:    shlq $40, %rsi
+; SSE42-NEXT:    orq %r10, %rsi
+; SSE42-NEXT:    shlq $48, %r9
+; SSE42-NEXT:    orq %rsi, %r9
+; SSE42-NEXT:    shlq $56, %r8
+; SSE42-NEXT:    orq %r9, %r8
+; SSE42-NEXT:    movq %r8, %xmm1
+; SSE42-NEXT:    movq %rdx, %xmm0
+; SSE42-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE42-NEXT:    popq %rbx
-; SSE42-NEXT:    popq %r14
 ; SSE42-NEXT:    retq
 ;
 ; AVX-LABEL: _clearupper16xi8b:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    pushq %rbp
-; AVX-NEXT:    pushq %r15
-; AVX-NEXT:    pushq %r14
-; AVX-NEXT:    pushq %r13
-; AVX-NEXT:    pushq %r12
 ; AVX-NEXT:    pushq %rbx
 ; AVX-NEXT:    vmovaps %xmm0, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx
+; AVX-NEXT:    movq -{{[0-9]+}}(%rsp), %r9
 ; AVX-NEXT:    movq -{{[0-9]+}}(%rsp), %rdx
-; AVX-NEXT:    movq %rcx, %r8
-; AVX-NEXT:    movq %rcx, %r9
-; AVX-NEXT:    movq %rcx, %r10
-; AVX-NEXT:    movq %rcx, %r11
-; AVX-NEXT:    movq %rcx, %r14
-; AVX-NEXT:    movq %rcx, %r15
-; AVX-NEXT:    movq %rdx, %r12
-; AVX-NEXT:    movq %rdx, %r13
+; AVX-NEXT:    movq %r9, %r8
+; AVX-NEXT:    shrq $56, %r8
+; AVX-NEXT:    andl $15, %r8d
+; AVX-NEXT:    movq %r9, %r10
+; AVX-NEXT:    shrq $48, %r10
+; AVX-NEXT:    andl $15, %r10d
+; AVX-NEXT:    movq %r9, %rsi
+; AVX-NEXT:    shrq $40, %rsi
+; AVX-NEXT:    andl $15, %esi
+; AVX-NEXT:    movq %r9, %r11
+; AVX-NEXT:    shrq $32, %r11
+; AVX-NEXT:    andl $15, %r11d
 ; AVX-NEXT:    movq %rdx, %rdi
+; AVX-NEXT:    shrq $56, %rdi
+; AVX-NEXT:    andl $15, %edi
 ; AVX-NEXT:    movq %rdx, %rax
-; AVX-NEXT:    movq %rdx, %rsi
+; AVX-NEXT:    shrq $48, %rax
+; AVX-NEXT:    andl $15, %eax
+; AVX-NEXT:    movq %rdx, %rcx
+; AVX-NEXT:    shrq $40, %rcx
+; AVX-NEXT:    andl $15, %ecx
 ; AVX-NEXT:    movq %rdx, %rbx
-; AVX-NEXT:    movq %rdx, %rbp
-; AVX-NEXT:    andb $15, %dl
-; AVX-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    movq %rcx, %rdx
-; AVX-NEXT:    andb $15, %cl
-; AVX-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $56, %rbp
-; AVX-NEXT:    andb $15, %bpl
-; AVX-NEXT:    movb %bpl, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $48, %rbx
-; AVX-NEXT:    andb $15, %bl
-; AVX-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $40, %rsi
-; AVX-NEXT:    andb $15, %sil
-; AVX-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $32, %rax
-; AVX-NEXT:    andb $15, %al
-; AVX-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $24, %rdi
-; AVX-NEXT:    andb $15, %dil
-; AVX-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $16, %r13
-; AVX-NEXT:    andb $15, %r13b
-; AVX-NEXT:    movb %r13b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $8, %r12
-; AVX-NEXT:    andb $15, %r12b
-; AVX-NEXT:    movb %r12b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $56, %rdx
-; AVX-NEXT:    andb $15, %dl
-; AVX-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $48, %r15
-; AVX-NEXT:    andb $15, %r15b
-; AVX-NEXT:    movb %r15b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $40, %r14
-; AVX-NEXT:    andb $15, %r14b
-; AVX-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $32, %r11
-; AVX-NEXT:    andb $15, %r11b
-; AVX-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $24, %r10
-; AVX-NEXT:    andb $15, %r10b
-; AVX-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $16, %r9
-; AVX-NEXT:    andb $15, %r9b
-; AVX-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    shrq $8, %r8
-; AVX-NEXT:    andb $15, %r8b
-; AVX-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
-; AVX-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
+; AVX-NEXT:    shrq $32, %rbx
+; AVX-NEXT:    andl $15, %ebx
+; AVX-NEXT:    shlq $32, %rbx
+; AVX-NEXT:    andl $252645135, %edx # imm = 0xF0F0F0F
+; AVX-NEXT:    orq %rbx, %rdx
+; AVX-NEXT:    shlq $40, %rcx
+; AVX-NEXT:    orq %rdx, %rcx
+; AVX-NEXT:    shlq $48, %rax
+; AVX-NEXT:    orq %rcx, %rax
+; AVX-NEXT:    shlq $56, %rdi
+; AVX-NEXT:    orq %rax, %rdi
+; AVX-NEXT:    movq %rdi, -{{[0-9]+}}(%rsp)
+; AVX-NEXT:    shlq $32, %r11
+; AVX-NEXT:    andl $252645135, %r9d # imm = 0xF0F0F0F
+; AVX-NEXT:    orq %r11, %r9
+; AVX-NEXT:    shlq $40, %rsi
+; AVX-NEXT:    orq %r9, %rsi
+; AVX-NEXT:    shlq $48, %r10
+; AVX-NEXT:    orq %rsi, %r10
+; AVX-NEXT:    shlq $56, %r8
+; AVX-NEXT:    orq %r10, %r8
+; AVX-NEXT:    movq %r8, -{{[0-9]+}}(%rsp)
 ; AVX-NEXT:    vmovaps -{{[0-9]+}}(%rsp), %xmm0
 ; AVX-NEXT:    popq %rbx
-; AVX-NEXT:    popq %r12
-; AVX-NEXT:    popq %r13
-; AVX-NEXT:    popq %r14
-; AVX-NEXT:    popq %r15
-; AVX-NEXT:    popq %rbp
 ; AVX-NEXT:    retq
   %x4  = bitcast <16 x i8> %0 to <32 x i4>
   %r0  = insertelement <32 x i4> %x4,  i4 zeroinitializer, i32 1
   %r1  = insertelement <32 x i4> %r0,  i4 zeroinitializer, i32 3
   %r2  = insertelement <32 x i4> %r1,  i4 zeroinitializer, i32 5
   %r3  = insertelement <32 x i4> %r2,  i4 zeroinitializer, i32 7
   %r4  = insertelement <32 x i4> %r3,  i4 zeroinitializer, i32 9
   %r5  = insertelement <32 x i4> %r4,  i4 zeroinitializer, i32 11
; AVX-NEXT:    retq
   %r15 = insertelement <32 x i4> %r14, i4 zeroinitializer, i32 31
   %r = bitcast <32 x i4> %r15 to <16 x i8>
   ret <16 x i8> %r
 }
 
 define <32 x i8> @_clearupper32xi8b(<32 x i8>) nounwind {
 ; SSE2-LABEL: _clearupper32xi8b:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pushq %r14
 ; SSE2-NEXT:    pushq %rbx
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
-; SSE2-NEXT:    movq %xmm0, %rcx
-; SSE2-NEXT:    movq %rcx, %r8
-; SSE2-NEXT:    movq %rcx, %r9
-; SSE2-NEXT:    movq %rcx, %r10
-; SSE2-NEXT:    movq %rcx, %rax
-; SSE2-NEXT:    movq %rcx, %rdx
-; SSE2-NEXT:    movq %rcx, %rsi
-; SSE2-NEXT:    movq %rcx, %rdi
-; SSE2-NEXT:    andb $15, %cl
-; SSE2-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %xmm2, %rcx
-; SSE2-NEXT:    shrq $56, %rdi
-; SSE2-NEXT:    andb $15, %dil
-; SSE2-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %r11
-; SSE2-NEXT:    shrq $48, %rsi
-; SSE2-NEXT:    andb $15, %sil
-; SSE2-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %r14
-; SSE2-NEXT:    shrq $40, %rdx
-; SSE2-NEXT:    andb $15, %dl
-; SSE2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rdx
-; SSE2-NEXT:    shrq $32, %rax
-; SSE2-NEXT:    andb $15, %al
-; SSE2-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rax
-; SSE2-NEXT:    shrq $24, %r10
-; SSE2-NEXT:    andb $15, %r10b
-; SSE2-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rdi
-; SSE2-NEXT:    shrq $16, %r9
-; SSE2-NEXT:    andb $15, %r9b
-; SSE2-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rsi
-; SSE2-NEXT:    shrq $8, %r8
-; SSE2-NEXT:    andb $15, %r8b
-; SSE2-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movq %rcx, %rbx
-; SSE2-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    andb $15, %cl
-; SSE2-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $56, %rbx
-; SSE2-NEXT:    andb $15, %bl
-; SSE2-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $48, %rsi
-; SSE2-NEXT:    andb $15, %sil
-; SSE2-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movq %xmm2, %r10
+; SSE2-NEXT:    movq %r10, %r8
+; SSE2-NEXT:    shrq $56, %r8
+; SSE2-NEXT:    andl $15, %r8d
+; SSE2-NEXT:    movq %r10, %r9
+; SSE2-NEXT:    shrq $48, %r9
+; SSE2-NEXT:    andl $15, %r9d
+; SSE2-NEXT:    movq %r10, %rsi
+; SSE2-NEXT:    shrq $40, %rsi
+; SSE2-NEXT:    andl $15, %esi
+; SSE2-NEXT:    movq %r10, %r11
+; SSE2-NEXT:    shrq $32, %r11
+; SSE2-NEXT:    andl $15, %r11d
+; SSE2-NEXT:    movq %xmm0, %rax
+; SSE2-NEXT:    movq %rax, %rdx
+; SSE2-NEXT:    shrq $56, %rdx
+; SSE2-NEXT:    andl $15, %edx
+; SSE2-NEXT:    movq %rax, %rcx
+; SSE2-NEXT:    shrq $48, %rcx
+; SSE2-NEXT:    andl $15, %ecx
+; SSE2-NEXT:    movq %rax, %rdi
 ; SSE2-NEXT:    shrq $40, %rdi
-; SSE2-NEXT:    andb $15, %dil
-; SSE2-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $32, %rax
-; SSE2-NEXT:    andb $15, %al
-; SSE2-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $24, %rdx
-; SSE2-NEXT:    andb $15, %dl
-; SSE2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $16, %r14
-; SSE2-NEXT:    andb $15, %r14b
-; SSE2-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    shrq $8, %r11
-; SSE2-NEXT:    andb $15, %r11b
-; SSE2-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; SSE2-NEXT:    movsd {{.*#+}} xmm2 = mem[0],zero
-; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm2[0]
+; SSE2-NEXT:    andl $15, %edi
+; SSE2-NEXT:    movq %rax, %rbx
+; SSE2-NEXT:    shrq $32, %rbx
+; SSE2-NEXT:    andl $15, %ebx
+; SSE2-NEXT:    shlq $32, %rbx
+; SSE2-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; SSE2-NEXT:    orq %rbx, %rax
+; SSE2-NEXT:    shlq $40, %rdi
+; SSE2-NEXT:    orq %rax, %rdi
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdi, %rcx
+; SSE2-NEXT:    shlq $56, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    shlq $32, %r11
+; SSE2-NEXT:    andl $252645135, %r10d # imm = 0xF0F0F0F
+; SSE2-NEXT:    orq %r11, %r10
+; SSE2-NEXT:    shlq $40, %rsi
+; SSE2-NEXT:    orq %r10, %rsi
+; SSE2-NEXT:    shlq $48, %r9
+; SSE2-NEXT:    orq %rsi, %r9
+; SSE2-NEXT:    shlq $56, %r8
+; SSE2-NEXT:    orq %r9, %r8
+; SSE2-NEXT:    movq %rdx, %xmm0
+; SSE2-NEXT:    movq %r8, %xmm2
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; SSE2-NEXT:    popq %rbx
-; SSE2-NEXT:    popq %r14
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: _clearupper32xi8b:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pushq %r14
 ; SSE42-NEXT:    pushq %rbx
-; SSE42-NEXT:    movq %xmm0, %rcx
-; SSE42-NEXT:    movq %rcx, %r8
-; SSE42-NEXT:    movq %rcx, %r9
-; SSE42-NEXT:    movq %rcx, %r10
-; SSE42-NEXT:    movq %rcx, %rax
-; SSE42-NEXT:    movq %rcx, %rdx
-; SSE42-NEXT:    movq %rcx, %rsi
-; SSE42-NEXT:    movq %rcx, %rdi
-; SSE42-NEXT:    andb $15, %cl
-; SSE42-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    pextrq $1, %xmm0, %rcx
-; SSE42-NEXT:    shrq $56, %rdi
-; SSE42-NEXT:    andb $15, %dil
-; SSE42-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %r11
-; SSE42-NEXT:    shrq $48, %rsi
-; SSE42-NEXT:    andb $15, %sil
-; SSE42-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %r14
-; SSE42-NEXT:    shrq $40, %rdx
-; SSE42-NEXT:    andb $15, %dl
-; SSE42-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rdx
-; SSE42-NEXT:    shrq $32, %rax
-; SSE42-NEXT:    andb $15, %al
-; SSE42-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rax
-; SSE42-NEXT:    shrq $24, %r10
-; SSE42-NEXT:    andb $15, %r10b
-; SSE42-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rdi
-; SSE42-NEXT:    shrq $16, %r9
-; SSE42-NEXT:    andb $15, %r9b
-; SSE42-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rsi
-; SSE42-NEXT:    shrq $8, %r8
-; SSE42-NEXT:    andb $15, %r8b
-; SSE42-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movq %rcx, %rbx
-; SSE42-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    andb $15, %cl
-; SSE42-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $56, %rbx
-; SSE42-NEXT:    andb $15, %bl
-; SSE42-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $48, %rsi
-; SSE42-NEXT:    andb $15, %sil
-; SSE42-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; SSE42-NEXT:    pextrq $1, %xmm0, %r10
+; SSE42-NEXT:    movq %r10, %r8
+; SSE42-NEXT:    shrq $56, %r8
+; SSE42-NEXT:    andl $15, %r8d
+; SSE42-NEXT:    movq %r10, %r9
+; SSE42-NEXT:    shrq $48, %r9
+; SSE42-NEXT:    andl $15, %r9d
+; SSE42-NEXT:    movq %r10, %rsi
+; SSE42-NEXT:    shrq $40, %rsi
+; SSE42-NEXT:    andl $15, %esi
+; SSE42-NEXT:    movq %r10, %r11
+; SSE42-NEXT:    shrq $32, %r11
+; SSE42-NEXT:    andl $15, %r11d
+; SSE42-NEXT:    movq %xmm0, %rax
+; SSE42-NEXT:    movq %rax, %rdx
+; SSE42-NEXT:    shrq $56, %rdx
+; SSE42-NEXT:    andl $15, %edx
+; SSE42-NEXT:    movq %rax, %rcx
+; SSE42-NEXT:    shrq $48, %rcx
+; SSE42-NEXT:    andl $15, %ecx
+; SSE42-NEXT:    movq %rax, %rdi
 ; SSE42-NEXT:    shrq $40, %rdi
-; SSE42-NEXT:    andb $15, %dil
-; SSE42-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $32, %rax
-; SSE42-NEXT:    andb $15, %al
-; SSE42-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $24, %rdx
-; SSE42-NEXT:    andb $15, %dl
-; SSE42-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $16, %r14
-; SSE42-NEXT:    andb $15, %r14b
-; SSE42-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    shrq $8, %r11
-; SSE42-NEXT:    andb $15, %r11b
-; SSE42-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
-; SSE42-NEXT:    movsd {{.*#+}} xmm2 = mem[0],zero
-; SSE42-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; SSE42-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm2[0]
+; SSE42-NEXT:    andl $15, %edi
+; SSE42-NEXT:    movq %rax, %rbx
+; SSE42-NEXT:    shrq $32, %rbx
+; SSE42-NEXT:    andl $15, %ebx
+; SSE42-NEXT:    shlq $32, %rbx
+; SSE42-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; SSE42-NEXT:    orq %rbx, %rax
+; SSE42-NEXT:    shlq $40, %rdi
+; SSE42-NEXT:    orq %rax, %rdi
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdi, %rcx
+; SSE42-NEXT:    shlq $56, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    shlq $32, %r11
+; SSE42-NEXT:    andl $252645135, %r10d # imm = 0xF0F0F0F
+; SSE42-NEXT:    orq %r11, %r10
+; SSE42-NEXT:    shlq $40, %rsi
+; SSE42-NEXT:    orq %r10, %rsi
+; SSE42-NEXT:    shlq $48, %r9
+; SSE42-NEXT:    orq %rsi, %r9
+; SSE42-NEXT:    shlq $56, %r8
+; SSE42-NEXT:    orq %r9, %r8
+; SSE42-NEXT:    movq %r8, %xmm2
+; SSE42-NEXT:    movq %rdx, %xmm0
+; SSE42-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; SSE42-NEXT:    popq %rbx
-; SSE42-NEXT:    popq %r14
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: _clearupper32xi8b:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    pushq %rbp
-; AVX1-NEXT:    pushq %r15
-; AVX1-NEXT:    pushq %r14
-; AVX1-NEXT:    pushq %r13
-; AVX1-NEXT:    pushq %r12
-; AVX1-NEXT:    pushq %rbx
 ; AVX1-NEXT:    vmovaps %xmm0, -{{[0-9]+}}(%rsp)
+; AVX1-NEXT:    movq -{{[0-9]+}}(%rsp), %r9
 ; AVX1-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx
-; AVX1-NEXT:    movq -{{[0-9]+}}(%rsp), %rdx
-; AVX1-NEXT:    movq %rcx, %r8
-; AVX1-NEXT:    movq %rcx, %r9
-; AVX1-NEXT:    movq %rcx, %r10
-; AVX1-NEXT:    movq %rcx, %r11
-; AVX1-NEXT:    movq %rcx, %r14
-; AVX1-NEXT:    movq %rcx, %r15
-; AVX1-NEXT:    movq %rdx, %r12
-; AVX1-NEXT:    movq %rdx, %r13
-; AVX1-NEXT:    movq %rdx, %rdi
-; AVX1-NEXT:    movq %rdx, %rax
-; AVX1-NEXT:    movq %rdx, %rsi
-; AVX1-NEXT:    movq %rdx, %rbx
-; AVX1-NEXT:    movq %rdx, %rbp
-; AVX1-NEXT:    andb $15, %dl
-; AVX1-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
+; AVX1-NEXT:    movq %r9, %r8
+; AVX1-NEXT:    shrq $56, %r8
+; AVX1-NEXT:    andl $15, %r8d
+; AVX1-NEXT:    movq %rcx, %rsi
+; AVX1-NEXT:    movq %rcx, %rdi
 ; AVX1-NEXT:    movq %rcx, %rdx
-; AVX1-NEXT:    andb $15, %cl
-; AVX1-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $56, %rbp
-; AVX1-NEXT:    andb $15, %bpl
-; AVX1-NEXT:    movb %bpl, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $48, %rbx
-; AVX1-NEXT:    andb $15, %bl
-; AVX1-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $40, %rsi
-; AVX1-NEXT:    andb $15, %sil
-; AVX1-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; AVX1-NEXT:    movq %rcx, %rax
 ; AVX1-NEXT:    shrq $32, %rax
-; AVX1-NEXT:    andb $15, %al
-; AVX1-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $24, %rdi
-; AVX1-NEXT:    andb $15, %dil
-; AVX1-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $16, %r13
-; AVX1-NEXT:    andb $15, %r13b
-; AVX1-NEXT:    movb %r13b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $8, %r12
-; AVX1-NEXT:    andb $15, %r12b
-; AVX1-NEXT:    movb %r12b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    shrq $8, %r8
-; AVX1-NEXT:    shrq $16, %r9
-; AVX1-NEXT:    shrq $24, %r10
-; AVX1-NEXT:    shrq $32, %r11
-; AVX1-NEXT:    shrq $40, %r14
-; AVX1-NEXT:    shrq $48, %r15
-; AVX1-NEXT:    shrq $56, %rdx
-; AVX1-NEXT:    andb $15, %dl
-; AVX1-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r15b
-; AVX1-NEXT:    movb %r15b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r14b
-; AVX1-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r11b
-; AVX1-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r10b
-; AVX1-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r9b
-; AVX1-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; AVX1-NEXT:    andb $15, %r8b
-; AVX1-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
+; AVX1-NEXT:    andl $15, %eax
+; AVX1-NEXT:    shlq $32, %rax
+; AVX1-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
+; AVX1-NEXT:    orq %rax, %rcx
+; AVX1-NEXT:    movq %r9, %rax
+; AVX1-NEXT:    shrq $48, %rax
+; AVX1-NEXT:    andl $15, %eax
+; AVX1-NEXT:    shrq $40, %rdx
+; AVX1-NEXT:    andl $15, %edx
+; AVX1-NEXT:    shlq $40, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    movq %r9, %rcx
+; AVX1-NEXT:    shrq $40, %rcx
+; AVX1-NEXT:    andl $15, %ecx
+; AVX1-NEXT:    shrq $48, %rdi
+; AVX1-NEXT:    andl $15, %edi
+; AVX1-NEXT:    shlq $48, %rdi
+; AVX1-NEXT:    orq %rdx, %rdi
+; AVX1-NEXT:    movq %r9, %rdx
+; AVX1-NEXT:    shrq $32, %rdx
+; AVX1-NEXT:    andl $15, %edx
+; AVX1-NEXT:    shrq $56, %rsi
+; AVX1-NEXT:    andl $15, %esi
+; AVX1-NEXT:    shlq $56, %rsi
+; AVX1-NEXT:    orq %rdi, %rsi
+; AVX1-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp)
+; AVX1-NEXT:    shlq $32, %rdx
+; AVX1-NEXT:    andl $252645135, %r9d # imm = 0xF0F0F0F
+; AVX1-NEXT:    orq %rdx, %r9
+; AVX1-NEXT:    shlq $40, %rcx
+; AVX1-NEXT:    orq %r9, %rcx
+; AVX1-NEXT:    shlq $48, %rax
+; AVX1-NEXT:    orq %rcx, %rax
+; AVX1-NEXT:    shlq $56, %r8
+; AVX1-NEXT:    orq %rax, %r8
+; AVX1-NEXT:    movq %r8, -{{[0-9]+}}(%rsp)
 ; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
 ; AVX1-NEXT:    vmovq %xmm0, %rax
-; AVX1-NEXT:    movq %rax, %rcx
-; AVX1-NEXT:    movq %rax, %rdx
+; AVX1-NEXT:    movq %rax, %r8
+; AVX1-NEXT:    movq %rax, %r9
 ; AVX1-NEXT:    movq %rax, %rsi
 ; AVX1-NEXT:    movq %rax, %rdi
-; AVX1-NEXT:    movl %eax, %ebp
-; AVX1-NEXT:    movl %eax, %ebx
+; AVX1-NEXT:    movl %eax, %ecx
+; AVX1-NEXT:    movl %eax, %edx
 ; AVX1-NEXT:    vmovd %eax, %xmm1
 ; AVX1-NEXT:    shrl $8, %eax
 ; AVX1-NEXT:    vpinsrb $1, %eax, %xmm1, %xmm1
-; AVX1-NEXT:    shrl $16, %ebx
-; AVX1-NEXT:    vpinsrb $2, %ebx, %xmm1, %xmm1
-; AVX1-NEXT:    shrl $24, %ebp
-; AVX1-NEXT:    vpinsrb $3, %ebp, %xmm1, %xmm1
+; AVX1-NEXT:    shrl $16, %edx
+; AVX1-NEXT:    vpinsrb $2, %edx, %xmm1, %xmm1
+; AVX1-NEXT:    shrl $24, %ecx
+; AVX1-NEXT:    vpinsrb $3, %ecx, %xmm1, %xmm1
 ; AVX1-NEXT:    shrq $32, %rdi
 ; AVX1-NEXT:    vpinsrb $4, %edi, %xmm1, %xmm1
 ; AVX1-NEXT:    shrq $40, %rsi
 ; AVX1-NEXT:    vpinsrb $5, %esi, %xmm1, %xmm1
-; AVX1-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
 ; AVX1-NEXT:    vmovdqa -{{[0-9]+}}(%rsp), %xmm2
-; AVX1-NEXT:    shrq $48, %rdx
-; AVX1-NEXT:    vpinsrb $6, %edx, %xmm1, %xmm1
+; AVX1-NEXT:    shrq $48, %r9
+; AVX1-NEXT:    vpinsrb $6, %r9d, %xmm1, %xmm1
 ; AVX1-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX1-NEXT:    shrq $56, %rcx
-; AVX1-NEXT:    vpinsrb $7, %ecx, %xmm1, %xmm0
+; AVX1-NEXT:    shrq $56, %r8
+; AVX1-NEXT:    vpinsrb $7, %r8d, %xmm1, %xmm0
 ; AVX1-NEXT:    movl %eax, %ecx
 ; AVX1-NEXT:    shrl $8, %ecx
 ; AVX1-NEXT:    vpinsrb $8, %eax, %xmm0, %xmm0
 ; AVX1-NEXT:    vpinsrb $9, %ecx, %xmm0, %xmm0
 ; AVX1-NEXT:    movl %eax, %ecx
 ; AVX1-NEXT:    shrl $16, %ecx
 ; AVX1-NEXT:    vpinsrb $10, %ecx, %xmm0, %xmm0
 ; AVX1-NEXT:    movl %eax, %ecx

 ; AVX1-NEXT:    shrq $40, %rcx
 ; AVX1-NEXT:    vpinsrb $13, %ecx, %xmm1, %xmm1
 ; AVX1-NEXT:    movq %rax, %rcx
 ; AVX1-NEXT:    shrq $48, %rcx
 ; AVX1-NEXT:    vpinsrb $14, %ecx, %xmm1, %xmm1
 ; AVX1-NEXT:    shrq $56, %rax
 ; AVX1-NEXT:    vpinsrb $15, %eax, %xmm1, %xmm1
 ; AVX1-NEXT:    vinsertf128 $1, %xmm0, %ymm1, %ymm0
-; AVX1-NEXT:    popq %rbx
-; AVX1-NEXT:    popq %r12
-; AVX1-NEXT:    popq %r13
-; AVX1-NEXT:    popq %r14
-; AVX1-NEXT:    popq %r15
-; AVX1-NEXT:    popq %rbp
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: _clearupper32xi8b:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    pushq %rbp
-; AVX2-NEXT:    pushq %r15
-; AVX2-NEXT:    pushq %r14
-; AVX2-NEXT:    pushq %r13
-; AVX2-NEXT:    pushq %r12
-; AVX2-NEXT:    pushq %rbx
 ; AVX2-NEXT:    vmovdqa %xmm0, -{{[0-9]+}}(%rsp)
+; AVX2-NEXT:    movq -{{[0-9]+}}(%rsp), %r9
 ; AVX2-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx
-; AVX2-NEXT:    movq -{{[0-9]+}}(%rsp), %rdx
-; AVX2-NEXT:    movq %rcx, %r8
-; AVX2-NEXT:    movq %rcx, %r9
-; AVX2-NEXT:    movq %rcx, %r10
-; AVX2-NEXT:    movq %rcx, %r11
-; AVX2-NEXT:    movq %rcx, %r14
-; AVX2-NEXT:    movq %rcx, %r15
-; AVX2-NEXT:    movq %rdx, %r12
-; AVX2-NEXT:    movq %rdx, %r13
-; AVX2-NEXT:    movq %rdx, %rdi
-; AVX2-NEXT:    movq %rdx, %rax
-; AVX2-NEXT:    movq %rdx, %rsi
-; AVX2-NEXT:    movq %rdx, %rbx
-; AVX2-NEXT:    movq %rdx, %rbp
-; AVX2-NEXT:    andb $15, %dl
-; AVX2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
+; AVX2-NEXT:    movq %r9, %r8
+; AVX2-NEXT:    shrq $56, %r8
+; AVX2-NEXT:    andl $15, %r8d
+; AVX2-NEXT:    movq %rcx, %rsi
+; AVX2-NEXT:    movq %rcx, %rdi
 ; AVX2-NEXT:    movq %rcx, %rdx
-; AVX2-NEXT:    andb $15, %cl
-; AVX2-NEXT:    movb %cl, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $56, %rbp
-; AVX2-NEXT:    andb $15, %bpl
-; AVX2-NEXT:    movb %bpl, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $48, %rbx
-; AVX2-NEXT:    andb $15, %bl
-; AVX2-NEXT:    movb %bl, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $40, %rsi
-; AVX2-NEXT:    andb $15, %sil
-; AVX2-NEXT:    movb %sil, -{{[0-9]+}}(%rsp)
+; AVX2-NEXT:    movq %rcx, %rax
 ; AVX2-NEXT:    shrq $32, %rax
-; AVX2-NEXT:    andb $15, %al
-; AVX2-NEXT:    movb %al, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $24, %rdi
-; AVX2-NEXT:    andb $15, %dil
-; AVX2-NEXT:    movb %dil, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $16, %r13
-; AVX2-NEXT:    andb $15, %r13b
-; AVX2-NEXT:    movb %r13b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $8, %r12
-; AVX2-NEXT:    andb $15, %r12b
-; AVX2-NEXT:    movb %r12b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    shrq $8, %r8
-; AVX2-NEXT:    shrq $16, %r9
-; AVX2-NEXT:    shrq $24, %r10
-; AVX2-NEXT:    shrq $32, %r11
-; AVX2-NEXT:    shrq $40, %r14
-; AVX2-NEXT:    shrq $48, %r15
-; AVX2-NEXT:    shrq $56, %rdx
-; AVX2-NEXT:    andb $15, %dl
-; AVX2-NEXT:    movb %dl, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r15b
-; AVX2-NEXT:    movb %r15b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r14b
-; AVX2-NEXT:    movb %r14b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r11b
-; AVX2-NEXT:    movb %r11b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r10b
-; AVX2-NEXT:    movb %r10b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r9b
-; AVX2-NEXT:    movb %r9b, -{{[0-9]+}}(%rsp)
-; AVX2-NEXT:    andb $15, %r8b
-; AVX2-NEXT:    movb %r8b, -{{[0-9]+}}(%rsp)
+; AVX2-NEXT:    andl $15, %eax
+; AVX2-NEXT:    shlq $32, %rax
+; AVX2-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
+; AVX2-NEXT:    orq %rax, %rcx
+; AVX2-NEXT:    movq %r9, %rax
+; AVX2-NEXT:    shrq $48, %rax
+; AVX2-NEXT:    andl $15, %eax
+; AVX2-NEXT:    shrq $40, %rdx
+; AVX2-NEXT:    andl $15, %edx
+; AVX2-NEXT:    shlq $40, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    movq %r9, %rcx
+; AVX2-NEXT:    shrq $40, %rcx
+; AVX2-NEXT:    andl $15, %ecx
+; AVX2-NEXT:    shrq $48, %rdi
+; AVX2-NEXT:    andl $15, %edi
+; AVX2-NEXT:    shlq $48, %rdi
+; AVX2-NEXT:    orq %rdx, %rdi
+; AVX2-NEXT:    movq %r9, %rdx
+; AVX2-NEXT:    shrq $32, %rdx
+; AVX2-NEXT:    andl $15, %edx
+; AVX2-NEXT:    shrq $56, %rsi
+; AVX2-NEXT:    andl $15, %esi
+; AVX2-NEXT:    shlq $56, %rsi
+; AVX2-NEXT:    orq %rdi, %rsi
+; AVX2-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp)
+; AVX2-NEXT:    shlq $32, %rdx
+; AVX2-NEXT:    andl $252645135, %r9d # imm = 0xF0F0F0F
+; AVX2-NEXT:    orq %rdx, %r9
+; AVX2-NEXT:    shlq $40, %rcx
+; AVX2-NEXT:    orq %r9, %rcx
+; AVX2-NEXT:    shlq $48, %rax
+; AVX2-NEXT:    orq %rcx, %rax
+; AVX2-NEXT:    shlq $56, %r8
+; AVX2-NEXT:    orq %rax, %r8
+; AVX2-NEXT:    movq %r8, -{{[0-9]+}}(%rsp)
 ; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
 ; AVX2-NEXT:    vmovq %xmm0, %rax
-; AVX2-NEXT:    movq %rax, %rcx
-; AVX2-NEXT:    movq %rax, %rdx
+; AVX2-NEXT:    movq %rax, %r8
+; AVX2-NEXT:    movq %rax, %r9
 ; AVX2-NEXT:    movq %rax, %rsi
 ; AVX2-NEXT:    movq %rax, %rdi
-; AVX2-NEXT:    movl %eax, %ebp
-; AVX2-NEXT:    movl %eax, %ebx
+; AVX2-NEXT:    movl %eax, %ecx
+; AVX2-NEXT:    movl %eax, %edx
 ; AVX2-NEXT:    vmovd %eax, %xmm1
 ; AVX2-NEXT:    shrl $8, %eax
 ; AVX2-NEXT:    vpinsrb $1, %eax, %xmm1, %xmm1
-; AVX2-NEXT:    shrl $16, %ebx
-; AVX2-NEXT:    vpinsrb $2, %ebx, %xmm1, %xmm1
-; AVX2-NEXT:    shrl $24, %ebp
-; AVX2-NEXT:    vpinsrb $3, %ebp, %xmm1, %xmm1
+; AVX2-NEXT:    shrl $16, %edx
+; AVX2-NEXT:    vpinsrb $2, %edx, %xmm1, %xmm1
+; AVX2-NEXT:    shrl $24, %ecx
+; AVX2-NEXT:    vpinsrb $3, %ecx, %xmm1, %xmm1
 ; AVX2-NEXT:    shrq $32, %rdi
 ; AVX2-NEXT:    vpinsrb $4, %edi, %xmm1, %xmm1
 ; AVX2-NEXT:    shrq $40, %rsi
 ; AVX2-NEXT:    vpinsrb $5, %esi, %xmm1, %xmm1
-; AVX2-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
 ; AVX2-NEXT:    vmovdqa -{{[0-9]+}}(%rsp), %xmm2
-; AVX2-NEXT:    shrq $48, %rdx
-; AVX2-NEXT:    vpinsrb $6, %edx, %xmm1, %xmm1
+; AVX2-NEXT:    shrq $48, %r9
+; AVX2-NEXT:    vpinsrb $6, %r9d, %xmm1, %xmm1
 ; AVX2-NEXT:    vpextrq $1, %xmm0, %rax
-; AVX2-NEXT:    shrq $56, %rcx
-; AVX2-NEXT:    vpinsrb $7, %ecx, %xmm1, %xmm0
+; AVX2-NEXT:    shrq $56, %r8
+; AVX2-NEXT:    vpinsrb $7, %r8d, %xmm1, %xmm0
 ; AVX2-NEXT:    movl %eax, %ecx
 ; AVX2-NEXT:    shrl $8, %ecx
 ; AVX2-NEXT:    vpinsrb $8, %eax, %xmm0, %xmm0
 ; AVX2-NEXT:    vpinsrb $9, %ecx, %xmm0, %xmm0
 ; AVX2-NEXT:    movl %eax, %ecx
 ; AVX2-NEXT:    shrl $16, %ecx
 ; AVX2-NEXT:    vpinsrb $10, %ecx, %xmm0, %xmm0
 ; AVX2-NEXT:    movl %eax, %ecx

 ; AVX2-NEXT:    shrq $40, %rcx
 ; AVX2-NEXT:    vpinsrb $13, %ecx, %xmm1, %xmm1
 ; AVX2-NEXT:    movq %rax, %rcx
 ; AVX2-NEXT:    shrq $48, %rcx
 ; AVX2-NEXT:    vpinsrb $14, %ecx, %xmm1, %xmm1
 ; AVX2-NEXT:    shrq $56, %rax
 ; AVX2-NEXT:    vpinsrb $15, %eax, %xmm1, %xmm1
 ; AVX2-NEXT:    vinserti128 $1, %xmm0, %ymm1, %ymm0
-; AVX2-NEXT:    popq %rbx
-; AVX2-NEXT:    popq %r12
-; AVX2-NEXT:    popq %r13
-; AVX2-NEXT:    popq %r14
-; AVX2-NEXT:    popq %r15
-; AVX2-NEXT:    popq %rbp
 ; AVX2-NEXT:    retq
   %x4  = bitcast <32 x i8> %0 to <64 x i4>
   %r0  = insertelement <64 x i4> %x4,  i4 zeroinitializer, i32 1
   %r1  = insertelement <64 x i4> %r0,  i4 zeroinitializer, i32 3
   %r2  = insertelement <64 x i4> %r1,  i4 zeroinitializer, i32 5
   %r3  = insertelement <64 x i4> %r2,  i4 zeroinitializer, i32 7
   %r4  = insertelement <64 x i4> %r3,  i4 zeroinitializer, i32 9
   %r5  = insertelement <64 x i4> %r4,  i4 zeroinitializer, i32 11

test/CodeGen/X86/pr20011.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-linux-gnu | FileCheck %s --check-prefix=X86
 ; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefix=X64
 
 %destTy = type { i2, i2 }
 
 define void @crash(i64 %x0, i64 %y0, %destTy* nocapture %dest) nounwind {
 ; X86-LABEL: crash:
 ; X86:       # %bb.0:
-; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movb {{[0-9]+}}(%esp), %cl
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %dl
-; X86-NEXT:    andb $3, %dl
-; X86-NEXT:    movb %dl, (%ecx)
-; X86-NEXT:    andb $3, %al
-; X86-NEXT:    movb %al, (%ecx)
+; X86-NEXT:    shlb $2, %dl
+; X86-NEXT:    andb $3, %cl
+; X86-NEXT:    orb %dl, %cl
+; X86-NEXT:    andb $15, %cl
+; X86-NEXT:    movb %cl, (%eax)
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: crash:
 ; X64:       # %bb.0:
-; X64-NEXT:    andl $3, %esi
-; X64-NEXT:    movb %sil, (%rdx)
-; X64-NEXT:    andl $3, %edi
+; X64-NEXT:    shlb $2, %sil
+; X64-NEXT:    andb $3, %dil
+; X64-NEXT:    orb %sil, %dil
+; X64-NEXT:    andb $15, %dil
 ; X64-NEXT:    movb %dil, (%rdx)
 ; X64-NEXT:    retq
   %x1 = trunc i64 %x0 to i2
   %y1 = trunc i64 %y0 to i2
   %1 = bitcast %destTy* %dest to <2 x i2>*
   %2 = insertelement <2 x i2> undef, i2 %x1, i32 0
   %3 = insertelement <2 x i2> %2, i2 %y1, i32 1
   store <2 x i2> %3, <2 x i2>* %1, align 1
   ret void
 }

test/CodeGen/X86/trunc-store.ll

 ; }
 
 define void @fn1() {
 ; CHECK-LABEL: fn1:
 ; CHECK:       # %bb.0: # %for.cond
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_1: # %vector.body
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    movb $0, -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    cmpq $8, %rax
 ; CHECK-NEXT:    jne .LBB0_1
 ; CHECK-NEXT:  # %bb.2: # %middle.block
 ; CHECK-NEXT:    retq
 for.cond:
   br label %vector.body
 
 vector.body:                                      ; preds = %vector.body, %for.cond

test/CodeGen/X86/vector-compare-results.ll

 ; AVX512BW-NEXT:    retq
   %1 = icmp sgt <16 x i16> %a0, %a1
   ret <16 x i1> %1
 }
 
 define <32 x i1> @test_cmp_v32i8(<32 x i8> %a0, <32 x i8> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v32i8:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pcmpgtb %xmm2, %xmm0
 ; SSE2-NEXT:    pcmpgtb %xmm3, %xmm1
-; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    pcmpgtb %xmm2, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32i8:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtb %xmm2, %xmm0
 ; SSE42-NEXT:    pcmpgtb %xmm3, %xmm1
-; SSE42-NEXT:    pextrb $15, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    pcmpgtb %xmm2, %xmm0
 ; SSE42-NEXT:    pextrb $1, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE42-NEXT:    pextrb $2, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE42-NEXT:    pextrb $3, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE42-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $4, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $5, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $5, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $6, %ecx
+; SSE42-NEXT:    pextrb $7, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $7, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $8, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $9, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $9, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $10, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $10, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $11, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $11, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $12, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $13, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $13, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $14, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $14, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $15, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $15, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $1, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $17, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $2, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $3, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $19, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $20, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $5, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $21, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $6, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $22, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $7, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $23, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $24, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $9, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $25, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $10, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $26, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $11, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $27, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $28, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $13, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $29, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $14, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $30, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $15, %xmm1, %edx
+; SSE42-NEXT:    shll $31, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    orl %eax, %edx
+; SSE42-NEXT:    movl %edx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v32i8:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm2
 ; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm3
 ; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm3, %xmm2

 ; AVX512BW-NEXT:    retq
   %1 = icmp sgt <16 x i32> %a0, %a1
   ret <16 x i1> %1
 }
 
 define <32 x i1> @test_cmp_v32i16(<32 x i16> %a0, <32 x i16> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v32i16:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pcmpgtw %xmm5, %xmm1
-; SSE2-NEXT:    pcmpgtw %xmm4, %xmm0
-; SSE2-NEXT:    packsswb %xmm1, %xmm0
 ; SSE2-NEXT:    pcmpgtw %xmm7, %xmm3
 ; SSE2-NEXT:    pcmpgtw %xmm6, %xmm2
-; SSE2-NEXT:    packsswb %xmm3, %xmm2
-; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    pcmpgtw %xmm5, %xmm1
+; SSE2-NEXT:    pcmpgtw %xmm4, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32i16:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtw %xmm4, %xmm0
-; SSE42-NEXT:    pcmpgtw %xmm5, %xmm1
-; SSE42-NEXT:    pcmpgtw %xmm6, %xmm2
 ; SSE42-NEXT:    pcmpgtw %xmm7, %xmm3
-; SSE42-NEXT:    pextrb $14, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    pcmpgtw %xmm6, %xmm2
+; SSE42-NEXT:    pcmpgtw %xmm5, %xmm1
+; SSE42-NEXT:    pcmpgtw %xmm4, %xmm0
 ; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE42-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE42-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $4, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $10, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $5, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $6, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $7, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $8, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $2, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $9, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $10, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $6, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $11, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $12, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $10, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $13, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $14, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $15, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $2, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $17, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $4, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $6, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $19, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $20, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $10, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $21, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $22, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $23, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $24, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $2, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $25, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $4, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $26, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $6, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $27, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $28, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $10, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $29, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $30, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm3, %edx
+; SSE42-NEXT:    shll $31, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    orl %eax, %edx
+; SSE42-NEXT:    movl %edx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v32i16:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm4
 ; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm5
 ; AVX1-NEXT:    vpcmpgtw %xmm4, %xmm5, %xmm4

 ; AVX512BW-NEXT:    retq
   %1 = icmp sgt <32 x i16> %a0, %a1
   ret <32 x i1> %1
 }
 
 define <64 x i1> @test_cmp_v64i8(<64 x i8> %a0, <64 x i8> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v64i8:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pcmpgtb %xmm4, %xmm0
-; SSE2-NEXT:    pcmpgtb %xmm5, %xmm1
-; SSE2-NEXT:    pcmpgtb %xmm6, %xmm2
 ; SSE2-NEXT:    pcmpgtb %xmm7, %xmm3
-; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 6(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 4(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    pcmpgtb %xmm6, %xmm2
+; SSE2-NEXT:    pcmpgtb %xmm5, %xmm1
+; SSE2-NEXT:    pcmpgtb %xmm4, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $4, %rcx
+; SSE2-NEXT:    orq %rax, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shlq $5, %rax
+; SSE2-NEXT:    orq %rcx, %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $6, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $7, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $8, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $9, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $10, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $11, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $12, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $13, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $14, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $15, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $16, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $17, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $18, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $19, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $20, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $21, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $22, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $23, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $24, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $25, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $26, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $27, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $28, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $29, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $30, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $31, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $32, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $33, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $34, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $35, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $36, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $37, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $38, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $39, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $40, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $41, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $42, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $43, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $44, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $45, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $46, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $47, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $49, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $50, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $51, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $52, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $53, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $54, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $55, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $56, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $57, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $58, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $59, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $60, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $61, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $62, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shlq $63, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    orq %rax, %rdx
+; SSE2-NEXT:    movq %rdx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v64i8:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtb %xmm4, %xmm0
-; SSE42-NEXT:    pcmpgtb %xmm5, %xmm1
-; SSE42-NEXT:    pcmpgtb %xmm6, %xmm2
 ; SSE42-NEXT:    pcmpgtb %xmm7, %xmm3
-; SSE42-NEXT:    pextrb $15, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    pcmpgtb %xmm6, %xmm2
+; SSE42-NEXT:    pcmpgtb %xmm5, %xmm1
+; SSE42-NEXT:    pcmpgtb %xmm4, %xmm0
 ; SSE42-NEXT:    pextrb $1, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE42-NEXT:    pextrb $2, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE42-NEXT:    pextrb $3, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE42-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $4, %rcx
+; SSE42-NEXT:    orq %rax, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shlq $5, %rax
+; SSE42-NEXT:    orq %rcx, %rax
+; SSE42-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $6, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $7, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $8, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $9, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $10, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $11, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $12, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $13, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $14, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $15, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $16, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $17, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $18, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $19, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $20, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $21, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $22, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $23, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $24, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $25, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $26, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $27, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $28, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $29, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $30, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $31, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $32, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $33, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $34, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $35, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $36, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $37, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $38, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $39, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $40, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $41, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $42, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $43, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $44, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $45, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $46, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $47, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $49, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $50, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $51, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $52, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $53, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $54, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $55, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $56, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $57, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $58, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $59, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $60, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $61, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $62, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm3, %edx
+; SSE42-NEXT:    shlq $63, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    orq %rax, %rdx
+; SSE42-NEXT:    movq %rdx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v64i8:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm0, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm2
-; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
-; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm0, %xmm0
-; AVX1-NEXT:    vpcmpgtb %xmm3, %xmm1, %xmm2
-; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm3
-; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm1
+; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm4
+; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm5
+; AVX1-NEXT:    vpcmpgtb %xmm4, %xmm5, %xmm4
 ; AVX1-NEXT:    vpcmpgtb %xmm3, %xmm1, %xmm1
-; AVX1-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm3
+; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm5
+; AVX1-NEXT:    vpcmpgtb %xmm3, %xmm5, %xmm3
+; AVX1-NEXT:    vpcmpgtb %xmm2, %xmm0, %xmm0
 ; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX1-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $4, %rcx
+; AVX1-NEXT:    orq %rax, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shlq $5, %rax
+; AVX1-NEXT:    orq %rcx, %rax
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $6, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $7, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $8, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $9, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $10, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $11, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $12, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $13, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $14, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $15, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $16, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $17, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $18, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $19, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $20, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $21, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $22, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $23, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $24, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $25, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $26, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $27, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $28, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $29, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $30, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $31, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $32, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $33, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $34, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $35, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $36, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $37, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $38, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $39, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $40, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $41, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $42, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $43, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $44, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $45, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $46, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $47, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $48, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $49, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $50, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $51, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $52, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $53, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $54, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $55, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $56, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $57, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $58, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $59, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $60, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $61, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $62, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm4, %edx
+; AVX1-NEXT:    shlq $63, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    orq %rax, %rdx
+; AVX1-NEXT:    movq %rdx, (%rdi)
 ; AVX1-NEXT:    movq %rdi, %rax
 ; AVX1-NEXT:    vzeroupper
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: test_cmp_v64i8:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    vpcmpgtb %ymm2, %ymm0, %ymm0
 ; AVX2-NEXT:    vpcmpgtb %ymm3, %ymm1, %ymm1
-; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm2
-; AVX2-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    vpcmpgtb %ymm2, %ymm0, %ymm0
 ; AVX2-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX2-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $4, %rcx
+; AVX2-NEXT:    orq %rax, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shlq $5, %rax
+; AVX2-NEXT:    orq %rcx, %rax
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $6, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $7, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $8, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $9, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $10, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $11, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $12, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $13, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $14, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $15, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $16, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $17, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $18, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $19, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $20, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $21, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $22, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $23, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $24, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $25, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $26, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $27, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $28, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $29, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $30, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $31, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $32, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $33, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $34, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $35, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $36, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $37, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $38, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $39, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $40, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $41, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $42, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $43, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $44, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $45, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $46, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $47, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $48, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $49, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $50, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $51, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $52, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $53, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $54, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $55, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $56, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $57, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $58, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $59, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $60, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $61, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $62, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    shlq $63, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    orq %rax, %rdx
+; AVX2-NEXT:    movq %rdx, (%rdi)
 ; AVX2-NEXT:    movq %rdi, %rax
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq
 ;
 ; AVX512F-LABEL: test_cmp_v64i8:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpcmpgtb %ymm3, %ymm1, %ymm4
 ; AVX512F-NEXT:    vpcmpgtb %ymm2, %ymm0, %ymm0

 ; AVX512BW-NEXT:    retq
   %1 = fcmp ogt <16 x double> %a0, %a1
   ret <16 x i1> %1
 }
 
 define <32 x i1> @test_cmp_v32f32(<32 x float> %a0, <32 x float> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v32f32:
 ; SSE2:       # %bb.0:
+; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm8
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm9
-; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm11
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm10
+; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm11
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm12
-; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm8
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm13
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm14
 ; SSE2-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm15
-; SSE2-NEXT:    cmpltps %xmm3, %xmm15
-; SSE2-NEXT:    cmpltps %xmm2, %xmm14
+; SSE2-NEXT:    cmpltps %xmm7, %xmm15
+; SSE2-NEXT:    cmpltps %xmm6, %xmm14
 ; SSE2-NEXT:    packssdw %xmm15, %xmm14
-; SSE2-NEXT:    cmpltps %xmm1, %xmm13
+; SSE2-NEXT:    cmpltps %xmm5, %xmm13
+; SSE2-NEXT:    cmpltps %xmm4, %xmm12
+; SSE2-NEXT:    packssdw %xmm13, %xmm12
+; SSE2-NEXT:    cmpltps %xmm3, %xmm11
+; SSE2-NEXT:    cmpltps %xmm2, %xmm10
+; SSE2-NEXT:    packssdw %xmm11, %xmm10
+; SSE2-NEXT:    cmpltps %xmm1, %xmm9
 ; SSE2-NEXT:    cmpltps %xmm0, %xmm8
-; SSE2-NEXT:    packssdw %xmm13, %xmm8
-; SSE2-NEXT:    packsswb %xmm14, %xmm8
-; SSE2-NEXT:    cmpltps %xmm7, %xmm12
-; SSE2-NEXT:    cmpltps %xmm6, %xmm10
-; SSE2-NEXT:    packssdw %xmm12, %xmm10
-; SSE2-NEXT:    cmpltps %xmm5, %xmm11
-; SSE2-NEXT:    cmpltps %xmm4, %xmm9
-; SSE2-NEXT:    packssdw %xmm11, %xmm9
-; SSE2-NEXT:    packsswb %xmm10, %xmm9
-; SSE2-NEXT:    movdqa %xmm9, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    packssdw %xmm9, %xmm8
 ; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm10, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm14, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32f32:
 ; SSE42:       # %bb.0:
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm15
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm14
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm13
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm12
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm11
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm10
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm9
 ; SSE42-NEXT:    movaps {{[0-9]+}}(%rsp), %xmm8
-; SSE42-NEXT:    cmpltps %xmm0, %xmm8
-; SSE42-NEXT:    cmpltps %xmm1, %xmm9
-; SSE42-NEXT:    cmpltps %xmm2, %xmm10
-; SSE42-NEXT:    cmpltps %xmm3, %xmm11
-; SSE42-NEXT:    cmpltps %xmm4, %xmm12
-; SSE42-NEXT:    cmpltps %xmm5, %xmm13
-; SSE42-NEXT:    cmpltps %xmm6, %xmm14
-; SSE42-NEXT:    cmpltps %xmm7, %xmm15
-; SSE42-NEXT:    pextrb $12, %xmm15, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm15, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
+; SSE42-NEXT:    cmpltps %xmm7, %xmm8
+; SSE42-NEXT:    cmpltps %xmm6, %xmm9
+; SSE42-NEXT:    cmpltps %xmm5, %xmm10
+; SSE42-NEXT:    cmpltps %xmm4, %xmm11
+; SSE42-NEXT:    cmpltps %xmm3, %xmm12
+; SSE42-NEXT:    cmpltps %xmm2, %xmm13
+; SSE42-NEXT:    cmpltps %xmm1, %xmm14
+; SSE42-NEXT:    cmpltps %xmm0, %xmm15
 ; SSE42-NEXT:    pextrb $4, %xmm15, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm15, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm14, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm14, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm15, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE42-NEXT:    pextrb $8, %xmm15, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE42-NEXT:    pextrb $12, %xmm15, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE42-NEXT:    pextrb $0, %xmm14, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $4, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
 ; SSE42-NEXT:    pextrb $4, %xmm14, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm14, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm13, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm13, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm13, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm13, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm12, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm12, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm12, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm12, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm11, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm11, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm11, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm11, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm10, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm10, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm10, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm10, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm9, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm9, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm9, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm9, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $5, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $8, %xmm14, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $6, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm14, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $7, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm13, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $8, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm13, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $9, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm13, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $10, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm13, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $11, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm12, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $12, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm12, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $13, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm12, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $14, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm12, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $15, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm11, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm11, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $17, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm11, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm11, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $19, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm10, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $20, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm10, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $21, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm10, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $22, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm10, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $23, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $24, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $25, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $26, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $27, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $28, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $29, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $30, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm8, %edx
+; SSE42-NEXT:    shll $31, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    orl %eax, %edx
+; SSE42-NEXT:    movl %edx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v32f32:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    vcmpltps %ymm3, %ymm7, %ymm3
 ; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm7
 ; AVX1-NEXT:    vpackssdw %xmm7, %xmm3, %xmm3

 ; AVX512BW-NEXT:    retq
   %1 = icmp sgt <16 x i64> %a0, %a1
   ret <16 x i1> %1
 }
 
 define <32 x i1> @test_cmp_v32i32(<32 x i32> %a0, <32 x i32> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v32i32:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    packssdw %xmm3, %xmm2
-; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT:    packssdw %xmm1, %xmm0
-; SSE2-NEXT:    packsswb %xmm2, %xmm0
 ; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm7
 ; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm6
 ; SSE2-NEXT:    packssdw %xmm7, %xmm6
 ; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm5
 ; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm4
 ; SSE2-NEXT:    packssdw %xmm5, %xmm4
-; SSE2-NEXT:    packsswb %xmm6, %xmm4
-; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm3
+; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm2
+; SSE2-NEXT:    packssdw %xmm3, %xmm2
+; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm1
+; SSE2-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm0
+; SSE2-NEXT:    packssdw %xmm1, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32i32:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm1
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm2
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm5
-; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm6
 ; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm7
-; SSE42-NEXT:    pextrb $12, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm6
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm4
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm3
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm2
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm1
+; SSE42-NEXT:    pcmpgtd {{[0-9]+}}(%rsp), %xmm0
 ; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $4, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm1, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $5, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $6, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $7, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $8, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $9, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $10, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $11, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $12, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $13, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $14, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $15, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $17, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $19, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $20, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $21, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $22, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $23, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $24, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $25, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $26, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $27, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $28, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $4, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $29, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $30, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $12, %xmm7, %edx
+; SSE42-NEXT:    shll $31, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    orl %eax, %edx
+; SSE42-NEXT:    movl %edx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v32i32:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm8
 ; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm9
 ; AVX1-NEXT:    vpcmpgtd %xmm8, %xmm9, %xmm8

 ; AVX512BW-NEXT:    retq
   %1 = icmp sgt <32 x i32> %a0, %a1
   ret <32 x i1> %1
 }
 
 define <64 x i1> @test_cmp_v64i16(<64 x i16> %a0, <64 x i16> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v64i16:
 ; SSE2:       # %bb.0:
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT:    packsswb %xmm1, %xmm0
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    packsswb %xmm3, %xmm2
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm5
-; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    packsswb %xmm5, %xmm4
+; SSE2-NEXT:    pushq %rax
 ; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm7
 ; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm6
-; SSE2-NEXT:    packsswb %xmm7, %xmm6
-; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 6(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 4(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm4
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm3
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm2
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm1
+; SSE2-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $4, %rcx
+; SSE2-NEXT:    orq %rax, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shlq $5, %rax
+; SSE2-NEXT:    orq %rcx, %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $6, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $7, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $8, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $9, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $10, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $11, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $12, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $13, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $14, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $15, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $16, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $17, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $18, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $19, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $20, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $21, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $22, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $23, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $24, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $25, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $26, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $27, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $28, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $29, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $30, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $31, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $32, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $33, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $34, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $35, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $36, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $37, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $38, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $39, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $40, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $41, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $42, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $43, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $44, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $45, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $46, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $47, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $49, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $50, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $51, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $52, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $53, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $54, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $55, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $56, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $57, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $58, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $59, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $60, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $61, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $62, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shlq $63, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    orq %rax, %rdx
+; SSE2-NEXT:    movq %rdx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
+; SSE2-NEXT:    popq %rcx
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v64i16:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm1
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm2
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm5
-; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm6
 ; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm7
-; SSE42-NEXT:    pextrb $14, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm6
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm4
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm3
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm2
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm1
+; SSE42-NEXT:    pcmpgtw {{[0-9]+}}(%rsp), %xmm0
 ; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE42-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE42-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $4, %rcx
+; SSE42-NEXT:    orq %rax, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shlq $5, %rax
+; SSE42-NEXT:    orq %rcx, %rax
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $6, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $7, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $8, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $9, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $10, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $11, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $12, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $13, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $14, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $15, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $16, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $17, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $18, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $19, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $20, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $21, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $22, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $23, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $24, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $25, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $26, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $27, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $28, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $29, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $30, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $31, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $32, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $33, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $34, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $35, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $36, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $37, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $38, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $39, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $40, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $41, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $42, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $43, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $44, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $45, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $46, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $47, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $49, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $50, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $51, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $52, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $53, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $54, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $55, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $56, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $2, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $57, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $58, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $6, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $59, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $60, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $10, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $61, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $62, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $14, %xmm7, %edx
+; SSE42-NEXT:    shlq $63, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    orq %rax, %rdx
+; SSE42-NEXT:    movq %rdx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v64i16:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vpcmpgtw %xmm4, %xmm0, %xmm8
-; AVX1-NEXT:    vextractf128 $1, %ymm4, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
-; AVX1-NEXT:    vpcmpgtw %xmm4, %xmm0, %xmm0
-; AVX1-NEXT:    vpcmpgtw %xmm5, %xmm1, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm5, %xmm5
-; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm1
-; AVX1-NEXT:    vpcmpgtw %xmm5, %xmm1, %xmm1
-; AVX1-NEXT:    vpcmpgtw %xmm6, %xmm2, %xmm5
-; AVX1-NEXT:    vextractf128 $1, %ymm6, %xmm6
-; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm2
+; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm8
+; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm9
+; AVX1-NEXT:    vpcmpgtw %xmm8, %xmm9, %xmm8
+; AVX1-NEXT:    vpcmpgtw %xmm7, %xmm3, %xmm9
+; AVX1-NEXT:    vextractf128 $1, %ymm6, %xmm7
+; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm3
+; AVX1-NEXT:    vpcmpgtw %xmm7, %xmm3, %xmm7
 ; AVX1-NEXT:    vpcmpgtw %xmm6, %xmm2, %xmm2
-; AVX1-NEXT:    vpcmpgtw %xmm7, %xmm3, %xmm6
-; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm7
-; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm3
-; AVX1-NEXT:    vpcmpgtw %xmm7, %xmm3, %xmm3
-; AVX1-NEXT:    vpextrb $14, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm3, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm6, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm5, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    vextractf128 $1, %ymm5, %xmm3
+; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm6
+; AVX1-NEXT:    vpcmpgtw %xmm3, %xmm6, %xmm6
+; AVX1-NEXT:    vpcmpgtw %xmm5, %xmm1, %xmm1
+; AVX1-NEXT:    vextractf128 $1, %ymm4, %xmm3
+; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm5
+; AVX1-NEXT:    vpcmpgtw %xmm3, %xmm5, %xmm5
+; AVX1-NEXT:    vpcmpgtw %xmm4, %xmm0, %xmm0
 ; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
 ; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm8, %eax
-; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm8, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $4, %rcx
+; AVX1-NEXT:    orq %rax, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
 ; AVX1-NEXT:    andl $1, %eax
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    shlq $5, %rax
+; AVX1-NEXT:    orq %rcx, %rax
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $6, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $7, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $8, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $9, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $10, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $11, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $12, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $13, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $14, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $15, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $16, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $17, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $18, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $19, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $20, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $21, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $22, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm1, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $23, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm6, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $24, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm6, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $25, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm6, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $26, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm6, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $27, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm6, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $28, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm6, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $29, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm6, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $30, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm6, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $31, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $32, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $33, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $34, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $35, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $36, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $37, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $38, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $39, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $40, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $41, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $42, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $43, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $44, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $45, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $46, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $47, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $48, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $49, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $50, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $51, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $52, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $53, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $54, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $55, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $56, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $2, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $57, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $58, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $6, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $59, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $60, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $10, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $61, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $62, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $14, %xmm8, %edx
+; AVX1-NEXT:    shlq $63, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    orq %rax, %rdx
+; AVX1-NEXT:    movq %rdx, (%rdi)
 ; AVX1-NEXT:    movq %rdi, %rax
 ; AVX1-NEXT:    vzeroupper
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: test_cmp_v64i16:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    vpcmpgtw %ymm4, %ymm0, %ymm0
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm4
-; AVX2-NEXT:    vpcmpgtw %ymm5, %ymm1, %ymm1
-; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm5
-; AVX2-NEXT:    vpcmpgtw %ymm6, %ymm2, %ymm2
-; AVX2-NEXT:    vextracti128 $1, %ymm2, %xmm6
-; AVX2-NEXT:    vpcmpgtw %ymm7, %ymm3, %ymm3
-; AVX2-NEXT:    vextracti128 $1, %ymm3, %xmm7
-; AVX2-NEXT:    vpextrb $14, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm7, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm3, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm6, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm5, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    vpcmpgtw %ymm7, %ymm3, %ymm7
+; AVX2-NEXT:    vextracti128 $1, %ymm7, %xmm3
+; AVX2-NEXT:    vpcmpgtw %ymm6, %ymm2, %ymm6
+; AVX2-NEXT:    vextracti128 $1, %ymm6, %xmm2
+; AVX2-NEXT:    vpcmpgtw %ymm5, %ymm1, %ymm5
+; AVX2-NEXT:    vextracti128 $1, %ymm5, %xmm1
+; AVX2-NEXT:    vpcmpgtw %ymm4, %ymm0, %ymm4
+; AVX2-NEXT:    vextracti128 $1, %ymm4, %xmm0
 ; AVX2-NEXT:    vpextrb $2, %xmm4, %eax
 ; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm4, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX2-NEXT:    vpextrb $4, %xmm4, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX2-NEXT:    vpextrb $6, %xmm4, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX2-NEXT:    vpextrb $8, %xmm4, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $4, %rcx
+; AVX2-NEXT:    orq %rax, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm4, %eax
 ; AVX2-NEXT:    andl $1, %eax
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    shlq $5, %rax
+; AVX2-NEXT:    orq %rcx, %rax
+; AVX2-NEXT:    vpextrb $12, %xmm4, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $6, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm4, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $7, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $8, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $9, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $10, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $11, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $12, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $13, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $14, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $15, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm5, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $16, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm5, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $17, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm5, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $18, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm5, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $19, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm5, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $20, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm5, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $21, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm5, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $22, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm5, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $23, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $24, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $25, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $26, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $27, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $28, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $29, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $30, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $31, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm6, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $32, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm6, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $33, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm6, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $34, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm6, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $35, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm6, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $36, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm6, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $37, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm6, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $38, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm6, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $39, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $40, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $41, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $42, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $43, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $44, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $45, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $46, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $47, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm7, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $48, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm7, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $49, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm7, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $50, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm7, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $51, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm7, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $52, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm7, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $53, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm7, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $54, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm7, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $55, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $56, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $2, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $57, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $58, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $6, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $59, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $60, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $10, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $61, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $62, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $14, %xmm3, %edx
+; AVX2-NEXT:    shlq $63, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    orq %rax, %rdx
+; AVX2-NEXT:    movq %rdx, (%rdi)
 ; AVX2-NEXT:    movq %rdi, %rax
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq
 ;
 ; AVX512F-LABEL: test_cmp_v64i16:
 ; AVX512F:       # %bb.0:
 ; AVX512F-NEXT:    vpcmpgtw %ymm7, %ymm3, %ymm3
 ; AVX512F-NEXT:    vpcmpgtw %ymm6, %ymm2, %ymm2
; AVX512BW-NEXT:    retq
   %1 = icmp sgt <64 x i16> %a0, %a1
   ret <64 x i1> %1
 }
 
 define <128 x i1> @test_cmp_v128i8(<128 x i8> %a0, <128 x i8> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v128i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    pushq %rax
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm2
 ; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm5
-; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm6
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm2
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm1
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm0
 ; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm7
-; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 14(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 14(%rdi)
-; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 12(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 12(%rdi)
-; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 10(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 10(%rdi)
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm6
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm4
 ; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 8(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 8(%rdi)
-; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 6(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 6(%rdi)
-; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 4(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 4(%rdi)
-; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
+; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $4, %rcx
+; SSE2-NEXT:    orq %rax, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shlq $5, %rax
+; SSE2-NEXT:    orq %rcx, %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $6, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $7, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $8, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $9, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $10, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $11, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $12, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $13, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $14, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $15, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $16, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $17, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $18, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $19, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $20, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $21, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $22, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $23, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $24, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $25, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $26, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $27, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $28, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $29, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $30, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $31, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $32, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $33, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $34, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $35, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $36, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $37, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $38, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $39, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $40, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $41, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $42, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $43, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $44, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $45, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $46, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $47, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $49, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $50, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $51, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $52, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $53, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $54, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $55, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $56, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $57, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $58, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $59, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $60, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $61, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $62, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shlq $63, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    orq %rax, %rdx
+; SSE2-NEXT:    movq %rdx, 8(%rdi)
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm2, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $4, %rcx
+; SSE2-NEXT:    orq %rax, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shlq $5, %rax
+; SSE2-NEXT:    orq %rcx, %rax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $6, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $7, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $8, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $9, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $10, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $11, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $12, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $13, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $14, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $15, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $16, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $17, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $18, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $19, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $20, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $21, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $22, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $23, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $24, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $25, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $26, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $27, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $28, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $29, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $30, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $31, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $32, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $33, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $34, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $35, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $36, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $37, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $38, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $39, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $40, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $41, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $42, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $43, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $44, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $45, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $46, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $47, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $48, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $49, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $50, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $51, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $52, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $53, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $54, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $55, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $56, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $57, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $58, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $59, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $60, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shlq $61, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shlq $62, %rcx
+; SSE2-NEXT:    orq %rdx, %rcx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shlq $63, %rdx
+; SSE2-NEXT:    orq %rcx, %rdx
+; SSE2-NEXT:    orq %rax, %rdx
+; SSE2-NEXT:    movq %rdx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    popq %rcx
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v128i8:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm1
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm2
 ; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm5
-; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm6
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm2
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm1
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm0
 ; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm7
-; SSE42-NEXT:    pextrb $15, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm7, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 14(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm6, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 12(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm5, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 10(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm6
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    pcmpgtb {{[0-9]+}}(%rsp), %xmm4
 ; SSE42-NEXT:    pextrb $1, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm4, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 8(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm3, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 6(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm2, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 4(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE42-NEXT:    pextrb $2, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE42-NEXT:    pextrb $3, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE42-NEXT:    pextrb $4, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $4, %rcx
+; SSE42-NEXT:    orq %rax, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm4, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shlq $5, %rax
+; SSE42-NEXT:    orq %rcx, %rax
+; SSE42-NEXT:    pextrb $6, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $6, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $7, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $8, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $9, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $10, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $11, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $12, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $13, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm4, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $14, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm4, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $15, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $16, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $17, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $18, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $19, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $20, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $21, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $22, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $23, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $24, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $25, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $26, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $27, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $28, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $29, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm5, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $30, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm5, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $31, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $32, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $33, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $34, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $35, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $36, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $37, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $38, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $39, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $40, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $41, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $42, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $43, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $44, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $45, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm6, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $46, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm6, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $47, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $49, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $50, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $51, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $52, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $53, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $54, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $55, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $56, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $57, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $58, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $59, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $60, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm7, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $61, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm7, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $62, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm7, %edx
+; SSE42-NEXT:    shlq $63, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    orq %rax, %rdx
+; SSE42-NEXT:    movq %rdx, 8(%rdi)
 ; SSE42-NEXT:    pextrb $1, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rcx,%rax,2), %rax
+; SSE42-NEXT:    pextrb $2, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,4), %rax
+; SSE42-NEXT:    pextrb $3, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leaq (%rax,%rcx,8), %rax
+; SSE42-NEXT:    pextrb $4, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $4, %rcx
+; SSE42-NEXT:    orq %rax, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shlq $5, %rax
+; SSE42-NEXT:    orq %rcx, %rax
+; SSE42-NEXT:    pextrb $6, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $6, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $7, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $8, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $9, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $10, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $11, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $12, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $13, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $14, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm0, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $15, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $16, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $17, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $18, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $19, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $20, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $21, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $22, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $23, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $24, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $25, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $26, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $27, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $28, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $29, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $30, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm1, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $31, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $32, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $33, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $34, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $35, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $36, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $37, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $38, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $39, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $40, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $41, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $42, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $43, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $44, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $45, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm2, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $46, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm2, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $47, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $0, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $48, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $1, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $49, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $2, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $50, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $3, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $51, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $4, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $52, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $5, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $53, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $6, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $54, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $7, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $55, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $8, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $56, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $9, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $57, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $10, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $58, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $11, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $59, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $12, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $60, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $13, %xmm3, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shlq $61, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    pextrb $14, %xmm3, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shlq $62, %rcx
+; SSE42-NEXT:    orq %rdx, %rcx
+; SSE42-NEXT:    pextrb $15, %xmm3, %edx
+; SSE42-NEXT:    shlq $63, %rdx
+; SSE42-NEXT:    orq %rcx, %rdx
+; SSE42-NEXT:    orq %rax, %rdx
+; SSE42-NEXT:    movq %rdx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v128i8:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vpcmpgtb %xmm4, %xmm0, %xmm8
-; AVX1-NEXT:    vextractf128 $1, %ymm4, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm0
+; AVX1-NEXT:    vextractf128 $1, %ymm5, %xmm8
+; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm9
+; AVX1-NEXT:    vpcmpgtb %xmm8, %xmm9, %xmm8
+; AVX1-NEXT:    vpcmpgtb %xmm5, %xmm1, %xmm9
+; AVX1-NEXT:    vextractf128 $1, %ymm4, %xmm5
+; AVX1-NEXT:    vextractf128 $1, %ymm0, %xmm1
+; AVX1-NEXT:    vpcmpgtb %xmm5, %xmm1, %xmm5
 ; AVX1-NEXT:    vpcmpgtb %xmm4, %xmm0, %xmm0
-; AVX1-NEXT:    vpcmpgtb %xmm5, %xmm1, %xmm4
-; AVX1-NEXT:    vextractf128 $1, %ymm5, %xmm5
-; AVX1-NEXT:    vextractf128 $1, %ymm1, %xmm1
-; AVX1-NEXT:    vpcmpgtb %xmm5, %xmm1, %xmm1
-; AVX1-NEXT:    vpcmpgtb %xmm6, %xmm2, %xmm5
-; AVX1-NEXT:    vextractf128 $1, %ymm6, %xmm6
-; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm2
-; AVX1-NEXT:    vpcmpgtb %xmm6, %xmm2, %xmm2
-; AVX1-NEXT:    vpcmpgtb %xmm7, %xmm3, %xmm6
-; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm7
-; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm3
+; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm1
+; AVX1-NEXT:    vextractf128 $1, %ymm3, %xmm4
+; AVX1-NEXT:    vpcmpgtb %xmm1, %xmm4, %xmm4
 ; AVX1-NEXT:    vpcmpgtb %xmm7, %xmm3, %xmm3
-; AVX1-NEXT:    vpextrb $15, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm3, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm6, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 12(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
+; AVX1-NEXT:    vextractf128 $1, %ymm6, %xmm1
+; AVX1-NEXT:    vextractf128 $1, %ymm2, %xmm7
+; AVX1-NEXT:    vpcmpgtb %xmm1, %xmm7, %xmm7
+; AVX1-NEXT:    vpcmpgtb %xmm6, %xmm2, %xmm2
 ; AVX1-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm5, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 8(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, 4(%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX1-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX1-NEXT:    vpextrb $3, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX1-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $4, %rcx
+; AVX1-NEXT:    orq %rax, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm2, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shlq $5, %rax
+; AVX1-NEXT:    orq %rcx, %rax
+; AVX1-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $6, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $7, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $8, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $9, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $10, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $11, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $12, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $13, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $14, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $15, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $16, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $17, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $18, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $19, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $20, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $21, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $22, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $23, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $24, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $25, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $26, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $27, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $28, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $29, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm7, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $30, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm7, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $31, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $32, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $33, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $34, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $35, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $36, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $37, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $38, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $39, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $40, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $41, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $42, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $43, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $44, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $45, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm3, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $46, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm3, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $47, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $48, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $49, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $50, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $51, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $52, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $53, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $54, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $55, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $56, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $57, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $58, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $59, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $60, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm4, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $61, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm4, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $62, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm4, %edx
+; AVX1-NEXT:    shlq $63, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    orq %rax, %rdx
+; AVX1-NEXT:    movq %rdx, 8(%rdi)
 ; AVX1-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $15, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $14, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $13, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $12, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $11, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $10, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $9, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $8, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $7, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $6, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $5, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $4, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $3, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $2, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $1, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
-; AVX1-NEXT:    vpextrb $0, %xmm8, %eax
-; AVX1-NEXT:    andb $1, %al
-; AVX1-NEXT:    movb %al, (%rdi)
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX1-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX1-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX1-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $4, %rcx
+; AVX1-NEXT:    orq %rax, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX1-NEXT:    andl $1, %eax
+; AVX1-NEXT:    shlq $5, %rax
+; AVX1-NEXT:    orq %rcx, %rax
+; AVX1-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $6, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $7, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $8, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $9, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $10, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $11, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $12, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $13, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $14, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $15, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $16, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $17, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $18, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $19, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $20, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $21, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $22, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $23, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $24, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $25, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $26, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $27, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $28, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $29, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm5, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $30, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm5, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $31, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $32, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $33, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $34, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $35, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $36, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $37, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $38, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $39, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $40, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $41, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $42, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $43, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $44, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $45, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm9, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $46, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm9, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $47, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $0, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $48, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $1, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $49, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $2, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $50, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $3, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $51, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $4, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $52, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $5, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $53, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $6, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $54, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $7, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $55, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $8, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $56, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $9, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $57, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $10, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $58, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $11, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $59, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $12, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $60, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $13, %xmm8, %edx
+; AVX1-NEXT:    andl $1, %edx
+; AVX1-NEXT:    shlq $61, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    vpextrb $14, %xmm8, %ecx
+; AVX1-NEXT:    andl $1, %ecx
+; AVX1-NEXT:    shlq $62, %rcx
+; AVX1-NEXT:    orq %rdx, %rcx
+; AVX1-NEXT:    vpextrb $15, %xmm8, %edx
+; AVX1-NEXT:    shlq $63, %rdx
+; AVX1-NEXT:    orq %rcx, %rdx
+; AVX1-NEXT:    orq %rax, %rdx
+; AVX1-NEXT:    movq %rdx, (%rdi)
 ; AVX1-NEXT:    movq %rdi, %rax
 ; AVX1-NEXT:    vzeroupper
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: test_cmp_v128i8:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX2-NEXT:    vpcmpgtb %ymm5, %ymm1, %ymm1
-; AVX2-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
+; AVX2-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX2-NEXT:    vpcmpgtb %ymm7, %ymm3, %ymm3
-; AVX2-NEXT:    vextracti128 $1, %ymm3, %xmm4
-; AVX2-NEXT:    vpextrb $15, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm4, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 12(%rdi)
-; AVX2-NEXT:    vextracti128 $1, %ymm2, %xmm3
-; AVX2-NEXT:    vpextrb $15, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm3, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
+; AVX2-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
 ; AVX2-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 8(%rdi)
-; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm2
-; AVX2-NEXT:    vpextrb $15, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX2-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX2-NEXT:    vpextrb $3, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX2-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $4, %rcx
+; AVX2-NEXT:    orq %rax, %rcx
 ; AVX2-NEXT:    vpextrb $5, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm2, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, 4(%rdi)
-; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX2-NEXT:    vpextrb $15, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $1, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm1, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $15, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $14, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $13, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $12, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $11, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $10, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $9, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $8, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $7, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $6, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $4, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $3, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $2, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shlq $5, %rax
+; AVX2-NEXT:    orq %rcx, %rax
+; AVX2-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $6, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $7, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $8, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $9, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $10, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $11, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $12, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $13, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $14, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $15, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm2, %xmm2
+; AVX2-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $16, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $17, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $18, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $19, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $20, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $21, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $22, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $23, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $24, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $25, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $26, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $27, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $28, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $29, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $30, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $31, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $32, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $33, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $34, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $35, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $36, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $37, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $38, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $39, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $40, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $41, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $42, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $43, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $44, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $45, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm3, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $46, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm3, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $47, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm3, %xmm2
+; AVX2-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $48, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $49, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $50, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $51, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $52, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $53, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $54, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $55, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $56, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $57, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $58, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $59, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $60, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $61, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $62, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX2-NEXT:    shlq $63, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    orq %rax, %rdx
+; AVX2-NEXT:    movq %rdx, 8(%rdi)
 ; AVX2-NEXT:    vpextrb $1, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
-; AVX2-NEXT:    vpextrb $0, %xmm0, %eax
-; AVX2-NEXT:    andb $1, %al
-; AVX2-NEXT:    movb %al, (%rdi)
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX2-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $4, %rcx
+; AVX2-NEXT:    orq %rax, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX2-NEXT:    andl $1, %eax
+; AVX2-NEXT:    shlq $5, %rax
+; AVX2-NEXT:    orq %rcx, %rax
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $6, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $7, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $8, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $9, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $10, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $11, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $12, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $13, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $14, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $15, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $16, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $17, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $18, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $19, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $20, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $21, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $22, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $23, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $24, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $25, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $26, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $27, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $28, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $29, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $30, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $31, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $32, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $33, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $34, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $35, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $36, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $37, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $38, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $39, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $40, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $41, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $42, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $43, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $44, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $45, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $46, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $47, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vextracti128 $1, %ymm1, %xmm0
+; AVX2-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $48, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $49, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $50, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $51, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $52, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $53, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $54, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $55, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $56, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $57, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $58, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $59, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $60, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX2-NEXT:    andl $1, %edx
+; AVX2-NEXT:    shlq $61, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX2-NEXT:    andl $1, %ecx
+; AVX2-NEXT:    shlq $62, %rcx
+; AVX2-NEXT:    orq %rdx, %rcx
+; AVX2-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX2-NEXT:    shlq $63, %rdx
+; AVX2-NEXT:    orq %rcx, %rdx
+; AVX2-NEXT:    orq %rax, %rdx
+; AVX2-NEXT:    movq %rdx, (%rdi)
 ; AVX2-NEXT:    movq %rdi, %rax
 ; AVX2-NEXT:    vzeroupper
 ; AVX2-NEXT:    retq
 ;
 ; AVX512F-LABEL: test_cmp_v128i8:
 ; AVX512F:       # %bb.0:
-; AVX512F-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX512F-NEXT:    vpcmpgtb %ymm5, %ymm1, %ymm1
-; AVX512F-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
+; AVX512F-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX512F-NEXT:    vpcmpgtb %ymm7, %ymm3, %ymm3
-; AVX512F-NEXT:    vextracti128 $1, %ymm3, %xmm4
-; AVX512F-NEXT:    vpmovsxbd %xmm4, %zmm4
-; AVX512F-NEXT:    vptestmd %zmm4, %zmm4, %k0
-; AVX512F-NEXT:    kmovw %k0, 14(%rdi)
-; AVX512F-NEXT:    vpmovsxbd %xmm3, %zmm3
-; AVX512F-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; AVX512F-NEXT:    kmovw %k0, 12(%rdi)
-; AVX512F-NEXT:    vextracti128 $1, %ymm2, %xmm3
-; AVX512F-NEXT:    vpmovsxbd %xmm3, %zmm3
-; AVX512F-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; AVX512F-NEXT:    kmovw %k0, 10(%rdi)
-; AVX512F-NEXT:    vpmovsxbd %xmm2, %zmm2
-; AVX512F-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; AVX512F-NEXT:    kmovw %k0, 8(%rdi)
-; AVX512F-NEXT:    vextracti128 $1, %ymm1, %xmm2
-; AVX512F-NEXT:    vpmovsxbd %xmm2, %zmm2
-; AVX512F-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; AVX512F-NEXT:    kmovw %k0, 6(%rdi)
-; AVX512F-NEXT:    vpmovsxbd %xmm1, %zmm1
-; AVX512F-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; AVX512F-NEXT:    kmovw %k0, 4(%rdi)
-; AVX512F-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX512F-NEXT:    vpmovsxbd %xmm1, %zmm1
-; AVX512F-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; AVX512F-NEXT:    kmovw %k0, 2(%rdi)
-; AVX512F-NEXT:    vpmovsxbd %xmm0, %zmm0
-; AVX512F-NEXT:    vptestmd %zmm0, %zmm0, %k0
-; AVX512F-NEXT:    kmovw %k0, (%rdi)
+; AVX512F-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
+; AVX512F-NEXT:    vpextrb $1, %xmm2, %eax
+; AVX512F-NEXT:    andl $1, %eax
+; AVX512F-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX512F-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX512F-NEXT:    vpextrb $3, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX512F-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $4, %rcx
+; AVX512F-NEXT:    orq %rax, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm2, %eax
+; AVX512F-NEXT:    andl $1, %eax
+; AVX512F-NEXT:    shlq $5, %rax
+; AVX512F-NEXT:    orq %rcx, %rax
+; AVX512F-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $6, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $7, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $8, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $9, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $10, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $11, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $12, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $13, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $14, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $15, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vextracti128 $1, %ymm2, %xmm2
+; AVX512F-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $16, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $17, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $18, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $19, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $20, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $21, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $22, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $23, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $24, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $25, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $26, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $27, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $28, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $29, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $30, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $31, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $32, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $33, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $34, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $35, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $36, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $37, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $38, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $39, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $40, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $41, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $42, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $43, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $44, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $45, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm3, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $46, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm3, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $47, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vextracti128 $1, %ymm3, %xmm2
+; AVX512F-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $48, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $49, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $50, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $51, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $52, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $53, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $54, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $55, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $56, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $57, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $58, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $59, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $60, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $61, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $62, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512F-NEXT:    shlq $63, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    orq %rax, %rdx
+; AVX512F-NEXT:    movq %rdx, 8(%rdi)
+; AVX512F-NEXT:    vpextrb $1, %xmm0, %eax
+; AVX512F-NEXT:    andl $1, %eax
+; AVX512F-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX512F-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX512F-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX512F-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $4, %rcx
+; AVX512F-NEXT:    orq %rax, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX512F-NEXT:    andl $1, %eax
+; AVX512F-NEXT:    shlq $5, %rax
+; AVX512F-NEXT:    orq %rcx, %rax
+; AVX512F-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $6, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $7, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $8, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $9, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $10, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $11, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $12, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $13, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $14, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $15, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX512F-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $16, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $17, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $18, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $19, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $20, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $21, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $22, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $23, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $24, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $25, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $26, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $27, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $28, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $29, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $30, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $31, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $32, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $33, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $34, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $35, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $36, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $37, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $38, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $39, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $40, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $41, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $42, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $43, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $44, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $45, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $46, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $47, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vextracti128 $1, %ymm1, %xmm0
+; AVX512F-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $48, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $49, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $50, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $51, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $52, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $53, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $54, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $55, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $56, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $57, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $58, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $59, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $60, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512F-NEXT:    andl $1, %edx
+; AVX512F-NEXT:    shlq $61, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512F-NEXT:    andl $1, %ecx
+; AVX512F-NEXT:    shlq $62, %rcx
+; AVX512F-NEXT:    orq %rdx, %rcx
+; AVX512F-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512F-NEXT:    shlq $63, %rdx
+; AVX512F-NEXT:    orq %rcx, %rdx
+; AVX512F-NEXT:    orq %rax, %rdx
+; AVX512F-NEXT:    movq %rdx, (%rdi)
 ; AVX512F-NEXT:    movq %rdi, %rax
 ; AVX512F-NEXT:    vzeroupper
 ; AVX512F-NEXT:    retq
 ;
 ; AVX512DQ-LABEL: test_cmp_v128i8:
 ; AVX512DQ:       # %bb.0:
-; AVX512DQ-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX512DQ-NEXT:    vpcmpgtb %ymm5, %ymm1, %ymm1
-; AVX512DQ-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
+; AVX512DQ-NEXT:    vpcmpgtb %ymm4, %ymm0, %ymm0
 ; AVX512DQ-NEXT:    vpcmpgtb %ymm7, %ymm3, %ymm3
-; AVX512DQ-NEXT:    vextracti128 $1, %ymm3, %xmm4
-; AVX512DQ-NEXT:    vpmovsxbd %xmm4, %zmm4
-; AVX512DQ-NEXT:    vptestmd %zmm4, %zmm4, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 14(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm3, %zmm3
-; AVX512DQ-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 12(%rdi)
-; AVX512DQ-NEXT:    vextracti128 $1, %ymm2, %xmm3
-; AVX512DQ-NEXT:    vpmovsxbd %xmm3, %zmm3
-; AVX512DQ-NEXT:    vptestmd %zmm3, %zmm3, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 10(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm2, %zmm2
-; AVX512DQ-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 8(%rdi)
-; AVX512DQ-NEXT:    vextracti128 $1, %ymm1, %xmm2
-; AVX512DQ-NEXT:    vpmovsxbd %xmm2, %zmm2
-; AVX512DQ-NEXT:    vptestmd %zmm2, %zmm2, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 6(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm1, %zmm1
-; AVX512DQ-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 4(%rdi)
-; AVX512DQ-NEXT:    vextracti128 $1, %ymm0, %xmm1
-; AVX512DQ-NEXT:    vpmovsxbd %xmm1, %zmm1
-; AVX512DQ-NEXT:    vptestmd %zmm1, %zmm1, %k0
-; AVX512DQ-NEXT:    kmovw %k0, 2(%rdi)
-; AVX512DQ-NEXT:    vpmovsxbd %xmm0, %zmm0
-; AVX512DQ-NEXT:    vptestmd %zmm0, %zmm0, %k0
-; AVX512DQ-NEXT:    kmovw %k0, (%rdi)
+; AVX512DQ-NEXT:    vpcmpgtb %ymm6, %ymm2, %ymm2
+; AVX512DQ-NEXT:    vpextrb $1, %xmm2, %eax
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX512DQ-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX512DQ-NEXT:    vpextrb $3, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX512DQ-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $4, %rcx
+; AVX512DQ-NEXT:    orq %rax, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm2, %eax
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    shlq $5, %rax
+; AVX512DQ-NEXT:    orq %rcx, %rax
+; AVX512DQ-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $6, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $7, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $8, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $9, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $10, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $11, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $12, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $13, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $14, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $15, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vextracti128 $1, %ymm2, %xmm2
+; AVX512DQ-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $16, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $17, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $18, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $19, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $20, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $21, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $22, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $23, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $24, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $25, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $26, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $27, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $28, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $29, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $30, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $31, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $0, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $32, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $33, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $34, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $35, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $36, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $37, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $38, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $39, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $40, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $41, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $42, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $43, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $44, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $45, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm3, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $46, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm3, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $47, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vextracti128 $1, %ymm3, %xmm2
+; AVX512DQ-NEXT:    vpextrb $0, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $48, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $49, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $50, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $51, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $52, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $53, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $54, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $55, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $56, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $57, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $58, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $59, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $60, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm2, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $61, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm2, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $62, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm2, %edx
+; AVX512DQ-NEXT:    shlq $63, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    orq %rax, %rdx
+; AVX512DQ-NEXT:    movq %rdx, 8(%rdi)
+; AVX512DQ-NEXT:    vpextrb $1, %xmm0, %eax
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rcx,%rax,2), %rax
+; AVX512DQ-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rax,%rcx,4), %rax
+; AVX512DQ-NEXT:    vpextrb $3, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    leaq (%rax,%rcx,8), %rax
+; AVX512DQ-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $4, %rcx
+; AVX512DQ-NEXT:    orq %rax, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm0, %eax
+; AVX512DQ-NEXT:    andl $1, %eax
+; AVX512DQ-NEXT:    shlq $5, %rax
+; AVX512DQ-NEXT:    orq %rcx, %rax
+; AVX512DQ-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $6, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $7, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $8, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $9, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $10, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $11, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $12, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $13, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $14, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $15, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vextracti128 $1, %ymm0, %xmm0
+; AVX512DQ-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $16, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $17, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $18, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $19, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $20, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $21, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $22, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $23, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $24, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $25, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $26, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $27, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $28, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $29, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $30, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $31, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $0, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $32, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $33, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $34, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $35, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $36, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $37, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $38, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $39, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $40, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $41, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $42, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $43, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $44, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $45, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm1, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $46, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm1, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $47, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vextracti128 $1, %ymm1, %xmm0
+; AVX512DQ-NEXT:    vpextrb $0, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $48, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $1, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $49, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $2, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $50, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $3, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $51, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $4, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $52, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $5, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $53, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $6, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $54, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $7, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $55, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $8, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $56, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $9, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $57, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $10, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $58, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $11, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $59, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $12, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $60, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $13, %xmm0, %edx
+; AVX512DQ-NEXT:    andl $1, %edx
+; AVX512DQ-NEXT:    shlq $61, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    vpextrb $14, %xmm0, %ecx
+; AVX512DQ-NEXT:    andl $1, %ecx
+; AVX512DQ-NEXT:    shlq $62, %rcx
+; AVX512DQ-NEXT:    orq %rdx, %rcx
+; AVX512DQ-NEXT:    vpextrb $15, %xmm0, %edx
+; AVX512DQ-NEXT:    shlq $63, %rdx
+; AVX512DQ-NEXT:    orq %rcx, %rdx
+; AVX512DQ-NEXT:    orq %rax, %rdx
+; AVX512DQ-NEXT:    movq %rdx, (%rdi)
 ; AVX512DQ-NEXT:    movq %rdi, %rax
 ; AVX512DQ-NEXT:    vzeroupper
 ; AVX512DQ-NEXT:    retq
 ;
 ; AVX512BW-LABEL: test_cmp_v128i8:
 ; AVX512BW:       # %bb.0:
 ; AVX512BW-NEXT:    vpcmpgtb %zmm3, %zmm1, %k0
 ; AVX512BW-NEXT:    vpcmpgtb %zmm2, %zmm0, %k1

 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
 ; SSE2-NEXT:    cmpltpd %xmm4, %xmm5
 ; SSE2-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm6[0,2]
 ; SSE2-NEXT:    packssdw %xmm7, %xmm5
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
 ; SSE2-NEXT:    cmpltpd %xmm3, %xmm4
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
 ; SSE2-NEXT:    cmpltpd %xmm2, %xmm3
-; SSE2-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    cmpltpd %xmm1, %xmm2
-; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    cmpltpd %xmm0, %xmm1
+; SSE2-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
+; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
+; SSE2-NEXT:    cmpltpd %xmm1, %xmm4
+; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm6
+; SSE2-NEXT:    cmpltpd %xmm0, %xmm6
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,2],xmm2[0,2]
-; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    packssdw %xmm3, %xmm1
-; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    packsswb %xmm5, %xmm1
+; SSE2-NEXT:    shufps {{.*#+}} xmm6 = xmm6[0,2],xmm4[0,2]
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
+; SSE2-NEXT:    packssdw %xmm3, %xmm6
+; SSE2-NEXT:    packsswb %xmm5, %xmm6
+; SSE2-NEXT:    psllw $7, %xmm6
+; SSE2-NEXT:    movdqa {{.*#+}} xmm1 = [128,128,128,128,128,128,128,128,128,128,128,128,128,128,128,128]
+; SSE2-NEXT:    pand %xmm1, %xmm6
+; SSE2-NEXT:    pxor %xmm3, %xmm3
+; SSE2-NEXT:    pcmpgtb %xmm6, %xmm3
+; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
 ; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
+; SSE2-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm5[0,2]
+; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm0
+; SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm5[0,2]
+; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    packssdw %xmm4, %xmm0
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
 ; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm4[0,2]
+; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm4[0,2]
 ; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    packssdw %xmm3, %xmm2
-; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
 ; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm3[0,2]
-; SSE2-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm0
-; SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm3[0,2]
-; SSE2-NEXT:    packssdw %xmm4, %xmm0
-; SSE2-NEXT:    packsswb %xmm2, %xmm0
+; SSE2-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm2
+; SSE2-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm4[0,2]
+; SSE2-NEXT:    packssdw %xmm5, %xmm2
+; SSE2-NEXT:    packsswb %xmm0, %xmm2
+; SSE2-NEXT:    psllw $7, %xmm2
+; SSE2-NEXT:    pand %xmm1, %xmm2
+; SSE2-NEXT:    pxor %xmm0, %xmm0
+; SSE2-NEXT:    pcmpgtb %xmm2, %xmm0
+; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32f64:
 ; SSE42:       # %bb.0:
 ; SSE42-NEXT:    pushq %rbp
 ; SSE42-NEXT:    pushq %r15
 ; SSE42-NEXT:    pushq %r14

 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
 ; SSE42-NEXT:    cmpltpd %xmm4, %xmm5
 ; SSE42-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm6[0,2]
 ; SSE42-NEXT:    packssdw %xmm7, %xmm5
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
 ; SSE42-NEXT:    cmpltpd %xmm3, %xmm4
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
 ; SSE42-NEXT:    cmpltpd %xmm2, %xmm3
-; SSE42-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm2
-; SSE42-NEXT:    cmpltpd %xmm1, %xmm2
-; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm1
-; SSE42-NEXT:    cmpltpd %xmm0, %xmm1
+; SSE42-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
+; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
+; SSE42-NEXT:    cmpltpd %xmm1, %xmm4
+; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm6
+; SSE42-NEXT:    cmpltpd %xmm0, %xmm6
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,2],xmm2[0,2]
-; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm2
-; SSE42-NEXT:    packssdw %xmm3, %xmm1
-; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    packsswb %xmm5, %xmm1
+; SSE42-NEXT:    shufps {{.*#+}} xmm6 = xmm6[0,2],xmm4[0,2]
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
+; SSE42-NEXT:    packssdw %xmm3, %xmm6
+; SSE42-NEXT:    packsswb %xmm5, %xmm6
+; SSE42-NEXT:    psllw $7, %xmm6
+; SSE42-NEXT:    movdqa {{.*#+}} xmm3 = [128,128,128,128,128,128,128,128,128,128,128,128,128,128,128,128]
+; SSE42-NEXT:    pand %xmm3, %xmm6
+; SSE42-NEXT:    xorpd %xmm1, %xmm1
+; SSE42-NEXT:    pcmpgtb %xmm6, %xmm1
+; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
 ; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
+; SSE42-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm5[0,2]
+; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm0
+; SSE42-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm5[0,2]
+; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    packssdw %xmm4, %xmm0
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
 ; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm2
-; SSE42-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm4[0,2]
+; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm5
+; SSE42-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm4[0,2]
 ; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    packssdw %xmm3, %xmm2
-; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
 ; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm4
-; SSE42-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm3[0,2]
-; SSE42-NEXT:    movapd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm3
-; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm0
-; SSE42-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm3[0,2]
-; SSE42-NEXT:    packssdw %xmm4, %xmm0
-; SSE42-NEXT:    packsswb %xmm2, %xmm0
-; SSE42-NEXT:    pextrb $15, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm0, %r8d
-; SSE42-NEXT:    pextrb $12, %xmm0, %r9d
-; SSE42-NEXT:    pextrb $11, %xmm0, %r10d
-; SSE42-NEXT:    pextrb $10, %xmm0, %r11d
-; SSE42-NEXT:    pextrb $9, %xmm0, %r14d
-; SSE42-NEXT:    pextrb $8, %xmm0, %r15d
-; SSE42-NEXT:    pextrb $7, %xmm0, %r12d
-; SSE42-NEXT:    pextrb $6, %xmm0, %r13d
-; SSE42-NEXT:    pextrb $5, %xmm0, %ebx
-; SSE42-NEXT:    pextrb $4, %xmm0, %ebp
-; SSE42-NEXT:    pextrb $3, %xmm0, %eax
-; SSE42-NEXT:    pextrb $2, %xmm0, %ecx
-; SSE42-NEXT:    pextrb $1, %xmm0, %edx
-; SSE42-NEXT:    pextrb $0, %xmm0, %esi
-; SSE42-NEXT:    andb $1, %r8b
-; SSE42-NEXT:    movb %r8b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r9b
-; SSE42-NEXT:    movb %r9b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r10b
-; SSE42-NEXT:    movb %r10b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r11b
-; SSE42-NEXT:    movb %r11b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r14b
-; SSE42-NEXT:    movb %r14b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r15b
-; SSE42-NEXT:    movb %r15b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r12b
-; SSE42-NEXT:    movb %r12b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %r13b
-; SSE42-NEXT:    movb %r13b, 2(%rdi)
-; SSE42-NEXT:    andb $1, %bl
-; SSE42-NEXT:    movb %bl, 2(%rdi)
-; SSE42-NEXT:    andb $1, %bpl
-; SSE42-NEXT:    movb %bpl, 2(%rdi)
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    andb $1, %cl
-; SSE42-NEXT:    movb %cl, 2(%rdi)
-; SSE42-NEXT:    andb $1, %dl
-; SSE42-NEXT:    movb %dl, 2(%rdi)
-; SSE42-NEXT:    andb $1, %sil
-; SSE42-NEXT:    movb %sil, 2(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm1, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm1, %r8d
-; SSE42-NEXT:    pextrb $12, %xmm1, %r9d
-; SSE42-NEXT:    pextrb $11, %xmm1, %r10d
-; SSE42-NEXT:    pextrb $10, %xmm1, %r11d
-; SSE42-NEXT:    pextrb $9, %xmm1, %r14d
-; SSE42-NEXT:    pextrb $8, %xmm1, %r15d
-; SSE42-NEXT:    pextrb $7, %xmm1, %r12d
-; SSE42-NEXT:    pextrb $6, %xmm1, %r13d
-; SSE42-NEXT:    pextrb $5, %xmm1, %ebx
-; SSE42-NEXT:    pextrb $4, %xmm1, %ebp
-; SSE42-NEXT:    pextrb $3, %xmm1, %eax
+; SSE42-NEXT:    cmpltpd {{[0-9]+}}(%rsp), %xmm2
+; SSE42-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm4[0,2]
+; SSE42-NEXT:    packssdw %xmm5, %xmm2
+; SSE42-NEXT:    packsswb %xmm0, %xmm2
+; SSE42-NEXT:    psllw $7, %xmm2
+; SSE42-NEXT:    pand %xmm3, %xmm2
+; SSE42-NEXT:    pxor %xmm0, %xmm0
+; SSE42-NEXT:    pcmpgtb %xmm2, %xmm0
+; SSE42-NEXT:    pextrb $1, %xmm1, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm1, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
 ; SSE42-NEXT:    pextrb $2, %xmm1, %ecx
-; SSE42-NEXT:    pextrb $1, %xmm1, %edx
-; SSE42-NEXT:    pextrb $0, %xmm1, %esi
-; SSE42-NEXT:    andb $1, %r8b
-; SSE42-NEXT:    movb %r8b, (%rdi)
-; SSE42-NEXT:    andb $1, %r9b
-; SSE42-NEXT:    movb %r9b, (%rdi)
-; SSE42-NEXT:    andb $1, %r10b
-; SSE42-NEXT:    movb %r10b, (%rdi)
-; SSE42-NEXT:    andb $1, %r11b
-; SSE42-NEXT:    movb %r11b, (%rdi)
-; SSE42-NEXT:    andb $1, %r14b
-; SSE42-NEXT:    movb %r14b, (%rdi)
-; SSE42-NEXT:    andb $1, %r15b
-; SSE42-NEXT:    movb %r15b, (%rdi)
-; SSE42-NEXT:    andb $1, %r12b
-; SSE42-NEXT:    movb %r12b, (%rdi)
-; SSE42-NEXT:    andb $1, %r13b
-; SSE42-NEXT:    movb %r13b, (%rdi)
-; SSE42-NEXT:    andb $1, %bl
-; SSE42-NEXT:    movb %bl, (%rdi)
-; SSE42-NEXT:    andb $1, %bpl
-; SSE42-NEXT:    movb %bpl, (%rdi)
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    andb $1, %cl
-; SSE42-NEXT:    movb %cl, (%rdi)
-; SSE42-NEXT:    andb $1, %dl
-; SSE42-NEXT:    movb %dl, (%rdi)
-; SSE42-NEXT:    andb $1, %sil
-; SSE42-NEXT:    movb %sil, (%rdi)
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %r9d
+; SSE42-NEXT:    pextrb $3, %xmm1, %r10d
+; SSE42-NEXT:    pextrb $4, %xmm1, %r14d
+; SSE42-NEXT:    pextrb $5, %xmm1, %r8d
+; SSE42-NEXT:    pextrb $6, %xmm1, %r11d
+; SSE42-NEXT:    pextrb $7, %xmm1, %r15d
+; SSE42-NEXT:    pextrb $8, %xmm1, %r12d
+; SSE42-NEXT:    pextrb $9, %xmm1, %r13d
+; SSE42-NEXT:    pextrb $10, %xmm1, %ebx
+; SSE42-NEXT:    pextrb $11, %xmm1, %ebp
+; SSE42-NEXT:    pextrb $12, %xmm1, %esi
+; SSE42-NEXT:    pextrb $13, %xmm1, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm1, %edx
+; SSE42-NEXT:    pextrb $15, %xmm1, %eax
+; SSE42-NEXT:    andl $1, %r10d
+; SSE42-NEXT:    leal (%r9,%r10,8), %r9d
+; SSE42-NEXT:    andl $1, %r14d
+; SSE42-NEXT:    shll $4, %r14d
+; SSE42-NEXT:    orl %r9d, %r14d
+; SSE42-NEXT:    andl $1, %r8d
+; SSE42-NEXT:    shll $5, %r8d
+; SSE42-NEXT:    orl %r14d, %r8d
+; SSE42-NEXT:    andl $1, %r11d
+; SSE42-NEXT:    shll $6, %r11d
+; SSE42-NEXT:    andl $1, %r15d
+; SSE42-NEXT:    shll $7, %r15d
+; SSE42-NEXT:    orl %r11d, %r15d
+; SSE42-NEXT:    andl $1, %r12d
+; SSE42-NEXT:    shll $8, %r12d
+; SSE42-NEXT:    orl %r15d, %r12d
+; SSE42-NEXT:    andl $1, %r13d
+; SSE42-NEXT:    shll $9, %r13d
+; SSE42-NEXT:    orl %r12d, %r13d
+; SSE42-NEXT:    andl $1, %ebx
+; SSE42-NEXT:    shll $10, %ebx
+; SSE42-NEXT:    orl %r13d, %ebx
+; SSE42-NEXT:    andl $1, %ebp
+; SSE42-NEXT:    shll $11, %ebp
+; SSE42-NEXT:    orl %ebx, %ebp
+; SSE42-NEXT:    andl $1, %esi
+; SSE42-NEXT:    shll $12, %esi
+; SSE42-NEXT:    orl %ebp, %esi
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $13, %ecx
+; SSE42-NEXT:    orl %esi, %ecx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $14, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $15, %eax
+; SSE42-NEXT:    orl %edx, %eax
+; SSE42-NEXT:    pextrb $0, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $1, %xmm0, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $17, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $2, %xmm0, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $3, %xmm0, %r9d
+; SSE42-NEXT:    andl $1, %r9d
+; SSE42-NEXT:    shll $19, %r9d
+; SSE42-NEXT:    orl %ecx, %r9d
+; SSE42-NEXT:    pextrb $4, %xmm0, %r10d
+; SSE42-NEXT:    pextrb $5, %xmm0, %r11d
+; SSE42-NEXT:    pextrb $6, %xmm0, %r14d
+; SSE42-NEXT:    pextrb $7, %xmm0, %r15d
+; SSE42-NEXT:    pextrb $8, %xmm0, %r12d
+; SSE42-NEXT:    pextrb $9, %xmm0, %r13d
+; SSE42-NEXT:    pextrb $10, %xmm0, %ebx
+; SSE42-NEXT:    pextrb $11, %xmm0, %ebp
+; SSE42-NEXT:    pextrb $12, %xmm0, %eax
+; SSE42-NEXT:    pextrb $13, %xmm0, %ecx
+; SSE42-NEXT:    pextrb $14, %xmm0, %edx
+; SSE42-NEXT:    pextrb $15, %xmm0, %esi
+; SSE42-NEXT:    andl $1, %r10d
+; SSE42-NEXT:    shll $20, %r10d
+; SSE42-NEXT:    orl %r9d, %r10d
+; SSE42-NEXT:    andl $1, %r11d
+; SSE42-NEXT:    shll $21, %r11d
+; SSE42-NEXT:    orl %r10d, %r11d
+; SSE42-NEXT:    andl $1, %r14d
+; SSE42-NEXT:    shll $22, %r14d
+; SSE42-NEXT:    orl %r11d, %r14d
+; SSE42-NEXT:    andl $1, %r15d
+; SSE42-NEXT:    shll $23, %r15d
+; SSE42-NEXT:    orl %r14d, %r15d
+; SSE42-NEXT:    andl $1, %r12d
+; SSE42-NEXT:    shll $24, %r12d
+; SSE42-NEXT:    orl %r15d, %r12d
+; SSE42-NEXT:    andl $1, %r13d
+; SSE42-NEXT:    shll $25, %r13d
+; SSE42-NEXT:    orl %r12d, %r13d
+; SSE42-NEXT:    andl $1, %ebx
+; SSE42-NEXT:    shll $26, %ebx
+; SSE42-NEXT:    orl %r13d, %ebx
+; SSE42-NEXT:    andl $1, %ebp
+; SSE42-NEXT:    shll $27, %ebp
+; SSE42-NEXT:    orl %ebx, %ebp
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $28, %eax
+; SSE42-NEXT:    orl %ebp, %eax
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $29, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $30, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    shll $31, %esi
+; SSE42-NEXT:    orl %edx, %esi
+; SSE42-NEXT:    orl %r8d, %esi
+; SSE42-NEXT:    movl %esi, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    popq %rbx
 ; SSE42-NEXT:    popq %r12
 ; SSE42-NEXT:    popq %r13
 ; SSE42-NEXT:    popq %r14
 ; SSE42-NEXT:    popq %r15
 ; SSE42-NEXT:    popq %rbp
 ; SSE42-NEXT:    retq
; AVX512BW-NEXT:    retq
   %1 = fcmp ogt <32 x double> %a0, %a1
   ret <32 x i1> %1
 }
 
 define <32 x i1> @test_cmp_v32i64(<32 x i64> %a0, <32 x i64> %a1) nounwind {
 ; SSE2-LABEL: test_cmp_v32i64:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm8 = [2147483648,0,2147483648,0]
-; SSE2-NEXT:    pxor %xmm8, %xmm7
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
 ; SSE2-NEXT:    pxor %xmm8, %xmm9
-; SSE2-NEXT:    movdqa %xmm7, %xmm10
-; SSE2-NEXT:    pcmpgtd %xmm9, %xmm10
-; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm10[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm7, %xmm9
-; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm9[1,1,3,3]
-; SSE2-NEXT:    pand %xmm11, %xmm7
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm10
+; SSE2-NEXT:    pxor %xmm8, %xmm10
+; SSE2-NEXT:    movdqa %xmm10, %xmm11
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm11
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm11[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm10
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm10[1,1,3,3]
-; SSE2-NEXT:    por %xmm7, %xmm9
+; SSE2-NEXT:    pand %xmm12, %xmm9
+; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm11[1,1,3,3]
+; SSE2-NEXT:    por %xmm9, %xmm10
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
+; SSE2-NEXT:    pxor %xmm8, %xmm9
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
+; SSE2-NEXT:    pxor %xmm8, %xmm11
+; SSE2-NEXT:    movdqa %xmm11, %xmm12
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm13 = xmm12[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm11
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm11[1,1,3,3]
+; SSE2-NEXT:    pand %xmm13, %xmm9
+; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm12[1,1,3,3]
+; SSE2-NEXT:    por %xmm9, %xmm11
+; SSE2-NEXT:    shufps {{.*#+}} xmm11 = xmm11[0,2],xmm10[0,2]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm9 = xmm11[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm9 = xmm9[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm9[0,2,2,3]
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm10
+; SSE2-NEXT:    pxor %xmm8, %xmm10
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
+; SSE2-NEXT:    pxor %xmm8, %xmm11
+; SSE2-NEXT:    movdqa %xmm11, %xmm12
+; SSE2-NEXT:    pcmpgtd %xmm10, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm13 = xmm12[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm10, %xmm11
+; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm11[1,1,3,3]
+; SSE2-NEXT:    pand %xmm13, %xmm10
+; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm12[1,1,3,3]
+; SSE2-NEXT:    por %xmm10, %xmm11
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm10
+; SSE2-NEXT:    pxor %xmm8, %xmm10
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm12
+; SSE2-NEXT:    pxor %xmm8, %xmm12
+; SSE2-NEXT:    movdqa %xmm12, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm10, %xmm13
+; SSE2-NEXT:    pshufd {{.*#+}} xmm14 = xmm13[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm10, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm12[1,1,3,3]
+; SSE2-NEXT:    pand %xmm14, %xmm10
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm13[1,1,3,3]
+; SSE2-NEXT:    por %xmm10, %xmm12
+; SSE2-NEXT:    shufps {{.*#+}} xmm12 = xmm12[0,2],xmm11[0,2]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm10 = xmm12[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm10 = xmm10[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm10[0,2,2,3]
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm10 = xmm10[0],xmm9[0]
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
+; SSE2-NEXT:    pxor %xmm8, %xmm9
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
+; SSE2-NEXT:    pxor %xmm8, %xmm11
+; SSE2-NEXT:    movdqa %xmm11, %xmm12
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm13 = xmm12[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm11
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm11[1,1,3,3]
+; SSE2-NEXT:    pand %xmm13, %xmm9
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm12[1,1,3,3]
+; SSE2-NEXT:    por %xmm9, %xmm12
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
+; SSE2-NEXT:    pxor %xmm8, %xmm9
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
+; SSE2-NEXT:    pxor %xmm8, %xmm11
+; SSE2-NEXT:    movdqa %xmm11, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm13
+; SSE2-NEXT:    pshufd {{.*#+}} xmm14 = xmm13[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm11
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm11[1,1,3,3]
+; SSE2-NEXT:    pand %xmm14, %xmm9
+; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm13[1,1,3,3]
+; SSE2-NEXT:    por %xmm9, %xmm11
+; SSE2-NEXT:    shufps {{.*#+}} xmm11 = xmm11[0,2],xmm12[0,2]
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
+; SSE2-NEXT:    pxor %xmm8, %xmm9
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm12
+; SSE2-NEXT:    pxor %xmm8, %xmm12
+; SSE2-NEXT:    movdqa %xmm12, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm13
+; SSE2-NEXT:    pshufd {{.*#+}} xmm14 = xmm13[0,0,2,2]
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm12[1,1,3,3]
+; SSE2-NEXT:    pand %xmm14, %xmm9
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm13[1,1,3,3]
+; SSE2-NEXT:    por %xmm9, %xmm12
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
+; SSE2-NEXT:    pxor %xmm8, %xmm9
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm13
+; SSE2-NEXT:    pxor %xmm8, %xmm13
+; SSE2-NEXT:    movdqa %xmm13, %xmm14
+; SSE2-NEXT:    pcmpgtd %xmm9, %xmm14
+; SSE2-NEXT:    pcmpeqd %xmm9, %xmm13
+; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm14[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm13 = xmm13[1,1,3,3]
+; SSE2-NEXT:    pand %xmm9, %xmm13
+; SSE2-NEXT:    pshufd {{.*#+}} xmm14 = xmm14[1,1,3,3]
+; SSE2-NEXT:    por %xmm13, %xmm14
+; SSE2-NEXT:    movdqa {{.*#+}} xmm9 = [255,255,255,255,255,255,255,255]
+; SSE2-NEXT:    pand %xmm9, %xmm10
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm11 = xmm11[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm11 = xmm11[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm11[0,2,2,3]
+; SSE2-NEXT:    shufps {{.*#+}} xmm14 = xmm14[0,2],xmm12[0,2]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm12 = xmm14[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm12 = xmm12[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm12[0,2,2,3]
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm12 = xmm12[0],xmm11[0]
+; SSE2-NEXT:    pand %xmm9, %xmm12
+; SSE2-NEXT:    packuswb %xmm10, %xmm12
+; SSE2-NEXT:    psllw $7, %xmm12
+; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [128,128,128,128,128,128,128,128,128,128,128,128,128,128,128,128]
+; SSE2-NEXT:    pand %xmm10, %xmm12
+; SSE2-NEXT:    pxor %xmm11, %xmm11
+; SSE2-NEXT:    pcmpgtb %xmm12, %xmm11
+; SSE2-NEXT:    pxor %xmm8, %xmm7
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm12
+; SSE2-NEXT:    pxor %xmm8, %xmm12
+; SSE2-NEXT:    movdqa %xmm7, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm12, %xmm13
+; SSE2-NEXT:    pcmpeqd %xmm7, %xmm12
+; SSE2-NEXT:    pshufd {{.*#+}} xmm14 = xmm13[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm12[1,1,3,3]
+; SSE2-NEXT:    pand %xmm14, %xmm7
+; SSE2-NEXT:    pshufd {{.*#+}} xmm12 = xmm13[1,1,3,3]
+; SSE2-NEXT:    por %xmm7, %xmm12
 ; SSE2-NEXT:    pxor %xmm8, %xmm6
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm7
 ; SSE2-NEXT:    pxor %xmm8, %xmm7
-; SSE2-NEXT:    movdqa %xmm6, %xmm10
-; SSE2-NEXT:    pcmpgtd %xmm7, %xmm10
-; SSE2-NEXT:    pshufd {{.*#+}} xmm11 = xmm10[0,0,2,2]
+; SSE2-NEXT:    movdqa %xmm6, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm7, %xmm13
 ; SSE2-NEXT:    pcmpeqd %xmm6, %xmm7
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm7[1,1,3,3]
-; SSE2-NEXT:    pand %xmm11, %xmm6
-; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm10[1,1,3,3]
-; SSE2-NEXT:    por %xmm6, %xmm7
-; SSE2-NEXT:    shufps {{.*#+}} xmm7 = xmm7[0,2],xmm9[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm6 = xmm7[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm6 = xmm6[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm9 = xmm6[0,2,2,3]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm13[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
+; SSE2-NEXT:    pand %xmm6, %xmm7
+; SSE2-NEXT:    pshufd {{.*#+}} xmm13 = xmm13[1,1,3,3]
+; SSE2-NEXT:    por %xmm7, %xmm13
+; SSE2-NEXT:    shufps {{.*#+}} xmm13 = xmm13[0,2],xmm12[0,2]
 ; SSE2-NEXT:    pxor %xmm8, %xmm5
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm7
 ; SSE2-NEXT:    pxor %xmm8, %xmm7
 ; SSE2-NEXT:    movdqa %xmm5, %xmm6
 ; SSE2-NEXT:    pcmpgtd %xmm7, %xmm6
-; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm6[0,0,2,2]
 ; SSE2-NEXT:    pcmpeqd %xmm5, %xmm7
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[1,1,3,3]
-; SSE2-NEXT:    pand %xmm10, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
-; SSE2-NEXT:    por %xmm5, %xmm6
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm6[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
+; SSE2-NEXT:    pand %xmm5, %xmm7
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm6[1,1,3,3]
+; SSE2-NEXT:    por %xmm7, %xmm5
 ; SSE2-NEXT:    pxor %xmm8, %xmm4
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm5
-; SSE2-NEXT:    pxor %xmm8, %xmm5
-; SSE2-NEXT:    movdqa %xmm4, %xmm7
-; SSE2-NEXT:    pcmpgtd %xmm5, %xmm7
-; SSE2-NEXT:    pshufd {{.*#+}} xmm10 = xmm7[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm4, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[1,1,3,3]
-; SSE2-NEXT:    pand %xmm10, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[1,1,3,3]
-; SSE2-NEXT:    por %xmm4, %xmm5
-; SSE2-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm6[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm4 = xmm5[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm4 = xmm4[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm4[0,2,2,3]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm5 = xmm5[0],xmm9[0]
-; SSE2-NEXT:    movdqa {{.*#+}} xmm9 = [255,255,255,255,255,255,255,255]
-; SSE2-NEXT:    pand %xmm9, %xmm5
-; SSE2-NEXT:    pxor %xmm8, %xmm3
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm6
 ; SSE2-NEXT:    pxor %xmm8, %xmm6
-; SSE2-NEXT:    movdqa %xmm3, %xmm7
+; SSE2-NEXT:    movdqa %xmm4, %xmm7
 ; SSE2-NEXT:    pcmpgtd %xmm6, %xmm7
+; SSE2-NEXT:    pcmpeqd %xmm4, %xmm6
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm7[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm3, %xmm6
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm6[1,1,3,3]
-; SSE2-NEXT:    pand %xmm4, %xmm3
+; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
+; SSE2-NEXT:    pand %xmm4, %xmm6
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm7[1,1,3,3]
-; SSE2-NEXT:    por %xmm3, %xmm4
-; SSE2-NEXT:    pxor %xmm8, %xmm2
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
+; SSE2-NEXT:    por %xmm6, %xmm4
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm6 = xmm13[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm6 = xmm6[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[0,2,2,3]
+; SSE2-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm5[0,2]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm4 = xmm4[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm4 = xmm4[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm4[0,2,2,3]
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm4 = xmm4[0],xmm6[0]
 ; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa %xmm2, %xmm6
-; SSE2-NEXT:    pcmpgtd %xmm3, %xmm6
-; SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm6[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm2, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm3[1,1,3,3]
-; SSE2-NEXT:    pand %xmm7, %xmm2
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    pxor %xmm8, %xmm5
+; SSE2-NEXT:    movdqa %xmm3, %xmm6
+; SSE2-NEXT:    pcmpgtd %xmm5, %xmm6
+; SSE2-NEXT:    pcmpeqd %xmm3, %xmm5
+; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm6[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
+; SSE2-NEXT:    pand %xmm3, %xmm5
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm6[1,1,3,3]
-; SSE2-NEXT:    por %xmm2, %xmm3
-; SSE2-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm4[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm2 = xmm3[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm2 = xmm2[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[0,2,2,3]
+; SSE2-NEXT:    por %xmm5, %xmm3
+; SSE2-NEXT:    pxor %xmm8, %xmm2
+; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm5
+; SSE2-NEXT:    pxor %xmm8, %xmm5
+; SSE2-NEXT:    movdqa %xmm2, %xmm6
+; SSE2-NEXT:    pcmpgtd %xmm5, %xmm6
+; SSE2-NEXT:    pcmpeqd %xmm2, %xmm5
+; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm6[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
+; SSE2-NEXT:    pand %xmm2, %xmm5
+; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm6[1,1,3,3]
+; SSE2-NEXT:    por %xmm5, %xmm2
+; SSE2-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm3[0,2]
 ; SSE2-NEXT:    pxor %xmm8, %xmm1
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
 ; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa %xmm1, %xmm4
-; SSE2-NEXT:    pcmpgtd %xmm3, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm4[0,0,2,2]
+; SSE2-NEXT:    movdqa %xmm1, %xmm5
+; SSE2-NEXT:    pcmpgtd %xmm3, %xmm5
 ; SSE2-NEXT:    pcmpeqd %xmm1, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm3[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm4[1,1,3,3]
-; SSE2-NEXT:    por %xmm1, %xmm3
+; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm5[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm3[1,1,3,3]
+; SSE2-NEXT:    pand %xmm1, %xmm3
+; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm5[1,1,3,3]
+; SSE2-NEXT:    por %xmm3, %xmm1
 ; SSE2-NEXT:    pxor %xmm8, %xmm0
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pxor %xmm8, %xmm1
-; SSE2-NEXT:    movdqa %xmm0, %xmm4
-; SSE2-NEXT:    pcmpgtd %xmm1, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm4[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm0, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm0
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm4[1,1,3,3]
-; SSE2-NEXT:    por %xmm0, %xmm1
-; SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[0,2],xmm3[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm0 = xmm1[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pxor {{[0-9]+}}(%rsp), %xmm8
+; SSE2-NEXT:    movdqa %xmm0, %xmm3
+; SSE2-NEXT:    pcmpgtd %xmm8, %xmm3
+; SSE2-NEXT:    pcmpeqd %xmm0, %xmm8
+; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[0,0,2,2]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm8[1,1,3,3]
+; SSE2-NEXT:    pand %xmm0, %xmm5
+; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[1,1,3,3]
+; SSE2-NEXT:    por %xmm5, %xmm0
+; SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm1 = xmm2[0,2,2,3,4,5,6,7]
+; SSE2-NEXT:    pshufhw {{.*#+}} xmm1 = xmm1[0,1,2,3,4,6,6,7]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[0,2,2,3]
+; SSE2-NEXT:    pshuflw {{.*#+}} xmm0 = xmm0[0,2,2,3,4,5,6,7]
 ; SSE2-NEXT:    pshufhw {{.*#+}} xmm0 = xmm0[0,1,2,3,4,6,6,7]
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,2,2,3]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm2[0]
+; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
+; SSE2-NEXT:    pand %xmm9, %xmm4
 ; SSE2-NEXT:    pand %xmm9, %xmm0
-; SSE2-NEXT:    packuswb %xmm5, %xmm0
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pxor %xmm8, %xmm1
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    pxor %xmm8, %xmm2
-; SSE2-NEXT:    movdqa %xmm2, %xmm3
-; SSE2-NEXT:    pcmpgtd %xmm1, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm3[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm1, %xmm2
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm2[1,1,3,3]
-; SSE2-NEXT:    pand %xmm4, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm3[1,1,3,3]
-; SSE2-NEXT:    por %xmm1, %xmm2
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pxor %xmm8, %xmm1
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa %xmm3, %xmm4
-; SSE2-NEXT:    pcmpgtd %xmm1, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm4[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm1, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm3[1,1,3,3]
-; SSE2-NEXT:    pand %xmm5, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm4[1,1,3,3]
-; SSE2-NEXT:    por %xmm1, %xmm3
-; SSE2-NEXT:    shufps {{.*#+}} xmm3 = xmm3[0,2],xmm2[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm1 = xmm3[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm1 = xmm1[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[0,2,2,3]
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pxor %xmm8, %xmm1
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa %xmm3, %xmm4
-; SSE2-NEXT:    pcmpgtd %xmm1, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm4[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm1, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm3[1,1,3,3]
-; SSE2-NEXT:    pand %xmm5, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm4[1,1,3,3]
-; SSE2-NEXT:    por %xmm1, %xmm3
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm1
-; SSE2-NEXT:    pxor %xmm8, %xmm1
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    pxor %xmm8, %xmm4
-; SSE2-NEXT:    movdqa %xmm4, %xmm5
-; SSE2-NEXT:    pcmpgtd %xmm1, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm5[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm1, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm4[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm1
-; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[1,1,3,3]
-; SSE2-NEXT:    por %xmm1, %xmm4
-; SSE2-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm3[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm1 = xmm4[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm1 = xmm1[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[0,2,2,3]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm2[0]
-; SSE2-NEXT:    pand %xmm9, %xmm1
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    pxor %xmm8, %xmm2
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa %xmm3, %xmm4
-; SSE2-NEXT:    pcmpgtd %xmm2, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm4[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm2, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm3[1,1,3,3]
-; SSE2-NEXT:    pand %xmm5, %xmm2
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm4[1,1,3,3]
-; SSE2-NEXT:    por %xmm2, %xmm3
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm2
-; SSE2-NEXT:    pxor %xmm8, %xmm2
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    pxor %xmm8, %xmm4
-; SSE2-NEXT:    movdqa %xmm4, %xmm5
-; SSE2-NEXT:    pcmpgtd %xmm2, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm5[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm2, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm4[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm2
-; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[1,1,3,3]
-; SSE2-NEXT:    por %xmm2, %xmm4
-; SSE2-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm3[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm2 = xmm4[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm2 = xmm2[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[0,2,2,3]
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm4
-; SSE2-NEXT:    pxor %xmm8, %xmm4
-; SSE2-NEXT:    movdqa %xmm4, %xmm5
-; SSE2-NEXT:    pcmpgtd %xmm3, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm5[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm3, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm4[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[1,1,3,3]
-; SSE2-NEXT:    por %xmm3, %xmm4
-; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm3
-; SSE2-NEXT:    pxor %xmm8, %xmm3
-; SSE2-NEXT:    pxor {{[0-9]+}}(%rsp), %xmm8
-; SSE2-NEXT:    movdqa %xmm8, %xmm5
-; SSE2-NEXT:    pcmpgtd %xmm3, %xmm5
-; SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm5[0,0,2,2]
-; SSE2-NEXT:    pcmpeqd %xmm3, %xmm8
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm8[1,1,3,3]
-; SSE2-NEXT:    pand %xmm6, %xmm3
-; SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
-; SSE2-NEXT:    por %xmm3, %xmm5
-; SSE2-NEXT:    shufps {{.*#+}} xmm5 = xmm5[0,2],xmm4[0,2]
-; SSE2-NEXT:    pshuflw {{.*#+}} xmm3 = xmm5[0,2,2,3,4,5,6,7]
-; SSE2-NEXT:    pshufhw {{.*#+}} xmm3 = xmm3[0,1,2,3,4,6,6,7]
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm3[0,2,2,3]
-; SSE2-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm2[0]
-; SSE2-NEXT:    pand %xmm9, %xmm3
-; SSE2-NEXT:    packuswb %xmm1, %xmm3
-; SSE2-NEXT:    movdqa %xmm3, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, 2(%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, 2(%rdi)
-; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %al
-; SSE2-NEXT:    movb -{{[0-9]+}}(%rsp), %cl
-; SSE2-NEXT:    andb $1, %cl
-; SSE2-NEXT:    movb %cl, (%rdi)
-; SSE2-NEXT:    andb $1, %al
-; SSE2-NEXT:    movb %al, (%rdi)
+; SSE2-NEXT:    packuswb %xmm4, %xmm0
+; SSE2-NEXT:    psllw $7, %xmm0
+; SSE2-NEXT:    pand %xmm10, %xmm0
+; SSE2-NEXT:    pxor %xmm1, %xmm1
+; SSE2-NEXT:    pcmpgtb %xmm0, %xmm1
+; SSE2-NEXT:    movdqa %xmm1, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp)
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $4, %ecx
+; SSE2-NEXT:    orl %eax, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %eax
+; SSE2-NEXT:    andl $1, %eax
+; SSE2-NEXT:    shll $5, %eax
+; SSE2-NEXT:    orl %ecx, %eax
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $6, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $7, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $8, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $9, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $10, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $11, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $12, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $13, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $14, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $15, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $16, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $17, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $18, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $19, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $20, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $21, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $22, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $23, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $24, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $25, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $26, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $27, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $28, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    andl $1, %edx
+; SSE2-NEXT:    shll $29, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %ecx
+; SSE2-NEXT:    andl $1, %ecx
+; SSE2-NEXT:    shll $30, %ecx
+; SSE2-NEXT:    orl %edx, %ecx
+; SSE2-NEXT:    movzbl -{{[0-9]+}}(%rsp), %edx
+; SSE2-NEXT:    shll $31, %edx
+; SSE2-NEXT:    orl %ecx, %edx
+; SSE2-NEXT:    orl %eax, %edx
+; SSE2-NEXT:    movl %edx, (%rdi)
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    retq
 ;
 ; SSE42-LABEL: test_cmp_v32i64:
 ; SSE42:       # %bb.0:
-; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm8
-; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
 ; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm10
-; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm12
+; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm11
 ; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm9
-; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm14
+; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm12
+; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm8
 ; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm13
+; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm14
 ; SSE42-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm15
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm15
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm14
+; SSE42-NEXT:    shufps {{.*#+}} xmm14 = xmm14[0,2],xmm15[0,2]
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm13
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm8
+; SSE42-NEXT:    shufps {{.*#+}} xmm8 = xmm8[0,2],xmm13[0,2]
+; SSE42-NEXT:    packssdw %xmm14, %xmm8
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm12
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm9
+; SSE42-NEXT:    shufps {{.*#+}} xmm9 = xmm9[0,2],xmm12[0,2]
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm11
+; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm10
+; SSE42-NEXT:    shufps {{.*#+}} xmm10 = xmm10[0,2],xmm11[0,2]
+; SSE42-NEXT:    packssdw %xmm9, %xmm10
+; SSE42-NEXT:    packsswb %xmm8, %xmm10
+; SSE42-NEXT:    psllw $7, %xmm10
+; SSE42-NEXT:    movdqa {{.*#+}} xmm11 = [128,128,128,128,128,128,128,128,128,128,128,128,128,128,128,128]
+; SSE42-NEXT:    pand %xmm11, %xmm10
+; SSE42-NEXT:    pxor %xmm9, %xmm9
+; SSE42-NEXT:    pxor %xmm8, %xmm8
+; SSE42-NEXT:    pcmpgtb %xmm10, %xmm8
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm7
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm6
 ; SSE42-NEXT:    shufps {{.*#+}} xmm6 = xmm6[0,2],xmm7[0,2]
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm5
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm4
 ; SSE42-NEXT:    shufps {{.*#+}} xmm4 = xmm4[0,2],xmm5[0,2]
 ; SSE42-NEXT:    packssdw %xmm6, %xmm4
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm3
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm2
 ; SSE42-NEXT:    shufps {{.*#+}} xmm2 = xmm2[0,2],xmm3[0,2]
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm1
 ; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm0
 ; SSE42-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2]
 ; SSE42-NEXT:    packssdw %xmm2, %xmm0
 ; SSE42-NEXT:    packsswb %xmm4, %xmm0
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm15
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm13
-; SSE42-NEXT:    shufps {{.*#+}} xmm13 = xmm13[0,2],xmm15[0,2]
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm14
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm9
-; SSE42-NEXT:    shufps {{.*#+}} xmm9 = xmm9[0,2],xmm14[0,2]
-; SSE42-NEXT:    packssdw %xmm13, %xmm9
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm12
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm10
-; SSE42-NEXT:    shufps {{.*#+}} xmm10 = xmm10[0,2],xmm12[0,2]
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm11
-; SSE42-NEXT:    pcmpgtq {{[0-9]+}}(%rsp), %xmm8
-; SSE42-NEXT:    shufps {{.*#+}} xmm8 = xmm8[0,2],xmm11[0,2]
-; SSE42-NEXT:    packssdw %xmm10, %xmm8
-; SSE42-NEXT:    packsswb %xmm9, %xmm8
-; SSE42-NEXT:    pextrb $15, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm8, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, 2(%rdi)
-; SSE42-NEXT:    pextrb $15, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $14, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $13, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $12, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $11, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $10, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $9, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $8, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $7, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $6, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $5, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $4, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $3, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $2, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $1, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
-; SSE42-NEXT:    pextrb $0, %xmm0, %eax
-; SSE42-NEXT:    andb $1, %al
-; SSE42-NEXT:    movb %al, (%rdi)
+; SSE42-NEXT:    psllw $7, %xmm0
+; SSE42-NEXT:    pand %xmm11, %xmm0
+; SSE42-NEXT:    pcmpgtb %xmm0, %xmm9
+; SSE42-NEXT:    pextrb $1, %xmm9, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    pextrb $0, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rcx,%rax,2), %eax
+; SSE42-NEXT:    pextrb $2, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,4), %eax
+; SSE42-NEXT:    pextrb $3, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    leal (%rax,%rcx,8), %eax
+; SSE42-NEXT:    pextrb $4, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $4, %ecx
+; SSE42-NEXT:    orl %eax, %ecx
+; SSE42-NEXT:    pextrb $5, %xmm9, %eax
+; SSE42-NEXT:    andl $1, %eax
+; SSE42-NEXT:    shll $5, %eax
+; SSE42-NEXT:    orl %ecx, %eax
+; SSE42-NEXT:    pextrb $6, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $6, %ecx
+; SSE42-NEXT:    pextrb $7, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $7, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $8, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $9, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $9, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $10, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $10, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $11, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $11, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $12, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $13, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $13, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $14, %xmm9, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $14, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $15, %xmm9, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $15, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $0, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $16, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $1, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $17, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $2, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $18, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $3, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $19, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $4, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $20, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $5, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $21, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $6, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $22, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $7, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $23, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $8, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $24, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $9, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $25, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $10, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $26, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $11, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $27, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $12, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $28, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $13, %xmm8, %edx
+; SSE42-NEXT:    andl $1, %edx
+; SSE42-NEXT:    shll $29, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    pextrb $14, %xmm8, %ecx
+; SSE42-NEXT:    andl $1, %ecx
+; SSE42-NEXT:    shll $30, %ecx
+; SSE42-NEXT:    orl %edx, %ecx
+; SSE42-NEXT:    pextrb $15, %xmm8, %edx
+; SSE42-NEXT:    shll $31, %edx
+; SSE42-NEXT:    orl %ecx, %edx
+; SSE42-NEXT:    orl %eax, %edx
+; SSE42-NEXT:    movl %edx, (%rdi)
 ; SSE42-NEXT:    movq %rdi, %rax
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: test_cmp_v32i64:
 ; AVX1:       # %bb.0:
 ; AVX1-NEXT:    pushq %rbp
 ; AVX1-NEXT:    movq %rsp, %rbp
 ; AVX1-NEXT:    andq $-32, %rsp