[PowerPC] Optimize VPERM and fix code order for swapping vector operands on LE

[maryammo]

Summary

This patch reverts commit 7614ba0a5db8 to optimize VPERM when one of its
vector operands is XXSWAPD, similar to XXPERM. It also reorganizes the
little-endian swap code on LE, swapping the vector operand after
adjusting the mask operand. This ensures that the vector operand is
swapped at the correct point in the code, resulting in a valid
constant pool for the mask operand.

[stefanp]

Overall I think that this makes sense to me. I do have a couple of comments but mostly nits.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10330

Now that the swap has been moved down should this also be moved down? Or at least part of it?
I ask because I've noticed that there are a number of tests where a new vmr has been introduced and I'm wondering if that's why.

10401

nit:
Since this guard is only around a smaller section of code I think you can take the isPPC64 part out and make it part of this guard.
Like:

if (isPPC64 && (V1HasXXSWAPD || V2HasXXSWAPD || Opcode == PPCISD::XXPERM))

That way if we are not PPC64 we won't need to check any of the rest.

llvm/test/CodeGen/PowerPC/xxperm-swap.ll

20 ↗

(On Diff #516474)

This is the kind of extra copy that I mean.

[maryammo]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10330

In the following code section (line 10286), it uses the NeedSwap to adjust the mask, so this single-use operand swap needs to happen before that.
You are right, there are new vmrs, they seem to be needed based on the new vector operand order in the xxperm instruction.

[stefanp]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10330

I see what you are saying. However, maybe something like this will work:

if ((!isLittleEndian && !V2->hasOneUse() && V1->hasOneUse()) ||
        (isLittleEndian && !V1->hasOneUse() && V2->hasOneUse())) {

I tried the above on one of the tests with the extra copy and it seems to get rid of it. I have not tested this extensively so please make sure that I have not broken anything.

[maryammo]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10330

I tried this by adding your code here and removing

// Only need to place items backwards in LE,
// the mask was properly calculated.
if (isLittleEndian)
  std::swap(V1, V2);

from line 10334. It regressed bunch of test cases, looking at them I realized we can not check the endianness here because this block of code is already guarded by

if (Subtarget.hasVSX() && Subtarget.hasP9Vector() &&
      (V1->hasOneUse() || V2->hasOneUse())) {

while this check if (isLittleEndian) is for all subtargets.

[maryammo]

ping for review please.

[stefanp]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10330

I think I didn't explain this properly.

I think that you should leave the code for:

// Only need to place items backwards in LE,
// the mask was properly calculated.
if (isLittleEndian)
  std::swap(V1, V2);

at the end. That accounts for Big vs. Little Endian and as you said should be for all subtargets.

What I was thinking is just changing this if statement here. This swap is only here to try to prevent copying registers. What I'm proposing is that you take into consideration whether or not there will be a final swap at the end when you check if you want to also do this swap here.

[maryammo]

Take into account the little-endian swap when deciding on single-use operand to be second one

[maryammo]

Address review comments

[stefanp]

I think the patch mostly looks good. I have one more minor comment.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

10402

Since you have removed Opcode == PPCISD::XXPERM from other places in the code is it safe to remove it from here as well?
Sorry, I know that I suggested it be written like this but looking at how the condition has been removed everywhere else it makes sense to me to remove this one as well..

[maryammo]

Address review comments

[stefanp]

Thank you for fixing this!
LGTM!

[maryammo]

Rebase to top of trun, resolve conflicts and update tests

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

SDValue PPCTargetLowering::LowerVPERM(SDValue Op, SelectionDAG &DAG,
                                       SDValue V1, SDValue V2) const {
   unsigned Opcode = PPCISD::VPERM;
   EVT ValType = V1.getValueType();
   SDLoc dl(Op);
   bool NeedSwap = false;
   bool isLittleEndian = Subtarget.isLittleEndian();
   bool isPPC64 = Subtarget.isPPC64();
 
-  // Only need to place items backwards in LE,
-  // the mask will be properly calculated.
-  if (isLittleEndian)
-    std::swap(V1, V2);
-
   if (Subtarget.hasVSX() && Subtarget.hasP9Vector() &&
       (V1->hasOneUse() || V2->hasOneUse())) {
     LLVM_DEBUG(dbgs() << "At least one of two input vectors are dead - using "
                          "XXPERM instead\n");
     Opcode = PPCISD::XXPERM;
 
     // The second input to XXPERM is also an output so if the second input has
     // multiple uses then copying is necessary, as a result we want the
     // single-use operand to be used as the second input to prevent copying.
-    if (!V2->hasOneUse() && V1->hasOneUse()) {
+    if ((!isLittleEndian && !V2->hasOneUse() && V1->hasOneUse()) ||
+        (isLittleEndian && !V1->hasOneUse() && V2->hasOneUse())) {
       std::swap(V1, V2);
       NeedSwap = !NeedSwap;
     }
   }
 
   // The SHUFFLE_VECTOR mask is almost exactly what we want for vperm, except
   // that it is in input element units, not in bytes.  Convert now.
 
   // For little endian, the order of the input vectors is reversed, and
   // the permutation mask is complemented with respect to 31.  This is
   // necessary to produce proper semantics with the big-endian-based vperm
[   V1   |   V2  ] -> [   V2   |   V1   ]
      0-15     16-31        0-15     16-31
   i.e.  index of V1 += 16, index of V2 -= 16
   */
 
   SmallVector<SDValue, 16> ResultMask;
   for (unsigned i = 0, e = VT.getVectorNumElements(); i != e; ++i) {
     unsigned SrcElt = PermMask[i] < 0 ? 0 : PermMask[i];
 
-    if (Opcode == PPCISD::XXPERM) {
-      if (V1HasXXSWAPD) {
-        if (SrcElt < 8)
-          SrcElt += 8;
-        else if (SrcElt < 16)
-          SrcElt -= 8;
-      }
-      if (V2HasXXSWAPD) {
-        if (SrcElt > 23)
-          SrcElt -= 8;
-        else if (SrcElt > 15)
-          SrcElt += 8;
-      }
-      if (NeedSwap) {
-        if (SrcElt < 16)
-          SrcElt += 16;
-        else
-          SrcElt -= 16;
-      }
-    }
-
     for (unsigned j = 0; j != BytesPerElement; ++j)
       if (isLittleEndian)
         ResultMask.push_back(
             DAG.getConstant(31 - (SrcElt * BytesPerElement + j), dl, MVT::i32));
       else
         ResultMask.push_back(
             DAG.getConstant(SrcElt * BytesPerElement + j, dl, MVT::i32));
   }
 
-  if (Opcode == PPCISD::XXPERM && (V1HasXXSWAPD || V2HasXXSWAPD)) {
-    if (V1HasXXSWAPD) {
-      dl = SDLoc(V1->getOperand(0));
-      V1 = V1->getOperand(0)->getOperand(1);
-    }
-    if (V2HasXXSWAPD) {
-      dl = SDLoc(V2->getOperand(0));
-      V2 = V2->getOperand(0)->getOperand(1);
-    }
-    if (isPPC64 && ValType != MVT::v2f64)
+
+  if (isPPC64 && (V1HasXXSWAPD || V2HasXXSWAPD)) {
+    if (ValType != MVT::v2f64)
       V1 = DAG.getBitcast(MVT::v2f64, V1);
-    if (isPPC64 && V2.getValueType() != MVT::v2f64)
+    if (V2.getValueType() != MVT::v2f64)
       V2 = DAG.getBitcast(MVT::v2f64, V2);
   }
 
   ShufflesHandledWithVPERM++;
   SDValue VPermMask = DAG.getBuildVector(MVT::v16i8, dl, ResultMask);
   LLVM_DEBUG({
     ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
     if (Opcode == PPCISD::XXPERM) {
       dbgs() << "Emitting a XXPERM for the following shuffle:\n";
     } else {
       dbgs() << "Emitting a VPERM for the following shuffle:\n";
     }
     SVOp->dump();
     dbgs() << "With the following permute control vector:\n";
     VPermMask.dump();
   });
 
   if (Opcode == PPCISD::XXPERM)
     VPermMask = DAG.getBitcast(MVT::v4i32, VPermMask);
 
+  // Only need to place items backwards in LE,
+  // the mask was properly calculated.
+  if (isLittleEndian)
+    std::swap(V1, V2);
+
   SDValue VPERMNode =
       DAG.getNode(Opcode, dl, V1.getValueType(), V1, V2, VPermMask);
 
   VPERMNode = DAG.getBitcast(ValType, VPERMNode);
   return VPERMNode;
 }
 
 /// getVectorCompareInfo - Given an intrinsic, return false if it is not a

llvm/test/CodeGen/PowerPC/build-vector-tests.ll

 ; P8BE-NEXT:    addis r3, r2, .LCPI9_0@toc@ha
 ; P8BE-NEXT:    addi r3, r3, .LCPI9_0@toc@l
 ; P8BE-NEXT:    lxvw4x v3, 0, r3
 ; P8BE-NEXT:    vperm v2, v2, v2, v3
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fromDiffMemVarDi:
 ; P8LE:       # %bb.0: # %entry
+; P8LE-NEXT:    addis r5, r2, .LCPI9_0@toc@ha
 ; P8LE-NEXT:    sldi r4, r4, 2
+; P8LE-NEXT:    addi r5, r5, .LCPI9_0@toc@l
 ; P8LE-NEXT:    add r3, r3, r4
+; P8LE-NEXT:    lxvd2x vs0, 0, r5
 ; P8LE-NEXT:    addi r3, r3, -12
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    addis r3, r2, .LCPI9_0@toc@ha
-; P8LE-NEXT:    addi r3, r3, .LCPI9_0@toc@l
+; P8LE-NEXT:    lxvd2x v3, 0, r3
 ; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    vperm v2, v2, v2, v3
+; P8LE-NEXT:    vperm v2, v3, v3, v2
 ; P8LE-NEXT:    blr
 entry:
   %idxprom = sext i32 %elem to i64
   %arrayidx = getelementptr inbounds i32, ptr %arr, i64 %idxprom
   %0 = load i32, ptr %arrayidx, align 4
   %vecinit = insertelement <4 x i32> undef, i32 %0, i32 0
   %sub = add nsw i32 %elem, -1
   %idxprom1 = sext i32 %sub to i64

 ; P8BE-NEXT:    addi r3, r3, .LCPI18_0@toc@l
 ; P8BE-NEXT:    lxvw4x v3, 0, r3
 ; P8BE-NEXT:    vperm v2, v2, v2, v3
 ; P8BE-NEXT:    xvcvspsxws v2, v2
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fromDiffMemConsDConvftoi:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    addis r3, r2, .LCPI18_0@toc@ha
-; P8LE-NEXT:    addi r3, r3, .LCPI18_0@toc@l
+; P8LE-NEXT:    addis r4, r2, .LCPI18_0@toc@ha
+; P8LE-NEXT:    lxvd2x v3, 0, r3
+; P8LE-NEXT:    addi r4, r4, .LCPI18_0@toc@l
+; P8LE-NEXT:    lxvd2x vs0, 0, r4
 ; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    vperm v2, v2, v2, v3
+; P8LE-NEXT:    vperm v2, v3, v3, v2
 ; P8LE-NEXT:    xvcvspsxws v2, v2
 ; P8LE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds float, ptr %ptr, i64 3
   %0 = load float, ptr %arrayidx, align 4
   %conv = fptosi float %0 to i32
   %vecinit = insertelement <4 x i32> undef, i32 %conv, i32 0
   %arrayidx1 = getelementptr inbounds float, ptr %ptr, i64 2

 ; P8BE-NEXT:    addis r3, r2, .LCPI41_0@toc@ha
 ; P8BE-NEXT:    addi r3, r3, .LCPI41_0@toc@l
 ; P8BE-NEXT:    lxvw4x v3, 0, r3
 ; P8BE-NEXT:    vperm v2, v2, v2, v3
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fromDiffMemVarDui:
 ; P8LE:       # %bb.0: # %entry
+; P8LE-NEXT:    addis r5, r2, .LCPI41_0@toc@ha
 ; P8LE-NEXT:    sldi r4, r4, 2
+; P8LE-NEXT:    addi r5, r5, .LCPI41_0@toc@l
 ; P8LE-NEXT:    add r3, r3, r4
+; P8LE-NEXT:    lxvd2x vs0, 0, r5
 ; P8LE-NEXT:    addi r3, r3, -12
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    addis r3, r2, .LCPI41_0@toc@ha
-; P8LE-NEXT:    addi r3, r3, .LCPI41_0@toc@l
+; P8LE-NEXT:    lxvd2x v3, 0, r3
 ; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    vperm v2, v2, v2, v3
+; P8LE-NEXT:    vperm v2, v3, v3, v2
 ; P8LE-NEXT:    blr
 entry:
   %idxprom = sext i32 %elem to i64
   %arrayidx = getelementptr inbounds i32, ptr %arr, i64 %idxprom
   %0 = load i32, ptr %arrayidx, align 4
   %vecinit = insertelement <4 x i32> undef, i32 %0, i32 0
   %sub = add nsw i32 %elem, -1
   %idxprom1 = sext i32 %sub to i64

 ; P8BE-NEXT:    addi r3, r3, .LCPI50_0@toc@l
 ; P8BE-NEXT:    lxvw4x v3, 0, r3
 ; P8BE-NEXT:    vperm v2, v2, v2, v3
 ; P8BE-NEXT:    xvcvspuxws v2, v2
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fromDiffMemConsDConvftoui:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    addis r3, r2, .LCPI50_0@toc@ha
-; P8LE-NEXT:    addi r3, r3, .LCPI50_0@toc@l
+; P8LE-NEXT:    addis r4, r2, .LCPI50_0@toc@ha
+; P8LE-NEXT:    lxvd2x v3, 0, r3
+; P8LE-NEXT:    addi r4, r4, .LCPI50_0@toc@l
+; P8LE-NEXT:    lxvd2x vs0, 0, r4
 ; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    lxvd2x vs0, 0, r3
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    vperm v2, v2, v2, v3
+; P8LE-NEXT:    vperm v2, v3, v3, v2
 ; P8LE-NEXT:    xvcvspuxws v2, v2
 ; P8LE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds float, ptr %ptr, i64 3
   %0 = load float, ptr %arrayidx, align 4
   %conv = fptoui float %0 to i32
   %vecinit = insertelement <4 x i32> undef, i32 %conv, i32 0
   %arrayidx1 = getelementptr inbounds float, ptr %ptr, i64 2

llvm/test/CodeGen/PowerPC/v16i8_scalar_to_vector_shuffle.ll

 entry:
   %vecins = insertelement <16 x i8> %a, i8 %b, i32 0
   ret <16 x i8> %vecins
 }
 
 define <16 x i8> @test_none_v16i8(i8 %arg, ptr nocapture noundef readonly %b) {
 ; CHECK-LE-P8-LABEL: test_none_v16i8:
 ; CHECK-LE-P8:       # %bb.0: # %entry
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
+; CHECK-LE-P8-NEXT:    addis r5, r2, .LCPI2_0@toc@ha
+; CHECK-LE-P8-NEXT:    lxvd2x v3, 0, r4
 ; CHECK-LE-P8-NEXT:    mtvsrd v4, r3
-; CHECK-LE-P8-NEXT:    addi r4, r4, .LCPI2_0@toc@l
+; CHECK-LE-P8-NEXT:    addi r5, r5, .LCPI2_0@toc@l
+; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r5
 ; CHECK-LE-P8-NEXT:    xxswapd v2, vs0
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    xxswapd v3, vs0
-; CHECK-LE-P8-NEXT:    vperm v2, v4, v2, v3
+; CHECK-LE-P8-NEXT:    vperm v2, v4, v3, v2
 ; CHECK-LE-P8-NEXT:    blr
 ;
 ; CHECK-LE-P9-LABEL: test_none_v16i8:
 ; CHECK-LE-P9:       # %bb.0: # %entry
 ; CHECK-LE-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-LE-P9-NEXT:    addis r3, r2, .LCPI2_0@toc@ha
 ; CHECK-LE-P9-NEXT:    lxv vs0, 0(r4)
 ; CHECK-LE-P9-NEXT:    addi r3, r3, .LCPI2_0@toc@l
entry:
   %lhs = bitcast <8 x i16> %lhs.tmp to <16 x i8>
   %shuffle = shufflevector <16 x i8> %lhs, <16 x i8> %rhs, <16 x i32> <i32 0, i32 1, i32 16, i32 17, i32 18, i32 19, i32 20, i32 21, i32 22, i32 23, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
   ret <16 x i8> %shuffle
 }
 
 define <16 x i8> @test_none_v8i16(i16 %arg, ptr nocapture noundef readonly %b) {
 ; CHECK-LE-P8-LABEL: test_none_v8i16:
 ; CHECK-LE-P8:       # %bb.0: # %entry
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    addis r4, r2, .LCPI5_0@toc@ha
+; CHECK-LE-P8-NEXT:    addis r5, r2, .LCPI5_0@toc@ha
+; CHECK-LE-P8-NEXT:    lxvd2x v3, 0, r4
 ; CHECK-LE-P8-NEXT:    mtvsrd v4, r3
-; CHECK-LE-P8-NEXT:    addi r4, r4, .LCPI5_0@toc@l
+; CHECK-LE-P8-NEXT:    addi r5, r5, .LCPI5_0@toc@l
+; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r5
 ; CHECK-LE-P8-NEXT:    xxswapd v2, vs0
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    xxswapd v3, vs0
-; CHECK-LE-P8-NEXT:    vperm v2, v4, v2, v3
+; CHECK-LE-P8-NEXT:    vperm v2, v4, v3, v2
 ; CHECK-LE-P8-NEXT:    blr
 ;
 ; CHECK-LE-P9-LABEL: test_none_v8i16:
 ; CHECK-LE-P9:       # %bb.0: # %entry
 ; CHECK-LE-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-LE-P9-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
 ; CHECK-LE-P9-NEXT:    lxv vs0, 0(r4)
 ; CHECK-LE-P9-NEXT:    addi r3, r3, .LCPI5_0@toc@l

llvm/test/CodeGen/PowerPC/v8i16_scalar_to_vector_shuffle.ll

entry:
   %2 = zext <4 x i16> %shuffle1 to <4 x i32>
   store <4 x i32> %2, ptr undef, align 16
   ret void
 }
 
 define void @test_none_v2i64(ptr nocapture readonly %ptr1, ptr nocapture readonly %ptr2) {
 ; CHECK-LE-P8-LABEL: test_none_v2i64:
 ; CHECK-LE-P8:       # %bb.0: # %entry
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    addis r4, r2, .LCPI4_0@toc@ha
-; CHECK-LE-P8-NEXT:    lxsdx v4, 0, r3
+; CHECK-LE-P8-NEXT:    addis r5, r2, .LCPI4_0@toc@ha
+; CHECK-LE-P8-NEXT:    lxsdx v3, 0, r3
 ; CHECK-LE-P8-NEXT:    addis r3, r2, .LCPI4_1@toc@ha
-; CHECK-LE-P8-NEXT:    addi r4, r4, .LCPI4_0@toc@l
+; CHECK-LE-P8-NEXT:    lxvd2x v4, 0, r4
+; CHECK-LE-P8-NEXT:    addi r5, r5, .LCPI4_0@toc@l
 ; CHECK-LE-P8-NEXT:    addi r3, r3, .LCPI4_1@toc@l
-; CHECK-LE-P8-NEXT:    lxvd2x vs1, 0, r4
+; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r5
 ; CHECK-LE-P8-NEXT:    xxswapd v2, vs0
 ; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r3
-; CHECK-LE-P8-NEXT:    xxswapd v3, vs1
-; CHECK-LE-P8-NEXT:    vperm v2, v4, v2, v3
-; CHECK-LE-P8-NEXT:    xxswapd v3, vs0
+; CHECK-LE-P8-NEXT:    vperm v2, v3, v4, v2
 ; CHECK-LE-P8-NEXT:    xxlxor v4, v4, v4
+; CHECK-LE-P8-NEXT:    xxswapd v3, vs0
 ; CHECK-LE-P8-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-LE-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-LE-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-LE-P8-NEXT:    blr
 ;
 ; CHECK-LE-P9-LABEL: test_none_v2i64:
 ; CHECK-LE-P9:       # %bb.0: # %entry
 ; CHECK-LE-P9-NEXT:    lfd f0, 0(r3)

llvm/test/CodeGen/PowerPC/vperm-swap.ll

 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -ppc-asm-full-reg-names \
 ; RUN:   -ppc-vsr-nums-as-vr -mtriple=powerpc64le-unknown-linux-gnu < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-LE-P8
 
 define <16 x i8> @test_none_v16i8(i8 %arg, ptr nocapture noundef readonly %b) {
 ; CHECK-LE-P8: .LCPI0_0:
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   30                              # 0x1e
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   22                              # 0x16
 ; CHECK-LE-P8-NEXT: .byte   7                               # 0x7
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
-; CHECK-LE-P8-NEXT: .byte   31                              # 0x1f
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
+; CHECK-LE-P8-NEXT: .byte   23                              # 0x17
 ; CHECK-LE-P8-LABEL: test_none_v16i8:
 ; CHECK-LE-P8:       # %bb.0: # %entry
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    addis r4, r2, .LCPI0_0@toc@ha
+; CHECK-LE-P8-NEXT:    addis r5, r2, .LCPI0_0@toc@ha
+; CHECK-LE-P8-NEXT:    lxvd2x v3, 0, r4
 ; CHECK-LE-P8-NEXT:    mtvsrd v4, r3
-; CHECK-LE-P8-NEXT:    addi r4, r4, .LCPI0_0@toc@l
+; CHECK-LE-P8-NEXT:    addi r5, r5, .LCPI0_0@toc@l
+; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r5
 ; CHECK-LE-P8-NEXT:    xxswapd v2, vs0
-; CHECK-LE-P8-NEXT:    lxvd2x vs0, 0, r4
-; CHECK-LE-P8-NEXT:    xxswapd v3, vs0
-; CHECK-LE-P8-NEXT:    vperm v2, v4, v2, v3
+; CHECK-LE-P8-NEXT:    vperm v2, v4, v3, v2
 ; CHECK-LE-P8-NEXT:    blr
 entry:
   %lhs = load <16 x i8>, ptr %b, align 4
   %rhs = insertelement <16 x i8> undef, i8 %arg, i32 0
   %shuffle = shufflevector <16 x i8> %lhs, <16 x i8> %rhs, <16 x i32> <i32 0, i32 1, i32 16, i32 17, i32 18, i32 19, i32 20, i32 21, i32 22, i32 23, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
   ret <16 x i8> %shuffle
 }