[PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

[nemanjai]

Summary

We currently miss a number of opportunities to emit single-instruction VMRG[LH][BHW] instructions for shuffles on little endian subtargets. Although this in itself is not a huge performance opportunity since loading the permute vector for a VPERM can always be pulled out of loops, producing such merge instructions is useful to downstream optimizations.
Since VPERM is essentially opaque to all subsequent optimizations, we want to avoid it as much as possible. Other permute instructions have semantics that can be reasoned about much more easily in later optimizations.

This patch does the following:

• Canonicalize shuffles so that the first element comes from the first vector (since that's what most of the mask matching functions want)
• Switch the elements that come from splat vectors so that they match the corresponding elements from the other vector (to allow for merges)
• Adds debugging messages for when a shuffle is matched to a VPERM so that anyone interested in improving this further can get the info for their code

[nemanjai]

Updated to reflect changes in the pre-committed new test case.

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14172

I see that our other combine functions have an assert in the beginning checking the opcode. It might make sense to have one here checking SVN->getOpcode() == ISD::VECTOR_SHUFFLE?

14190

Is it possible to add a comment regarding the splats here? Or is the comment above supposed to explain this bit, too?

14206

s/condusive/conducive

[nemanjai]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14172

The reason for the asserts in other combines is that they do not take a specialized node. This takes a ShuffleVectorSDNode pointer. It is not possible for a shuffle vector node to have an opcode other than vector shuffle.

[nemanjai]

Add handling for shuffles that are fed by SCALAR_TO_VECTOR to avoid the swap before the shuffle.

[lei]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14162

Can this and the for-loop below be a range based for-loop?

for (auto FirstOp : Op->op_values())
  if (!FirstOp.isUndef())
    break;

llvm/lib/Target/PowerPC/PPCISelLowering.h

226

Maybe consider using existing naming used for scalar and vector ISD nodes: SCALAR_TO_VEC_PERMUTED

[RolandF]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

9735

I don't think it is safe to use a load and splat for a smaller than word sized load. The larger load might now cross a page boundary.

14202

This code is only used on the early, conditional returns, and not on the final return. The function could be refactored such that the final return case is moved first, or the early returns could return the result of a function that generates the instruction, to prevent unused code.

[RolandF]

llvm/test/CodeGen/PowerPC/build-vector-tests.ll

1724

Looks like the code got worse here?

3243

This code is also worse.

4917

Okay it looks like there are a number of similar cases where the code is slightly worse. I will stop flagging them individually.

[RolandF]

llvm/lib/Target/PowerPC/PPCISelLowering.h

226

Can we rename this opcode? S and V are too short to have meaning and permute is too general. Maybe SCALAR_TO_VECTOR_BE or _RIGHT or _UPPER or something?

[NeHuang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14193

nit: Is it possible to combine line 14068 and 14066 into one if check since the two operations are same?

14231

nit: Little -> little

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14162

Can we use llvm::any_of to make the code more compact ?

[nemanjai]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

9735

I think this code is a bit confusing because of the <=. I actually don't really know why I used <= rather than == since the memory value type can never be narrower than the splat width since we are checking for normal (unindexed, non-extending) loads.

14162

How? I need to find the first operand that is *not* undef.

14162

But then FirstOp is out of scope after the loop and we need it.

14202

I am not sure this would be an improvement. The early exit conditions are the common case. The final return is only for the pattern:
(<n x Ty> (scalar_to_vector (Ty (extract_elt <n x Ty> %a, C)))).

I'll refactor it as per your suggestion and if it is more readable we'll go with that.
Unused nodes aren't really a problem - the SDAG will just get rid of them, but I agree that it is less than ideal to create nodes that will just be discarded.

llvm/lib/Target/PowerPC/PPCISelLowering.h

226

SCALAR_TO_VECTOR_BE is close but the problem is that SCALAR_TO_VECTOR assumes the scalar is placed into vector element zero. However, (almost) none of the instructions we have actually put the scalar into vector element zero for either endianness. The value always goes to the least significant portion of the most significant doubleword. It is very hard to encapsulate that level of weirdness in a name.

I think I will opt for either SCALAR_TO_VECTOR_PERMUTED or SCALAR_TO_VECTOR_PPC with a more detailed comments:

/// PowerPC instructions that have SCALAR_TO_VECTOR semantics tend to
/// place the value into the least significant element of the most significant
/// doubleword in the vector. This is not element zero for anything smaller
/// than a doubleword on either endianness. This node has the same semantics
/// as SCALAR_TO_VECTOR except that the value remains in the
/// aforementioned location in the vector register.

If you have a preference for either of those, please let me know.

[anil9]

llvm/lib/Target/PowerPC/PPCInstrVSX.td

2179–2180

Indentation discrepancy.

2179–2180

Indentation discrepancy.

2507–2508

Same as before.

[nemanjai]

• Fix some cases where code got worse
• Refactor some code for mode readability
• Fix some naming and comments

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14162

Ah, sorry. Maybe, llvm::find_if_not is the right one. But it seems not help too much for this case. So, it depends on you.

[RolandF]

LGTM

[nathanchance]

Hi! This patch causes a crash when compiling the Linux kernel with certain distribution configs:

$ git bisect log
# bad: [b885b1b92d3d9bfcc8229476f4ad4660305d6160] [mlir] Fix gcc build break due to previous commit
# good: [158e734af19d6be206f80c213a028b569c441b24] [ARM] Adjust AND/OR combines to not call isConstantSplat on i1 vectors. NFC.
git bisect start 'b885b1b92d3d9bfcc8229476f4ad4660305d6160' '158e734af19d6be206f80c213a028b569c441b24'
# good: [fcd67665a8de61223313e1e1582faf17d9ee76b8] [StackSafety] Add "Must Live" logic
git bisect good fcd67665a8de61223313e1e1582faf17d9ee76b8
# bad: [d3b752845df0331348dad48000fc8b82afb3de5b] [clang][test][NFC] Also test for serialization in AST dump tests, part 1/n.
git bisect bad d3b752845df0331348dad48000fc8b82afb3de5b
# bad: [bb480056602daab86fbcd6aac5c6bc92ce350bb3] [NFC] Make AST_BLOCK_HASH test more robust with downstream changes
git bisect bad bb480056602daab86fbcd6aac5c6bc92ce350bb3
# bad: [d938ec4509c47d461377527fc2877ae14b91275c] [AArch64] Avoid incompatibility between SLSBLR mitigation and BTI codegen.
git bisect bad d938ec4509c47d461377527fc2877ae14b91275c
# good: [9ca50e887db7f903c04a90593d2beed8a96794f1] [libTooling] Add parser for string representation of `RangeSelector`.
git bisect good 9ca50e887db7f903c04a90593d2beed8a96794f1
# bad: [1fed131660b2c5d3ea7007e273a7a5da80699445] [PowerPC] Canonicalize shuffles to match more single-instruction masks on LE
git bisect bad 1fed131660b2c5d3ea7007e273a7a5da80699445
# good: [9c9b71a2908d47ebd65cb7c2e1e499484aaa547e] [gn build] Port 9ca50e887db
git bisect good 9c9b71a2908d47ebd65cb7c2e1e499484aaa547e
# good: [8f3b2c8aa3175628128a32a6bcaecc67efd03514] AMDGPU/GlobalISel: Remove selection of MAD/MAC when not available
git bisect good 8f3b2c8aa3175628128a32a6bcaecc67efd03514
# first bad commit: [1fed131660b2c5d3ea7007e273a7a5da80699445] [PowerPC] Canonicalize shuffles to match more single-instruction masks on LE

To reproduce with the Linux kernel source tree:

$ mkdir -p out/ppc64le

$ curl -LSso out/ppc64le/.config 'https://git.kernel.org/pub/scm/linux/kernel/git/jwboyer/fedora.git/plain/fedora/configs/kernel-5.6.17-ppc64le.config?h=kernel-5.6.17-300.fc32'

$ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc64le-linux-gnu- LD=powerpc64le-linux-gnu-ld LLVM=1 O=out/ppc64le OBJDUMP=powerpc64le-linux-gnu-objdump olddefconfig drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.o
...
fatal error: error in backend: Cannot select: t146: v16i8 = PPCISD::SCALAR_TO_VECTOR_PERMUTED t110, drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:2855:3 @[ drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:241:2 ]
  t110: i32 = any_extend t32, drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:2855:3 @[ drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:241:2 ]
    t32: i1,ch = CopyFromReg t0, Register:i1 %31, drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:2855:3 @[ drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.c:241:2 ]
      t31: i1 = Register %31
In function: dml20_recalculate
clang-11: error: clang frontend command failed with exit code 70 (use -v to see invocation)
ClangBuiltLinux clang version 11.0.0 (https://github.com/llvm/llvm-project b885b1b92d3d9bfcc8229476f4ad4660305d6160)
...

cvise spits out (not cleaned up or anything):

enum { a, b } c;
d;
e() {
  _Bool f = a;
  for (; d; ++d)
    if (c)
      f = b;
  if (f)
    g();
}

Interestingness test and original preprocessed file available here: https://github.com/nathanchance/creduce-files/tree/ac42e9b24f969b3bd8a72d0a8be0bccb88e570d2/D77448

[nathanchance]

Gentle ping. This is still an issue at 4c6548222b3c41d024581d28f42b3f02510bcfe3 and I have not heard anything about a fix. I know I sent that message on Friday night before the weekend so I am hoping that it did not get lost. If I need to file a bug report, I am more than happy to do so.

[nemanjai]

In D77448#2116050, @nathanchance wrote:

Gentle ping. This is still an issue at 4c6548222b3c41d024581d28f42b3f02510bcfe3 and I have not heard anything about a fix. I know I sent that message on Friday night before the weekend so I am hoping that it did not get lost. If I need to file a bug report, I am more than happy to do so.

I am really sorry I missed this message. Should be fixed in https://reviews.llvm.org/rG57ad8f4.

[nathanchance]

@nemanjai thanks for the fix, I can confirm all of my PowerPC builds are successful with LLVM at 4772b99dffec4f87bb7bc9273495066058ac0186!

[MaskRay]

Our internal testing finds a correctness bug related to this patch (after the last known fix eafe7c14ea38946e8c1fb64d548effaee7614718)

The code is at https://github.com/google/dimsum/blob/master/dimsum_fuzz.cc#L402

With the following patch it can build with -mcpu=pwr9 -O0. When I feed the executable dimsum_fuzz itself to dimsum_fuzz, it triggers __builtin_trap because hmin (horizontal min) on typedef signed char simd_char __attribute__((vector_size__(8))); computed result is different from a naive algorithm. I'll inform you when I can get a minimized reproduce or in the meantime if you can find problem that'd also be nice:)

diff --git i/dimsum_fuzz.cc w/dimsum_fuzz.cc
index fafd1fa..b7188fd 100644
--- i/dimsum_fuzz.cc
+++ w/dimsum_fuzz.cc
@@ -532 +532,6 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
 }
+
+int main() {
+  const long data[] = {0x00010102464c457f, 0x0000000000000000};
+  LLVMFuzzerTestOneInput((const uint8_t*)data, 16);
+}
diff --git i/simd_vsx.h w/simd_vsx.h
index 5c59e2a..15a32fe 100644
--- i/simd_vsx.h
+++ w/simd_vsx.h
@@ -23,2 +23,6 @@
 
+inline __vector long vec_abs(__vector long a) {
+  return (__vector long)vec_abs((__vector long long)a);
+}
+
 namespace dimsum {

cgdb session

2004x typename std::enable_if<(_SimdType::size() > 1 && is_simd<_SimdType>::value &&
2005x                          __floor_pow_of_2(_SimdType::size()) ==
2006x                              _SimdType::size()),
2007x                         typename _SimdType::value_type>::type
2008x __hmin(const _SimdType& __v) {
2009x   auto __arr = split_by<2>(__v);
2010x   return __hmin(min(__arr[0], __arr[1]));                                                                                                       
2011x }
2012x
2013x template <class _SimdType>
2014x typename std::enable_if<(_SimdType::size() > 1 &&
2015x                          !(is_simd<_SimdType>::value &&
2016x                            __floor_pow_of_2(_SimdType::size()) ==
/home/maskray/test/src/third_party/dimsum/simd.h                                                                                                      
t third_party/dimsum/simd.h:2079
0x0000000010083c40 in std::experimental::__hmin<std::experimental::simd<signed char, std::experimental::__simd_abi<(std::experimental::_StorageKind)2,
 16> > > (__v=...) at third_party/dimsum/simd.h:2010
Value returned is $2 = <incomplete type>
(gdb) s
std::experimental::__hmin<std::experimental::simd<signed char, std::experimental::__simd_abi<(std::experimental::_StorageKind)2, 8> > > (__v=...) at t
hird_party/dimsum/simd.h:2009
(gdb) n
(gdb) x/2xg &__v
0x7fffffffed88: 0x0000000000000000      0x00010102464c457f
(gdb) x/2xg &__arr
0x7fffffffecf8: 0x00007fff00000000      0x00007fffffffed88
(gdb) p sizeof(__arr)            # under the hood this is a signed char vector_size(8) type
$3 = 8
(gdb) 

------

142x template <typename T>
143x void TestHMin(const uint8_t* data) {
144x   NativeSimd<T> simd;
145x   LoadFromRaw(data, &simd);
146x   if (!IsNormal(simd)) return;
147x   if (dimsum::simulated::hmin(simd) != dimsum::hmin(simd))     ######   dimsum::hmin(simd) is wrong                                                                                
148x     __builtin_trap();
149x }

[MaskRay]

@nemanjai See https://gist.github.com/MaskRay/ba7547f6a587176666ff77527cf12c4d for a reproduce.

[nemanjai]

In D77448#2165900, @MaskRay wrote:

@nemanjai See https://gist.github.com/MaskRay/ba7547f6a587176666ff77527cf12c4d for a reproduce.

Fixed in https://reviews.llvm.org/rG7d076e19e31a
I'll wait for it to go through the bots and then I'll open a PR to ask for a merge into 11.0. Thanks for reporting this and providing the repro.

llvm/lib/Target/PowerPC/PPCISelLowering.h

enum NodeType : unsigned {
     /// type to a legal floating point vector type.
     /// For example v2i32 -> widened to v4i32 -> v2f64
     SINT_VEC_TO_FP,
 
     /// Extract a subvector from unsigned integer vector and convert to FP.
     /// As with SINT_VEC_TO_FP, used for converting illegal types.
     UINT_VEC_TO_FP,
 
+    /// PowerPC instructions that have SCALAR_TO_VECTOR semantics tend to
+    /// place the value into the least significant element of the most
+    /// significant doubleword in the vector. This is not element zero for
+    /// anything smaller than a doubleword on either endianness. This node has
+    /// the same semantics as SCALAR_TO_VECTOR except that the value remains in
+    /// the aforementioned location in the vector register.
+    SCALAR_TO_VECTOR_PERMUTED,
+
     // FIXME: Remove these once the ANDI glue bug is fixed:
     /// i1 = ANDI_rec_1_[EQ|GT]_BIT(i32 or i64 x) - Represents the result of the
     /// eq or gt bit of CR0 after executing andi. x, 1. This is used to
     /// implement truncation of i32 or i64 to i1.
     ANDI_rec_1_EQ_BIT,
     ANDI_rec_1_GT_BIT,
 
     // READ_TIME_BASE - A read of the 64-bit time-base register on a 32-bit
private:
     SDValue combineSRL(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineMUL(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineADD(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineFMALike(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineTRUNCATE(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineSetCC(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineABS(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineVSelect(SDNode *N, DAGCombinerInfo &DCI) const;
+    SDValue combineVectorShuffle(ShuffleVectorSDNode *SVN,
+                                 SelectionDAG &DAG) const;
     SDValue combineVReverseMemOP(ShuffleVectorSDNode *SVN, LSBaseSDNode *LSBase,
                                  DAGCombinerInfo &DCI) const;
 
     /// ConvertSETCCToSubtract - looks at SETCC that compares ints. It replaces
     /// SETCC with integer subtraction when (1) there is a legal way of doing it
     /// (2) keeping the result of comparison in GPR has performance benefit.
     SDValue ConvertSETCCToSubtract(SDNode *N, DAGCombinerInfo &DCI) const;
 

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

 static cl::opt<bool> EnableQuadPrecision("enable-ppc-quad-precision",
 cl::desc("enable quad precision float support on ppc"), cl::Hidden);
 
 static cl::opt<bool> UseAbsoluteJumpTables("ppc-use-absolute-jumptables",
 cl::desc("use absolute jump tables on ppc"), cl::Hidden);
 
 STATISTIC(NumTailCalls, "Number of tail calls");
 STATISTIC(NumSiblingCalls, "Number of sibling calls");
+STATISTIC(ShufflesHandledWithVPERM, "Number of shuffles lowered to a VPERM");
 
 static bool isNByteElemShuffleMask(ShuffleVectorSDNode *, unsigned, int);
 
 static SDValue widenVec(SelectionDAG &DAG, SDValue Vec, const SDLoc &dl);
 
 // FIXME: Remove this once the bug has been fixed!
 extern cl::opt<bool> ANDIGlueBug;
 
const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   case PPCISD::EH_SJLJ_SETJMP:  return "PPCISD::EH_SJLJ_SETJMP";
   case PPCISD::EH_SJLJ_LONGJMP: return "PPCISD::EH_SJLJ_LONGJMP";
   case PPCISD::MFOCRF:          return "PPCISD::MFOCRF";
   case PPCISD::MFVSR:           return "PPCISD::MFVSR";
   case PPCISD::MTVSRA:          return "PPCISD::MTVSRA";
   case PPCISD::MTVSRZ:          return "PPCISD::MTVSRZ";
   case PPCISD::SINT_VEC_TO_FP:  return "PPCISD::SINT_VEC_TO_FP";
   case PPCISD::UINT_VEC_TO_FP:  return "PPCISD::UINT_VEC_TO_FP";
+  case PPCISD::SCALAR_TO_VECTOR_PERMUTED:
+    return "PPCISD::SCALAR_TO_VECTOR_PERMUTED";
   case PPCISD::ANDI_rec_1_EQ_BIT:
     return "PPCISD::ANDI_rec_1_EQ_BIT";
   case PPCISD::ANDI_rec_1_GT_BIT:
     return "PPCISD::ANDI_rec_1_GT_BIT";
   case PPCISD::VCMP:            return "PPCISD::VCMP";
   case PPCISD::VCMPo:           return "PPCISD::VCMPo";
   case PPCISD::LBRX:            return "PPCISD::LBRX";
   case PPCISD::STBRX:           return "PPCISD::STBRX";
static bool usePartialVectorLoads(SDNode *N, const PPCSubtarget& ST) {
 
   SDValue LoadedVal(N, 0);
   if (!LoadedVal.hasOneUse())
     return false;
 
   for (SDNode::use_iterator UI = LD->use_begin(), UE = LD->use_end();
        UI != UE; ++UI)
     if (UI.getUse().get().getResNo() == 0 &&
-        UI->getOpcode() != ISD::SCALAR_TO_VECTOR)
+        UI->getOpcode() != ISD::SCALAR_TO_VECTOR &&
+        UI->getOpcode() != PPCISD::SCALAR_TO_VECTOR_PERMUTED)
       return false;
 
   return true;
 }
 
 /// getPreIndexedAddressParts - returns true by value, base pointer and
 /// offset pointer and addressing mode by reference if the node's address
 /// can be legally represented as pre-indexed load / store address.
SDValue PPCTargetLowering::LowerBITCAST(SDValue Op, SelectionDAG &DAG) const {
   return DAG.getNode(PPCISD::BUILD_FP128, dl, MVT::f128, Op0.getOperand(0),
                      Op0.getOperand(1));
 }
 
 static const SDValue *getNormalLoadInput(const SDValue &Op) {
   const SDValue *InputLoad = &Op;
   if (InputLoad->getOpcode() == ISD::BITCAST)
     InputLoad = &InputLoad->getOperand(0);
-  if (InputLoad->getOpcode() == ISD::SCALAR_TO_VECTOR)
+  if (InputLoad->getOpcode() == ISD::SCALAR_TO_VECTOR ||
+      InputLoad->getOpcode() == PPCISD::SCALAR_TO_VECTOR_PERMUTED)
     InputLoad = &InputLoad->getOperand(0);
   if (InputLoad->getOpcode() != ISD::LOAD)
     return nullptr;
   LoadSDNode *LD = cast<LoadSDNode>(*InputLoad);
   return ISD::isNormalLoad(LD) ? InputLoad : nullptr;
 }
 
 // If this is a case we can't handle, return null and let the default

 /// return the code it can be lowered into.  Worst case, it can always be
 /// lowered into a vperm.
 SDValue PPCTargetLowering::LowerVECTOR_SHUFFLE(SDValue Op,
                                                SelectionDAG &DAG) const {
   SDLoc dl(Op);
   SDValue V1 = Op.getOperand(0);
   SDValue V2 = Op.getOperand(1);
   ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
+
+  // Any nodes that were combined in the target-independent combiner prior
+  // to vector legalization will not be sent to the target combine. Try to
+  // combine it here.
+  if (SDValue NewShuffle = combineVectorShuffle(SVOp, DAG)) {
+    DAG.ReplaceAllUsesOfValueWith(Op, NewShuffle);
+    Op = NewShuffle;
+    SVOp = cast<ShuffleVectorSDNode>(Op);
+  }
   EVT VT = Op.getValueType();
   bool isLittleEndian = Subtarget.isLittleEndian();
 
   unsigned ShiftElts, InsertAtByte;
   bool Swap = false;
 
   // If this is a load-and-splat, we can do that with a single instruction
   // in some cases. However if the load has multiple uses, we don't want to
if (InputLoad && Subtarget.hasVSX() && V2.isUndef() &&
     LoadSDNode *LD = cast<LoadSDNode>(*InputLoad);
     // For 4-byte load-and-splat, we need Power9.
     if ((IsFourByte && Subtarget.hasP9Vector()) || !IsFourByte) {
       uint64_t Offset = 0;
       if (IsFourByte)
         Offset = isLittleEndian ? (3 - SplatIdx) * 4 : SplatIdx * 4;
       else
         Offset = isLittleEndian ? (1 - SplatIdx) * 8 : SplatIdx * 8;
+
+      // If we are loading a partial vector, it does not make sense to adjust
+      // the base pointer. This happens with (splat (s_to_v_permuted (ld))).
+      if (LD->getMemoryVT().getSizeInBits() == (IsFourByte ? 32 : 64))
+        Offset = 0;
       SDValue BasePtr = LD->getBasePtr();
       if (Offset != 0)
         BasePtr = DAG.getNode(ISD::ADD, dl, getPointerTy(DAG.getDataLayout()),
                               BasePtr, DAG.getIntPtrConstant(Offset, dl));
       SDValue Ops[] = {
         LD->getChain(),    // Chain
         BasePtr,           // BasePtr
         DAG.getValueType(Op.getValueType()) // VT
for (unsigned j = 0; j != BytesPerElement; ++j)
       if (isLittleEndian)
         ResultMask.push_back(DAG.getConstant(31 - (SrcElt*BytesPerElement + j),
                                              dl, MVT::i32));
       else
         ResultMask.push_back(DAG.getConstant(SrcElt*BytesPerElement + j, dl,
                                              MVT::i32));
   }
 
+  ShufflesHandledWithVPERM++;
   SDValue VPermMask = DAG.getBuildVector(MVT::v16i8, dl, ResultMask);
+  LLVM_DEBUG(dbgs() << "Emitting a VPERM for the following shuffle:\n");
+  LLVM_DEBUG(SVOp->dump());
+  LLVM_DEBUG(dbgs() << "With the following permute control vector:\n");
+  LLVM_DEBUG(VPermMask.dump());
+
   if (isLittleEndian)
     return DAG.getNode(PPCISD::VPERM, dl, V1.getValueType(),
                        V2, V1, VPermMask);
   else
     return DAG.getNode(PPCISD::VPERM, dl, V1.getValueType(),
                        V1, V2, VPermMask);
 }
 
Val = DAG.getMemIntrinsicNode(PPCISD::ST_VSR_SCAL_INT, dl,
           DAG.getVTList(MVT::Other), Ops,
           cast<StoreSDNode>(N)->getMemoryVT(),
           cast<StoreSDNode>(N)->getMemOperand());
 
   DCI.AddToWorklist(Val.getNode());
   return Val;
 }
 
+static bool isAlternatingShuffMask(const ArrayRef<int> &Mask, int NumElts) {
+  // Check that the source of the element keeps flipping
+  // (i.e. Mask[i] < NumElts -> Mask[i+i] >= NumElts).
+  bool PrevElemFromFirstVec = Mask[0] < NumElts;
+  for (int i = 1, e = Mask.size(); i < e; i++) {
+    if (PrevElemFromFirstVec && Mask[i] < NumElts)
+      return false;
+    if (!PrevElemFromFirstVec && Mask[i] >= NumElts)
+      return false;
+    PrevElemFromFirstVec = !PrevElemFromFirstVec;
+  }
+  return true;
+}
+
+static bool isSplatBV(SDValue Op) {
+  if (Op.getOpcode() != ISD::BUILD_VECTOR)
+    return false;
+  SDValue FirstOp;
+
+  // Find first non-undef input.
+  for (int i = 0, e = Op.getNumOperands(); i < e; i++) {
+    FirstOp = Op.getOperand(i);
+    if (!FirstOp.isUndef())
+      break;
+  }
+
+  // All inputs are undef or the same as the first non-undef input.
+  for (int i = 1, e = Op.getNumOperands(); i < e; i++)
+    if (Op.getOperand(i) != FirstOp && !Op.getOperand(i).isUndef())
+      return false;
+  return true;
+}
+
+static SDValue isScalarToVec(SDValue Op) {
+  if (Op.getOpcode() == ISD::SCALAR_TO_VECTOR)
+    return Op;
+  if (Op.getOpcode() != ISD::BITCAST)
+    return SDValue();
+  Op = Op.getOperand(0);
+  if (Op.getOpcode() == ISD::SCALAR_TO_VECTOR)
+    return Op;
+  return SDValue();
+}
+
+static void fixupShuffleMaskForPermutedSToV(SmallVectorImpl<int> &ShuffV,
+                                            int LHSMaxIdx, int RHSMinIdx,
+                                            int RHSMaxIdx, int HalfVec) {
+  for (int i = 0, e = ShuffV.size(); i < e; i++) {
+    int Idx = ShuffV[i];
+    if ((Idx >= 0 && Idx < LHSMaxIdx) || (Idx >= RHSMinIdx && Idx < RHSMaxIdx))
+      ShuffV[i] += HalfVec;
+  }
+  return;
+}
+
+// Replace a SCALAR_TO_VECTOR with a SCALAR_TO_VECTOR_PERMUTED except if
+// the original is:
+// (<n x Ty> (scalar_to_vector (Ty (extract_elt <n x Ty> %a, C))))
+// In such a case, just change the shuffle mask to extract the element
+// from the permuted index.
+static SDValue getSToVPermuted(SDValue OrigSToV, SelectionDAG &DAG) {
+  SDLoc dl(OrigSToV);
+  EVT VT = OrigSToV.getValueType();
+  assert(OrigSToV.getOpcode() == ISD::SCALAR_TO_VECTOR &&
+         "Expecting a SCALAR_TO_VECTOR here");
+  SDValue Input = OrigSToV.getOperand(0);
+
+  if (Input.getOpcode() == ISD::EXTRACT_VECTOR_ELT) {
+    ConstantSDNode *Idx = dyn_cast<ConstantSDNode>(Input.getOperand(1));
+    SDValue OrigVector = Input.getOperand(0);
+
+    // Can't handle non-const element indices or different vector types
+    // for the input to the extract and the output of the scalar_to_vector.
+    if (Idx && VT == OrigVector.getValueType()) {
+      SmallVector<int, 16> NewMask(VT.getVectorNumElements(), -1);
+      NewMask[VT.getVectorNumElements() / 2] = Idx->getZExtValue();
+      return DAG.getVectorShuffle(VT, dl, OrigVector, OrigVector, NewMask);
+    }
+  }
+  return DAG.getNode(PPCISD::SCALAR_TO_VECTOR_PERMUTED, dl, VT,
+                     OrigSToV.getOperand(0));
+}
+
+// On little endian subtargets, combine shuffles such as:
+// vector_shuffle<16,1,17,3,18,5,19,7,20,9,21,11,22,13,23,15>, <zero>, %b
+// into:
+// vector_shuffle<16,0,17,1,18,2,19,3,20,4,21,5,22,6,23,7>, <zero>, %b
+// because the latter can be matched to a single instruction merge.
+// Furthermore, SCALAR_TO_VECTOR on little endian always involves a permute
+// to put the value into element zero. Adjust the shuffle mask so that the
+// vector can remain in permuted form (to prevent a swap prior to a shuffle).
+SDValue PPCTargetLowering::combineVectorShuffle(ShuffleVectorSDNode *SVN,
+                                                SelectionDAG &DAG) const {
+  SDValue LHS = SVN->getOperand(0);
+  SDValue RHS = SVN->getOperand(1);
+  auto Mask = SVN->getMask();
+  int NumElts = LHS.getValueType().getVectorNumElements();
+  SDValue Res(SVN, 0);
+  SDLoc dl(SVN);
+
+  // None of these combines are useful on big endian systems since the ISA
+  // already has a big endian bias.
+  if (!Subtarget.isLittleEndian())
+    return Res;
+
+  // If this is not a shuffle of a shuffle and the first element comes from
+  // the second vector, canonicalize to the commuted form. This will make it
+  // more likely to match one of the single instruction patterns.
+  if (Mask[0] >= NumElts && LHS.getOpcode() != ISD::VECTOR_SHUFFLE &&
+      RHS.getOpcode() != ISD::VECTOR_SHUFFLE) {
+    std::swap(LHS, RHS);
+    Res = DAG.getCommutedVectorShuffle(*SVN);
+    Mask = cast<ShuffleVectorSDNode>(Res)->getMask();
+  }
+
+  // Adjust the shuffle mask if either input vector comes from a
+  // SCALAR_TO_VECTOR and keep the respective input vector in permuted
+  // form (to prevent the need for a swap).
+  SmallVector<int, 16> ShuffV(Mask.begin(), Mask.end());
+  SDValue SToVLHS = isScalarToVec(LHS);
+  SDValue SToVRHS = isScalarToVec(RHS);
+  if (SToVLHS || SToVRHS) {
+    int NumEltsIn = SToVLHS ? SToVLHS.getValueType().getVectorNumElements()
+                            : SToVRHS.getValueType().getVectorNumElements();
+    int NumEltsOut = ShuffV.size();
+
+    // Initially assume that neither input is permuted. These will be adjusted
+    // accordingly if either input is.
+    int LHSMaxIdx = -1;
+    int RHSMinIdx = -1;
+    int RHSMaxIdx = -1;
+    int HalfVec = LHS.getValueType().getVectorNumElements() / 2;
+
+    // Get the permuted scalar to vector nodes for the source(s) that come from
+    // ISD::SCALAR_TO_VECTOR.
+    if (SToVLHS) {
+      // Set up the values for the shuffle vector fixup.
+      LHSMaxIdx = NumEltsOut / NumEltsIn;
+      SToVLHS = getSToVPermuted(SToVLHS, DAG);
+      if (SToVLHS.getValueType() != LHS.getValueType())
+        SToVLHS = DAG.getBitcast(LHS.getValueType(), SToVLHS);
+      LHS = SToVLHS;
+    }
+    if (SToVRHS) {
+      RHSMinIdx = NumEltsOut;
+      RHSMaxIdx = NumEltsOut / NumEltsIn + RHSMinIdx;
+      SToVRHS = getSToVPermuted(SToVRHS, DAG);
+      if (SToVRHS.getValueType() != RHS.getValueType())
+        SToVRHS = DAG.getBitcast(RHS.getValueType(), SToVRHS);
+      RHS = SToVRHS;
+    }
+
+    // Fix up the shuffle mask to reflect where the desired element actually is.
+    // The minimum and maximum indices that correspond to element zero for both
+    // the LHS and RHS are computed and will control which shuffle mask entries
+    // are to be changed. For example, if the RHS is permuted, any shuffle mask
+    // entries in the range [RHSMinIdx,RHSMaxIdx) will be incremented by
+    // HalfVec to refer to the corresponding element in the permuted vector.
+    fixupShuffleMaskForPermutedSToV(ShuffV, LHSMaxIdx, RHSMinIdx, RHSMaxIdx,
+                                    HalfVec);
+    Res = DAG.getVectorShuffle(SVN->getValueType(0), dl, LHS, RHS, ShuffV);
+
+    // We may have simplified away the shuffle. We won't be able to do anything
+    // further with it here.
+    if (!isa<ShuffleVectorSDNode>(Res))
+      return Res;
+    Mask = cast<ShuffleVectorSDNode>(Res)->getMask();
+  }
+
+  // The common case after we commuted the shuffle is that the RHS is a splat
+  // and we have elements coming in from the splat at indices that are not
+  // conducive to using a merge.
+  // Example:
+  // vector_shuffle<0,17,1,19,2,21,3,23,4,25,5,27,6,29,7,31> t1, <zero>
+  if (!isSplatBV(RHS))
+    return Res;
+
+  // We are looking for a mask such that all even elements are from
+  // one vector and all odd elements from the other.
+  if (!isAlternatingShuffMask(Mask, NumElts))
+    return Res;
+
+  // Adjust the mask so we are pulling in the same index from the splat
+  // as the index from the interesting vector in consecutive elements.
+  // Example:
+  // vector_shuffle<0,16,1,17,2,18,3,19,4,20,5,21,6,22,7,23> t1, <zero>
+  for (int i = 1, e = Mask.size(); i < e; i += 2)
+    ShuffV[i] = (ShuffV[i - 1] + NumElts);
+
+  Res = DAG.getVectorShuffle(SVN->getValueType(0), dl, LHS, RHS, ShuffV);
+  return Res;
+}
+
 SDValue PPCTargetLowering::combineVReverseMemOP(ShuffleVectorSDNode *SVN,
                                                 LSBaseSDNode *LSBase,
                                                 DAGCombinerInfo &DCI) const {
   assert((ISD::isNormalLoad(LSBase) || ISD::isNormalStore(LSBase)) &&
         "Not a reverse memop pattern!");
 
   auto IsElementReverse = [](const ShuffleVectorSDNode *SVN) -> bool {
     auto Mask = SVN->getMask();
SDValue PPCTargetLowering::PerformDAGCombine(SDNode *N,
   case ISD::SINT_TO_FP:
   case ISD::UINT_TO_FP:
     return combineFPToIntToFP(N, DCI);
   case ISD::VECTOR_SHUFFLE:
     if (ISD::isNormalLoad(N->getOperand(0).getNode())) {
       LSBaseSDNode* LSBase = cast<LSBaseSDNode>(N->getOperand(0));
       return combineVReverseMemOP(cast<ShuffleVectorSDNode>(N), LSBase, DCI);
     }
-    break;
+    return combineVectorShuffle(cast<ShuffleVectorSDNode>(N), DCI.DAG);
   case ISD::STORE: {
 
     EVT Op1VT = N->getOperand(1).getValueType();
     unsigned Opcode = N->getOperand(1).getOpcode();
 
     if (Opcode == ISD::FP_TO_SINT || Opcode == ISD::FP_TO_UINT) {
       SDValue Val= combineStoreFPToInt(N, DCI);
       if (Val)

llvm/lib/Target/PowerPC/PPCInstrVSX.td

 def PPCswapNoChain : SDNode<"PPCISD::SWAP_NO_CHAIN", SDT_PPCxxswapd>;
 def PPCvabsd : SDNode<"PPCISD::VABSD", SDTVabsd, []>;
 
 def PPCfpexth : SDNode<"PPCISD::FP_EXTEND_HALF", SDT_PPCfpexth, []>;
 def PPCldvsxlh : SDNode<"PPCISD::LD_VSX_LH", SDT_PPCldvsxlh,
                         [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def PPCldsplat : SDNode<"PPCISD::LD_SPLAT", SDT_PPCldsplat,
                         [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
+def PPCSToV : SDNode<"PPCISD::SCALAR_TO_VECTOR_PERMUTED",
+                     SDTypeProfile<1, 1, []>, []>;
 
 //-------------------------- Predicate definitions ---------------------------//
 def HasVSX : Predicate<"PPCSubTarget->hasVSX()">;
 def IsLittleEndian : Predicate<"PPCSubTarget->isLittleEndian()">;
 def IsBigEndian : Predicate<"!PPCSubTarget->isLittleEndian()">;
 def HasOnlySwappingMemOps : Predicate<"!PPCSubTarget->hasP9Vector()">;
 def HasP8Vector : Predicate<"PPCSubTarget->hasP8Vector()">;
 def HasDirectMove : Predicate<"PPCSubTarget->hasDirectMove()">;

 // [PO S RA RB XO SX]
 class X_XS6_RA5_RB5<bits<6> opcode, bits<10> xo, string opc,
                     RegisterOperand vtype, list<dag> pattern>
   : XX1Form_memOp<opcode, xo, (outs), (ins vtype:$XT, memrr:$dst),
             !strconcat(opc, " $XT, $dst"), IIC_LdStSTFD, pattern>;
 } // Predicates = HasP9Vector
 } // AddedComplexity = 400, hasSideEffects = 0
 
+multiclass ScalToVecWPermute<ValueType Ty, dag In, dag NonPermOut, dag PermOut> {
+  def : Pat<(Ty (scalar_to_vector In)), (Ty NonPermOut)>;
+  def : Pat<(Ty (PPCSToV In)), (Ty PermOut)>;
+}
+
 //-------------------------- Instruction definitions -------------------------//
 // VSX instructions require the VSX feature, they are to be selected over
 // equivalent Altivec patterns (as they address a larger register set) and
 // they do not have unmodeled side effects.
 let Predicates = [HasVSX], AddedComplexity = 400 in {
 let hasSideEffects = 0 in {
 
   // Load indexed instructions
dag BE_VDOUBLE_PERMUTE = (v16i8 (VPERM (v16i8 (COPY_TO_REGCLASS $S, VRRC)),
                                          BE_VDWORD_PERM_VEC));
   dag BE_VARIABLE_DOUBLE = (COPY_TO_REGCLASS BE_VDOUBLE_PERMUTE, VSRC);
 }
 
 def AlignValues {
   dag F32_TO_BE_WORD1 = (v4f32 (XXSLDWI (XSCVDPSPN $B), (XSCVDPSPN $B), 3));
   dag I32_TO_BE_WORD1 = (COPY_TO_REGCLASS (MTVSRWZ $B), VSRC);
 }
 
 // Integer extend helper dags 32 -> 64
 def AnyExts {
   dag A = (INSERT_SUBREG (i64 (IMPLICIT_DEF)), $A, sub_32);
   dag B = (INSERT_SUBREG (i64 (IMPLICIT_DEF)), $B, sub_32);
   dag C = (INSERT_SUBREG (i64 (IMPLICIT_DEF)), $C, sub_32);
   dag D = (INSERT_SUBREG (i64 (IMPLICIT_DEF)), $D, sub_32);
 }
 
 def DblToFlt {

 def : Pat<(v2i64 (bitconvert f128:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 def : Pat<(v4i32 (bitconvert f128:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 def : Pat<(v8i16 (bitconvert f128:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 def : Pat<(v16i8 (bitconvert f128:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 
 def : Pat<(v2f64 (PPCsvec2fp v4i32:$C, 0)),
           (v2f64 (XVCVSXWDP (v2i64 (XXMRGHW $C, $C))))>;
 def : Pat<(v2f64 (PPCsvec2fp v4i32:$C, 1)),
           (v2f64 (XVCVSXWDP (v2i64 (XXMRGLW $C, $C))))>;
 
 def : Pat<(v2f64 (PPCuvec2fp v4i32:$C, 0)),
           (v2f64 (XVCVUXWDP (v2i64 (XXMRGHW $C, $C))))>;
 def : Pat<(v2f64 (PPCuvec2fp v4i32:$C, 1)),
           (v2f64 (XVCVUXWDP (v2i64 (XXMRGLW $C, $C))))>;
def : Pat<(v4i32 (build_vector DblToUInt.A, DblToUInt.A,
                                DblToUInt.A, DblToUInt.A)),
           (v4i32 (XXSPLTW (COPY_TO_REGCLASS (XSCVDPUXWS $A), VSRC), 1))>;
 def : Pat<(v2i64 (build_vector DblToLong.A, DblToLong.A)),
           (v2i64 (XXPERMDI (COPY_TO_REGCLASS (XSCVDPSXDS $A), VSRC),
                            (COPY_TO_REGCLASS (XSCVDPSXDS $A), VSRC), 0))>;
 def : Pat<(v2i64 (build_vector DblToULong.A, DblToULong.A)),
           (v2i64 (XXPERMDI (COPY_TO_REGCLASS (XSCVDPUXDS $A), VSRC),
                            (COPY_TO_REGCLASS (XSCVDPUXDS $A), VSRC), 0))>;
-def : Pat<(v4i32 (scalar_to_vector FltToIntLoad.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPSXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1))>;
-def : Pat<(v4i32 (scalar_to_vector FltToUIntLoad.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPUXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1))>;
+defm : ScalToVecWPermute<
+  v4i32, FltToIntLoad.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPSXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1),
+  (COPY_TO_REGCLASS (XSCVDPSXWSs (XFLOADf32 xoaddr:$A)), VSRC)>;
+defm : ScalToVecWPermute<
+  v4i32, FltToUIntLoad.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPUXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1),
+  (COPY_TO_REGCLASS (XSCVDPUXWSs (XFLOADf32 xoaddr:$A)), VSRC)>;
 def : Pat<(v4f32 (build_vector f32:$A, f32:$A, f32:$A, f32:$A)),
           (v4f32 (XXSPLTW (v4f32 (XSCVDPSPN $A)), 0))>;
 def : Pat<(v2f64 (PPCldsplat xoaddr:$A)),
           (v2f64 (LXVDSX xoaddr:$A))>;
 def : Pat<(v2i64 (PPCldsplat xoaddr:$A)),
           (v2i64 (LXVDSX xoaddr:$A))>;
 
 // Build vectors of floating point converted to i64.
 def : Pat<(v2i64 (build_vector FltToLong.A, FltToLong.A)),
           (v2i64 (XXPERMDIs
                    (COPY_TO_REGCLASS (XSCVDPSXDSs $A), VSFRC), 0))>;
 def : Pat<(v2i64 (build_vector FltToULong.A, FltToULong.A)),
           (v2i64 (XXPERMDIs
                    (COPY_TO_REGCLASS (XSCVDPUXDSs $A), VSFRC), 0))>;
-def : Pat<(v2i64 (scalar_to_vector DblToLongLoad.A)),
-          (v2i64 (XVCVDPSXDS (LXVDSX xoaddr:$A)))>;
-def : Pat<(v2i64 (scalar_to_vector DblToULongLoad.A)),
-          (v2i64 (XVCVDPUXDS (LXVDSX xoaddr:$A)))>;
+defm : ScalToVecWPermute<
+  v2i64, DblToLongLoad.A,
+  (XVCVDPSXDS (LXVDSX xoaddr:$A)), (XVCVDPSXDS (LXVDSX xoaddr:$A))>;
+defm : ScalToVecWPermute<
+  v2i64, DblToULongLoad.A,
+  (XVCVDPUXDS (LXVDSX xoaddr:$A)), (XVCVDPUXDS (LXVDSX xoaddr:$A))>;
 } // HasVSX
 
 // Any big endian VSX subtarget.
 let Predicates = [HasVSX, IsBigEndian] in {
 def : Pat<(v2f64 (scalar_to_vector f64:$A)),
           (v2f64 (SUBREG_TO_REG (i64 1), $A, sub_64))>;
 
 def : Pat<(f64 (extractelt v2f64:$S, 0)),

 def : Pat<WToDPExtractConv.BV02U,
           (v2f64 (XVCVUXWDP $A))>;
 def : Pat<WToDPExtractConv.BV13U,
           (v2f64 (XVCVUXWDP (XXSLDWI $A, $A, 3)))>;
 } // HasVSX, IsBigEndian
 
 // Any little endian VSX subtarget.
 let Predicates = [HasVSX, IsLittleEndian] in {
-def : Pat<(v2f64 (scalar_to_vector f64:$A)),
-          (v2f64 (XXPERMDI (SUBREG_TO_REG (i64 1), $A, sub_64),
-                           (SUBREG_TO_REG (i64 1), $A, sub_64), 0))>;
+defm : ScalToVecWPermute<v2f64, (f64 f64:$A),
+                         (XXPERMDI (SUBREG_TO_REG (i64 1), $A, sub_64),
+                                   (SUBREG_TO_REG (i64 1), $A, sub_64), 0),
+                         (SUBREG_TO_REG (i64 1), $A, sub_64)>;
 
 def : Pat<(f64 (extractelt v2f64:$S, 0)),
           (f64 (EXTRACT_SUBREG (XXPERMDI $S, $S, 2), sub_64))>;
 def : Pat<(f64 (extractelt v2f64:$S, 1)),
           (f64 (EXTRACT_SUBREG $S, sub_64))>;
 
 def : Pat<(v2f64 (PPCld_vec_be xoaddr:$src)), (LXVD2X xoaddr:$src)>;
 def : Pat<(PPCst_vec_be v2f64:$rS, xoaddr:$dst), (STXVD2X $rS, xoaddr:$dst)>;

 def : Pat<(PPCstore_scal_int_from_vsr
             (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), xoaddr:$dst, 8),
           (STXSDX (XSCVDPSXDS f64:$src), xoaddr:$dst)>;
 def : Pat<(PPCstore_scal_int_from_vsr
             (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), xoaddr:$dst, 8),
           (STXSDX (XSCVDPUXDS f64:$src), xoaddr:$dst)>;
 
 // Load-and-splat with fp-to-int conversion (using X-Form VSX/FP loads).
-def : Pat<(v4i32 (scalar_to_vector DblToIntLoad.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPSXWS (XFLOADf64 xoaddr:$A)), VSRC), 1))>;
-def : Pat<(v4i32 (scalar_to_vector DblToUIntLoad.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPUXWS (XFLOADf64 xoaddr:$A)), VSRC), 1))>;
-def : Pat<(v2i64 (scalar_to_vector FltToLongLoad.A)),
-          (v2i64 (XXPERMDIs (XSCVDPSXDS (COPY_TO_REGCLASS
-                                          (XFLOADf32 xoaddr:$A), VSFRC)), 0))>;
-def : Pat<(v2i64 (scalar_to_vector FltToULongLoad.A)),
-          (v2i64 (XXPERMDIs (XSCVDPUXDS (COPY_TO_REGCLASS
-                                          (XFLOADf32 xoaddr:$A), VSFRC)), 0))>;
+defm : ScalToVecWPermute<
+  v4i32, DblToIntLoad.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPSXWS (XFLOADf64 xoaddr:$A)), VSRC), 1),
+  (COPY_TO_REGCLASS (XSCVDPSXWS (XFLOADf64 xoaddr:$A)), VSRC)>;
+defm : ScalToVecWPermute<
+  v4i32, DblToUIntLoad.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPUXWS (XFLOADf64 xoaddr:$A)), VSRC), 1),
+  (COPY_TO_REGCLASS (XSCVDPUXWS (XFLOADf64 xoaddr:$A)), VSRC)>;
+defm : ScalToVecWPermute<
+  v2i64, FltToLongLoad.A,
+  (XXPERMDIs (XSCVDPSXDS (COPY_TO_REGCLASS (XFLOADf32 xoaddr:$A), VSFRC)), 0),
+  (SUBREG_TO_REG (i64 1), (XSCVDPSXDS (COPY_TO_REGCLASS (XFLOADf32 xoaddr:$A),
+                                                        VSFRC)), sub_64)>;
+defm : ScalToVecWPermute<
+  v2i64, FltToULongLoad.A,
+  (XXPERMDIs (XSCVDPUXDS (COPY_TO_REGCLASS (XFLOADf32 xoaddr:$A), VSFRC)), 0),
+  (SUBREG_TO_REG (i64 1), (XSCVDPUXDS (COPY_TO_REGCLASS (XFLOADf32 xoaddr:$A),
+                                                        VSFRC)), sub_64)>;
 } // HasVSX, NoP9Vector
 
 // Any VSX subtarget that only has loads and stores that load in big endian
 // order regardless of endianness. This is really pre-Power9 subtargets.
 let Predicates = [HasVSX, HasOnlySwappingMemOps] in {
   def : Pat<(v2f64 (PPClxvd2x xoaddr:$src)), (LXVD2X xoaddr:$src)>;
 
   // Stores.
def : Pat<DWToSPExtractConv.El1SS1,
                             (f64 (COPY_TO_REGCLASS $S1, VSRC)), VSFRC)))>;
 def : Pat<DWToSPExtractConv.El0US1,
           (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S1, $S1, 2), VSFRC)))>;
 def : Pat<DWToSPExtractConv.El1US1,
           (f32 (XSCVUXDSP (COPY_TO_REGCLASS
                             (f64 (COPY_TO_REGCLASS $S1, VSRC)), VSFRC)))>;
 
 // v4f32 scalar <-> vector conversions (LE)
-def : Pat<(v4f32 (scalar_to_vector f32:$A)),
-          (v4f32 (XXSLDWI (XSCVDPSPN $A), (XSCVDPSPN $A), 1))>;
+  // The permuted version is no better than the version that puts the value
+  // into the right element because XSCVDPSPN is different from all the other
+  // instructions used for PPCSToV.
+  defm : ScalToVecWPermute<v4f32, (f32 f32:$A),
+                           (XXSLDWI (XSCVDPSPN $A), (XSCVDPSPN $A), 1),
+                           (XXSLDWI (XSCVDPSPN $A), (XSCVDPSPN $A), 3)>;
 def : Pat<(f32 (vector_extract v4f32:$S, 0)),
           (f32 (XSCVSPDPN (XXSLDWI $S, $S, 3)))>;
 def : Pat<(f32 (vector_extract v4f32:$S, 1)),
           (f32 (XSCVSPDPN (XXPERMDI $S, $S, 2)))>;
 def : Pat<(f32 (vector_extract v4f32:$S, 2)),
           (f32 (XSCVSPDPN (XXSLDWI $S, $S, 1)))>;
 def : Pat<(f32 (vector_extract v4f32:$S, 3)),
           (f32 (XSCVSPDPN $S))>;

 def : Pat<(f64 (PPCfcfid (f64 (PPCmtvsra (i32 (extractelt v4i32:$A, 2)))))),
           (f64 (COPY_TO_REGCLASS (XVCVSXWDP (XXSPLTW $A, 1)), VSFRC))>;
 def : Pat<(f64 (PPCfcfid (f64 (PPCmtvsra (i32 (extractelt v4i32:$A, 3)))))),
           (f64 (COPY_TO_REGCLASS (XVCVSXWDP (XXSPLTW $A, 0)), VSFRC))>;
 
 // LIWAX - This instruction is used for sign extending i32 -> i64.
 // LIWZX - This instruction will be emitted for i32, f32, and when
 //         zero-extending i32 to i64 (zext i32 -> i64).
-def : Pat<(v2i64 (scalar_to_vector (i64 (sextloadi32 xoaddr:$src)))),
-          (v2i64 (XXPERMDIs
-          (COPY_TO_REGCLASS (LIWAX xoaddr:$src), VSFRC), 2))>;
-def : Pat<(v2i64 (scalar_to_vector (i64 (zextloadi32 xoaddr:$src)))),
-          (v2i64 (XXPERMDIs
-          (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2))>;
-def : Pat<(v4i32 (scalar_to_vector (i32 (load xoaddr:$src)))),
-          (v4i32 (XXPERMDIs
-          (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2))>;
-def : Pat<(v4f32 (scalar_to_vector (f32 (load xoaddr:$src)))),
-          (v4f32 (XXPERMDIs
-          (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2))>;
+defm : ScalToVecWPermute<
+  v2i64, (i64 (sextloadi32 xoaddr:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (LIWAX xoaddr:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (LIWAX xoaddr:$src), sub_64)>;
+
+defm : ScalToVecWPermute<
+  v2i64, (i64 (zextloadi32 xoaddr:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (LIWZX xoaddr:$src), sub_64)>;
+
+defm : ScalToVecWPermute<
+  v4i32, (i32 (load xoaddr:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (LIWZX xoaddr:$src), sub_64)>;
+
+defm : ScalToVecWPermute<
+  v4f32, (f32 (load xoaddr:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (LIWZX xoaddr:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (LIWZX xoaddr:$src), sub_64)>;
 
 def : Pat<DWToSPExtractConv.BVU,
           (v4f32 (VPKUDUM (XXSLDWI (XVCVUXDSP $S2), (XVCVUXDSP $S2), 3),
                           (XXSLDWI (XVCVUXDSP $S1), (XVCVUXDSP $S1), 3)))>;
 def : Pat<DWToSPExtractConv.BVS,
           (v4f32 (VPKUDUM (XXSLDWI (XVCVSXDSP $S2), (XVCVSXDSP $S2), 3),
                           (XXSLDWI (XVCVSXDSP $S1), (XVCVSXDSP $S1), 3)))>;
 def : Pat<(store (i32 (extractelt v4i32:$A, 2)), xoaddr:$src),
def : Pat<(i64 (vector_extract v2i64:$S, 1)),
           (i64 VectorExtractions.LE_DWORD_0)>;
 def : Pat<(i64 (vector_extract v2i64:$S, i64:$Idx)),
           (i64 VectorExtractions.BE_VARIABLE_DWORD)>;
 } // HasVSX, HasDirectMove, IsBigEndian
 
 // Little endian VSX subtarget with direct moves.
 let Predicates = [HasVSX, HasDirectMove, IsLittleEndian] in {
   // v16i8 scalar <-> vector conversions (LE)
-  def : Pat<(v16i8 (scalar_to_vector i32:$A)),
-            (v16i8 (COPY_TO_REGCLASS MovesToVSR.LE_WORD_0, VSRC))>;
-  def : Pat<(v8i16 (scalar_to_vector i32:$A)),
-            (v8i16 (COPY_TO_REGCLASS MovesToVSR.LE_WORD_0, VSRC))>;
-  def : Pat<(v4i32 (scalar_to_vector i32:$A)),
-            (v4i32 MovesToVSR.LE_WORD_0)>;
-  def : Pat<(v2i64 (scalar_to_vector i64:$A)),
-            (v2i64 MovesToVSR.LE_DWORD_0)>;
+  defm : ScalToVecWPermute<v16i8, (i32 i32:$A),
+                           (COPY_TO_REGCLASS MovesToVSR.LE_WORD_0, VSRC),
+                           (COPY_TO_REGCLASS MovesToVSR.LE_WORD_1, VSRC)>;
+  defm : ScalToVecWPermute<v8i16, (i32 i32:$A),
+                           (COPY_TO_REGCLASS MovesToVSR.LE_WORD_0, VSRC),
+                           (COPY_TO_REGCLASS MovesToVSR.LE_WORD_1, VSRC)>;
+  defm : ScalToVecWPermute<v4i32, (i32 i32:$A), MovesToVSR.LE_WORD_0,
+                           (SUBREG_TO_REG (i64 1), (MTVSRWZ $A), sub_64)>;
+  defm : ScalToVecWPermute<v2i64, (i64 i64:$A), MovesToVSR.LE_DWORD_0,
+                           MovesToVSR.LE_DWORD_1>;
+
   // v2i64 scalar <-> vector conversions (LE)
   def : Pat<(i64 (vector_extract v2i64:$S, 0)),
             (i64 VectorExtractions.LE_DWORD_0)>;
   def : Pat<(i64 (vector_extract v2i64:$S, 1)),
             (i64 VectorExtractions.LE_DWORD_1)>;
   def : Pat<(i64 (vector_extract v2i64:$S, i64:$Idx)),
             (i64 VectorExtractions.LE_VARIABLE_DWORD)>;
 } // HasVSX, HasDirectMove, IsLittleEndian

 def : Pat<(nonQuadwOffsetStore v4i32:$rS, xoaddr:$dst),
           (STXVX $rS, xoaddr:$dst)>;
 def : Pat<(int_ppc_vsx_stxvw4x v4i32:$rS, xoaddr:$dst),
           (STXVX $rS, xoaddr:$dst)>;
 def : Pat<(int_ppc_vsx_stxvd2x v2f64:$rS, xoaddr:$dst),
           (STXVX $rS, xoaddr:$dst)>;
 
 // Build vectors from i8 loads
-def : Pat<(v16i8 (scalar_to_vector ScalarLoads.Li8)),
-          (v16i8 (VSPLTBs 7, (LXSIBZX xoaddr:$src)))>;
-def : Pat<(v8i16 (scalar_to_vector ScalarLoads.ZELi8)),
-          (v8i16 (VSPLTHs 3, (LXSIBZX xoaddr:$src)))>;
-def : Pat<(v4i32 (scalar_to_vector ScalarLoads.ZELi8)),
-         (v4i32 (XXSPLTWs (LXSIBZX xoaddr:$src), 1))>;
-def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi8i64)),
-          (v2i64 (XXPERMDIs (LXSIBZX xoaddr:$src), 0))>;
-def : Pat<(v4i32 (scalar_to_vector ScalarLoads.SELi8)),
-          (v4i32 (XXSPLTWs (VEXTSB2Ws (LXSIBZX xoaddr:$src)), 1))>;
-def : Pat<(v2i64 (scalar_to_vector ScalarLoads.SELi8i64)),
-          (v2i64 (XXPERMDIs (VEXTSB2Ds (LXSIBZX xoaddr:$src)), 0))>;
+defm : ScalToVecWPermute<v16i8, ScalarLoads.Li8,
+                         (VSPLTBs 7, (LXSIBZX xoaddr:$src)),
+                         (VSPLTBs 7, (LXSIBZX xoaddr:$src))>;
+defm : ScalToVecWPermute<v8i16, ScalarLoads.ZELi8,
+                         (VSPLTHs 3, (LXSIBZX xoaddr:$src)),
+                         (VSPLTHs 3, (LXSIBZX xoaddr:$src))>;
+defm : ScalToVecWPermute<v4i32, ScalarLoads.ZELi8,
+                         (XXSPLTWs (LXSIBZX xoaddr:$src), 1),
+                         (XXSPLTWs (LXSIBZX xoaddr:$src), 1)>;
+defm : ScalToVecWPermute<v2i64, ScalarLoads.ZELi8i64,
+                         (XXPERMDIs (LXSIBZX xoaddr:$src), 0),
+                         (XXPERMDIs (LXSIBZX xoaddr:$src), 0)>;
+defm : ScalToVecWPermute<v4i32, ScalarLoads.SELi8,
+                         (XXSPLTWs (VEXTSB2Ws (LXSIBZX xoaddr:$src)), 1),
+                         (XXSPLTWs (VEXTSB2Ws (LXSIBZX xoaddr:$src)), 1)>;
+defm : ScalToVecWPermute<v2i64, ScalarLoads.SELi8i64,
+                         (XXPERMDIs (VEXTSB2Ds (LXSIBZX xoaddr:$src)), 0),
+                         (XXPERMDIs (VEXTSB2Ds (LXSIBZX xoaddr:$src)), 0)>;
 
 // Build vectors from i16 loads
-def : Pat<(v8i16 (scalar_to_vector ScalarLoads.Li16)),
-          (v8i16 (VSPLTHs 3, (LXSIHZX xoaddr:$src)))>;
-def : Pat<(v4i32 (scalar_to_vector ScalarLoads.ZELi16)),
-          (v4i32 (XXSPLTWs (LXSIHZX xoaddr:$src), 1))>;
-def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi16i64)),
-         (v2i64 (XXPERMDIs (LXSIHZX xoaddr:$src), 0))>;
-def : Pat<(v4i32 (scalar_to_vector ScalarLoads.SELi16)),
-          (v4i32 (XXSPLTWs (VEXTSH2Ws (LXSIHZX xoaddr:$src)), 1))>;
-def : Pat<(v2i64 (scalar_to_vector ScalarLoads.SELi16i64)),
-          (v2i64 (XXPERMDIs (VEXTSH2Ds (LXSIHZX xoaddr:$src)), 0))>;
+defm : ScalToVecWPermute<v8i16, ScalarLoads.Li16,
+                         (VSPLTHs 3, (LXSIHZX xoaddr:$src)),
+                         (VSPLTHs 3, (LXSIHZX xoaddr:$src))>;
+defm : ScalToVecWPermute<v4i32, ScalarLoads.ZELi16,
+                         (XXSPLTWs (LXSIHZX xoaddr:$src), 1),
+                         (XXSPLTWs (LXSIHZX xoaddr:$src), 1)>;
+defm : ScalToVecWPermute<v2i64, ScalarLoads.ZELi16i64,
+                         (XXPERMDIs (LXSIHZX xoaddr:$src), 0),
+                         (XXPERMDIs (LXSIHZX xoaddr:$src), 0)>;
+defm : ScalToVecWPermute<v4i32, ScalarLoads.SELi16,
+                         (XXSPLTWs (VEXTSH2Ws (LXSIHZX xoaddr:$src)), 1),
+                         (XXSPLTWs (VEXTSH2Ws (LXSIHZX xoaddr:$src)), 1)>;
+defm : ScalToVecWPermute<v2i64, ScalarLoads.SELi16i64,
+                         (XXPERMDIs (VEXTSH2Ds (LXSIHZX xoaddr:$src)), 0),
+                         (XXPERMDIs (VEXTSH2Ds (LXSIHZX xoaddr:$src)), 0)>;
 
 // Load/convert and convert/store patterns for f16.
 def : Pat<(f64 (extloadf16 xoaddr:$src)),
           (f64 (XSCVHPDP (LXSIHZX xoaddr:$src)))>;
 def : Pat<(truncstoref16 f64:$src, xoaddr:$dst),
           (STXSIHX (XSCVDPHP $src), xoaddr:$dst)>;
 def : Pat<(f32 (extloadf16 xoaddr:$src)),
           (f32 (COPY_TO_REGCLASS (XSCVHPDP (LXSIHZX xoaddr:$src)), VSSRC))>;
def : Pat<(f32 (PPCxsmaxc f32:$XA, f32:$XB)),
                                            (COPY_TO_REGCLASS $XB, VSSRC)),
                                  VSSRC))>;
 def : Pat<(f32 (PPCxsminc f32:$XA, f32:$XB)),
           (f32 (COPY_TO_REGCLASS (XSMINCDP (COPY_TO_REGCLASS $XA, VSSRC),
                                            (COPY_TO_REGCLASS $XB, VSSRC)),
                                  VSSRC))>;
 
 // Endianness-neutral patterns for const splats with ISA 3.0 instructions.
-def : Pat<(v4i32 (scalar_to_vector i32:$A)),
-          (v4i32 (MTVSRWS $A))>;
+defm : ScalToVecWPermute<v4i32, (i32 i32:$A), (MTVSRWS $A), (MTVSRWS $A)>;
 def : Pat<(v4i32 (build_vector i32:$A, i32:$A, i32:$A, i32:$A)),
           (v4i32 (MTVSRWS $A))>;
 def : Pat<(v16i8 (build_vector immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A,
                                immNonAllOneAnyExt8:$A, immNonAllOneAnyExt8:$A)),
           (v16i8 (COPY_TO_REGCLASS (XXSPLTIB imm:$A), VSRC))>;
-def : Pat<(v4i32 (scalar_to_vector FltToIntLoad.A)),
-          (v4i32 (XVCVSPSXWS (LXVWSX xoaddr:$A)))>;
-def : Pat<(v4i32 (scalar_to_vector FltToUIntLoad.A)),
-          (v4i32 (XVCVSPUXWS (LXVWSX xoaddr:$A)))>;
-def : Pat<(v4i32 (scalar_to_vector DblToIntLoadP9.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPSXWS (DFLOADf64 iaddrX4:$A)), VSRC), 1))>;
-def : Pat<(v4i32 (scalar_to_vector DblToUIntLoadP9.A)),
-          (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                            (XSCVDPUXWS (DFLOADf64 iaddrX4:$A)), VSRC), 1))>;
-def : Pat<(v2i64 (scalar_to_vector FltToLongLoadP9.A)),
-          (v2i64 (XXPERMDIs (XSCVDPSXDS (COPY_TO_REGCLASS
-                                          (DFLOADf32 iaddrX4:$A),
-                                          VSFRC)), 0))>;
-def : Pat<(v2i64 (scalar_to_vector FltToULongLoadP9.A)),
-          (v2i64 (XXPERMDIs (XSCVDPUXDS (COPY_TO_REGCLASS
-                                          (DFLOADf32 iaddrX4:$A),
-                                          VSFRC)), 0))>;
+defm : ScalToVecWPermute<v4i32, FltToIntLoad.A,
+                         (XVCVSPSXWS (LXVWSX xoaddr:$A)),
+                         (XVCVSPSXWS (LXVWSX xoaddr:$A))>;
+defm : ScalToVecWPermute<v4i32, FltToUIntLoad.A,
+                         (XVCVSPUXWS (LXVWSX xoaddr:$A)),
+                         (XVCVSPUXWS (LXVWSX xoaddr:$A))>;
+defm : ScalToVecWPermute<
+  v4i32, DblToIntLoadP9.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPSXWS (DFLOADf64 iaddrX4:$A)), VSRC), 1),
+  (SUBREG_TO_REG (i64 1), (XSCVDPSXWS (DFLOADf64 iaddrX4:$A)), sub_64)>;
+defm : ScalToVecWPermute<
+  v4i32, DblToUIntLoadP9.A,
+  (XXSPLTW (COPY_TO_REGCLASS (XSCVDPUXWS (DFLOADf64 iaddrX4:$A)), VSRC), 1),
+  (SUBREG_TO_REG (i64 1), (XSCVDPUXWS (DFLOADf64 iaddrX4:$A)), sub_64)>;
+defm : ScalToVecWPermute<
+  v2i64, FltToLongLoadP9.A,
+  (XXPERMDIs (XSCVDPSXDS (COPY_TO_REGCLASS (DFLOADf32 iaddrX4:$A), VSFRC)), 0),
+  (SUBREG_TO_REG
+     (i64 1),
+     (XSCVDPSXDS (COPY_TO_REGCLASS (DFLOADf32 iaddrX4:$A), VSFRC)), sub_64)>;
+defm : ScalToVecWPermute<
+  v2i64, FltToULongLoadP9.A,
+  (XXPERMDIs (XSCVDPUXDS (COPY_TO_REGCLASS (DFLOADf32 iaddrX4:$A), VSFRC)), 0),
+  (SUBREG_TO_REG
+     (i64 1),
+     (XSCVDPUXDS (COPY_TO_REGCLASS (DFLOADf32 iaddrX4:$A), VSFRC)), sub_64)>;
 def : Pat<(v4f32 (PPCldsplat xoaddr:$A)),
           (v4f32 (LXVWSX xoaddr:$A))>;
 def : Pat<(v4i32 (PPCldsplat xoaddr:$A)),
           (v4i32 (LXVWSX xoaddr:$A))>;
 } // HasVSX, HasP9Vector
 
 // Big endian Power9 subtarget.
 let Predicates = [HasVSX, HasP9Vector, IsBigEndian] in {
def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 4)), xoaddr:$dst),
           (STXSIHXv (COPY_TO_REGCLASS $S, VSRC), xoaddr:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 5)), xoaddr:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 14)), VSRC), xoaddr:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 6)), xoaddr:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 12)), VSRC), xoaddr:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 7)), xoaddr:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 10)), VSRC), xoaddr:$dst)>;
 
-def : Pat<(v2i64 (scalar_to_vector (i64 (load iaddrX4:$src)))),
-          (v2i64 (XXPERMDIs
-          (COPY_TO_REGCLASS (DFLOADf64 iaddrX4:$src), VSFRC), 2))>;
-def : Pat<(v2i64 (scalar_to_vector (i64 (load xaddrX4:$src)))),
-          (v2i64 (XXPERMDIs
-          (COPY_TO_REGCLASS (XFLOADf64 xaddrX4:$src), VSFRC), 2))>;
+defm : ScalToVecWPermute<
+  v2i64, (i64 (load iaddrX4:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (DFLOADf64 iaddrX4:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (DFLOADf64 iaddrX4:$src), sub_64)>;
+defm : ScalToVecWPermute<
+  v2i64, (i64 (load xaddrX4:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (XFLOADf64 xaddrX4:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (XFLOADf64 xaddrX4:$src), sub_64)>;
+defm : ScalToVecWPermute<
+  v2f64, (f64 (load iaddrX4:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (DFLOADf64 iaddrX4:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (DFLOADf64 iaddrX4:$src), sub_64)>;
+defm : ScalToVecWPermute<
+  v2f64, (f64 (load xaddrX4:$src)),
+  (XXPERMDIs (COPY_TO_REGCLASS (XFLOADf64 xaddrX4:$src), VSFRC), 2),
+  (SUBREG_TO_REG (i64 1), (XFLOADf64 xaddrX4:$src), sub_64)>;
 
-def : Pat<(v2f64 (scalar_to_vector (f64 (load iaddrX4:$src)))),
-          (v2f64 (XXPERMDIs
-          (COPY_TO_REGCLASS (DFLOADf64 iaddrX4:$src), VSFRC), 2))>;
-def : Pat<(v2f64 (scalar_to_vector (f64 (load xaddrX4:$src)))),
-          (v2f64 (XXPERMDIs
-          (COPY_TO_REGCLASS (XFLOADf64 xaddrX4:$src), VSFRC), 2))>;
 def : Pat<(store (i64 (extractelt v2i64:$A, 0)), xaddrX4:$src),
           (XFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
                        sub_64), xaddrX4:$src)>;
 def : Pat<(store (f64 (extractelt v2f64:$A, 0)), xaddrX4:$src),
           (XFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
                        sub_64), xaddrX4:$src)>;
 def : Pat<(store (i64 (extractelt v2i64:$A, 1)), xaddrX4:$src),
           (XFSTOREf64 (EXTRACT_SUBREG $A, sub_64), xaddrX4:$src)>;

llvm/test/CodeGen/PowerPC/VSX-XForm-Scalars.ll

 ; RUN: llc < %s -mcpu=pwr8 -mtriple=powerpc64le-unknown-unknown \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names -verify-machineinstrs \
 ; RUN:   | FileCheck %s --check-prefix=CHECK-P8
 ; RUN: llc < %s -mcpu=pwr9 -mtriple=powerpc64le-unknown-unknown \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names -verify-machineinstrs \
 ; RUN:   | FileCheck %s --check-prefix=CHECK-P9
 
 @a = external local_unnamed_addr global <4 x i32>, align 16
 @pb = external local_unnamed_addr global float*, align 8
 
 define void @testExpandPostRAPseudo(i32* nocapture readonly %ptr) {
 ; CHECK-P8-LABEL: testExpandPostRAPseudo:
 ; CHECK-P8:  # %bb.0: # %entry
 ; CHECK-P8:    lfiwzx f0, 0, r3
 ; CHECK-P8:    ld r4, .LC0@toc@l(r4)
-; CHECK-P8:    xxswapd vs0, f0
-; CHECK-P8:    xxspltw v2, vs0, 3
+; CHECK-P8:    xxspltw v2, vs0, 1
 ; CHECK-P8:    stvx v2, 0, r4
 ; CHECK-P8:    lis r4, 1024
 ; CHECK-P8:    lfiwax f0, 0, r3
 ; CHECK-P8:    addis r3, r2, .LC1@toc@ha
 ; CHECK-P8:    ld r3, .LC1@toc@l(r3)
 ; CHECK-P8:    xscvsxdsp f0, f0
 ; CHECK-P8:    ld r3, 0(r3)
 ; CHECK-P8:    stfsx f0, r3, r4

llvm/test/CodeGen/PowerPC/build-vector-tests.ll

 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE-NEXT:    xxspltw v2, vs0, 0
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemVali:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    xxswapd vs0, f0
-; P8LE-NEXT:    xxspltw v2, vs0, 3
+; P8LE-NEXT:    xxspltw v2, vs0, 1
 ; P8LE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr, align 4
   %splat.splatinsert = insertelement <4 x i32> undef, i32 %0, i32 0
   %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
   ret <4 x i32> %splat.splat
 }
 

 ; P9BE-NEXT:    xvcvspsxws v2, vs0
 ; P9BE-NEXT:    blr
 ;
 ; P9LE-LABEL: spltMemValConvftoi:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lxvwsx vs0, 0, r3
 ; P9LE-NEXT:    xvcvspsxws v2, vs0
 ; P9LE-NEXT:    blr
 ;
 ; P8BE-LABEL: spltMemValConvftoi:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfsx f0, 0, r3
 ; P8BE-NEXT:    xscvdpsxws f0, f0
 ; P8BE-NEXT:    xxspltw v2, vs0, 1
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValConvftoi:

 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE-NEXT:    xxspltw v2, vs0, 0
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValui:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    xxswapd vs0, f0
-; P8LE-NEXT:    xxspltw v2, vs0, 3
+; P8LE-NEXT:    xxspltw v2, vs0, 1
 ; P8LE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr, align 4
   %splat.splatinsert = insertelement <4 x i32> undef, i32 %0, i32 0
   %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
   ret <4 x i32> %splat.splat
 }
 

 ; P9BE-NEXT:    xvcvspuxws v2, vs0
 ; P9BE-NEXT:    blr
 ;
 ; P9LE-LABEL: spltMemValConvftoui:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lxvwsx vs0, 0, r3
 ; P9LE-NEXT:    xvcvspuxws v2, vs0
 ; P9LE-NEXT:    blr
 ;
 ; P8BE-LABEL: spltMemValConvftoui:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfsx f0, 0, r3
 ; P8BE-NEXT:    xscvdpuxws f0, f0
 ; P8BE-NEXT:    xxspltw v2, vs0, 1
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValConvftoui:

 ; P9BE-NEXT:    xscvdpsxds f0, f0
 ; P9BE-NEXT:    xxspltd v2, f0, 0
 ; P9BE-NEXT:    blr
 ;
 ; P9LE-LABEL: spltMemValConvftoll:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfs f0, 0(r3)
 ; P9LE-NEXT:    xscvdpsxds f0, f0
-; P9LE-NEXT:    xxspltd v2, f0, 0
+; P9LE-NEXT:    xxspltd v2, vs0, 0
 ; P9LE-NEXT:    blr
 ;
 ; P8BE-LABEL: spltMemValConvftoll:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfsx f0, 0, r3
 ; P8BE-NEXT:    xscvdpsxds f0, f0
 ; P8BE-NEXT:    xxspltd v2, f0, 0
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValConvftoll:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfsx f0, 0, r3
 ; P8LE-NEXT:    xscvdpsxds f0, f0
-; P8LE-NEXT:    xxspltd v2, f0, 0
+; P8LE-NEXT:    xxspltd v2, vs0, 0
 ; P8LE-NEXT:    blr
 entry:
   %0 = load float, float* %ptr, align 4
   %conv = fptosi float %0 to i64
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 }

 ; P9BE-NEXT:    xvcvdpsxds v2, vs0
 ; P9BE-NEXT:    blr
 ;
 ; P9LE-LABEL: spltMemValConvdtoll:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lxvdsx vs0, 0, r3
 ; P9LE-NEXT:    xvcvdpsxds v2, vs0
 ; P9LE-NEXT:    blr
 ;
 ; P8BE-LABEL: spltMemValConvdtoll:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lxvdsx vs0, 0, r3
 ; P8BE-NEXT:    xvcvdpsxds v2, vs0
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValConvdtoll:
 ; P8LE:       # %bb.0: # %entry

 ; P9BE-NEXT:    xscvdpuxds f0, f0
 ; P9BE-NEXT:    xxspltd v2, f0, 0
 ; P9BE-NEXT:    blr
 ;
 ; P9LE-LABEL: spltMemValConvftoull:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfs f0, 0(r3)
 ; P9LE-NEXT:    xscvdpuxds f0, f0
-; P9LE-NEXT:    xxspltd v2, f0, 0
+; P9LE-NEXT:    xxspltd v2, vs0, 0
 ; P9LE-NEXT:    blr
 ;
 ; P8BE-LABEL: spltMemValConvftoull:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfsx f0, 0, r3
 ; P8BE-NEXT:    xscvdpuxds f0, f0
 ; P8BE-NEXT:    xxspltd v2, f0, 0
 ; P8BE-NEXT:    blr
 ;
 ; P8LE-LABEL: spltMemValConvftoull:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfsx f0, 0, r3
 ; P8LE-NEXT:    xscvdpuxds f0, f0
-; P8LE-NEXT:    xxspltd v2, f0, 0
+; P8LE-NEXT:    xxspltd v2, vs0, 0
 ; P8LE-NEXT:    blr
 entry:
   %0 = load float, float* %ptr, align 4
   %conv = fptoui float %0 to i64
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 }

llvm/test/CodeGen/PowerPC/canonical-merge-shuffles.ll

 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 8, i32 24, i32 9, i32 25, i32 10, i32 26, i32 11, i32 27, i32 12, i32 28, i32 13, i32 29, i32 14, i32 30, i32 15, i32 31>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrghb2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrghb2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI1_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrghb2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI1_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 24, i32 8, i32 25, i32 9, i32 26, i32 10, i32 27, i32 11, i32 28, i32 12, i32 29, i32 13, i32 30, i32 14, i32 31, i32 15>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrghh(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrghh:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrghh:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 8, i32 9, i32 24, i32 25, i32 10, i32 11, i32 26, i32 27, i32 12, i32 13, i32 28, i32 29, i32 14, i32 15, i32 30, i32 31>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrghh2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrghh2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI3_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrghh2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI3_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 24, i32 25, i32 8, i32 9, i32 26, i32 27, i32 10, i32 11, i32 28, i32 29, i32 12, i32 13, i32 30, i32 31, i32 14, i32 15>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglb(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglb:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    vmrglb v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglb:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    vmrglb v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 0, i32 16, i32 1, i32 17, i32 2, i32 18, i32 3, i32 19, i32 4, i32 20, i32 5, i32 21, i32 6, i32 22, i32 7, i32 23>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglb2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglb2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI5_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrglb v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglb2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI5_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI5_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrglb v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 16, i32 0, i32 17, i32 1, i32 18, i32 2, i32 19, i32 3, i32 20, i32 4, i32 21, i32 5, i32 22, i32 6, i32 23, i32 7>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglh(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglh:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglh:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 0, i32 1, i32 16, i32 17, i32 2, i32 3, i32 18, i32 19, i32 4, i32 5, i32 20, i32 21, i32 6, i32 7, i32 22, i32 23>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglh2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglh2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI7_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI7_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglh2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI7_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI7_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 16, i32 17, i32 0, i32 1, i32 18, i32 19, i32 2, i32 3, i32 20, i32 21, i32 4, i32 5, i32 22, i32 23, i32 6, i32 7>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrghw(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrghw:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrghw:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 8, i32 9, i32 10, i32 11, i32 24, i32 25, i32 26, i32 27, i32 12, i32 13, i32 14, i32 15, i32 28, i32 29, i32 30, i32 31>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrghw2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrghw2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI9_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI9_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrghw2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI9_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI9_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 24, i32 25, i32 26, i32 27, i32 8, i32 9, i32 10, i32 11, i32 28, i32 29, i32 30, i32 31, i32 12, i32 13, i32 14, i32 15>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglw(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglw:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglw:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 0, i32 1, i32 2, i32 3, i32 16, i32 17, i32 18, i32 19, i32 4, i32 5, i32 6, i32 7, i32 20, i32 21, i32 22, i32 23>
   ret <16 x i8> %shuffle
 }
 define dso_local <16 x i8> @testmrglw2(<16 x i8> %a, <16 x i8> %b) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglw2:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r3, r2, .LCPI11_0@toc@ha
-; CHECK-P8-NEXT:    addi r3, r3, .LCPI11_0@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P8-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglw2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI11_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI11_0@toc@l
-; CHECK-P9-NEXT:    lxvx v4, 0, r3
-; CHECK-P9-NEXT:    vperm v2, v3, v2, v4
+; CHECK-P9-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P9-NEXT:    blr
 entry:
   %shuffle = shufflevector <16 x i8> %a, <16 x i8> %b, <16 x i32> <i32 16, i32 17, i32 18, i32 19, i32 0, i32 1, i32 2, i32 3, i32 20, i32 21, i32 22, i32 23, i32 4, i32 5, i32 6, i32 7>
   ret <16 x i8> %shuffle
 }
 
 define dso_local <8 x i16> @testmrglb3(<8 x i8>* nocapture readonly %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: testmrglb3:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    ld r3, 0(r3)
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI12_0@toc@ha
-; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI12_0@toc@l
-; CHECK-P8-NEXT:    lvx v3, 0, r3
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    vperm v2, v2, v4, v3
+; CHECK-P8-NEXT:    xxlxor v2, v2, v2
+; CHECK-P8-NEXT:    mtvsrd v3, r3
+; CHECK-P8-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testmrglb3:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lfd f0, 0(r3)
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI12_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI12_0@toc@l
-; CHECK-P9-NEXT:    lxvx v3, 0, r3
-; CHECK-P9-NEXT:    xxswapd v2, f0
-; CHECK-P9-NEXT:    xxlxor v4, v4, v4
-; CHECK-P9-NEXT:    vperm v2, v2, v4, v3
+; CHECK-P9-NEXT:    lxsd v2, 0(r3)
+; CHECK-P9-NEXT:    xxlxor v3, v3, v3
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 entry:
   %0 = load <8 x i8>, <8 x i8>* %a, align 8
   %1 = zext <8 x i8> %0 to <8 x i16>
   ret <8 x i16> %1
 }
 
 attributes #0 = { nounwind }

llvm/test/CodeGen/PowerPC/fp-strict-round.ll

 ; P8-NEXT:    xscvdpspn v3, f0
 ; P8-NEXT:    vmrghw v2, v2, v3
 ; P8-NEXT:    blr
 ;
 ; P9-LABEL: fptrunc_v2f32_v2f64:
 ; P9:       # %bb.0:
 ; P9-NEXT:    xsrsp f0, v2
 ; P9-NEXT:    xscvdpspn vs0, f0
-; P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; P9-NEXT:    xxswapd vs0, v2
 ; P9-NEXT:    xsrsp f0, f0
 ; P9-NEXT:    xscvdpspn vs0, f0
-; P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; P9-NEXT:    vmrglw v2, v3, v2
+; P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; P9-NEXT:    vmrghw v2, v3, v2
 ; P9-NEXT:    blr
   %res = call <2 x float> @llvm.experimental.constrained.fptrunc.v2f32.v2f64(
                         <2 x double> %vf1,
                         metadata !"round.dynamic",
                         metadata !"fpexcept.strict")
   ret <2 x float> %res
 }
 

llvm/test/CodeGen/PowerPC/load-and-splat.ll

 ; P9-NEXT:    lxvwsx vs0, 0, r4
 ; P9-NEXT:    stxv vs0, 0(r3)
 ; P9-NEXT:    blr
 ;
 ; P8-LABEL: test2:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    addi r4, r4, 12
 ; P8-NEXT:    lfiwzx f0, 0, r4
-; P8-NEXT:    xxswapd vs0, f0
-; P8-NEXT:    xxspltw v2, vs0, 3
+; P8-NEXT:    xxspltw v2, vs0, 1
 ; P8-NEXT:    stvx v2, 0, r3
 ; P8-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds float, float* %a, i64 3
   %0 = load float, float* %arrayidx, align 4
   %splat.splatinsert.i = insertelement <4 x float> undef, float %0, i32 0
   %splat.splat.i = shufflevector <4 x float> %splat.splatinsert.i, <4 x float> undef, <4 x i32> zeroinitializer
   store <4 x float> %splat.splat.i, <4 x float>* %c, align 16
   ret void
 }
 
 define dso_local void @test3(<4 x i32>* nocapture %c, i32* nocapture readonly %a) local_unnamed_addr {
 ; P9-LABEL: test3:
 ; P9:       # %bb.0: # %entry
 ; P9-NEXT:    addi r4, r4, 12
 ; P9-NEXT:    lxvwsx vs0, 0, r4
 ; P9-NEXT:    stxv vs0, 0(r3)
 ; P9-NEXT:    blr
 ;
 ; P8-LABEL: test3:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    addi r4, r4, 12
 ; P8-NEXT:    lfiwzx f0, 0, r4
-; P8-NEXT:    xxswapd vs0, f0
-; P8-NEXT:    xxspltw v2, vs0, 3
+; P8-NEXT:    xxspltw v2, vs0, 1
 ; P8-NEXT:    stvx v2, 0, r3
 ; P8-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %a, i64 3
   %0 = load i32, i32* %arrayidx, align 4
   %splat.splatinsert.i = insertelement <4 x i32> undef, i32 %0, i32 0
   %splat.splat.i = shufflevector <4 x i32> %splat.splatinsert.i, <4 x i32> undef, <4 x i32> zeroinitializer
   store <4 x i32> %splat.splat.i, <4 x i32>* %c, align 16

 ; P9-LABEL: unadjusted_lxvwsx:
 ; P9:       # %bb.0: # %entry
 ; P9-NEXT:    lxvwsx v2, 0, r3
 ; P9-NEXT:    blr
 ;
 ; P8-LABEL: unadjusted_lxvwsx:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    lfiwzx f0, 0, r3
-; P8-NEXT:    xxswapd vs0, f0
-; P8-NEXT:    xxspltw v2, vs0, 3
+; P8-NEXT:    xxspltw v2, vs0, 1
 ; P8-NEXT:    blr
   entry:
     %0 = bitcast i32* %s to <4 x i8>*
     %1 = load <4 x i8>, <4 x i8>* %0, align 4
     %2 = shufflevector <4 x i8> %1, <4 x i8> undef, <16 x i32> <i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3>
     ret <16 x i8> %2
 }
 
 define <16 x i8> @adjusted_lxvwsx(i64* %s, i64* %t) {
 ; P9-LABEL: adjusted_lxvwsx:
 ; P9:       # %bb.0: # %entry
 ; P9-NEXT:    addi r3, r3, 4
 ; P9-NEXT:    lxvwsx v2, 0, r3
 ; P9-NEXT:    blr
 ;
 ; P8-LABEL: adjusted_lxvwsx:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    ld r3, 0(r3)
 ; P8-NEXT:    mtfprd f0, r3
-; P8-NEXT:    xxswapd v2, vs0
-; P8-NEXT:    xxspltw v2, v2, 2
+; P8-NEXT:    xxspltw v2, vs0, 0
 ; P8-NEXT:    blr
   entry:
     %0 = bitcast i64* %s to <8 x i8>*
     %1 = load <8 x i8>, <8 x i8>* %0, align 8
     %2 = shufflevector <8 x i8> %1, <8 x i8> undef, <16 x i32> <i32 4, i32 5, i32 6, i32 7, i32 4, i32 5, i32 6, i32 7, i32 4, i32 5, i32 6, i32 7, i32 4, i32 5, i32 6, i32 7>
     ret <16 x i8> %2
 }
 

llvm/test/CodeGen/PowerPC/load-v4i8-improved.ll

 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -mtriple=powerpc64le-unknown-linux-gnu < %s \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names | FileCheck --check-prefix=CHECK-LE \
 ; RUN:   -implicit-check-not vmrg -implicit-check-not=vperm %s
 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -mtriple=powerpc64-unknown-linux-gnu < %s \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names | FileCheck \
 ; RUN:   -implicit-check-not vmrg -implicit-check-not=vperm %s
 
 define <16 x i8> @test(i32* %s, i32* %t) {
 ; CHECK-LE-LABEL: test:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    lfiwzx f0, 0, r3
-; CHECK-LE-NEXT:    xxswapd vs0, f0
-; CHECK-LE-NEXT:    xxspltw v2, vs0, 3
+; CHECK-LE-NEXT:    xxspltw v2, vs0, 1
 ; CHECK-LE-NEXT:    blr
 
 ; CHECK-LABEL: test:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfiwzx f0, 0, r3
 ; CHECK-NEXT:    xxsldwi vs0, f0, f0, 1
 ; CHECK-NEXT:    xxspltw v2, vs0, 0
 ; CHECK-NEXT:    blr
 entry:
   %0 = bitcast i32* %s to <4 x i8>*
   %1 = load <4 x i8>, <4 x i8>* %0, align 4
   %2 = shufflevector <4 x i8> %1, <4 x i8> undef, <16 x i32> <i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3>
   ret <16 x i8> %2
 }

llvm/test/CodeGen/PowerPC/p8-scalar_vector_conversions.ll

 entry:
   %splat.splatinsert = insertelement <16 x i8> undef, i8 %a, i32 0
   %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
   ret <16 x i8> %splat.splat
 ; CHECK-LABEL: buildc
 ; CHECK: sldi r3, r3, 56
 ; CHECK: mtvsrd v2, r3
 ; CHECK-LE-LABEL: buildc
-; CHECK-LE: mtfprd f0, r3
-; CHECK-LE: xxswapd v2, vs0
+; CHECK-LE: mtvsrd v2, r3
+; CHECK-LE: vspltb v2, v2, 7
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <8 x i16> @builds(i16 zeroext %a) {
 entry:
   %splat.splatinsert = insertelement <8 x i16> undef, i16 %a, i32 0
   %splat.splat = shufflevector <8 x i16> %splat.splatinsert, <8 x i16> undef, <8 x i32> zeroinitializer
   ret <8 x i16> %splat.splat
 ; CHECK-LABEL: builds
 ; CHECK: sldi r3, r3, 48
 ; CHECK: mtvsrd v2, r3
 ; CHECK-LE-LABEL: builds
-; CHECK-LE: mtfprd f0, r3
-; CHECK-LE: xxswapd v2, vs0
+; CHECK-LE: mtvsrd v2, r3
+; CHECK-LE: vsplth v2, v2, 3
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <4 x i32> @buildi(i32 zeroext %a) {
 entry:
   %splat.splatinsert = insertelement <4 x i32> undef, i32 %a, i32 0
   %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
   ret <4 x i32> %splat.splat

llvm/test/CodeGen/PowerPC/pr25080.ll

 ; LE-NEXT:    xxland 34, 34, 36
 ; LE-NEXT:    xxland 35, 35, 36
 ; LE-NEXT:    vcmpequw 2, 2, 5
 ; LE-NEXT:    vcmpequw 3, 3, 5
 ; LE-NEXT:    xxswapd 0, 34
 ; LE-NEXT:    mfvsrwz 3, 34
 ; LE-NEXT:    xxsldwi 1, 34, 34, 1
 ; LE-NEXT:    mfvsrwz 4, 35
-; LE-NEXT:    xxsldwi 4, 34, 34, 3
-; LE-NEXT:    mtfprd 2, 3
+; LE-NEXT:    xxsldwi 2, 34, 34, 3
+; LE-NEXT:    mtvsrd 36, 3
 ; LE-NEXT:    mffprwz 3, 0
 ; LE-NEXT:    xxswapd 0, 35
-; LE-NEXT:    mtfprd 3, 4
-; LE-NEXT:    xxsldwi 5, 35, 35, 1
+; LE-NEXT:    mtvsrd 37, 4
 ; LE-NEXT:    mffprwz 4, 1
-; LE-NEXT:    xxsldwi 7, 35, 35, 3
-; LE-NEXT:    mtfprd 1, 3
-; LE-NEXT:    xxswapd 33, 3
-; LE-NEXT:    mffprwz 3, 4
-; LE-NEXT:    mtfprd 4, 4
-; LE-NEXT:    xxswapd 34, 1
+; LE-NEXT:    xxsldwi 1, 35, 35, 1
+; LE-NEXT:    mtvsrd 34, 3
+; LE-NEXT:    mffprwz 3, 2
+; LE-NEXT:    mtvsrd 32, 4
 ; LE-NEXT:    mffprwz 4, 0
-; LE-NEXT:    mtfprd 0, 3
-; LE-NEXT:    xxswapd 35, 4
-; LE-NEXT:    mffprwz 3, 5
-; LE-NEXT:    mtfprd 6, 4
-; LE-NEXT:    xxswapd 36, 0
-; LE-NEXT:    mtfprd 1, 3
-; LE-NEXT:    mffprwz 3, 7
-; LE-NEXT:    xxswapd 37, 6
-; LE-NEXT:    vmrglh 2, 3, 2
-; LE-NEXT:    xxswapd 35, 2
-; LE-NEXT:    mtfprd 2, 3
-; LE-NEXT:    xxswapd 32, 1
+; LE-NEXT:    xxsldwi 0, 35, 35, 3
+; LE-NEXT:    mtvsrd 33, 3
+; LE-NEXT:    mffprwz 3, 1
+; LE-NEXT:    mtvsrd 38, 4
+; LE-NEXT:    mtvsrd 35, 3
+; LE-NEXT:    mffprwz 3, 0
+; LE-NEXT:    vmrghh 2, 0, 2
+; LE-NEXT:    mtvsrd 32, 3
 ; LE-NEXT:    addis 3, 2, .LCPI0_1@toc@ha
+; LE-NEXT:    vmrghh 4, 1, 4
 ; LE-NEXT:    addi 3, 3, .LCPI0_1@toc@l
-; LE-NEXT:    xxswapd 38, 2
-; LE-NEXT:    vmrglh 3, 4, 3
-; LE-NEXT:    vmrglh 4, 0, 5
-; LE-NEXT:    vmrglh 5, 6, 1
-; LE-NEXT:    vmrglw 2, 3, 2
-; LE-NEXT:    vmrglw 3, 5, 4
+; LE-NEXT:    vmrghh 3, 3, 6
+; LE-NEXT:    vmrghh 5, 0, 5
+; LE-NEXT:    vmrglw 2, 4, 2
 ; LE-NEXT:    vspltish 4, 15
+; LE-NEXT:    vmrglw 3, 5, 3
 ; LE-NEXT:    xxmrgld 34, 35, 34
 ; LE-NEXT:    lvx 3, 0, 3
 ; LE-NEXT:    xxlor 34, 34, 35
 ; LE-NEXT:    vslh 2, 2, 4
 ; LE-NEXT:    vsrah 2, 2, 4
 ; LE-NEXT:    blr
 ;
 ; BE-LABEL: pr25080:

llvm/test/CodeGen/PowerPC/pr25157-peephole.ll

L.LB38_2452:
   %1 = fpext float %0 to double
   %2 = insertelement <2 x double> undef, double %1, i32 1
   store <2 x double> %2, <2 x double>* bitcast (i8* getelementptr inbounds (%struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626, %struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626* @_main1_2_, i64 0, i32 0, i64 32) to <2 x double>*), align 16
   unreachable
 }
 
 ; CHECK-LABEL: @aercalc_
 ; CHECK: lfs
-; CHECK: xxspltd
+; CHECK: xxswapd
 ; CHECK: stxvd2x
 ; CHECK-NOT: xxswapd
 
 ; CHECK-P9-LABEL: @aercalc_
 ; CHECK-P9: lfs
-; CHECK-P9: xxspltd
 ; CHECK-P9: stxv
 ; CHECK-P9-NOT: xxswapd

llvm/test/CodeGen/PowerPC/pr38087.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mcpu=pwr9 -ppc-vsr-nums-as-vr \
 ; RUN:   -mtriple=powerpc64le-unknown-unknown -ppc-asm-full-reg-names < %s | \
 ; RUN:   FileCheck %s
 ; Function Attrs: nounwind readnone speculatable
 declare <4 x float> @llvm.fmuladd.v4f32(<4 x float>, <4 x float>, <4 x float>) #0
 
 ; Function Attrs: nounwind readnone speculatable
 declare { i32, i1 } @llvm.usub.with.overflow.i32(i32, i32) #0
 
 define void @draw_llvm_vs_variant0(<4 x float> %x) {
 ; CHECK-LABEL: draw_llvm_vs_variant0:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    lfd f0, 0(r3)
-; CHECK-NEXT:    xxswapd v3, f0
-; CHECK-NEXT:    vmrglh v3, v3, v3
+; CHECK-NEXT:    lxsd v3, 0(r3)
+; CHECK-NEXT:    vmrghh v3, v3, v3
 ; CHECK-NEXT:    vextsh2w v3, v3
 ; CHECK-NEXT:    xvcvsxwsp vs0, v3
 ; CHECK-NEXT:    xxspltw vs0, vs0, 2
 ; CHECK-NEXT:    xvmaddasp vs0, v2, v2
 ; CHECK-NEXT:    stxvx vs0, 0, r3
 ; CHECK-NEXT:    blr
 entry:
   %.size = load i32, i32* undef

llvm/test/CodeGen/PowerPC/pre-inc-disable.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mcpu=pwr9 -O3 -verify-machineinstrs -ppc-vsr-nums-as-vr \
 ; RUN:     -ppc-asm-full-reg-names -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     < %s | FileCheck %s
 
 ; RUN: llc -mcpu=pwr9 -O3 -verify-machineinstrs -ppc-vsr-nums-as-vr \
 ; RUN:     -ppc-asm-full-reg-names -mtriple=powerpc64-unknown-linux-gnu \
 ; RUN:     < %s | FileCheck %s --check-prefix=P9BE
 
 ; Function Attrs: norecurse nounwind readonly
 define signext i32 @test_pre_inc_disable_1(i8* nocapture readonly %pix1, i32 signext %i_stride_pix1, i8* nocapture readonly %pix2) {
 ; CHECK-LABEL: test_pre_inc_disable_1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    lfd f0, 0(r5)
+; CHECK-NEXT:    lxsd v5, 0(r5)
 ; CHECK-NEXT:    addis r5, r2, .LCPI0_0@toc@ha
 ; CHECK-NEXT:    addi r5, r5, .LCPI0_0@toc@l
 ; CHECK-NEXT:    lxvx v2, 0, r5
 ; CHECK-NEXT:    addis r5, r2, .LCPI0_1@toc@ha
 ; CHECK-NEXT:    addi r5, r5, .LCPI0_1@toc@l
 ; CHECK-NEXT:    lxvx v4, 0, r5
-; CHECK-NEXT:    xxswapd v5, f0
-; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    li r5, 4
+; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    vperm v0, v3, v5, v2
 ; CHECK-NEXT:    mtctr r5
 ; CHECK-NEXT:    li r5, 0
-; CHECK-NEXT:    vperm v1, v5, v3, v4
+; CHECK-NEXT:    vperm v1, v3, v5, v4
 ; CHECK-NEXT:    li r6, 0
 ; CHECK-NEXT:    xvnegsp v5, v0
 ; CHECK-NEXT:    xvnegsp v0, v1
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  .LBB0_1: # %for.cond1.preheader
 ; CHECK-NEXT:    #
-; CHECK-NEXT:    lfd f0, 0(r3)
-; CHECK-NEXT:    xxswapd v1, f0
-; CHECK-NEXT:    lfdx f0, r3, r4
-; CHECK-NEXT:    vperm v6, v1, v3, v4
+; CHECK-NEXT:    lxsd v1, 0(r3)
+; CHECK-NEXT:    vperm v6, v3, v1, v4
 ; CHECK-NEXT:    vperm v1, v3, v1, v2
 ; CHECK-NEXT:    xvnegsp v1, v1
-; CHECK-NEXT:    add r7, r3, r4
 ; CHECK-NEXT:    xvnegsp v6, v6
+; CHECK-NEXT:    add r7, r3, r4
 ; CHECK-NEXT:    vabsduw v1, v1, v5
 ; CHECK-NEXT:    vabsduw v6, v6, v0
 ; CHECK-NEXT:    vadduwm v1, v6, v1
 ; CHECK-NEXT:    xxswapd v6, v1
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxspltw v6, v1, 2
 ; CHECK-NEXT:    vadduwm v1, v1, v6
-; CHECK-NEXT:    xxswapd v6, f0
+; CHECK-NEXT:    lxsdx v6, r3, r4
 ; CHECK-NEXT:    vextuwrx r3, r5, v1
-; CHECK-NEXT:    vperm v7, v6, v3, v4
+; CHECK-NEXT:    vperm v7, v3, v6, v4
 ; CHECK-NEXT:    vperm v6, v3, v6, v2
-; CHECK-NEXT:    add r6, r3, r6
-; CHECK-NEXT:    add r3, r7, r4
 ; CHECK-NEXT:    xvnegsp v6, v6
 ; CHECK-NEXT:    xvnegsp v1, v7
 ; CHECK-NEXT:    vabsduw v6, v6, v5
+; CHECK-NEXT:    add r6, r3, r6
 ; CHECK-NEXT:    vabsduw v1, v1, v0
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxswapd v6, v1
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    xxspltw v6, v1, 2
 ; CHECK-NEXT:    vadduwm v1, v1, v6
 ; CHECK-NEXT:    vextuwrx r8, r5, v1
+; CHECK-NEXT:    add r3, r7, r4
 ; CHECK-NEXT:    add r6, r8, r6
 ; CHECK-NEXT:    bdnz .LBB0_1
 ; CHECK-NEXT:  # %bb.2: # %for.cond.cleanup
 ; CHECK-NEXT:    extsw r3, r6
 ; CHECK-NEXT:    blr
 ;
 ; P9BE-LABEL: test_pre_inc_disable_1:
 ; P9BE:       # %bb.0: # %entry

 for.cond.cleanup:                                 ; preds = %for.cond1.preheader
   ret i32 %op.extra.1
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define signext i32 @test_pre_inc_disable_2(i8* nocapture readonly %pix1, i8* nocapture readonly %pix2) {
 ; CHECK-LABEL: test_pre_inc_disable_2:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    lfd f0, 0(r3)
+; CHECK-NEXT:    lxsd v2, 0(r3)
 ; CHECK-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
 ; CHECK-NEXT:    addi r3, r3, .LCPI1_0@toc@l
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    addis r3, r2, .LCPI1_1@toc@ha
-; CHECK-NEXT:    xxswapd v2, f0
-; CHECK-NEXT:    lfd f0, 0(r4)
 ; CHECK-NEXT:    addi r3, r3, .LCPI1_1@toc@l
-; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    lxvx v0, 0, r3
-; CHECK-NEXT:    xxswapd v1, f0
-; CHECK-NEXT:    vperm v5, v2, v3, v4
+; CHECK-NEXT:    lxsd v1, 0(r4)
+; CHECK-NEXT:    xxlxor v3, v3, v3
+; CHECK-NEXT:    vperm v5, v3, v2, v4
 ; CHECK-NEXT:    vperm v2, v3, v2, v0
 ; CHECK-NEXT:    vperm v0, v3, v1, v0
-; CHECK-NEXT:    vperm v3, v1, v3, v4
+; CHECK-NEXT:    vperm v3, v3, v1, v4
 ; CHECK-NEXT:    vabsduw v2, v2, v0
 ; CHECK-NEXT:    vabsduw v3, v5, v3
 ; CHECK-NEXT:    vadduwm v2, v3, v2
 ; CHECK-NEXT:    xxswapd v3, v2
-; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    vadduwm v2, v2, v3
 ; CHECK-NEXT:    xxspltw v3, v2, 2
 ; CHECK-NEXT:    vadduwm v2, v2, v3
+; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-NEXT:    extsw r3, r3
 ; CHECK-NEXT:    blr
 ;
 ; P9BE-LABEL: test_pre_inc_disable_2:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r3)
 ; P9BE-NEXT:    addis r3, r2, .LCPI1_0@toc@ha

 ;
 ;  return i_sum;
 ;}
 
 define void @test32(i8* nocapture readonly %pix2, i32 signext %i_pix2) {
 ; CHECK-LABEL: test32:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    add r5, r3, r4
-; CHECK-NEXT:    lfiwzx f0, r3, r4
+; CHECK-NEXT:    lxsiwzx v2, r3, r4
 ; CHECK-NEXT:    addis r3, r2, .LCPI2_0@toc@ha
 ; CHECK-NEXT:    addi r3, r3, .LCPI2_0@toc@l
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    li r3, 4
-; CHECK-NEXT:    xxswapd v2, f0
-; CHECK-NEXT:    lfiwzx f0, r5, r3
+; CHECK-NEXT:    lxsiwzx v5, r5, r3
 ; CHECK-NEXT:    xxlxor v3, v3, v3
 ; CHECK-NEXT:    vperm v2, v2, v3, v4
-; CHECK-NEXT:    xxswapd v5, f0
 ; CHECK-NEXT:    vperm v3, v5, v3, v4
 ; CHECK-NEXT:    vspltisw v4, 8
 ; CHECK-NEXT:    vnegw v3, v3
 ; CHECK-NEXT:    vadduwm v4, v4, v4
 ; CHECK-NEXT:    vslw v3, v3, v4
 ; CHECK-NEXT:    vsubuwm v2, v3, v2
 ; CHECK-NEXT:    xxswapd vs0, v2
 ; CHECK-NEXT:    stxvx vs0, 0, r3

 ; CHECK-LABEL: test16:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    sldi r4, r4, 1
 ; CHECK-NEXT:    add r6, r3, r4
 ; CHECK-NEXT:    li r7, 16
 ; CHECK-NEXT:    lxsihzx v2, r6, r7
 ; CHECK-NEXT:    lxsihzx v4, r3, r4
 ; CHECK-NEXT:    li r6, 0
-; CHECK-NEXT:    mtfprd f0, r6
+; CHECK-NEXT:    mtvsrd v3, r6
 ; CHECK-NEXT:    vsplth v4, v4, 3
-; CHECK-NEXT:    xxswapd v3, vs0
 ; CHECK-NEXT:    vsplth v2, v2, 3
 ; CHECK-NEXT:    addis r3, r2, .LCPI3_0@toc@ha
 ; CHECK-NEXT:    addi r3, r3, .LCPI3_0@toc@l
-; CHECK-NEXT:    vmrglh v2, v3, v2
-; CHECK-NEXT:    vmrglh v3, v3, v4
-; CHECK-NEXT:    xxlxor v4, v4, v4
-; CHECK-NEXT:    vmrglw v3, v3, v4
+; CHECK-NEXT:    vmrghh v4, v3, v4
+; CHECK-NEXT:    vmrghh v2, v3, v2
+; CHECK-NEXT:    vsplth v3, v3, 3
+; CHECK-NEXT:    vmrglw v3, v4, v3
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    vperm v2, v2, v3, v4
 ; CHECK-NEXT:    xxspltw v3, v2, 2
 ; CHECK-NEXT:    vadduwm v2, v2, v3
 ; CHECK-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-NEXT:    cmpw r3, r5
 ; CHECK-NEXT:    bgelr+ cr0

 }
 
 define void @test8(i8* nocapture readonly %sums, i32 signext %delta, i32 signext %thresh) {
 ; CHECK-LABEL: test8:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    add r6, r3, r4
 ; CHECK-NEXT:    lxsibzx v2, r3, r4
 ; CHECK-NEXT:    li r3, 0
-; CHECK-NEXT:    mtfprd f0, r3
+; CHECK-NEXT:    mtvsrd v3, r3
 ; CHECK-NEXT:    li r3, 8
 ; CHECK-NEXT:    lxsibzx v5, r6, r3
-; CHECK-NEXT:    xxswapd v3, vs0
-; CHECK-NEXT:    vspltb v4, v3, 15
-; CHECK-NEXT:    vspltb v2, v2, 7
-; CHECK-NEXT:    vmrglb v2, v3, v2
 ; CHECK-NEXT:    addis r3, r2, .LCPI4_0@toc@ha
 ; CHECK-NEXT:    addi r3, r3, .LCPI4_0@toc@l
+; CHECK-NEXT:    vspltb v2, v2, 7
+; CHECK-NEXT:    vmrghb v2, v3, v2
+; CHECK-NEXT:    vspltb v4, v3, 7
 ; CHECK-NEXT:    vspltb v5, v5, 7
 ; CHECK-NEXT:    vmrglh v2, v2, v4
-; CHECK-NEXT:    vmrglb v3, v3, v5
+; CHECK-NEXT:    vmrghb v3, v3, v5
 ; CHECK-NEXT:    vmrglw v2, v2, v4
 ; CHECK-NEXT:    vmrglh v3, v3, v4
 ; CHECK-NEXT:    vmrglw v3, v4, v3
 ; CHECK-NEXT:    lxvx v4, 0, r3
 ; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    vperm v2, v3, v2, v4
 ; CHECK-NEXT:    xxspltw v3, v2, 2
 ; CHECK-NEXT:    vadduwm v2, v2, v3

llvm/test/CodeGen/PowerPC/qpx-load-splat.ll

entry:
   store double* %p, double** %pptr, align 8
   ret <4 x double> %shuffle.i
 }
 
 define <4 x float> @foof(float* nocapture readonly %a) #0 {
 ; CHECK-LABEL: foof:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfiwzx f0, 0, r3
-; CHECK-NEXT:    xxswapd vs0, f0
-; CHECK-NEXT:    xxspltw v2, vs0, 3
+; CHECK-NEXT:    xxspltw v2, vs0, 1
 ; CHECK-NEXT:    blr
 entry:
   %0 = load float, float* %a, align 4
   %vecinit.i = insertelement <4 x float> undef, float %0, i32 0
   %shuffle.i = shufflevector <4 x float> %vecinit.i, <4 x float> undef, <4 x i32> zeroinitializer
   ret <4 x float> %shuffle.i
 }
 
 define <4 x float> @foofx(float* nocapture readonly %a, i64 %idx) #0 {
 ; CHECK-LABEL: foofx:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    sldi r4, r4, 2
 ; CHECK-NEXT:    lfiwzx f0, r3, r4
-; CHECK-NEXT:    xxswapd vs0, f0
-; CHECK-NEXT:    xxspltw v2, vs0, 3
+; CHECK-NEXT:    xxspltw v2, vs0, 1
 ; CHECK-NEXT:    blr
 entry:
   %p = getelementptr float, float* %a, i64 %idx
   %0 = load float, float* %p, align 4
   %vecinit.i = insertelement <4 x float> undef, float %0, i32 0
   %shuffle.i = shufflevector <4 x float> %vecinit.i, <4 x float> undef, <4 x i32> zeroinitializer
   ret <4 x float> %shuffle.i
 }
 
 

llvm/test/CodeGen/PowerPC/scalar_vector_test_1.ll

 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9LE
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9BE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8LE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8BE
 
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test1(i64* nocapture readonly %int64, <2 x i64> %vec) {
 ; P9LE-LABEL: s2v_test1:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 0(r3)
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test1:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 entry:
   %0 = load i64, i64* %int64, align 8
   %vecins = insertelement <2 x i64> %vec, i64 %0, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test2(i64* nocapture readonly %int64, <2 x i64> %vec)  {
 ; P9LE-LABEL: s2v_test2:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 8(r3)
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test2:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 8(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i64, i64* %int64, i64 1
   %0 = load i64, i64* %arrayidx, align 8
   %vecins = insertelement <2 x i64> %vec, i64 %0, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test3(i64* nocapture readonly %int64, <2 x i64> %vec, i32 signext %Idx)  {
 ; P9LE-LABEL: s2v_test3:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    sldi r4, r7, 3
 ; P9LE-NEXT:    lfdx f0, r3, r4
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test3
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    sldi r4, r7, 3
 ; P9BE-NEXT:    lfdx f0, r3, r4
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 entry:
   %idxprom = sext i32 %Idx to i64
   %arrayidx = getelementptr inbounds i64, i64* %int64, i64 %idxprom
   %0 = load i64, i64* %arrayidx, align 8
   %vecins = insertelement <2 x i64> %vec, i64 %0, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test4(i64* nocapture readonly %int64, <2 x i64> %vec)  {
 ; P9LE-LABEL: s2v_test4:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 8(r3)
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test4:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 8(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i64, i64* %int64, i64 1
   %0 = load i64, i64* %arrayidx, align 8
   %vecins = insertelement <2 x i64> %vec, i64 %0, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test5(<2 x i64> %vec, i64* nocapture readonly %ptr1)  {
 ; P9LE-LABEL: s2v_test5:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 0(r5)
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test5:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r5)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 entry:
   %0 = load i64, i64* %ptr1, align 8
   %vecins = insertelement <2 x i64> %vec, i64 %0, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x double> @s2v_test_f1(double* nocapture readonly %f64, <2 x double> %vec)  {
 ; P9LE-LABEL: s2v_test_f1:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 0(r3)
-; P9LE-NEXT:    xxswapd vs0, f0
-; P9LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f1:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f1:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfdx f0, 0, r3
-; P8LE-NEXT:    xxspltd vs0, vs0, 0
-; P8LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f1:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfdx f0, 0, r3
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %0 = load double, double* %f64, align 8
   %vecins = insertelement <2 x double> %vec, double %0, i32 0
   ret <2 x double> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x double> @s2v_test_f2(double* nocapture readonly %f64, <2 x double> %vec)  {
 ; P9LE-LABEL: s2v_test_f2:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 8(r3)
-; P9LE-NEXT:    xxswapd vs0, f0
-; P9LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f2:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 8(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f2:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfd f0, 8(r3)
-; P8LE-NEXT:    xxspltd vs0, vs0, 0
-; P8LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f2:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfd f0, 8(r3)
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds double, double* %f64, i64 1
   %0 = load double, double* %arrayidx, align 8
   %vecins = insertelement <2 x double> %vec, double %0, i32 0
   ret <2 x double> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x double> @s2v_test_f3(double* nocapture readonly %f64, <2 x double> %vec, i32 signext %Idx)  {
 ; P9LE-LABEL: s2v_test_f3:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    sldi r4, r7, 3
 ; P9LE-NEXT:    lfdx f0, r3, r4
-; P9LE-NEXT:    xxswapd vs0, f0
-; P9LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f3:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    sldi r4, r7, 3
 ; P9BE-NEXT:    lfdx f0, r3, r4
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f3:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    sldi r4, r7, 3
 ; P8LE-NEXT:    lfdx f0, r3, r4
-; P8LE-NEXT:    xxspltd vs0, vs0, 0
-; P8LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f3:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    sldi r4, r7, 3
 ; P8BE-NEXT:    lfdx f0, r3, r4
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %idxprom = sext i32 %Idx to i64
   %arrayidx = getelementptr inbounds double, double* %f64, i64 %idxprom
   %0 = load double, double* %arrayidx, align 8
   %vecins = insertelement <2 x double> %vec, double %0, i32 0
   ret <2 x double> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x double> @s2v_test_f4(double* nocapture readonly %f64, <2 x double> %vec)  {
 ; P9LE-LABEL: s2v_test_f4:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 8(r3)
-; P9LE-NEXT:    xxswapd vs0, f0
-; P9LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f4:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 8(r3)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f4:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfd f0, 8(r3)
-; P8LE-NEXT:    xxspltd vs0, vs0, 0
-; P8LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f4:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfd f0, 8(r3)
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds double, double* %f64, i64 1
   %0 = load double, double* %arrayidx, align 8
   %vecins = insertelement <2 x double> %vec, double %0, i32 0
   ret <2 x double> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x double> @s2v_test_f5(<2 x double> %vec, double* nocapture readonly %ptr1)  {
 ; P9LE-LABEL: s2v_test_f5:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfd f0, 0(r5)
-; P9LE-NEXT:    xxswapd vs0, f0
-; P9LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f5:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfd f0, 0(r5)
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f5:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfdx f0, 0, r5
-; P8LE-NEXT:    xxspltd vs0, vs0, 0
-; P8LE-NEXT:    xxpermdi v2, v2, vs0, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f5:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfdx f0, 0, r5
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %0 = load double, double* %ptr1, align 8
   %vecins = insertelement <2 x double> %vec, double %0, i32 0
   ret <2 x double> %vecins
 }
 

llvm/test/CodeGen/PowerPC/scalar_vector_test_3.ll

 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9LE
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9BE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8LE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8BE
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test1(i32* nocapture readonly %int32, <2 x i64> %vec)  {
 ; P9LE-LABEL: s2v_test1:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfiwax f0, 0, r3
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test1:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfiwax f0, 0, r3
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test1:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwax f0, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test1:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfiwax f0, 0, r3
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %int32, align 4
   %conv = sext i32 %0 to i64
   %vecins = insertelement <2 x i64> %vec, i64 %conv, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test2(i32* nocapture readonly %int32, <2 x i64> %vec)  {
 ; P9LE-LABEL: s2v_test2:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    addi r3, r3, 4
 ; P9LE-NEXT:    lfiwax f0, 0, r3
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test2:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    addi r3, r3, 4
 ; P9BE-NEXT:    lfiwax f0, 0, r3
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test2:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    addi r3, r3, 4
 ; P8LE-NEXT:    lfiwax f0, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test2:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    addi r3, r3, 4
 ; P8BE-NEXT:    lfiwax f0, 0, r3
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 1
   %0 = load i32, i32* %arrayidx, align 4
   %conv = sext i32 %0 to i64
   %vecins = insertelement <2 x i64> %vec, i64 %conv, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test3(i32* nocapture readonly %int32, <2 x i64> %vec, i32 signext %Idx)  {
 ; P9LE-LABEL: s2v_test3:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    sldi r4, r7, 2
 ; P9LE-NEXT:    lfiwax f0, r3, r4
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test3:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    sldi r4, r7, 2
 ; P9BE-NEXT:    lfiwax f0, r3, r4
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test3:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    sldi r4, r7, 2
 ; P8LE-NEXT:    lfiwax f0, r3, r4
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test3:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    sldi r4, r7, 2
 ; P8BE-NEXT:    lfiwax f0, r3, r4
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %idxprom = sext i32 %Idx to i64
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 %idxprom
   %0 = load i32, i32* %arrayidx, align 4
   %conv = sext i32 %0 to i64
   %vecins = insertelement <2 x i64> %vec, i64 %conv, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test4(i32* nocapture readonly %int32, <2 x i64> %vec)  {
 ; P9LE-LABEL: s2v_test4:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    addi r3, r3, 4
 ; P9LE-NEXT:    lfiwax f0, 0, r3
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test4:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    addi r3, r3, 4
 ; P9BE-NEXT:    lfiwax f0, 0, r3
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test4:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    addi r3, r3, 4
 ; P8LE-NEXT:    lfiwax f0, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test4:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    addi r3, r3, 4
 ; P8BE-NEXT:    lfiwax f0, 0, r3
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 1
   %0 = load i32, i32* %arrayidx, align 4
   %conv = sext i32 %0 to i64
   %vecins = insertelement <2 x i64> %vec, i64 %conv, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test5(<2 x i64> %vec, i32* nocapture readonly %ptr1)  {
 ; P9LE-LABEL: s2v_test5:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfiwax f0, 0, r5
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P9LE-NEXT:    xxmrghd v2, v2, vs0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test5:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfiwax f0, 0, r5
 ; P9BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test5:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwax f0, 0, r5
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    xxpermdi v2, v2, v3, 1
+; P8LE-NEXT:    xxmrghd v2, v2, vs0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test5:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfiwax f0, 0, r5
 ; P8BE-NEXT:    xxpermdi v2, vs0, v2, 1
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr1, align 4
   %conv = sext i32 %0 to i64
   %vecins = insertelement <2 x i64> %vec, i64 %conv, i32 0
   ret <2 x i64> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test6(i32* nocapture readonly %ptr)  {
 ; P9LE-LABEL: s2v_test6:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfiwax f0, 0, r3
-; P9LE-NEXT:    xxswapd v2, f0
-; P9LE-NEXT:    xxspltd v2, v2, 1
+; P9LE-NEXT:    xxspltd v2, vs0, 0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test6:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfiwax f0, 0, r3
 ; P9BE-NEXT:    xxspltd v2, vs0, 0
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test6:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwax f0, 0, r3
-; P8LE-NEXT:    xxswapd v2, f0
-; P8LE-NEXT:    xxspltd v2, v2, 1
+; P8LE-NEXT:    xxspltd v2, vs0, 0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test6:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfiwax f0, 0, r3
 ; P8BE-NEXT:    xxspltd v2, vs0, 0
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr, align 4
   %conv = sext i32 %0 to i64
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @s2v_test7(i32* nocapture readonly %ptr)  {
 ; P9LE-LABEL: s2v_test7:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    lfiwax f0, 0, r3
-; P9LE-NEXT:    xxswapd v2, f0
-; P9LE-NEXT:    xxspltd v2, v2, 1
+; P9LE-NEXT:    xxspltd v2, vs0, 0
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test7:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE-NEXT:    lfiwax f0, 0, r3
 ; P9BE-NEXT:    xxspltd v2, vs0, 0
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test7:
 ; P8LE:       # %bb.0: # %entry
 ; P8LE-NEXT:    lfiwax f0, 0, r3
-; P8LE-NEXT:    xxswapd v2, f0
-; P8LE-NEXT:    xxspltd v2, v2, 1
+; P8LE-NEXT:    xxspltd v2, vs0, 0
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test7:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfiwax f0, 0, r3
 ; P8BE-NEXT:    xxspltd v2, vs0, 0
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr, align 4
   %conv = sext i32 %0 to i64
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 }
 

llvm/test/CodeGen/PowerPC/scalar_vector_test_4.ll

 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9LE
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P9BE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8LE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefix=P8BE
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x i32> @s2v_test1(i32* nocapture readonly %int32, <4 x i32> %vec)  {
 ; P8LE-LABEL: s2v_test1:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lfiwzx f0, 0, r3
 ; P8LE-NEXT:    addis r4, r2, .LCPI0_0@toc@ha
-; P8LE-NEXT:    addi r3, r4, .LCPI0_0@toc@l
-; P8LE-NEXT:    lvx v3, 0, r3
-; P8LE-NEXT:    xxswapd v4, f0
-; P8LE-NEXT:    vperm v2, v4, v2, v3
+; P8LE-NEXT:    lxsiwzx v4, 0, r3
+; P8LE-NEXT:    addi r4, r4, .LCPI0_0@toc@l
+; P8LE-NEXT:    lvx v3, 0, r4
+; P8LE-NEXT:    vperm v2, v2, v4, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test1:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %int32, align 4
   %vecins = insertelement <4 x i32> %vec, i32 %0, i32 0
   ret <4 x i32> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x i32> @s2v_test2(i32* nocapture readonly %int32, <4 x i32> %vec)  {
 ; P8LE-LABEL: s2v_test2:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    addi r3, r3, 4
 ; P8LE-NEXT:    addis r4, r2, .LCPI1_0@toc@ha
-; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    addi r3, r4, .LCPI1_0@toc@l
-; P8LE-NEXT:    lvx v3, 0, r3
-; P8LE-NEXT:    xxswapd v4, f0
-; P8LE-NEXT:    vperm v2, v4, v2, v3
+; P8LE-NEXT:    addi r3, r3, 4
+; P8LE-NEXT:    addi r4, r4, .LCPI1_0@toc@l
+; P8LE-NEXT:    lxsiwzx v4, 0, r3
+; P8LE-NEXT:    lvx v3, 0, r4
+; P8LE-NEXT:    vperm v2, v2, v4, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test2:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         addi r3, r3, 4
 ; P8BE:         lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 1
   %0 = load i32, i32* %arrayidx, align 4
   %vecins = insertelement <4 x i32> %vec, i32 %0, i32 0
   ret <4 x i32> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x i32> @s2v_test3(i32* nocapture readonly %int32, <4 x i32> %vec, i32 signext %Idx)  {
 ; P8LE-LABEL: s2v_test3:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    sldi r5, r7, 2
 ; P8LE-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
-; P8LE-NEXT:    lfiwzx f0, r3, r5
-; P8LE-NEXT:    addi r3, r4, .LCPI2_0@toc@l
-; P8LE-NEXT:    lvx v4, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    vperm v2, v3, v2, v4
+; P8LE-NEXT:    sldi r5, r7, 2
+; P8LE-NEXT:    addi r4, r4, .LCPI2_0@toc@l
+; P8LE-NEXT:    lxsiwzx v3, r3, r5
+; P8LE-NEXT:    lvx v4, 0, r4
+; P8LE-NEXT:    vperm v2, v2, v3, v4
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test3:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         sldi r4, r7, 2
 ; P8BE:         lfiwzx f0, r3, r4
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %idxprom = sext i32 %Idx to i64
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 %idxprom
   %0 = load i32, i32* %arrayidx, align 4
   %vecins = insertelement <4 x i32> %vec, i32 %0, i32 0
   ret <4 x i32> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x i32> @s2v_test4(i32* nocapture readonly %int32, <4 x i32> %vec)  {
 ; P8LE-LABEL: s2v_test4:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    addi r3, r3, 4
 ; P8LE-NEXT:    addis r4, r2, .LCPI3_0@toc@ha
-; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    addi r3, r4, .LCPI3_0@toc@l
-; P8LE-NEXT:    lvx v3, 0, r3
-; P8LE-NEXT:    xxswapd v4, f0
-; P8LE-NEXT:    vperm v2, v4, v2, v3
+; P8LE-NEXT:    addi r3, r3, 4
+; P8LE-NEXT:    addi r4, r4, .LCPI3_0@toc@l
+; P8LE-NEXT:    lxsiwzx v4, 0, r3
+; P8LE-NEXT:    lvx v3, 0, r4
+; P8LE-NEXT:    vperm v2, v2, v4, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test4:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         addi r3, r3, 4
 ; P8BE:         lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %int32, i64 1
   %0 = load i32, i32* %arrayidx, align 4
   %vecins = insertelement <4 x i32> %vec, i32 %0, i32 0
   ret <4 x i32> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x i32> @s2v_test5(<4 x i32> %vec, i32* nocapture readonly %ptr1)  {
 ; P8LE-LABEL: s2v_test5:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lfiwzx f0, 0, r5
 ; P8LE-NEXT:    addis r3, r2, .LCPI4_0@toc@ha
+; P8LE-NEXT:    lxsiwzx v4, 0, r5
 ; P8LE-NEXT:    addi r3, r3, .LCPI4_0@toc@l
 ; P8LE-NEXT:    lvx v3, 0, r3
-; P8LE-NEXT:    xxswapd v4, f0
-; P8LE-NEXT:    vperm v2, v4, v2, v3
+; P8LE-NEXT:    vperm v2, v2, v4, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test5:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         lfiwzx f0, 0, r5
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %0 = load i32, i32* %ptr1, align 4
   %vecins = insertelement <4 x i32> %vec, i32 %0, i32 0
   ret <4 x i32> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <4 x float> @s2v_test_f1(float* nocapture readonly %f64, <4 x float> %vec)  {
 ; P8LE-LABEL: s2v_test_f1:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lfiwzx f0, 0, r3
 ; P8LE-NEXT:    addis r4, r2, .LCPI5_0@toc@ha
-; P8LE-NEXT:    addi r3, r4, .LCPI5_0@toc@l
-; P8LE-NEXT:    lvx v3, 0, r3
-; P8LE-NEXT:    xxswapd v4, f0
-; P8LE-NEXT:    vperm v2, v4, v2, v3
+; P8LE-NEXT:    lxsiwzx v4, 0, r3
+; P8LE-NEXT:    addi r4, r4, .LCPI5_0@toc@l
+; P8LE-NEXT:    lvx v3, 0, r4
+; P8LE-NEXT:    vperm v2, v2, v4, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f1:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE:         lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE:         xxsldwi vs0, v2, vs0, 1
 ; P8BE:         xxsldwi v2, vs0, vs0, 3
 ; P8BE-NEXT:    blr
 entry:
   %0 = load float, float* %f64, align 4
   %vecins = insertelement <4 x float> %vec, float %0, i32 0
   ret <4 x float> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x float> @s2v_test_f2(float* nocapture readonly %f64, <2 x float> %vec)  {
 ; P9LE-LABEL: s2v_test_f2:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    addi r3, r3, 4
-; P9LE-DAG:     xxspltw v2, v2, 2
-; P9LE-DAG:     lfiwzx f0, 0, r3
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    vmrglw v2, v2, v3
+; P9LE-NEXT:    lxsiwzx v3, 0, r3
+; P9LE-NEXT:    vmrglw v2, v2, v2
+; P9LE-NEXT:    vmrghw v2, v2, v3
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f2:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE:       addi r3, r3, 4
 ; P9BE-DAG:   xxspltw v2, v2, 1
 ; P9BE-DAG:   lfiwzx f0, 0, r3
 ; P9BE-NEXT:  xxsldwi v3, f0, f0, 1
 ; P9BE:       vmrghw v2, v3, v2
 ; P9BE-NEXT:  blr
 
 ; P8LE-LABEL: s2v_test_f2:
 ; P8LE:       # %bb.0: # %entry
+; P8LE-NEXT:    vmrglw v2, v2, v2
 ; P8LE-NEXT:    addi r3, r3, 4
-; P8LE-NEXT:    xxspltw v2, v2, 2
-; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    lxsiwzx v3, 0, r3
+; P8LE-NEXT:    vmrghw v2, v2, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f2:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    addi r3, r3, 4
 ; P8BE-NEXT:    xxspltw v2, v2, 1
 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi v3, f0, f0, 1
 ; P8BE-NEXT:    vmrghw v2, v3, v2
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds float, float* %f64, i64 1
   %0 = load float, float* %arrayidx, align 8
   %vecins = insertelement <2 x float> %vec, float %0, i32 0
   ret <2 x float> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x float> @s2v_test_f3(float* nocapture readonly %f64, <2 x float> %vec, i32 signext %Idx)  {
 ; P9LE-LABEL: s2v_test_f3:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    sldi r4, r7, 2
-; P9LE-NEXT:    lfiwzx f0, r3, r4
-; P9LE-DAG:     xxspltw v2, v2, 2
-; P9LE-DAG:     xxswapd v3, f0
-; P9LE-NEXT:    vmrglw v2, v2, v3
+; P9LE-NEXT:    lxsiwzx v3, r3, r4
+; P9LE-NEXT:    vmrglw v2, v2, v2
+; P9LE-NEXT:    vmrghw v2, v2, v3
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f3:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE:         sldi r4, r7, 2
 ; P9BE:         lfiwzx f0, r3, r4
 ; P9BE-DAG:     xxspltw v2, v2, 1
 ; P9BE-DAG:     xxsldwi v3, f0, f0, 1
 ; P9BE:         vmrghw v2, v3, v2
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f3:
 ; P8LE:       # %bb.0: # %entry
+; P8LE-NEXT:    vmrglw v2, v2, v2
 ; P8LE-NEXT:    sldi r4, r7, 2
-; P8LE-NEXT:    xxspltw v2, v2, 2
-; P8LE-NEXT:    lfiwzx f0, r3, r4
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    lxsiwzx v3, r3, r4
+; P8LE-NEXT:    vmrghw v2, v2, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f3:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    sldi r4, r7, 2
 ; P8BE-NEXT:    xxspltw v2, v2, 1
 ; P8BE-NEXT:    lfiwzx f0, r3, r4
 ; P8BE-NEXT:    xxsldwi v3, f0, f0, 1
 ; P8BE-NEXT:    vmrghw v2, v3, v2
 ; P8BE-NEXT:    blr
 entry:
   %idxprom = sext i32 %Idx to i64
   %arrayidx = getelementptr inbounds float, float* %f64, i64 %idxprom
   %0 = load float, float* %arrayidx, align 8
   %vecins = insertelement <2 x float> %vec, float %0, i32 0
   ret <2 x float> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x float> @s2v_test_f4(float* nocapture readonly %f64, <2 x float> %vec)  {
 ; P9LE-LABEL: s2v_test_f4:
 ; P9LE:       # %bb.0: # %entry
 ; P9LE-NEXT:    addi r3, r3, 4
-; P9LE-NEXT:    lfiwzx f0, 0, r3
-; P9LE-DAG:     xxspltw v2, v2, 2
-; P9LE-DAG:     xxswapd v3, f0
-; P9LE-NEXT:    vmrglw v2, v2, v3
+; P9LE-NEXT:    lxsiwzx v3, 0, r3
+; P9LE-NEXT:    vmrglw v2, v2, v2
+; P9LE-NEXT:    vmrghw v2, v2, v3
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f4:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE:         addi r3, r3, 4
 ; P9BE:         lfiwzx f0, 0, r3
 ; P9BE-DAG:     xxspltw v2, v2, 1
 ; P9BE-DAG:     xxsldwi v3, f0, f0, 1
 ; P9BE:         vmrghw v2, v3, v2
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f4:
 ; P8LE:       # %bb.0: # %entry
+; P8LE-NEXT:    vmrglw v2, v2, v2
 ; P8LE-NEXT:    addi r3, r3, 4
-; P8LE-NEXT:    xxspltw v2, v2, 2
-; P8LE-NEXT:    lfiwzx f0, 0, r3
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    lxsiwzx v3, 0, r3
+; P8LE-NEXT:    vmrghw v2, v2, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f4:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    addi r3, r3, 4
 ; P8BE-NEXT:    xxspltw v2, v2, 1
 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    xxsldwi v3, f0, f0, 1
 ; P8BE-NEXT:    vmrghw v2, v3, v2
 ; P8BE-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds float, float* %f64, i64 1
   %0 = load float, float* %arrayidx, align 8
   %vecins = insertelement <2 x float> %vec, float %0, i32 0
   ret <2 x float> %vecins
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x float> @s2v_test_f5(<2 x float> %vec, float* nocapture readonly %ptr1)  {
 ; P9LE-LABEL: s2v_test_f5:
 ; P9LE:       # %bb.0: # %entry
-; P9LE-NEXT:    lfiwzx f0, 0, r5
-; P9LE-NEXT:    xxspltw v2, v2, 2
-; P9LE-NEXT:    xxswapd v3, f0
-; P9LE-NEXT:    vmrglw v2, v2, v3
+; P9LE-NEXT:    lxsiwzx v3, 0, r5
+; P9LE-NEXT:    vmrglw v2, v2, v2
+; P9LE-NEXT:    vmrghw v2, v2, v3
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: s2v_test_f5:
 ; P9BE:       # %bb.0: # %entry
 ; P9BE:         lfiwzx f0, 0, r5
 ; P9BE:         xxspltw v2, v2, 1
 ; P9BE-NEXT:    xxsldwi v3, f0, f0, 1
 ; P9BE:         vmrghw v2, v3, v2
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: s2v_test_f5:
 ; P8LE:       # %bb.0: # %entry
-; P8LE-NEXT:    lfiwzx f0, 0, r5
-; P8LE-NEXT:    xxspltw v2, v2, 2
-; P8LE-NEXT:    xxswapd v3, f0
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    vmrglw v2, v2, v2
+; P8LE-NEXT:    lxsiwzx v3, 0, r5
+; P8LE-NEXT:    vmrghw v2, v2, v3
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: s2v_test_f5:
 ; P8BE:       # %bb.0: # %entry
 ; P8BE-NEXT:    lfiwzx f0, 0, r5
 ; P8BE-NEXT:    xxspltw v2, v2, 1
 ; P8BE-NEXT:    xxsldwi v3, f0, f0, 1
 ; P8BE-NEXT:    vmrghw v2, v3, v2
 ; P8BE-NEXT:    blr
 entry:
   %0 = load float, float* %ptr1, align 8
   %vecins = insertelement <2 x float> %vec, float %0, i32 0
   ret <2 x float> %vecins
 }
 

llvm/test/CodeGen/PowerPC/srem-vector-lkk.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P9LE
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P9BE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P8LE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P8BE
 
 define <4 x i16> @fold_srem_vec_1(<4 x i16> %x) {
 ; P9LE-LABEL: fold_srem_vec_1:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -21386
-; P9LE-NEXT:    ori r5, r5, 37253
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    lis r4, -21386
+; P9LE-NEXT:    ori r4, r4, 37253
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 6
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, 31710
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 31710
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    ori r5, r5, 63421
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    sub r4, r5, r4
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    ori r4, r4, 63421
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    sub r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 6
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, 21399
 ; P9LE-NEXT:    mulli r4, r4, -124
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 21399
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    ori r5, r5, 33437
-; P9LE-NEXT:    mulhw r4, r4, r5
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    ori r4, r4, 33437
+; P9LE-NEXT:    mulhw r4, r3, r4
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 5
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, -16728
 ; P9LE-NEXT:    mulli r4, r4, 98
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    ori r5, r5, 63249
-; P9LE-NEXT:    mulhw r4, r4, r5
+; P9LE-NEXT:    lis r4, -16728
+; P9LE-NEXT:    ori r4, r4, 63249
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 8
 ; P9LE-NEXT:    add r4, r4, r5
 ; P9LE-NEXT:    mulli r4, r4, -1003
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: fold_srem_vec_1:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 2
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, 31710

 ; P9BE-NEXT:    vmrghh v2, v2, v4
 ; P9BE-NEXT:    vmrghw v2, v3, v2
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fold_srem_vec_1:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, 21399
-; P8LE-NEXT:    lis r9, -21386
-; P8LE-NEXT:    lis r11, 31710
 ; P8LE-NEXT:    lis r8, -16728
+; P8LE-NEXT:    lis r9, -21386
+; P8LE-NEXT:    lis r10, 31710
 ; P8LE-NEXT:    ori r3, r3, 33437
-; P8LE-NEXT:    ori r9, r9, 37253
 ; P8LE-NEXT:    ori r8, r8, 63249
+; P8LE-NEXT:    ori r9, r9, 37253
+; P8LE-NEXT:    ori r10, r10, 63421
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    rldicl r5, r4, 32, 48
-; P8LE-NEXT:    clrldi r7, r4, 48
 ; P8LE-NEXT:    rldicl r6, r4, 16, 48
+; P8LE-NEXT:    clrldi r7, r4, 48
+; P8LE-NEXT:    extsh r5, r5
+; P8LE-NEXT:    extsh r6, r6
 ; P8LE-NEXT:    rldicl r4, r4, 48, 48
-; P8LE-NEXT:    extsh r10, r5
-; P8LE-NEXT:    extsh r0, r7
-; P8LE-NEXT:    mulhw r3, r10, r3
-; P8LE-NEXT:    ori r10, r11, 63421
-; P8LE-NEXT:    extsh r11, r4
-; P8LE-NEXT:    extsh r12, r6
-; P8LE-NEXT:    mulhw r9, r0, r9
-; P8LE-NEXT:    mulhw r10, r11, r10
-; P8LE-NEXT:    mulhw r8, r12, r8
-; P8LE-NEXT:    srwi r12, r3, 31
+; P8LE-NEXT:    extsh r7, r7
+; P8LE-NEXT:    mulhw r3, r5, r3
+; P8LE-NEXT:    extsh r4, r4
+; P8LE-NEXT:    mulhw r8, r6, r8
+; P8LE-NEXT:    mulhw r9, r7, r9
+; P8LE-NEXT:    mulhw r10, r4, r10
+; P8LE-NEXT:    srwi r11, r3, 31
 ; P8LE-NEXT:    srawi r3, r3, 5
-; P8LE-NEXT:    add r9, r9, r0
-; P8LE-NEXT:    sub r10, r10, r11
-; P8LE-NEXT:    add r3, r3, r12
+; P8LE-NEXT:    add r3, r3, r11
+; P8LE-NEXT:    srwi r11, r8, 31
+; P8LE-NEXT:    add r9, r9, r7
+; P8LE-NEXT:    srawi r8, r8, 8
+; P8LE-NEXT:    sub r10, r10, r4
+; P8LE-NEXT:    add r8, r8, r11
 ; P8LE-NEXT:    srwi r11, r9, 31
 ; P8LE-NEXT:    srawi r9, r9, 6
-; P8LE-NEXT:    srwi r12, r8, 31
-; P8LE-NEXT:    srawi r8, r8, 8
+; P8LE-NEXT:    mulli r3, r3, 98
 ; P8LE-NEXT:    add r9, r9, r11
 ; P8LE-NEXT:    srwi r11, r10, 31
 ; P8LE-NEXT:    srawi r10, r10, 6
-; P8LE-NEXT:    add r8, r8, r12
-; P8LE-NEXT:    mulli r3, r3, 98
-; P8LE-NEXT:    add r10, r10, r11
 ; P8LE-NEXT:    mulli r8, r8, -1003
+; P8LE-NEXT:    add r10, r10, r11
 ; P8LE-NEXT:    mulli r9, r9, 95
 ; P8LE-NEXT:    mulli r10, r10, -124
 ; P8LE-NEXT:    sub r3, r5, r3
+; P8LE-NEXT:    mtvsrd v2, r3
 ; P8LE-NEXT:    sub r5, r6, r8
-; P8LE-NEXT:    mtfprd f0, r3
 ; P8LE-NEXT:    sub r3, r7, r9
+; P8LE-NEXT:    mtvsrd v3, r5
 ; P8LE-NEXT:    sub r4, r4, r10
-; P8LE-NEXT:    mtfprd f1, r5
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    vmrglh v3, v5, v4
+; P8LE-NEXT:    mtvsrd v4, r3
+; P8LE-NEXT:    mtvsrd v5, r4
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    vmrghh v3, v5, v4
 ; P8LE-NEXT:    vmrglw v2, v2, v3
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: fold_srem_vec_1:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, -16728
 ; P8BE-NEXT:    lis r8, 21399
; P8BE-NEXT:    blr
   ret <4 x i16> %1
 }
 
 define <4 x i16> @fold_srem_vec_2(<4 x i16> %x) {
 ; P9LE-LABEL: fold_srem_vec_2:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -21386
-; P9LE-NEXT:    ori r5, r5, 37253
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r6, r4, r5
-; P9LE-NEXT:    add r4, r6, r4
-; P9LE-NEXT:    srwi r6, r4, 31
-; P9LE-NEXT:    srawi r4, r4, 6
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, -21386
+; P9LE-NEXT:    ori r4, r4, 37253
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r5, r3, r4
+; P9LE-NEXT:    add r5, r5, r3
+; P9LE-NEXT:    srwi r6, r5, 31
+; P9LE-NEXT:    srawi r5, r5, 6
+; P9LE-NEXT:    add r5, r5, r6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r6, r4, r5
-; P9LE-NEXT:    add r4, r6, r4
-; P9LE-NEXT:    srwi r6, r4, 31
-; P9LE-NEXT:    srawi r4, r4, 6
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r5, r3, r4
+; P9LE-NEXT:    add r5, r5, r3
+; P9LE-NEXT:    srwi r6, r5, 31
+; P9LE-NEXT:    srawi r5, r5, 6
+; P9LE-NEXT:    add r5, r5, r6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r6, r4, r5
-; P9LE-NEXT:    add r4, r6, r4
-; P9LE-NEXT:    srwi r6, r4, 31
-; P9LE-NEXT:    srawi r4, r4, 6
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r5, r3, r4
+; P9LE-NEXT:    add r5, r5, r3
+; P9LE-NEXT:    srwi r6, r5, 31
+; P9LE-NEXT:    srawi r5, r5, 6
+; P9LE-NEXT:    add r5, r5, r6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 6
 ; P9LE-NEXT:    add r4, r4, r5
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: fold_srem_vec_2:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, -21386

 ; P9BE-NEXT:    vmrghh v2, v2, v4
 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fold_srem_vec_2:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, -21386
-; P8LE-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
 ; P8LE-NEXT:    ori r3, r3, 37253
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    clrldi r5, r4, 48
 ; P8LE-NEXT:    rldicl r6, r4, 48, 48
-; P8LE-NEXT:    extsh r8, r5
+; P8LE-NEXT:    extsh r5, r5
 ; P8LE-NEXT:    rldicl r7, r4, 32, 48
-; P8LE-NEXT:    extsh r9, r6
-; P8LE-NEXT:    mulhw r10, r8, r3
+; P8LE-NEXT:    extsh r6, r6
+; P8LE-NEXT:    mulhw r8, r5, r3
 ; P8LE-NEXT:    rldicl r4, r4, 16, 48
-; P8LE-NEXT:    extsh r11, r7
-; P8LE-NEXT:    mulhw r12, r9, r3
-; P8LE-NEXT:    extsh r0, r4
-; P8LE-NEXT:    mulhw r30, r11, r3
-; P8LE-NEXT:    mulhw r3, r0, r3
-; P8LE-NEXT:    add r8, r10, r8
-; P8LE-NEXT:    add r9, r12, r9
-; P8LE-NEXT:    srwi r10, r8, 31
+; P8LE-NEXT:    extsh r7, r7
+; P8LE-NEXT:    mulhw r9, r6, r3
+; P8LE-NEXT:    extsh r4, r4
+; P8LE-NEXT:    mulhw r10, r7, r3
+; P8LE-NEXT:    mulhw r3, r4, r3
+; P8LE-NEXT:    add r8, r8, r5
+; P8LE-NEXT:    add r9, r9, r6
+; P8LE-NEXT:    srwi r11, r8, 31
 ; P8LE-NEXT:    srawi r8, r8, 6
-; P8LE-NEXT:    add r11, r30, r11
-; P8LE-NEXT:    add r3, r3, r0
-; P8LE-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
-; P8LE-NEXT:    add r8, r8, r10
-; P8LE-NEXT:    srwi r10, r9, 31
+; P8LE-NEXT:    add r10, r10, r7
+; P8LE-NEXT:    add r3, r3, r4
+; P8LE-NEXT:    add r8, r8, r11
+; P8LE-NEXT:    srwi r11, r9, 31
 ; P8LE-NEXT:    srawi r9, r9, 6
 ; P8LE-NEXT:    mulli r8, r8, 95
-; P8LE-NEXT:    add r9, r9, r10
-; P8LE-NEXT:    srwi r10, r11, 31
-; P8LE-NEXT:    srawi r11, r11, 6
+; P8LE-NEXT:    add r9, r9, r11
+; P8LE-NEXT:    srwi r11, r10, 31
+; P8LE-NEXT:    srawi r10, r10, 6
 ; P8LE-NEXT:    mulli r9, r9, 95
-; P8LE-NEXT:    add r10, r11, r10
+; P8LE-NEXT:    add r10, r10, r11
 ; P8LE-NEXT:    srwi r11, r3, 31
 ; P8LE-NEXT:    srawi r3, r3, 6
 ; P8LE-NEXT:    mulli r10, r10, 95
 ; P8LE-NEXT:    sub r5, r5, r8
 ; P8LE-NEXT:    add r3, r3, r11
-; P8LE-NEXT:    mtfprd f0, r5
+; P8LE-NEXT:    mtvsrd v2, r5
 ; P8LE-NEXT:    mulli r3, r3, 95
 ; P8LE-NEXT:    sub r6, r6, r9
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    xxswapd v2, vs0
+; P8LE-NEXT:    mtvsrd v3, r6
 ; P8LE-NEXT:    sub r5, r7, r10
-; P8LE-NEXT:    mtfprd f2, r5
-; P8LE-NEXT:    xxswapd v3, vs1
+; P8LE-NEXT:    mtvsrd v4, r5
 ; P8LE-NEXT:    sub r3, r4, r3
-; P8LE-NEXT:    mtfprd f3, r3
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    vmrglh v3, v5, v4
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v3, v5, v4
 ; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: fold_srem_vec_2:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, -21386
 ; P8BE-NEXT:    ori r3, r3, 37253

 
 
 ; Don't fold if we can combine srem with sdiv.
 define <4 x i16> @combine_srem_sdiv(<4 x i16> %x) {
 ; P9LE-LABEL: combine_srem_sdiv:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -21386
-; P9LE-NEXT:    ori r5, r5, 37253
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r6, r4, r5
-; P9LE-NEXT:    add r4, r6, r4
-; P9LE-NEXT:    srwi r6, r4, 31
-; P9LE-NEXT:    srawi r4, r4, 6
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    mulli r6, r4, 95
+; P9LE-NEXT:    lis r4, -21386
+; P9LE-NEXT:    ori r4, r4, 37253
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r5, r3, r4
+; P9LE-NEXT:    add r5, r5, r3
+; P9LE-NEXT:    srwi r6, r5, 31
+; P9LE-NEXT:    srawi r5, r5, 6
+; P9LE-NEXT:    add r5, r5, r6
+; P9LE-NEXT:    mulli r6, r5, 95
 ; P9LE-NEXT:    sub r3, r3, r6
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    extsh r6, r3
-; P9LE-NEXT:    mulhw r7, r6, r5
+; P9LE-NEXT:    mulhw r7, r6, r4
 ; P9LE-NEXT:    add r6, r7, r6
 ; P9LE-NEXT:    srwi r7, r6, 31
 ; P9LE-NEXT:    srawi r6, r6, 6
 ; P9LE-NEXT:    add r6, r6, r7
 ; P9LE-NEXT:    mulli r7, r6, 95
 ; P9LE-NEXT:    sub r3, r3, r7
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    extsh r7, r3
-; P9LE-NEXT:    mulhw r8, r7, r5
+; P9LE-NEXT:    mulhw r8, r7, r4
 ; P9LE-NEXT:    add r7, r8, r7
 ; P9LE-NEXT:    srwi r8, r7, 31
 ; P9LE-NEXT:    srawi r7, r7, 6
 ; P9LE-NEXT:    add r7, r7, r8
 ; P9LE-NEXT:    mulli r8, r7, 95
 ; P9LE-NEXT:    sub r3, r3, r8
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    extsh r8, r3
-; P9LE-NEXT:    mulhw r5, r8, r5
-; P9LE-NEXT:    add r5, r5, r8
-; P9LE-NEXT:    srwi r8, r5, 31
-; P9LE-NEXT:    srawi r5, r5, 6
-; P9LE-NEXT:    add r5, r5, r8
-; P9LE-NEXT:    mulli r8, r5, 95
+; P9LE-NEXT:    mulhw r4, r8, r4
+; P9LE-NEXT:    add r4, r4, r8
+; P9LE-NEXT:    srwi r8, r4, 31
+; P9LE-NEXT:    srawi r4, r4, 6
+; P9LE-NEXT:    add r4, r4, r8
+; P9LE-NEXT:    mulli r8, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r8
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    mtfprd f0, r4
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v4, r6
 ; P9LE-NEXT:    vmrglw v2, v2, v3
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r6
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r7
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r5
-; P9LE-NEXT:    xxswapd v5, vs0
-; P9LE-NEXT:    vmrglh v4, v5, v4
+; P9LE-NEXT:    mtvsrd v3, r5
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r7
+; P9LE-NEXT:    mtvsrd v5, r4
+; P9LE-NEXT:    vmrghh v4, v5, v4
 ; P9LE-NEXT:    vmrglw v3, v4, v3
 ; P9LE-NEXT:    vadduhm v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: combine_srem_sdiv:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2

 ; P9BE-NEXT:    vmrghh v4, v5, v4
 ; P9BE-NEXT:    vmrghw v3, v4, v3
 ; P9BE-NEXT:    vadduhm v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: combine_srem_sdiv:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
-; P8LE-NEXT:    lis r4, -21386
-; P8LE-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
-; P8LE-NEXT:    ori r4, r4, 37253
-; P8LE-NEXT:    mffprd r5, f0
-; P8LE-NEXT:    clrldi r3, r5, 48
-; P8LE-NEXT:    rldicl r6, r5, 48, 48
-; P8LE-NEXT:    rldicl r7, r5, 32, 48
-; P8LE-NEXT:    extsh r8, r3
-; P8LE-NEXT:    extsh r9, r6
-; P8LE-NEXT:    extsh r10, r7
-; P8LE-NEXT:    mulhw r11, r8, r4
-; P8LE-NEXT:    rldicl r5, r5, 16, 48
-; P8LE-NEXT:    mulhw r12, r9, r4
-; P8LE-NEXT:    mulhw r0, r10, r4
-; P8LE-NEXT:    extsh r30, r5
-; P8LE-NEXT:    mulhw r4, r30, r4
+; P8LE-NEXT:    lis r3, -21386
+; P8LE-NEXT:    ori r3, r3, 37253
+; P8LE-NEXT:    mffprd r4, f0
+; P8LE-NEXT:    clrldi r5, r4, 48
+; P8LE-NEXT:    rldicl r6, r4, 48, 48
+; P8LE-NEXT:    rldicl r7, r4, 32, 48
+; P8LE-NEXT:    extsh r5, r5
+; P8LE-NEXT:    extsh r8, r6
+; P8LE-NEXT:    extsh r9, r7
+; P8LE-NEXT:    mulhw r10, r5, r3
+; P8LE-NEXT:    mulhw r11, r8, r3
+; P8LE-NEXT:    rldicl r4, r4, 16, 48
+; P8LE-NEXT:    mulhw r12, r9, r3
+; P8LE-NEXT:    extsh r0, r4
+; P8LE-NEXT:    mulhw r3, r0, r3
+; P8LE-NEXT:    add r10, r10, r5
 ; P8LE-NEXT:    add r8, r11, r8
+; P8LE-NEXT:    srwi r11, r10, 31
 ; P8LE-NEXT:    add r9, r12, r9
-; P8LE-NEXT:    srwi r11, r8, 31
-; P8LE-NEXT:    add r10, r0, r10
-; P8LE-NEXT:    srawi r8, r8, 6
-; P8LE-NEXT:    srawi r12, r9, 6
+; P8LE-NEXT:    srawi r10, r10, 6
+; P8LE-NEXT:    srawi r12, r8, 6
+; P8LE-NEXT:    srwi r8, r8, 31
+; P8LE-NEXT:    add r10, r10, r11
+; P8LE-NEXT:    add r3, r3, r0
+; P8LE-NEXT:    srawi r11, r9, 6
 ; P8LE-NEXT:    srwi r9, r9, 31
-; P8LE-NEXT:    add r8, r8, r11
-; P8LE-NEXT:    add r4, r4, r30
-; P8LE-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
-; P8LE-NEXT:    srawi r11, r10, 6
-; P8LE-NEXT:    srwi r10, r10, 31
-; P8LE-NEXT:    add r9, r12, r9
-; P8LE-NEXT:    mtfprd f0, r8
-; P8LE-NEXT:    mulli r12, r8, 95
-; P8LE-NEXT:    add r10, r11, r10
-; P8LE-NEXT:    srwi r8, r4, 31
-; P8LE-NEXT:    mtfprd f1, r9
-; P8LE-NEXT:    srawi r4, r4, 6
-; P8LE-NEXT:    mulli r11, r9, 95
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f2, r10
-; P8LE-NEXT:    mulli r9, r10, 95
-; P8LE-NEXT:    add r4, r4, r8
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    mulli r4, r4, 95
-; P8LE-NEXT:    xxswapd v1, vs2
-; P8LE-NEXT:    sub r3, r3, r12
-; P8LE-NEXT:    mtfprd f0, r3
-; P8LE-NEXT:    sub r6, r6, r11
-; P8LE-NEXT:    xxswapd v6, vs3
-; P8LE-NEXT:    sub r3, r7, r9
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    mtfprd f4, r3
-; P8LE-NEXT:    sub r3, r5, r4
-; P8LE-NEXT:    mtfprd f5, r3
-; P8LE-NEXT:    xxswapd v4, vs1
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    xxswapd v5, vs4
-; P8LE-NEXT:    xxswapd v0, vs5
-; P8LE-NEXT:    vmrglh v3, v4, v3
-; P8LE-NEXT:    vmrglh v4, v0, v5
-; P8LE-NEXT:    vmrglh v5, v6, v1
-; P8LE-NEXT:    vmrglw v3, v4, v3
-; P8LE-NEXT:    vmrglw v2, v5, v2
+; P8LE-NEXT:    add r8, r12, r8
+; P8LE-NEXT:    mtvsrd v2, r10
+; P8LE-NEXT:    mulli r12, r10, 95
+; P8LE-NEXT:    add r9, r11, r9
+; P8LE-NEXT:    srwi r11, r3, 31
+; P8LE-NEXT:    mtvsrd v3, r8
+; P8LE-NEXT:    srawi r3, r3, 6
+; P8LE-NEXT:    mulli r10, r8, 95
+; P8LE-NEXT:    mtvsrd v4, r9
+; P8LE-NEXT:    add r3, r3, r11
+; P8LE-NEXT:    mulli r8, r9, 95
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    mulli r9, r3, 95
+; P8LE-NEXT:    sub r5, r5, r12
+; P8LE-NEXT:    sub r6, r6, r10
+; P8LE-NEXT:    mtvsrd v3, r5
+; P8LE-NEXT:    mtvsrd v5, r6
+; P8LE-NEXT:    sub r5, r7, r8
+; P8LE-NEXT:    sub r4, r4, r9
+; P8LE-NEXT:    mtvsrd v0, r5
+; P8LE-NEXT:    mtvsrd v1, r4
+; P8LE-NEXT:    vmrghh v3, v5, v3
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v0, v1, v0
+; P8LE-NEXT:    vmrghh v4, v5, v4
+; P8LE-NEXT:    vmrglw v3, v0, v3
+; P8LE-NEXT:    vmrglw v2, v4, v2
 ; P8LE-NEXT:    vadduhm v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: combine_srem_sdiv:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r5, v2
 ; P8BE-NEXT:    lis r4, -21386
 ; P8BE-NEXT:    ori r4, r4, 37253

 }
 
 ; Don't fold for divisors that are a power of two.
 define <4 x i16> @dont_fold_srem_power_of_two(<4 x i16> %x) {
 ; P9LE-LABEL: dont_fold_srem_power_of_two:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    srawi r4, r4, 6
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    srawi r4, r3, 6
 ; P9LE-NEXT:    addze r4, r4
 ; P9LE-NEXT:    slwi r4, r4, 6
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    srawi r4, r4, 5
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    srawi r4, r3, 5
 ; P9LE-NEXT:    addze r4, r4
 ; P9LE-NEXT:    slwi r4, r4, 5
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, -21386
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -21386
-; P9LE-NEXT:    ori r5, r5, 37253
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    ori r4, r4, 37253
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 6
 ; P9LE-NEXT:    add r4, r4, r5
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    srawi r4, r4, 3
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    srawi r4, r3, 3
 ; P9LE-NEXT:    addze r4, r4
 ; P9LE-NEXT:    slwi r4, r4, 3
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v4, v2
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v4, v2
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: dont_fold_srem_power_of_two:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 2
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    extsh r3, r3

 ;
 ; P8LE-LABEL: dont_fold_srem_power_of_two:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, -21386
 ; P8LE-NEXT:    ori r3, r3, 37253
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    rldicl r5, r4, 16, 48
-; P8LE-NEXT:    clrldi r7, r4, 48
-; P8LE-NEXT:    extsh r6, r5
-; P8LE-NEXT:    extsh r8, r7
-; P8LE-NEXT:    mulhw r3, r6, r3
-; P8LE-NEXT:    rldicl r9, r4, 48, 48
-; P8LE-NEXT:    srawi r8, r8, 6
-; P8LE-NEXT:    extsh r10, r9
+; P8LE-NEXT:    clrldi r6, r4, 48
+; P8LE-NEXT:    extsh r5, r5
+; P8LE-NEXT:    extsh r6, r6
+; P8LE-NEXT:    mulhw r3, r5, r3
+; P8LE-NEXT:    rldicl r7, r4, 48, 48
+; P8LE-NEXT:    srawi r8, r6, 6
+; P8LE-NEXT:    extsh r7, r7
 ; P8LE-NEXT:    addze r8, r8
 ; P8LE-NEXT:    rldicl r4, r4, 32, 48
-; P8LE-NEXT:    srawi r10, r10, 5
+; P8LE-NEXT:    srawi r9, r7, 5
+; P8LE-NEXT:    extsh r4, r4
 ; P8LE-NEXT:    slwi r8, r8, 6
-; P8LE-NEXT:    add r3, r3, r6
-; P8LE-NEXT:    addze r6, r10
-; P8LE-NEXT:    sub r7, r7, r8
+; P8LE-NEXT:    add r3, r3, r5
+; P8LE-NEXT:    addze r9, r9
+; P8LE-NEXT:    sub r6, r6, r8
 ; P8LE-NEXT:    srwi r10, r3, 31
 ; P8LE-NEXT:    srawi r3, r3, 6
-; P8LE-NEXT:    mtfprd f0, r7
-; P8LE-NEXT:    slwi r6, r6, 5
+; P8LE-NEXT:    slwi r8, r9, 5
+; P8LE-NEXT:    mtvsrd v2, r6
 ; P8LE-NEXT:    add r3, r3, r10
-; P8LE-NEXT:    extsh r10, r4
-; P8LE-NEXT:    sub r6, r9, r6
+; P8LE-NEXT:    srawi r9, r4, 3
+; P8LE-NEXT:    sub r6, r7, r8
 ; P8LE-NEXT:    mulli r3, r3, 95
-; P8LE-NEXT:    srawi r8, r10, 3
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    addze r7, r8
-; P8LE-NEXT:    xxswapd v3, vs1
+; P8LE-NEXT:    addze r7, r9
+; P8LE-NEXT:    mtvsrd v3, r6
+; P8LE-NEXT:    vmrghh v2, v3, v2
 ; P8LE-NEXT:    sub r3, r5, r3
 ; P8LE-NEXT:    slwi r5, r7, 3
 ; P8LE-NEXT:    sub r4, r4, r5
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    vmrglh v3, v4, v5
+; P8LE-NEXT:    mtvsrd v4, r3
+; P8LE-NEXT:    mtvsrd v5, r4
+; P8LE-NEXT:    vmrghh v3, v4, v5
 ; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: dont_fold_srem_power_of_two:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, -21386
 ; P8BE-NEXT:    ori r3, r3, 37253

 }
 
 ; Don't fold if the divisor is one.
 define <4 x i16> @dont_fold_srem_one(<4 x i16> %x) {
 ; P9LE-LABEL: dont_fold_srem_one:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -14230
-; P9LE-NEXT:    ori r5, r5, 30865
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    lis r4, -14230
+; P9LE-NEXT:    ori r4, r4, 30865
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 9
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, -19946
 ; P9LE-NEXT:    mulli r4, r4, 654
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, -19946
+; P9LE-NEXT:    mtvsrd v3, r3
+; P9LE-NEXT:    li r3, 0
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
-; P9LE-NEXT:    ori r5, r5, 17097
-; P9LE-NEXT:    xxlxor v3, v3, v3
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    ori r4, r4, 17097
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 4
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, 24749
 ; P9LE-NEXT:    mulli r4, r4, 23
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    vmrghh v3, v3, v4
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    ori r5, r5, 47143
-; P9LE-NEXT:    mulhw r4, r4, r5
+; P9LE-NEXT:    lis r4, 24749
+; P9LE-NEXT:    ori r4, r4, 47143
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 11
 ; P9LE-NEXT:    add r4, r4, r5
 ; P9LE-NEXT:    mulli r4, r4, 5423
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: dont_fold_srem_one:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 4
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, -19946

 ; P9BE-NEXT:    mtvsrd v4, r3
 ; P9BE-NEXT:    vmrghh v2, v4, v2
 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: dont_fold_srem_one:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
-; P8LE-NEXT:    lis r3, 24749
-; P8LE-NEXT:    lis r7, -19946
-; P8LE-NEXT:    lis r9, -14230
-; P8LE-NEXT:    xxlxor v5, v5, v5
-; P8LE-NEXT:    ori r3, r3, 47143
-; P8LE-NEXT:    ori r7, r7, 17097
-; P8LE-NEXT:    mffprd r4, f0
-; P8LE-NEXT:    rldicl r5, r4, 16, 48
-; P8LE-NEXT:    rldicl r6, r4, 32, 48
-; P8LE-NEXT:    rldicl r4, r4, 48, 48
-; P8LE-NEXT:    extsh r8, r5
-; P8LE-NEXT:    extsh r10, r6
-; P8LE-NEXT:    mulhw r3, r8, r3
-; P8LE-NEXT:    ori r8, r9, 30865
-; P8LE-NEXT:    extsh r9, r4
-; P8LE-NEXT:    mulhw r7, r10, r7
-; P8LE-NEXT:    mulhw r8, r9, r8
-; P8LE-NEXT:    add r7, r7, r10
-; P8LE-NEXT:    srwi r10, r3, 31
-; P8LE-NEXT:    add r8, r8, r9
-; P8LE-NEXT:    srawi r3, r3, 11
-; P8LE-NEXT:    srwi r9, r7, 31
-; P8LE-NEXT:    srawi r7, r7, 4
-; P8LE-NEXT:    add r3, r3, r10
-; P8LE-NEXT:    add r7, r7, r9
+; P8LE-NEXT:    lis r5, 24749
+; P8LE-NEXT:    lis r6, -19946
+; P8LE-NEXT:    lis r8, -14230
+; P8LE-NEXT:    ori r5, r5, 47143
+; P8LE-NEXT:    ori r6, r6, 17097
+; P8LE-NEXT:    ori r8, r8, 30865
+; P8LE-NEXT:    mffprd r3, f0
+; P8LE-NEXT:    rldicl r4, r3, 16, 48
+; P8LE-NEXT:    rldicl r7, r3, 32, 48
+; P8LE-NEXT:    rldicl r3, r3, 48, 48
+; P8LE-NEXT:    extsh r4, r4
+; P8LE-NEXT:    extsh r7, r7
+; P8LE-NEXT:    extsh r3, r3
+; P8LE-NEXT:    mulhw r5, r4, r5
+; P8LE-NEXT:    mulhw r6, r7, r6
+; P8LE-NEXT:    mulhw r8, r3, r8
+; P8LE-NEXT:    srwi r9, r5, 31
+; P8LE-NEXT:    srawi r5, r5, 11
+; P8LE-NEXT:    add r6, r6, r7
+; P8LE-NEXT:    add r8, r8, r3
+; P8LE-NEXT:    add r5, r5, r9
+; P8LE-NEXT:    srwi r9, r6, 31
+; P8LE-NEXT:    srawi r6, r6, 4
+; P8LE-NEXT:    add r6, r6, r9
 ; P8LE-NEXT:    srwi r9, r8, 31
 ; P8LE-NEXT:    srawi r8, r8, 9
-; P8LE-NEXT:    mulli r3, r3, 5423
+; P8LE-NEXT:    mulli r5, r5, 5423
 ; P8LE-NEXT:    add r8, r8, r9
-; P8LE-NEXT:    mulli r7, r7, 23
+; P8LE-NEXT:    mulli r6, r6, 23
+; P8LE-NEXT:    li r9, 0
 ; P8LE-NEXT:    mulli r8, r8, 654
-; P8LE-NEXT:    sub r3, r5, r3
-; P8LE-NEXT:    mtfprd f0, r3
-; P8LE-NEXT:    sub r3, r6, r7
-; P8LE-NEXT:    sub r4, r4, r8
-; P8LE-NEXT:    mtfprd f1, r3
-; P8LE-NEXT:    mtfprd f2, r4
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v2, v2, v3
-; P8LE-NEXT:    vmrglh v3, v4, v5
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    mtvsrd v2, r9
+; P8LE-NEXT:    sub r4, r4, r5
+; P8LE-NEXT:    sub r5, r7, r6
+; P8LE-NEXT:    mtvsrd v3, r4
+; P8LE-NEXT:    sub r3, r3, r8
+; P8LE-NEXT:    mtvsrd v4, r5
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v3, v3, v4
+; P8LE-NEXT:    vmrghh v2, v5, v2
+; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: dont_fold_srem_one:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r3, v2
 ; P8BE-NEXT:    lis r5, 24749
 ; P8BE-NEXT:    lis r6, -19946
 ; P8BE-NEXT:    lis r8, -14230

 }
 
 ; Don't fold if the divisor is 2^15.
 define <4 x i16> @dont_fold_urem_i16_smax(<4 x i16> %x) {
 ; P9LE-LABEL: dont_fold_urem_i16_smax:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -19946
-; P9LE-NEXT:    ori r5, r5, 17097
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    mulhw r5, r4, r5
-; P9LE-NEXT:    add r4, r5, r4
+; P9LE-NEXT:    lis r4, -19946
+; P9LE-NEXT:    ori r4, r4, 17097
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    mulhw r4, r3, r4
+; P9LE-NEXT:    add r4, r4, r3
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 4
 ; P9LE-NEXT:    add r4, r4, r5
-; P9LE-NEXT:    lis r5, 24749
 ; P9LE-NEXT:    mulli r4, r4, 23
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 24749
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    ori r5, r5, 47143
-; P9LE-NEXT:    mulhw r4, r4, r5
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    ori r4, r4, 47143
+; P9LE-NEXT:    mulhw r4, r3, r4
 ; P9LE-NEXT:    srwi r5, r4, 31
 ; P9LE-NEXT:    srawi r4, r4, 11
 ; P9LE-NEXT:    add r4, r4, r5
 ; P9LE-NEXT:    mulli r4, r4, 5423
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    extsh r4, r3
-; P9LE-NEXT:    srawi r4, r4, 15
+; P9LE-NEXT:    extsh r3, r3
+; P9LE-NEXT:    srawi r4, r3, 15
 ; P9LE-NEXT:    addze r4, r4
 ; P9LE-NEXT:    slwi r4, r4, 15
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxlxor v4, v4, v4
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    li r3, 0
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v3, v2
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: dont_fold_urem_i16_smax:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 4
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, -19946

 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: dont_fold_urem_i16_smax:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r4, 24749
 ; P8LE-NEXT:    lis r5, -19946
-; P8LE-NEXT:    xxlxor v5, v5, v5
 ; P8LE-NEXT:    ori r4, r4, 47143
 ; P8LE-NEXT:    ori r5, r5, 17097
 ; P8LE-NEXT:    mffprd r3, f0
 ; P8LE-NEXT:    rldicl r6, r3, 16, 48
 ; P8LE-NEXT:    rldicl r7, r3, 32, 48
-; P8LE-NEXT:    extsh r8, r6
-; P8LE-NEXT:    extsh r9, r7
-; P8LE-NEXT:    mulhw r4, r8, r4
-; P8LE-NEXT:    mulhw r5, r9, r5
+; P8LE-NEXT:    extsh r6, r6
+; P8LE-NEXT:    extsh r7, r7
+; P8LE-NEXT:    mulhw r4, r6, r4
+; P8LE-NEXT:    mulhw r5, r7, r5
 ; P8LE-NEXT:    rldicl r3, r3, 48, 48
+; P8LE-NEXT:    extsh r3, r3
 ; P8LE-NEXT:    srwi r8, r4, 31
 ; P8LE-NEXT:    srawi r4, r4, 11
-; P8LE-NEXT:    add r5, r5, r9
+; P8LE-NEXT:    add r5, r5, r7
 ; P8LE-NEXT:    add r4, r4, r8
 ; P8LE-NEXT:    srwi r8, r5, 31
 ; P8LE-NEXT:    srawi r5, r5, 4
 ; P8LE-NEXT:    mulli r4, r4, 5423
 ; P8LE-NEXT:    add r5, r5, r8
-; P8LE-NEXT:    extsh r8, r3
+; P8LE-NEXT:    srawi r9, r3, 15
+; P8LE-NEXT:    li r8, 0
 ; P8LE-NEXT:    mulli r5, r5, 23
-; P8LE-NEXT:    srawi r8, r8, 15
+; P8LE-NEXT:    mtvsrd v2, r8
 ; P8LE-NEXT:    sub r4, r6, r4
-; P8LE-NEXT:    addze r6, r8
-; P8LE-NEXT:    mtfprd f0, r4
-; P8LE-NEXT:    slwi r4, r6, 15
+; P8LE-NEXT:    addze r6, r9
+; P8LE-NEXT:    slwi r6, r6, 15
+; P8LE-NEXT:    mtvsrd v3, r4
 ; P8LE-NEXT:    sub r5, r7, r5
-; P8LE-NEXT:    sub r3, r3, r4
-; P8LE-NEXT:    mtfprd f1, r5
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v2, v2, v3
-; P8LE-NEXT:    vmrglh v3, v4, v5
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    sub r3, r3, r6
+; P8LE-NEXT:    mtvsrd v4, r5
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v3, v3, v4
+; P8LE-NEXT:    vmrghh v2, v5, v2
+; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: dont_fold_urem_i16_smax:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r3, v2
 ; P8BE-NEXT:    lis r4, 24749
 ; P8BE-NEXT:    lis r5, -19946
 ; P8BE-NEXT:    ori r4, r4, 47143

llvm/test/CodeGen/PowerPC/swaps-le-5.ll

 entry:
   %0 = load <2 x double>, <2 x double>* @x, align 16
   %vecins = insertelement <2 x double> %0, double %y, i32 0
   store <2 x double> %vecins, <2 x double>* @z, align 16
   ret void
 }
 
 ; CHECK-LABEL: @bar0
+; CHECK-DAG: xxswapd 1, 1
 ; CHECK-DAG: lxvd2x [[REG1:[0-9]+]]
-; CHECK-DAG: xxspltd [[REG2:[0-9]+]]
-; CHECK: xxpermdi [[REG3:[0-9]+]], [[REG2]], [[REG1]], 1
-; CHECK: stxvd2x [[REG3]]
+; CHECK: xxmrgld [[REG2:[0-9]+]], 1, [[REG1]]
+; CHECK: stxvd2x [[REG2]]
 ; CHECK-NOT: xxswapd
 
 define void @bar1(double %y) {
 entry:
   %0 = load <2 x double>, <2 x double>* @x, align 16
   %vecins = insertelement <2 x double> %0, double %y, i32 1
   store <2 x double> %vecins, <2 x double>* @z, align 16
   ret void
 }
 
 ; CHECK-LABEL: @bar1
+; CHECK-DAG: xxswapd 1, 1
 ; CHECK-DAG: lxvd2x [[REG1:[0-9]+]]
-; CHECK-DAG: xxspltd [[REG2:[0-9]+]]
-; CHECK: xxmrghd [[REG3:[0-9]+]], [[REG1]], [[REG2]]
-; CHECK: stxvd2x [[REG3]]
+; CHECK: xxpermdi [[REG2:[0-9]+]], [[REG1]], 1, 1
+; CHECK: stxvd2x [[REG2]]
 ; CHECK-NOT: xxswapd
 
 define void @baz0() {
 entry:
   %0 = load <2 x double>, <2 x double>* @z, align 16
   %1 = load <2 x double>, <2 x double>* @x, align 16
   %vecins = shufflevector <2 x double> %0, <2 x double> %1, <2 x i32> <i32 0, i32 2>
   store <2 x double> %vecins, <2 x double>* @z, align 16

llvm/test/CodeGen/PowerPC/swaps-le-6.ll

 define void @bar0() {
 ; CHECK-LABEL: bar0:
 ; CHECK:   # %bb.0: # %entry
 ; CHECK:     addis r3, r2, .LC0@toc@ha
 ; CHECK:     addis r4, r2, .LC1@toc@ha
 ; CHECK:     ld r3, .LC0@toc@l(r3)
 ; CHECK:     addis r3, r2, .LC2@toc@ha
 ; CHECK:     ld r3, .LC2@toc@l(r3)
-; CHECK:     xxpermdi vs0, vs0, vs1, 1
+; CHECK:     xxmrgld vs0, vs0, vs1
 ; CHECK:     stxvd2x vs0, 0, r3
 ; CHECK:     blr
 ;
 ; CHECK-P9-NOVECTOR-LABEL: bar0:
 ; CHECK-P9-NOVECTOR:   # %bb.0: # %entry
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC0@toc@ha
 ; CHECK-P9-NOVECTOR:     ld r3, .LC0@toc@l(r3)
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC1@toc@ha
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC2@toc@ha
 ; CHECK-P9-NOVECTOR:     ld r3, .LC2@toc@l(r3)
-; CHECK-P9-NOVECTOR:     xxpermdi vs0, vs1, vs0, 1
+; CHECK-P9-NOVECTOR:     xxmrgld vs0, vs1, vs0
 ; CHECK-P9-NOVECTOR:     stxvd2x vs0, 0, r3
 ; CHECK-P9-NOVECTOR:     blr
 ;
 ; CHECK-P9-LABEL: bar0:
 ; CHECK-P9:   # %bb.0: # %entry
 ; CHECK-P9:     addis r3, r2, .LC0@toc@ha
 ; CHECK-P9:     ld r3, .LC0@toc@l(r3)
 ; CHECK-P9:     lxvx vs0, 0, r3

 define void @bar1() {
 ; CHECK-LABEL: bar1:
 ; CHECK:   # %bb.0: # %entry
 ; CHECK:     addis r3, r2, .LC0@toc@ha
 ; CHECK:     addis r4, r2, .LC1@toc@ha
 ; CHECK:     ld r3, .LC0@toc@l(r3)
 ; CHECK:     addis r3, r2, .LC2@toc@ha
 ; CHECK:     ld r3, .LC2@toc@l(r3)
-; CHECK:     xxmrghd vs0, vs1, vs0
+; CHECK:     xxpermdi vs0, vs1, vs0, 1
 ; CHECK:     stxvd2x vs0, 0, r3
 ; CHECK:     blr
 ;
 ; CHECK-P9-NOVECTOR-LABEL: bar1:
 ; CHECK-P9-NOVECTOR:   # %bb.0: # %entry
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC0@toc@ha
 ; CHECK-P9-NOVECTOR:     ld r3, .LC0@toc@l(r3)
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC1@toc@ha
 ; CHECK-P9-NOVECTOR:     addis r3, r2, .LC2@toc@ha
 ; CHECK-P9-NOVECTOR:     ld r3, .LC2@toc@l(r3)
-; CHECK-P9-NOVECTOR:     xxmrghd vs0, vs0, vs1
+; CHECK-P9-NOVECTOR:     xxpermdi vs0, vs0, vs1, 1
 ; CHECK-P9-NOVECTOR:     stxvd2x vs0, 0, r3
 ; CHECK-P9-NOVECTOR:     blr
 ;
 ; CHECK-P9-LABEL: bar1:
 ; CHECK-P9:   # %bb.0: # %entry
 ; CHECK-P9:     addis r3, r2, .LC0@toc@ha
 ; CHECK-P9:     ld r3, .LC0@toc@l(r3)
 ; CHECK-P9:     lxvx vs0, 0, r3

llvm/test/CodeGen/PowerPC/urem-vector-lkk.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P9LE
 ; RUN: llc -mcpu=pwr9 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P9BE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P8LE
 ; RUN: llc -mcpu=pwr8 -verify-machineinstrs -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names \
 ; RUN:    -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s --check-prefixes=CHECK,P8BE
 
 define <4 x i16> @fold_urem_vec_1(<4 x i16> %x) {
 ; P9LE-LABEL: fold_urem_vec_1:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, 21399
-; P9LE-NEXT:    ori r5, r5, 33437
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    lis r5, 16727
-; P9LE-NEXT:    ori r5, r5, 2287
+; P9LE-NEXT:    lis r4, 21399
+; P9LE-NEXT:    ori r4, r4, 33437
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r4, r3, r4
 ; P9LE-NEXT:    srwi r4, r4, 5
 ; P9LE-NEXT:    mulli r4, r4, 98
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 16727
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    lis r5, 8456
-; P9LE-NEXT:    ori r5, r5, 16913
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    ori r4, r4, 2287
+; P9LE-NEXT:    mulhwu r4, r3, r4
 ; P9LE-NEXT:    srwi r4, r4, 8
 ; P9LE-NEXT:    mulli r4, r4, 1003
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    rlwinm r4, r3, 30, 18, 31
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    lis r5, 22765
-; P9LE-NEXT:    ori r5, r5, 8969
-; P9LE-NEXT:    srwi r4, r4, 2
-; P9LE-NEXT:    mulli r4, r4, 124
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r5, 8456
+; P9LE-NEXT:    ori r5, r5, 16913
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    clrlwi r4, r3, 16
+; P9LE-NEXT:    rlwinm r3, r3, 30, 18, 31
+; P9LE-NEXT:    mulhwu r3, r3, r5
+; P9LE-NEXT:    srwi r3, r3, 2
+; P9LE-NEXT:    mulli r3, r3, 124
+; P9LE-NEXT:    sub r3, r4, r3
+; P9LE-NEXT:    lis r4, 22765
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r5, r4, r5
-; P9LE-NEXT:    sub r4, r4, r5
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r5
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    ori r4, r4, 8969
+; P9LE-NEXT:    mulhwu r4, r3, r4
+; P9LE-NEXT:    sub r5, r3, r4
+; P9LE-NEXT:    srwi r5, r5, 1
+; P9LE-NEXT:    add r4, r5, r4
 ; P9LE-NEXT:    srwi r4, r4, 6
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v4, v2
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v4, v2
 ; P9LE-NEXT:    vmrglw v2, v3, v2
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: fold_urem_vec_1:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, 16727

 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fold_urem_vec_1:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, 22765
 ; P8LE-NEXT:    lis r7, 21399
-; P8LE-NEXT:    lis r10, 16727
+; P8LE-NEXT:    lis r9, 16727
+; P8LE-NEXT:    lis r10, 8456
 ; P8LE-NEXT:    ori r3, r3, 8969
 ; P8LE-NEXT:    ori r7, r7, 33437
-; P8LE-NEXT:    ori r10, r10, 2287
+; P8LE-NEXT:    ori r9, r9, 2287
+; P8LE-NEXT:    ori r10, r10, 16913
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    clrldi r6, r4, 48
 ; P8LE-NEXT:    rldicl r5, r4, 32, 48
-; P8LE-NEXT:    clrlwi r9, r6, 16
+; P8LE-NEXT:    clrlwi r6, r6, 16
 ; P8LE-NEXT:    rldicl r8, r4, 16, 48
-; P8LE-NEXT:    clrlwi r11, r5, 16
-; P8LE-NEXT:    mulhwu r3, r9, r3
-; P8LE-NEXT:    clrlwi r12, r8, 16
-; P8LE-NEXT:    mulhwu r7, r11, r7
-; P8LE-NEXT:    lis r11, 8456
+; P8LE-NEXT:    clrlwi r5, r5, 16
+; P8LE-NEXT:    mulhwu r3, r6, r3
 ; P8LE-NEXT:    rldicl r4, r4, 48, 48
-; P8LE-NEXT:    mulhwu r10, r12, r10
-; P8LE-NEXT:    ori r11, r11, 16913
-; P8LE-NEXT:    rlwinm r12, r4, 30, 18, 31
-; P8LE-NEXT:    mulhwu r11, r12, r11
-; P8LE-NEXT:    sub r9, r9, r3
-; P8LE-NEXT:    srwi r9, r9, 1
+; P8LE-NEXT:    clrlwi r8, r8, 16
+; P8LE-NEXT:    rlwinm r11, r4, 30, 18, 31
+; P8LE-NEXT:    mulhwu r7, r5, r7
+; P8LE-NEXT:    clrlwi r4, r4, 16
+; P8LE-NEXT:    mulhwu r9, r8, r9
+; P8LE-NEXT:    mulhwu r10, r11, r10
+; P8LE-NEXT:    sub r11, r6, r3
+; P8LE-NEXT:    srwi r11, r11, 1
 ; P8LE-NEXT:    srwi r7, r7, 5
-; P8LE-NEXT:    add r3, r9, r3
-; P8LE-NEXT:    srwi r9, r10, 8
+; P8LE-NEXT:    add r3, r11, r3
+; P8LE-NEXT:    srwi r9, r9, 8
+; P8LE-NEXT:    srwi r10, r10, 2
 ; P8LE-NEXT:    srwi r3, r3, 6
 ; P8LE-NEXT:    mulli r7, r7, 98
-; P8LE-NEXT:    srwi r10, r11, 2
 ; P8LE-NEXT:    mulli r9, r9, 1003
 ; P8LE-NEXT:    mulli r3, r3, 95
 ; P8LE-NEXT:    mulli r10, r10, 124
 ; P8LE-NEXT:    sub r5, r5, r7
 ; P8LE-NEXT:    sub r7, r8, r9
-; P8LE-NEXT:    mtfprd f0, r5
 ; P8LE-NEXT:    sub r3, r6, r3
+; P8LE-NEXT:    mtvsrd v2, r5
 ; P8LE-NEXT:    sub r4, r4, r10
-; P8LE-NEXT:    mtfprd f1, r7
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    vmrglh v3, v5, v4
+; P8LE-NEXT:    mtvsrd v3, r7
+; P8LE-NEXT:    mtvsrd v4, r3
+; P8LE-NEXT:    mtvsrd v5, r4
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    vmrghh v3, v5, v4
 ; P8LE-NEXT:    vmrglw v2, v2, v3
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: fold_urem_vec_1:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, 22765
 ; P8BE-NEXT:    lis r7, 16727
; P8BE-NEXT:    blr
   ret <4 x i16> %1
 }
 
 define <4 x i16> @fold_urem_vec_2(<4 x i16> %x) {
 ; P9LE-LABEL: fold_urem_vec_2:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, 22765
-; P9LE-NEXT:    ori r5, r5, 8969
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r6, r4, r5
-; P9LE-NEXT:    sub r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 22765
+; P9LE-NEXT:    ori r4, r4, 8969
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r5, r3, r4
+; P9LE-NEXT:    sub r6, r3, r5
+; P9LE-NEXT:    srwi r6, r6, 1
+; P9LE-NEXT:    add r5, r6, r5
+; P9LE-NEXT:    srwi r5, r5, 6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r6, r4, r5
-; P9LE-NEXT:    sub r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r5, r3, r4
+; P9LE-NEXT:    sub r6, r3, r5
+; P9LE-NEXT:    srwi r6, r6, 1
+; P9LE-NEXT:    add r5, r6, r5
+; P9LE-NEXT:    srwi r5, r5, 6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r6, r4, r5
-; P9LE-NEXT:    sub r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 6
-; P9LE-NEXT:    mulli r4, r4, 95
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r5, r3, r4
+; P9LE-NEXT:    sub r6, r3, r5
+; P9LE-NEXT:    srwi r6, r6, 1
+; P9LE-NEXT:    add r5, r6, r5
+; P9LE-NEXT:    srwi r5, r5, 6
+; P9LE-NEXT:    mulli r5, r5, 95
+; P9LE-NEXT:    sub r3, r3, r5
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r5, r4, r5
-; P9LE-NEXT:    sub r4, r4, r5
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r5
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r4, r3, r4
+; P9LE-NEXT:    sub r5, r3, r4
+; P9LE-NEXT:    srwi r5, r5, 1
+; P9LE-NEXT:    add r4, r5, r4
 ; P9LE-NEXT:    srwi r4, r4, 6
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: fold_urem_vec_2:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, 22765

 ; P9BE-NEXT:    vmrghh v2, v2, v4
 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: fold_urem_vec_2:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, 22765
-; P8LE-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
 ; P8LE-NEXT:    ori r3, r3, 8969
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    clrldi r5, r4, 48
 ; P8LE-NEXT:    rldicl r6, r4, 48, 48
-; P8LE-NEXT:    clrlwi r8, r5, 16
+; P8LE-NEXT:    clrlwi r5, r5, 16
 ; P8LE-NEXT:    rldicl r7, r4, 32, 48
-; P8LE-NEXT:    clrlwi r9, r6, 16
+; P8LE-NEXT:    clrlwi r6, r6, 16
+; P8LE-NEXT:    mulhwu r8, r5, r3
 ; P8LE-NEXT:    rldicl r4, r4, 16, 48
-; P8LE-NEXT:    mulhwu r10, r8, r3
-; P8LE-NEXT:    clrlwi r11, r7, 16
-; P8LE-NEXT:    clrlwi r0, r4, 16
-; P8LE-NEXT:    mulhwu r12, r9, r3
-; P8LE-NEXT:    mulhwu r30, r11, r3
-; P8LE-NEXT:    mulhwu r3, r0, r3
-; P8LE-NEXT:    sub r8, r8, r10
-; P8LE-NEXT:    srwi r8, r8, 1
-; P8LE-NEXT:    sub r9, r9, r12
-; P8LE-NEXT:    add r8, r8, r10
-; P8LE-NEXT:    sub r10, r11, r30
-; P8LE-NEXT:    sub r11, r0, r3
-; P8LE-NEXT:    srwi r9, r9, 1
-; P8LE-NEXT:    srwi r10, r10, 1
+; P8LE-NEXT:    clrlwi r7, r7, 16
+; P8LE-NEXT:    mulhwu r9, r6, r3
+; P8LE-NEXT:    clrlwi r4, r4, 16
+; P8LE-NEXT:    mulhwu r10, r7, r3
+; P8LE-NEXT:    mulhwu r3, r4, r3
+; P8LE-NEXT:    sub r11, r5, r8
+; P8LE-NEXT:    sub r12, r6, r9
+; P8LE-NEXT:    srwi r11, r11, 1
+; P8LE-NEXT:    add r8, r11, r8
+; P8LE-NEXT:    sub r11, r7, r10
+; P8LE-NEXT:    srwi r12, r12, 1
+; P8LE-NEXT:    add r9, r12, r9
+; P8LE-NEXT:    sub r12, r4, r3
 ; P8LE-NEXT:    srwi r11, r11, 1
-; P8LE-NEXT:    add r9, r9, r12
 ; P8LE-NEXT:    srwi r8, r8, 6
-; P8LE-NEXT:    add r10, r10, r30
-; P8LE-NEXT:    add r3, r11, r3
+; P8LE-NEXT:    add r10, r11, r10
+; P8LE-NEXT:    srwi r11, r12, 1
 ; P8LE-NEXT:    srwi r9, r9, 6
-; P8LE-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
+; P8LE-NEXT:    add r3, r11, r3
 ; P8LE-NEXT:    mulli r8, r8, 95
 ; P8LE-NEXT:    srwi r10, r10, 6
 ; P8LE-NEXT:    srwi r3, r3, 6
 ; P8LE-NEXT:    mulli r9, r9, 95
 ; P8LE-NEXT:    mulli r10, r10, 95
 ; P8LE-NEXT:    mulli r3, r3, 95
 ; P8LE-NEXT:    sub r5, r5, r8
 ; P8LE-NEXT:    sub r6, r6, r9
-; P8LE-NEXT:    mtfprd f0, r5
+; P8LE-NEXT:    mtvsrd v2, r5
 ; P8LE-NEXT:    sub r5, r7, r10
 ; P8LE-NEXT:    sub r3, r4, r3
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    mtfprd f2, r5
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f3, r3
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    vmrglh v3, v5, v4
+; P8LE-NEXT:    mtvsrd v3, r6
+; P8LE-NEXT:    mtvsrd v4, r5
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    vmrghh v3, v5, v4
 ; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: fold_urem_vec_2:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, 22765
 ; P8BE-NEXT:    ori r3, r3, 8969

 
 
 ; Don't fold if we can combine urem with udiv.
 define <4 x i16> @combine_urem_udiv(<4 x i16> %x) {
 ; P9LE-LABEL: combine_urem_udiv:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, 22765
-; P9LE-NEXT:    ori r5, r5, 8969
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r6, r4, r5
-; P9LE-NEXT:    sub r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r6
-; P9LE-NEXT:    srwi r4, r4, 6
-; P9LE-NEXT:    mulli r6, r4, 95
+; P9LE-NEXT:    lis r4, 22765
+; P9LE-NEXT:    ori r4, r4, 8969
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r5, r3, r4
+; P9LE-NEXT:    sub r6, r3, r5
+; P9LE-NEXT:    srwi r6, r6, 1
+; P9LE-NEXT:    add r5, r6, r5
+; P9LE-NEXT:    srwi r5, r5, 6
+; P9LE-NEXT:    mulli r6, r5, 95
 ; P9LE-NEXT:    sub r3, r3, r6
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r6, r3, 16
-; P9LE-NEXT:    mulhwu r7, r6, r5
+; P9LE-NEXT:    mulhwu r7, r6, r4
 ; P9LE-NEXT:    sub r6, r6, r7
 ; P9LE-NEXT:    srwi r6, r6, 1
 ; P9LE-NEXT:    add r6, r6, r7
 ; P9LE-NEXT:    srwi r6, r6, 6
 ; P9LE-NEXT:    mulli r7, r6, 95
 ; P9LE-NEXT:    sub r3, r3, r7
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r7, r3, 16
-; P9LE-NEXT:    mulhwu r8, r7, r5
+; P9LE-NEXT:    mulhwu r8, r7, r4
 ; P9LE-NEXT:    sub r7, r7, r8
 ; P9LE-NEXT:    srwi r7, r7, 1
 ; P9LE-NEXT:    add r7, r7, r8
 ; P9LE-NEXT:    srwi r7, r7, 6
 ; P9LE-NEXT:    mulli r8, r7, 95
 ; P9LE-NEXT:    sub r3, r3, r8
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r8, r3, 16
-; P9LE-NEXT:    mulhwu r5, r8, r5
-; P9LE-NEXT:    sub r8, r8, r5
+; P9LE-NEXT:    mulhwu r4, r8, r4
+; P9LE-NEXT:    sub r8, r8, r4
 ; P9LE-NEXT:    srwi r8, r8, 1
-; P9LE-NEXT:    add r5, r8, r5
-; P9LE-NEXT:    srwi r5, r5, 6
-; P9LE-NEXT:    mulli r8, r5, 95
+; P9LE-NEXT:    add r4, r8, r4
+; P9LE-NEXT:    srwi r4, r4, 6
+; P9LE-NEXT:    mulli r8, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r8
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    mtfprd f0, r4
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
+; P9LE-NEXT:    mtvsrd v4, r6
 ; P9LE-NEXT:    vmrglw v2, v2, v3
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r6
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r7
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r5
-; P9LE-NEXT:    xxswapd v5, vs0
-; P9LE-NEXT:    vmrglh v4, v5, v4
+; P9LE-NEXT:    mtvsrd v3, r5
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    mtvsrd v4, r7
+; P9LE-NEXT:    mtvsrd v5, r4
+; P9LE-NEXT:    vmrghh v4, v5, v4
 ; P9LE-NEXT:    vmrglw v3, v4, v3
 ; P9LE-NEXT:    vadduhm v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: combine_urem_udiv:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2

 ; P9BE-NEXT:    vmrghh v4, v5, v4
 ; P9BE-NEXT:    vmrghw v3, v4, v3
 ; P9BE-NEXT:    vadduhm v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: combine_urem_udiv:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
-; P8LE-NEXT:    lis r4, 22765
+; P8LE-NEXT:    lis r3, 22765
 ; P8LE-NEXT:    std r30, -16(r1) # 8-byte Folded Spill
-; P8LE-NEXT:    ori r4, r4, 8969
-; P8LE-NEXT:    mffprd r5, f0
-; P8LE-NEXT:    clrldi r3, r5, 48
-; P8LE-NEXT:    rldicl r6, r5, 48, 48
-; P8LE-NEXT:    clrlwi r8, r3, 16
-; P8LE-NEXT:    rldicl r7, r5, 32, 48
-; P8LE-NEXT:    clrlwi r9, r6, 16
-; P8LE-NEXT:    mulhwu r10, r8, r4
-; P8LE-NEXT:    clrlwi r11, r7, 16
-; P8LE-NEXT:    rldicl r5, r5, 16, 48
-; P8LE-NEXT:    mulhwu r12, r9, r4
-; P8LE-NEXT:    mulhwu r0, r11, r4
-; P8LE-NEXT:    clrlwi r30, r5, 16
-; P8LE-NEXT:    mulhwu r4, r30, r4
-; P8LE-NEXT:    sub r8, r8, r10
+; P8LE-NEXT:    ori r3, r3, 8969
+; P8LE-NEXT:    mffprd r4, f0
+; P8LE-NEXT:    clrldi r5, r4, 48
+; P8LE-NEXT:    rldicl r6, r4, 48, 48
+; P8LE-NEXT:    clrlwi r5, r5, 16
+; P8LE-NEXT:    clrlwi r8, r6, 16
+; P8LE-NEXT:    rldicl r7, r4, 32, 48
+; P8LE-NEXT:    rldicl r4, r4, 16, 48
+; P8LE-NEXT:    mulhwu r9, r5, r3
+; P8LE-NEXT:    mulhwu r11, r8, r3
+; P8LE-NEXT:    clrlwi r10, r7, 16
+; P8LE-NEXT:    clrlwi r12, r4, 16
+; P8LE-NEXT:    mulhwu r0, r10, r3
+; P8LE-NEXT:    mulhwu r3, r12, r3
+; P8LE-NEXT:    sub r30, r5, r9
+; P8LE-NEXT:    sub r8, r8, r11
+; P8LE-NEXT:    srwi r30, r30, 1
 ; P8LE-NEXT:    srwi r8, r8, 1
-; P8LE-NEXT:    sub r9, r9, r12
-; P8LE-NEXT:    add r8, r8, r10
-; P8LE-NEXT:    sub r10, r11, r0
-; P8LE-NEXT:    srwi r9, r9, 1
+; P8LE-NEXT:    sub r10, r10, r0
+; P8LE-NEXT:    add r9, r30, r9
+; P8LE-NEXT:    add r8, r8, r11
+; P8LE-NEXT:    sub r11, r12, r3
 ; P8LE-NEXT:    srwi r10, r10, 1
-; P8LE-NEXT:    sub r11, r30, r4
-; P8LE-NEXT:    add r9, r9, r12
-; P8LE-NEXT:    srwi r8, r8, 6
 ; P8LE-NEXT:    ld r30, -16(r1) # 8-byte Folded Reload
-; P8LE-NEXT:    add r10, r10, r0
-; P8LE-NEXT:    srwi r11, r11, 1
 ; P8LE-NEXT:    srwi r9, r9, 6
-; P8LE-NEXT:    mtfprd f0, r8
-; P8LE-NEXT:    mulli r12, r8, 95
+; P8LE-NEXT:    srwi r11, r11, 1
+; P8LE-NEXT:    srwi r8, r8, 6
+; P8LE-NEXT:    add r10, r10, r0
+; P8LE-NEXT:    mulli r12, r9, 95
+; P8LE-NEXT:    add r3, r11, r3
+; P8LE-NEXT:    mtvsrd v2, r9
 ; P8LE-NEXT:    srwi r10, r10, 6
-; P8LE-NEXT:    add r4, r11, r4
-; P8LE-NEXT:    mtfprd f1, r9
-; P8LE-NEXT:    mulli r8, r9, 95
-; P8LE-NEXT:    mulli r9, r10, 95
-; P8LE-NEXT:    srwi r4, r4, 6
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    mtfprd f2, r10
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    mulli r4, r4, 95
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v1, vs2
-; P8LE-NEXT:    sub r3, r3, r12
-; P8LE-NEXT:    xxswapd v6, vs3
-; P8LE-NEXT:    mtfprd f0, r3
-; P8LE-NEXT:    sub r3, r7, r9
-; P8LE-NEXT:    sub r6, r6, r8
-; P8LE-NEXT:    mtfprd f4, r3
-; P8LE-NEXT:    sub r3, r5, r4
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    mtfprd f5, r3
-; P8LE-NEXT:    xxswapd v5, vs4
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    xxswapd v3, vs0
-; P8LE-NEXT:    xxswapd v4, vs1
-; P8LE-NEXT:    xxswapd v0, vs5
-; P8LE-NEXT:    vmrglh v3, v4, v3
-; P8LE-NEXT:    vmrglh v4, v0, v5
-; P8LE-NEXT:    vmrglh v5, v6, v1
-; P8LE-NEXT:    vmrglw v3, v4, v3
-; P8LE-NEXT:    vmrglw v2, v5, v2
+; P8LE-NEXT:    mulli r9, r8, 95
+; P8LE-NEXT:    srwi r3, r3, 6
+; P8LE-NEXT:    mtvsrd v3, r8
+; P8LE-NEXT:    mulli r8, r10, 95
+; P8LE-NEXT:    mtvsrd v4, r10
+; P8LE-NEXT:    mulli r10, r3, 95
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    sub r5, r5, r12
+; P8LE-NEXT:    sub r6, r6, r9
+; P8LE-NEXT:    mtvsrd v3, r5
+; P8LE-NEXT:    mtvsrd v5, r6
+; P8LE-NEXT:    sub r5, r7, r8
+; P8LE-NEXT:    sub r4, r4, r10
+; P8LE-NEXT:    mtvsrd v0, r5
+; P8LE-NEXT:    mtvsrd v1, r4
+; P8LE-NEXT:    vmrghh v3, v5, v3
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v0, v1, v0
+; P8LE-NEXT:    vmrghh v4, v5, v4
+; P8LE-NEXT:    vmrglw v3, v0, v3
+; P8LE-NEXT:    vmrglw v2, v4, v2
 ; P8LE-NEXT:    vadduhm v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: combine_urem_udiv:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r5, v2
 ; P8BE-NEXT:    lis r4, 22765
 ; P8BE-NEXT:    ori r4, r4, 8969

 
 ; Don't fold for divisors that are a power of two.
 define <4 x i16> @dont_fold_urem_power_of_two(<4 x i16> %x) {
 ; P9LE-LABEL: dont_fold_urem_power_of_two:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 0
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r3, r3, 26
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r3, r3, 27
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, 22765
-; P9LE-NEXT:    ori r5, r5, 8969
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r5, r4, r5
-; P9LE-NEXT:    sub r4, r4, r5
-; P9LE-NEXT:    srwi r4, r4, 1
-; P9LE-NEXT:    add r4, r4, r5
+; P9LE-NEXT:    lis r4, 22765
+; P9LE-NEXT:    ori r4, r4, 8969
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r4, r3, r4
+; P9LE-NEXT:    sub r5, r3, r4
+; P9LE-NEXT:    srwi r5, r5, 1
+; P9LE-NEXT:    add r4, r5, r4
 ; P9LE-NEXT:    srwi r4, r4, 6
 ; P9LE-NEXT:    mulli r4, r4, 95
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
 ; P9LE-NEXT:    clrlwi r3, r3, 29
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v2, v4, v2
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    vmrghh v2, v4, v2
 ; P9LE-NEXT:    vmrglw v2, v2, v3
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: dont_fold_urem_power_of_two:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 2
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    clrlwi r3, r3, 27

 ; P8LE-LABEL: dont_fold_urem_power_of_two:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
 ; P8LE-NEXT:    lis r3, 22765
 ; P8LE-NEXT:    ori r3, r3, 8969
 ; P8LE-NEXT:    mffprd r4, f0
 ; P8LE-NEXT:    rldicl r5, r4, 16, 48
 ; P8LE-NEXT:    rldicl r7, r4, 48, 48
-; P8LE-NEXT:    clrlwi r6, r5, 16
-; P8LE-NEXT:    mulhwu r3, r6, r3
-; P8LE-NEXT:    sub r6, r6, r3
+; P8LE-NEXT:    clrlwi r5, r5, 16
+; P8LE-NEXT:    mulhwu r3, r5, r3
+; P8LE-NEXT:    sub r6, r5, r3
 ; P8LE-NEXT:    srwi r6, r6, 1
 ; P8LE-NEXT:    add r3, r6, r3
 ; P8LE-NEXT:    clrldi r6, r4, 48
 ; P8LE-NEXT:    srwi r3, r3, 6
 ; P8LE-NEXT:    clrlwi r6, r6, 26
 ; P8LE-NEXT:    mulli r3, r3, 95
 ; P8LE-NEXT:    rldicl r4, r4, 32, 48
-; P8LE-NEXT:    mtfprd f0, r6
+; P8LE-NEXT:    mtvsrd v2, r6
 ; P8LE-NEXT:    clrlwi r6, r7, 27
 ; P8LE-NEXT:    clrlwi r4, r4, 29
-; P8LE-NEXT:    mtfprd f1, r6
-; P8LE-NEXT:    mtfprd f3, r4
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    xxswapd v3, vs1
+; P8LE-NEXT:    mtvsrd v3, r6
+; P8LE-NEXT:    mtvsrd v5, r4
+; P8LE-NEXT:    vmrghh v2, v3, v2
 ; P8LE-NEXT:    sub r3, r5, r3
-; P8LE-NEXT:    xxswapd v5, vs3
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v3, v4, v5
+; P8LE-NEXT:    mtvsrd v4, r3
+; P8LE-NEXT:    vmrghh v3, v4, v5
 ; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: dont_fold_urem_power_of_two:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, 22765
 ; P8BE-NEXT:    ori r3, r3, 8969

 }
 
 ; Don't fold if the divisor is one.
 define <4 x i16> @dont_fold_urem_one(<4 x i16> %x) {
 ; P9LE-LABEL: dont_fold_urem_one:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    li r3, 4
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    lis r5, -19946
-; P9LE-NEXT:    ori r5, r5, 17097
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    lis r5, 24749
-; P9LE-NEXT:    ori r5, r5, 47143
+; P9LE-NEXT:    lis r4, -19946
+; P9LE-NEXT:    ori r4, r4, 17097
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    mulhwu r4, r3, r4
 ; P9LE-NEXT:    srwi r4, r4, 4
 ; P9LE-NEXT:    mulli r4, r4, 23
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    lis r4, 24749
+; P9LE-NEXT:    mtvsrd v3, r3
 ; P9LE-NEXT:    li r3, 6
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    clrlwi r4, r3, 16
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    lis r5, -14230
-; P9LE-NEXT:    ori r5, r5, 30865
+; P9LE-NEXT:    clrlwi r3, r3, 16
+; P9LE-NEXT:    ori r4, r4, 47143
+; P9LE-NEXT:    mulhwu r4, r3, r4
 ; P9LE-NEXT:    srwi r4, r4, 11
 ; P9LE-NEXT:    mulli r4, r4, 5423
 ; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v3, vs0
-; P9LE-NEXT:    mtfprd f0, r3
+; P9LE-NEXT:    mtvsrd v4, r3
 ; P9LE-NEXT:    li r3, 2
 ; P9LE-NEXT:    vextuhrx r3, r3, v2
-; P9LE-NEXT:    rlwinm r4, r3, 31, 17, 31
-; P9LE-NEXT:    mulhwu r4, r4, r5
-; P9LE-NEXT:    srwi r4, r4, 8
-; P9LE-NEXT:    mulli r4, r4, 654
-; P9LE-NEXT:    sub r3, r3, r4
-; P9LE-NEXT:    xxswapd v4, vs0
-; P9LE-NEXT:    mtfprd f0, r3
-; P9LE-NEXT:    xxswapd v2, vs0
-; P9LE-NEXT:    vmrglh v3, v4, v3
-; P9LE-NEXT:    xxlxor v4, v4, v4
-; P9LE-NEXT:    vmrglh v2, v2, v4
+; P9LE-NEXT:    lis r5, -14230
+; P9LE-NEXT:    ori r5, r5, 30865
+; P9LE-NEXT:    vmrghh v3, v4, v3
+; P9LE-NEXT:    clrlwi r4, r3, 16
+; P9LE-NEXT:    rlwinm r3, r3, 31, 17, 31
+; P9LE-NEXT:    mulhwu r3, r3, r5
+; P9LE-NEXT:    srwi r3, r3, 8
+; P9LE-NEXT:    mulli r3, r3, 654
+; P9LE-NEXT:    sub r3, r4, r3
+; P9LE-NEXT:    mtvsrd v2, r3
+; P9LE-NEXT:    li r3, 0
+; P9LE-NEXT:    mtvsrd v4, r3
+; P9LE-NEXT:    vmrghh v2, v2, v4
 ; P9LE-NEXT:    vmrglw v2, v3, v2
 ; P9LE-NEXT:    blr
 ;
 ; P9BE-LABEL: dont_fold_urem_one:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    li r3, 6
 ; P9BE-NEXT:    vextuhlx r3, r3, v2
 ; P9BE-NEXT:    lis r4, 24749

 ; P9BE-NEXT:    mtvsrd v4, r3
 ; P9BE-NEXT:    vmrghh v2, v4, v2
 ; P9BE-NEXT:    vmrghw v2, v2, v3
 ; P9BE-NEXT:    blr
 ;
 ; P8LE-LABEL: dont_fold_urem_one:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    xxswapd vs0, v2
-; P8LE-NEXT:    lis r3, -19946
-; P8LE-NEXT:    lis r7, 24749
-; P8LE-NEXT:    lis r9, -14230
-; P8LE-NEXT:    xxlxor v5, v5, v5
-; P8LE-NEXT:    ori r3, r3, 17097
-; P8LE-NEXT:    ori r7, r7, 47143
-; P8LE-NEXT:    ori r9, r9, 30865
+; P8LE-NEXT:    lis r3, -14230
+; P8LE-NEXT:    lis r7, -19946
+; P8LE-NEXT:    lis r9, 24749
+; P8LE-NEXT:    ori r3, r3, 30865
+; P8LE-NEXT:    ori r7, r7, 17097
 ; P8LE-NEXT:    mffprd r4, f0
-; P8LE-NEXT:    rldicl r5, r4, 32, 48
-; P8LE-NEXT:    rldicl r6, r4, 16, 48
-; P8LE-NEXT:    clrlwi r8, r5, 16
-; P8LE-NEXT:    rldicl r4, r4, 48, 48
+; P8LE-NEXT:    rldicl r5, r4, 48, 48
+; P8LE-NEXT:    rldicl r6, r4, 32, 48
+; P8LE-NEXT:    rldicl r4, r4, 16, 48
+; P8LE-NEXT:    rlwinm r8, r5, 31, 17, 31
+; P8LE-NEXT:    clrlwi r6, r6, 16
+; P8LE-NEXT:    clrlwi r5, r5, 16
 ; P8LE-NEXT:    mulhwu r3, r8, r3
-; P8LE-NEXT:    clrlwi r8, r6, 16
-; P8LE-NEXT:    mulhwu r7, r8, r7
-; P8LE-NEXT:    rlwinm r8, r4, 31, 17, 31
-; P8LE-NEXT:    mulhwu r8, r8, r9
-; P8LE-NEXT:    srwi r3, r3, 4
-; P8LE-NEXT:    srwi r7, r7, 11
-; P8LE-NEXT:    mulli r3, r3, 23
-; P8LE-NEXT:    srwi r8, r8, 8
-; P8LE-NEXT:    mulli r7, r7, 5423
-; P8LE-NEXT:    mulli r8, r8, 654
+; P8LE-NEXT:    ori r8, r9, 47143
+; P8LE-NEXT:    clrlwi r4, r4, 16
+; P8LE-NEXT:    li r9, 0
+; P8LE-NEXT:    mulhwu r7, r6, r7
+; P8LE-NEXT:    mulhwu r8, r4, r8
+; P8LE-NEXT:    mtvsrd v2, r9
+; P8LE-NEXT:    srwi r3, r3, 8
+; P8LE-NEXT:    srwi r7, r7, 4
+; P8LE-NEXT:    mulli r3, r3, 654
+; P8LE-NEXT:    srwi r8, r8, 11
+; P8LE-NEXT:    mulli r7, r7, 23
+; P8LE-NEXT:    mulli r8, r8, 5423
 ; P8LE-NEXT:    sub r3, r5, r3
 ; P8LE-NEXT:    sub r5, r6, r7
-; P8LE-NEXT:    mtfprd f0, r3
+; P8LE-NEXT:    mtvsrd v3, r3
 ; P8LE-NEXT:    sub r3, r4, r8
-; P8LE-NEXT:    mtfprd f1, r5
-; P8LE-NEXT:    mtfprd f2, r3
-; P8LE-NEXT:    xxswapd v2, vs0
-; P8LE-NEXT:    xxswapd v3, vs1
-; P8LE-NEXT:    xxswapd v4, vs2
-; P8LE-NEXT:    vmrglh v2, v3, v2
-; P8LE-NEXT:    vmrglh v3, v4, v5
-; P8LE-NEXT:    vmrglw v2, v2, v3
+; P8LE-NEXT:    mtvsrd v4, r5
+; P8LE-NEXT:    mtvsrd v5, r3
+; P8LE-NEXT:    vmrghh v2, v3, v2
+; P8LE-NEXT:    vmrghh v3, v5, v4
+; P8LE-NEXT:    vmrglw v2, v3, v2
 ; P8LE-NEXT:    blr
 ;
 ; P8BE-LABEL: dont_fold_urem_one:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    mfvsrd r4, v2
 ; P8BE-NEXT:    lis r3, 24749
 ; P8BE-NEXT:    lis r7, -19946
 ; P8BE-NEXT:    lis r8, -14230

llvm/test/CodeGen/PowerPC/vec_conv_fp32_to_i16_elts.ll

 ; CHECK-P8-NEXT:    mtfprd f0, r3
 ; CHECK-P8-NEXT:    xxswapd v2, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtfprd f0, r3
 ; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs1
-; CHECK-P9-NEXT:    xxswapd v3, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    li r3, 0
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtfprd f0, r3
 ; CHECK-BE-NEXT:    xscvspdpn f1, vs0
 ; CHECK-BE-NEXT:    xxsldwi vs0, vs0, vs0, 1

 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f3, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs2
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v4, v5
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    vmrghh v3, v4, v3
+; CHECK-P8-NEXT:    vmrghh v2, v2, v5
+; CHECK-P8-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v4, v2
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    vmrghh v2, v4, v2
 ; CHECK-P9-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-BE-NEXT:    xscvspdpn f0, vs0
entry:
   ret i64 %1
 }
 
 define <8 x i16> @test8elt(<8 x float>* nocapture readonly) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v2, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    lvx v5, r3, r4
-; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    lvx v3, r3, r4
 ; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xxsldwi vs2, v5, v5, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v5
-; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    xscvspdpn f2, v2
+; CHECK-P8-NEXT:    xxsldwi vs4, v2, v2, 1
+; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mffprwz r5, f0
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    mtfprd f0, r5
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglh v2, v4, v3
-; CHECK-P8-NEXT:    mtfprd f2, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v5, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    xxswapd vs0, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xxsldwi vs1, v3, v3, 1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    mffprwz r3, f2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f4
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvdpsxws f4, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v2, v4, v2
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v6, vs3
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    vmrglh v3, v3, v4
-; CHECK-P8-NEXT:    vmrglh v4, v0, v5
-; CHECK-P8-NEXT:    vmrglh v5, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghh v3, v3, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghh v5, v0, v5
+; CHECK-P8-NEXT:    mtvsrd v1, r3
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
+; CHECK-P8-NEXT:    vmrghh v4, v4, v1
+; CHECK-P8-NEXT:    vmrglw v3, v4, v5
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xxsldwi vs2, vs1, vs1, 3
entry:
   %1 = fptoui <8 x float> %a to <8 x i16>
   ret <8 x i16> %1
 }
 
 define void @test16elt(<16 x i16>* noalias nocapture sret %agg.result, <16 x float>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v5, 0, r4
-; CHECK-P8-NEXT:    li r6, 32
 ; CHECK-P8-NEXT:    li r5, 16
-; CHECK-P8-NEXT:    lvx v2, r4, r6
+; CHECK-P8-NEXT:    li r6, 32
 ; CHECK-P8-NEXT:    lvx v3, r4, r5
+; CHECK-P8-NEXT:    lvx v2, r4, r6
 ; CHECK-P8-NEXT:    li r6, 48
-; CHECK-P8-NEXT:    xscvspdpn f0, v5
-; CHECK-P8-NEXT:    xxsldwi vs1, v5, v5, 3
+; CHECK-P8-NEXT:    xxsldwi vs0, v5, v5, 3
+; CHECK-P8-NEXT:    xscvspdpn f1, v5
 ; CHECK-P8-NEXT:    lvx v4, r4, r6
-; CHECK-P8-NEXT:    xscvspdpn f4, v2
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f2, v3
 ; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xxswapd vs8, v3
-; CHECK-P8-NEXT:    xscvspdpn f6, v4
+; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
 ; CHECK-P8-NEXT:    xxsldwi vs7, v3, v3, 3
+; CHECK-P8-NEXT:    xxswapd vs8, v3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xxsldwi vs10, v2, v2, 3
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    xscvspdpn f7, vs7
+; CHECK-P8-NEXT:    xscvspdpn f8, vs8
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    xxsldwi vs9, v3, v3, 1
+; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    xscvspdpn f2, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f1, f5
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxsldwi vs0, v3, v3, 1
+; CHECK-P8-NEXT:    xscvspdpn f4, v2
+; CHECK-P8-NEXT:    xscvdpsxws f5, f7
+; CHECK-P8-NEXT:    xxsldwi vs7, v4, v4, 3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xxsldwi vs3, v2, v2, 3
+; CHECK-P8-NEXT:    xscvspdpn f6, v4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f1, f8
+; CHECK-P8-NEXT:    xxswapd vs8, v4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f5
+; CHECK-P8-NEXT:    xxswapd vs5, v2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xxsldwi vs12, v2, v2, 1
-; CHECK-P8-NEXT:    xscvspdpn f8, vs8
-; CHECK-P8-NEXT:    xxswapd vs11, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xxswapd v2, v4
+; CHECK-P8-NEXT:    vmrghh v3, v0, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f6, f6
+; CHECK-P8-NEXT:    xscvspdpn f1, vs5
+; CHECK-P8-NEXT:    xxsldwi vs5, v2, v2, 1
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v2, v5, v1
+; CHECK-P8-NEXT:    vmrghh v5, v6, v0
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f2, f3
+; CHECK-P8-NEXT:    xscvspdpn f5, vs5
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
 ; CHECK-P8-NEXT:    xscvspdpn f7, vs7
-; CHECK-P8-NEXT:    xxsldwi vs13, v4, v4, 3
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxsldwi v3, v4, v4, 1
-; CHECK-P8-NEXT:    xscvspdpn f10, vs10
+; CHECK-P8-NEXT:    mtvsrd v7, r4
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xxsldwi vs2, v4, v4, 1
+; CHECK-P8-NEXT:    xscvspdpn f8, vs8
+; CHECK-P8-NEXT:    xscvdpsxws f0, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvspdpn f1, vs2
+; CHECK-P8-NEXT:    xscvdpsxws f3, f7
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xscvdpsxws f0, f8
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvspdpn f9, vs9
-; CHECK-P8-NEXT:    xscvdpsxws f6, f6
-; CHECK-P8-NEXT:    xscvspdpn f12, vs12
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    vmrghh v0, v0, v7
+; CHECK-P8-NEXT:    mtvsrd v7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvspdpn f11, vs11
-; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvspdpn v2, v2
-; CHECK-P8-NEXT:    xscvdpsxws f8, f8
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    mffprwz r6, f2
-; CHECK-P8-NEXT:    xscvspdpn f13, vs13
-; CHECK-P8-NEXT:    xscvspdpn v3, v3
-; CHECK-P8-NEXT:    xscvdpsxws f10, f10
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    vmrghh v4, v8, v4
+; CHECK-P8-NEXT:    mtvsrd v8, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    mtfprd f2, r6
-; CHECK-P8-NEXT:    mffprwz r6, f6
-; CHECK-P8-NEXT:    xscvdpsxws f12, f12
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f6, r6
-; CHECK-P8-NEXT:    mffprwz r6, f3
-; CHECK-P8-NEXT:    xscvdpsxws v2, v2
-; CHECK-P8-NEXT:    xxswapd v9, vs6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f8
-; CHECK-P8-NEXT:    mtfprd f3, r6
-; CHECK-P8-NEXT:    xxswapd v0, vs5
-; CHECK-P8-NEXT:    mffprwz r6, f7
-; CHECK-P8-NEXT:    xscvdpsxws f13, f13
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    xscvdpsxws v3, v3
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    mffprwz r4, f10
-; CHECK-P8-NEXT:    mtfprd f7, r6
-; CHECK-P8-NEXT:    mffprwz r6, f9
-; CHECK-P8-NEXT:    mtfprd f10, r4
-; CHECK-P8-NEXT:    mffprwz r4, f12
-; CHECK-P8-NEXT:    mtfprd f9, r6
-; CHECK-P8-NEXT:    xxswapd v6, vs10
-; CHECK-P8-NEXT:    mffprwz r6, f11
-; CHECK-P8-NEXT:    mtfprd f12, r4
-; CHECK-P8-NEXT:    xxswapd v1, vs9
-; CHECK-P8-NEXT:    mfvsrwz r4, v2
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f11, r6
-; CHECK-P8-NEXT:    mffprwz r6, f13
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v7, vs11
-; CHECK-P8-NEXT:    mfvsrwz r4, v3
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
-; CHECK-P8-NEXT:    xxswapd v4, vs7
-; CHECK-P8-NEXT:    vmrglh v2, v2, v0
-; CHECK-P8-NEXT:    xxswapd v5, vs8
-; CHECK-P8-NEXT:    xxswapd v0, vs2
-; CHECK-P8-NEXT:    mtfprd f13, r6
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    vmrglh v4, v5, v4
-; CHECK-P8-NEXT:    vmrglh v5, v0, v1
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglh v0, v7, v6
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    xxswapd v7, vs13
-; CHECK-P8-NEXT:    xxswapd v10, vs1
+; CHECK-P8-NEXT:    vmrghh v1, v1, v9
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghh v7, v8, v7
+; CHECK-P8-NEXT:    vmrghh v6, v6, v9
 ; CHECK-P8-NEXT:    vmrglw v2, v2, v3
-; CHECK-P8-NEXT:    vmrglh v1, v1, v6
-; CHECK-P8-NEXT:    vmrglh v6, v8, v7
-; CHECK-P8-NEXT:    vmrglh v7, v9, v10
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
-; CHECK-P8-NEXT:    vmrglw v4, v1, v0
-; CHECK-P8-NEXT:    vmrglw v5, v7, v6
+; CHECK-P8-NEXT:    vmrglw v3, v0, v5
+; CHECK-P8-NEXT:    vmrglw v4, v1, v4
+; CHECK-P8-NEXT:    vmrglw v5, v6, v7
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    stvx v2, 0, r3
 ; CHECK-P8-NEXT:    xxmrgld v3, v5, v4
 ; CHECK-P8-NEXT:    stvx v3, r3, r5
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs1, 0(r4)
-; CHECK-P9-NEXT:    lxv vs3, 16(r4)
-; CHECK-P9-NEXT:    xscvspdpn f5, vs1
-; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
-; CHECK-P9-NEXT:    xscvspdpn f8, vs3
-; CHECK-P9-NEXT:    xxswapd vs4, vs1
-; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    lxv vs2, 0(r4)
+; CHECK-P9-NEXT:    xxsldwi vs3, vs2, vs2, 3
+; CHECK-P9-NEXT:    xxswapd vs4, vs2
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvspdpn f4, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f5, f5
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    xscvspdpn f5, vs2
+; CHECK-P9-NEXT:    xxsldwi vs2, vs2, vs2, 1
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f8, f8
-; CHECK-P9-NEXT:    xxsldwi vs6, vs3, vs3, 3
-; CHECK-P9-NEXT:    xxswapd vs7, vs3
-; CHECK-P9-NEXT:    xscvspdpn f6, vs6
-; CHECK-P9-NEXT:    xxsldwi vs3, vs3, vs3, 1
-; CHECK-P9-NEXT:    xscvspdpn f7, vs7
-; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    lxv vs1, 16(r4)
+; CHECK-P9-NEXT:    xxsldwi vs6, vs1, vs1, 3
+; CHECK-P9-NEXT:    xxswapd vs3, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r5
+; CHECK-P9-NEXT:    mffprwz r5, f4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f5
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
+; CHECK-P9-NEXT:    mffprwz r5, f4
+; CHECK-P9-NEXT:    xscvspdpn f4, vs6
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    mffprwz r5, f2
+; CHECK-P9-NEXT:    xscvspdpn f2, vs1
+; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f4
-; CHECK-P9-NEXT:    xscvdpsxws f6, f6
-; CHECK-P9-NEXT:    mffprwz r5, f5
-; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    xscvdpsxws f7, f7
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    mtfprd f5, r5
-; CHECK-P9-NEXT:    mffprwz r5, f8
-; CHECK-P9-NEXT:    mtfprd f8, r5
-; CHECK-P9-NEXT:    mffprwz r5, f2
 ; CHECK-P9-NEXT:    lxv vs0, 32(r4)
-; CHECK-P9-NEXT:    xxsldwi vs9, vs0, vs0, 3
-; CHECK-P9-NEXT:    xxswapd vs10, vs0
-; CHECK-P9-NEXT:    xscvspdpn f9, vs9
-; CHECK-P9-NEXT:    xscvspdpn f10, vs10
-; CHECK-P9-NEXT:    xscvdpsxws f9, f9
-; CHECK-P9-NEXT:    xscvdpsxws f10, f10
-; CHECK-P9-NEXT:    mtfprd f2, r5
+; CHECK-P9-NEXT:    mtvsrd v4, r5
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r5, f4
-; CHECK-P9-NEXT:    mtfprd f4, r5
+; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    mtvsrd v4, r5
+; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    xxsldwi vs3, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r5
+; CHECK-P9-NEXT:    mffprwz r5, f2
+; CHECK-P9-NEXT:    xscvspdpn f2, vs3
+; CHECK-P9-NEXT:    vmrghh v4, v5, v4
+; CHECK-P9-NEXT:    mtvsrd v5, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f1
-; CHECK-P9-NEXT:    mtfprd f1, r5
-; CHECK-P9-NEXT:    mffprwz r5, f6
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    xxswapd v3, vs4
+; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    mtvsrd v0, r5
+; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghh v5, v5, v0
+; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrglw v3, v5, v4
+; CHECK-P9-NEXT:    mffprwz r5, f2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mtfprd f6, r5
-; CHECK-P9-NEXT:    mffprwz r5, f7
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mffprwz r5, f1
 ; CHECK-P9-NEXT:    lxv vs1, 48(r4)
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs5
-; CHECK-P9-NEXT:    mtfprd f7, r5
-; CHECK-P9-NEXT:    mffprwz r5, f3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs6
-; CHECK-P9-NEXT:    xxswapd v5, vs7
-; CHECK-P9-NEXT:    mtfprd f3, r5
-; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    xxswapd v0, vs3
-; CHECK-P9-NEXT:    vmrglh v4, v5, v4
-; CHECK-P9-NEXT:    xxswapd v5, vs8
-; CHECK-P9-NEXT:    vmrglh v5, v5, v0
+; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    mtvsrd v1, r5
+; CHECK-P9-NEXT:    vmrghh v0, v1, v0
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    mtfprd f2, r4
-; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglw v3, v5, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xxmrgld vs2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v2, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r4
+; CHECK-P9-NEXT:    mffprwz r4, f0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs1, vs1, 3
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
+; CHECK-P9-NEXT:    vmrghh v2, v4, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrglw v2, v2, v0
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglh v2, v4, v2
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    mffprwz r5, f9
-; CHECK-P9-NEXT:    mtfprd f9, r5
-; CHECK-P9-NEXT:    mffprwz r5, f10
-; CHECK-P9-NEXT:    mtfprd f10, r5
-; CHECK-P9-NEXT:    xxswapd v0, vs9
-; CHECK-P9-NEXT:    xxswapd v1, vs10
-; CHECK-P9-NEXT:    vmrglh v0, v1, v0
-; CHECK-P9-NEXT:    vmrglw v2, v2, v0
-; CHECK-P9-NEXT:    stxv vs2, 0(r3)
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r4
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld vs0, v3, v2
 ; CHECK-P9-NEXT:    stxv vs0, 16(r3)
+; CHECK-P9-NEXT:    stxv vs2, 0(r3)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r4)
 ; CHECK-BE-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-BE-NEXT:    xscvspdpn f2, vs2
 ; CHECK-BE-NEXT:    xxswapd vs3, vs1

 ; CHECK-P8-NEXT:    mtfprd f0, r3
 ; CHECK-P8-NEXT:    xxswapd v2, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtfprd f0, r3
 ; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs1
-; CHECK-P9-NEXT:    xxswapd v3, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    li r3, 0
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtfprd f0, r3
 ; CHECK-BE-NEXT:    xscvspdpn f1, vs0
 ; CHECK-BE-NEXT:    xxsldwi vs0, vs0, vs0, 1

 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f3, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs2
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v4, v5
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    vmrghh v3, v4, v3
+; CHECK-P8-NEXT:    vmrghh v2, v2, v5
+; CHECK-P8-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v4, v2
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    vmrghh v2, v4, v2
 ; CHECK-P9-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-BE-NEXT:    xscvspdpn f0, vs0
entry:
   ret i64 %1
 }
 
 define <8 x i16> @test8elt_signed(<8 x float>* nocapture readonly) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v2, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    lvx v5, r3, r4
-; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    lvx v3, r3, r4
 ; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xxsldwi vs2, v5, v5, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v5
-; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    xscvspdpn f2, v2
+; CHECK-P8-NEXT:    xxsldwi vs4, v2, v2, 1
+; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mffprwz r5, f0
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    mtfprd f0, r5
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglh v2, v4, v3
-; CHECK-P8-NEXT:    mtfprd f2, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v5, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    xxswapd vs0, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xxsldwi vs1, v3, v3, 1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    mffprwz r3, f2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f4
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvdpsxws f4, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v2, v4, v2
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v6, vs3
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    vmrglh v3, v3, v4
-; CHECK-P8-NEXT:    vmrglh v4, v0, v5
-; CHECK-P8-NEXT:    vmrglh v5, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghh v3, v3, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghh v5, v0, v5
+; CHECK-P8-NEXT:    mtvsrd v1, r3
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
+; CHECK-P8-NEXT:    vmrghh v4, v4, v1
+; CHECK-P8-NEXT:    vmrglw v3, v4, v5
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xxsldwi vs2, vs1, vs1, 3
entry:
   %1 = fptosi <8 x float> %a to <8 x i16>
   ret <8 x i16> %1
 }
 
 define void @test16elt_signed(<16 x i16>* noalias nocapture sret %agg.result, <16 x float>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v5, 0, r4
-; CHECK-P8-NEXT:    li r6, 32
 ; CHECK-P8-NEXT:    li r5, 16
-; CHECK-P8-NEXT:    lvx v2, r4, r6
+; CHECK-P8-NEXT:    li r6, 32
 ; CHECK-P8-NEXT:    lvx v3, r4, r5
+; CHECK-P8-NEXT:    lvx v2, r4, r6
 ; CHECK-P8-NEXT:    li r6, 48
-; CHECK-P8-NEXT:    xscvspdpn f0, v5
-; CHECK-P8-NEXT:    xxsldwi vs1, v5, v5, 3
+; CHECK-P8-NEXT:    xxsldwi vs0, v5, v5, 3
+; CHECK-P8-NEXT:    xscvspdpn f1, v5
 ; CHECK-P8-NEXT:    lvx v4, r4, r6
-; CHECK-P8-NEXT:    xscvspdpn f4, v2
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f2, v3
 ; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xxswapd vs8, v3
-; CHECK-P8-NEXT:    xscvspdpn f6, v4
+; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
 ; CHECK-P8-NEXT:    xxsldwi vs7, v3, v3, 3
+; CHECK-P8-NEXT:    xxswapd vs8, v3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xxsldwi vs10, v2, v2, 3
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    xscvspdpn f7, vs7
+; CHECK-P8-NEXT:    xscvspdpn f8, vs8
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    xxsldwi vs9, v3, v3, 1
+; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    xscvspdpn f2, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f1, f5
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxsldwi vs0, v3, v3, 1
+; CHECK-P8-NEXT:    xscvspdpn f4, v2
+; CHECK-P8-NEXT:    xscvdpsxws f5, f7
+; CHECK-P8-NEXT:    xxsldwi vs7, v4, v4, 3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xxsldwi vs3, v2, v2, 3
+; CHECK-P8-NEXT:    xscvspdpn f6, v4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f1, f8
+; CHECK-P8-NEXT:    xxswapd vs8, v4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f5
+; CHECK-P8-NEXT:    xxswapd vs5, v2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xxsldwi vs12, v2, v2, 1
-; CHECK-P8-NEXT:    xscvspdpn f8, vs8
-; CHECK-P8-NEXT:    xxswapd vs11, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xxswapd v2, v4
+; CHECK-P8-NEXT:    vmrghh v3, v0, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvdpsxws f6, f6
+; CHECK-P8-NEXT:    xscvspdpn f1, vs5
+; CHECK-P8-NEXT:    xxsldwi vs5, v2, v2, 1
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v2, v5, v1
+; CHECK-P8-NEXT:    vmrghh v5, v6, v0
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f2, f3
+; CHECK-P8-NEXT:    xscvspdpn f5, vs5
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
 ; CHECK-P8-NEXT:    xscvspdpn f7, vs7
-; CHECK-P8-NEXT:    xxsldwi vs13, v4, v4, 3
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxsldwi v3, v4, v4, 1
-; CHECK-P8-NEXT:    xscvspdpn f10, vs10
+; CHECK-P8-NEXT:    mtvsrd v7, r4
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xxsldwi vs2, v4, v4, 1
+; CHECK-P8-NEXT:    xscvspdpn f8, vs8
+; CHECK-P8-NEXT:    xscvdpsxws f0, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xscvspdpn f1, vs2
+; CHECK-P8-NEXT:    xscvdpsxws f3, f7
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xscvdpsxws f0, f8
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvspdpn f9, vs9
-; CHECK-P8-NEXT:    xscvdpsxws f6, f6
-; CHECK-P8-NEXT:    xscvspdpn f12, vs12
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    vmrghh v0, v0, v7
+; CHECK-P8-NEXT:    mtvsrd v7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvspdpn f11, vs11
-; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvspdpn v2, v2
-; CHECK-P8-NEXT:    xscvdpsxws f8, f8
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    mffprwz r6, f2
-; CHECK-P8-NEXT:    xscvspdpn f13, vs13
-; CHECK-P8-NEXT:    xscvspdpn v3, v3
-; CHECK-P8-NEXT:    xscvdpsxws f10, f10
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    vmrghh v4, v8, v4
+; CHECK-P8-NEXT:    mtvsrd v8, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    mtfprd f2, r6
-; CHECK-P8-NEXT:    mffprwz r6, f6
-; CHECK-P8-NEXT:    xscvdpsxws f12, f12
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f6, r6
-; CHECK-P8-NEXT:    mffprwz r6, f3
-; CHECK-P8-NEXT:    xscvdpsxws v2, v2
-; CHECK-P8-NEXT:    xxswapd v9, vs6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f8
-; CHECK-P8-NEXT:    mtfprd f3, r6
-; CHECK-P8-NEXT:    xxswapd v0, vs5
-; CHECK-P8-NEXT:    mffprwz r6, f7
-; CHECK-P8-NEXT:    xscvdpsxws f13, f13
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    xscvdpsxws v3, v3
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    mffprwz r4, f10
-; CHECK-P8-NEXT:    mtfprd f7, r6
-; CHECK-P8-NEXT:    mffprwz r6, f9
-; CHECK-P8-NEXT:    mtfprd f10, r4
-; CHECK-P8-NEXT:    mffprwz r4, f12
-; CHECK-P8-NEXT:    mtfprd f9, r6
-; CHECK-P8-NEXT:    xxswapd v6, vs10
-; CHECK-P8-NEXT:    mffprwz r6, f11
-; CHECK-P8-NEXT:    mtfprd f12, r4
-; CHECK-P8-NEXT:    xxswapd v1, vs9
-; CHECK-P8-NEXT:    mfvsrwz r4, v2
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f11, r6
-; CHECK-P8-NEXT:    mffprwz r6, f13
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v7, vs11
-; CHECK-P8-NEXT:    mfvsrwz r4, v3
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
-; CHECK-P8-NEXT:    xxswapd v4, vs7
-; CHECK-P8-NEXT:    vmrglh v2, v2, v0
-; CHECK-P8-NEXT:    xxswapd v5, vs8
-; CHECK-P8-NEXT:    xxswapd v0, vs2
-; CHECK-P8-NEXT:    mtfprd f13, r6
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    vmrglh v4, v5, v4
-; CHECK-P8-NEXT:    vmrglh v5, v0, v1
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglh v0, v7, v6
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    xxswapd v7, vs13
-; CHECK-P8-NEXT:    xxswapd v10, vs1
+; CHECK-P8-NEXT:    vmrghh v1, v1, v9
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghh v7, v8, v7
+; CHECK-P8-NEXT:    vmrghh v6, v6, v9
 ; CHECK-P8-NEXT:    vmrglw v2, v2, v3
-; CHECK-P8-NEXT:    vmrglh v1, v1, v6
-; CHECK-P8-NEXT:    vmrglh v6, v8, v7
-; CHECK-P8-NEXT:    vmrglh v7, v9, v10
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
-; CHECK-P8-NEXT:    vmrglw v4, v1, v0
-; CHECK-P8-NEXT:    vmrglw v5, v7, v6
+; CHECK-P8-NEXT:    vmrglw v3, v0, v5
+; CHECK-P8-NEXT:    vmrglw v4, v1, v4
+; CHECK-P8-NEXT:    vmrglw v5, v6, v7
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    stvx v2, 0, r3
 ; CHECK-P8-NEXT:    xxmrgld v3, v5, v4
 ; CHECK-P8-NEXT:    stvx v3, r3, r5
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs1, 0(r4)
-; CHECK-P9-NEXT:    lxv vs3, 16(r4)
-; CHECK-P9-NEXT:    xscvspdpn f5, vs1
-; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
-; CHECK-P9-NEXT:    xscvspdpn f8, vs3
-; CHECK-P9-NEXT:    xxswapd vs4, vs1
-; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    lxv vs2, 0(r4)
+; CHECK-P9-NEXT:    xxsldwi vs3, vs2, vs2, 3
+; CHECK-P9-NEXT:    xxswapd vs4, vs2
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvspdpn f4, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f5, f5
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    xscvspdpn f5, vs2
+; CHECK-P9-NEXT:    xxsldwi vs2, vs2, vs2, 1
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f8, f8
-; CHECK-P9-NEXT:    xxsldwi vs6, vs3, vs3, 3
-; CHECK-P9-NEXT:    xxswapd vs7, vs3
-; CHECK-P9-NEXT:    xscvspdpn f6, vs6
-; CHECK-P9-NEXT:    xxsldwi vs3, vs3, vs3, 1
-; CHECK-P9-NEXT:    xscvspdpn f7, vs7
-; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    lxv vs1, 16(r4)
+; CHECK-P9-NEXT:    xxsldwi vs6, vs1, vs1, 3
+; CHECK-P9-NEXT:    xxswapd vs3, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r5
+; CHECK-P9-NEXT:    mffprwz r5, f4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f5
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
+; CHECK-P9-NEXT:    mffprwz r5, f4
+; CHECK-P9-NEXT:    xscvspdpn f4, vs6
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    mffprwz r5, f2
+; CHECK-P9-NEXT:    xscvspdpn f2, vs1
+; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f4
-; CHECK-P9-NEXT:    xscvdpsxws f6, f6
-; CHECK-P9-NEXT:    mffprwz r5, f5
-; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    xscvdpsxws f7, f7
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    mtfprd f5, r5
-; CHECK-P9-NEXT:    mffprwz r5, f8
-; CHECK-P9-NEXT:    mtfprd f8, r5
-; CHECK-P9-NEXT:    mffprwz r5, f2
 ; CHECK-P9-NEXT:    lxv vs0, 32(r4)
-; CHECK-P9-NEXT:    xxsldwi vs9, vs0, vs0, 3
-; CHECK-P9-NEXT:    xxswapd vs10, vs0
-; CHECK-P9-NEXT:    xscvspdpn f9, vs9
-; CHECK-P9-NEXT:    xscvspdpn f10, vs10
-; CHECK-P9-NEXT:    xscvdpsxws f9, f9
-; CHECK-P9-NEXT:    xscvdpsxws f10, f10
-; CHECK-P9-NEXT:    mtfprd f2, r5
+; CHECK-P9-NEXT:    mtvsrd v4, r5
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r5, f4
-; CHECK-P9-NEXT:    mtfprd f4, r5
+; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    mtvsrd v4, r5
+; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    xxsldwi vs3, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r5
+; CHECK-P9-NEXT:    mffprwz r5, f2
+; CHECK-P9-NEXT:    xscvspdpn f2, vs3
+; CHECK-P9-NEXT:    vmrghh v4, v5, v4
+; CHECK-P9-NEXT:    mtvsrd v5, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f1
-; CHECK-P9-NEXT:    mtfprd f1, r5
-; CHECK-P9-NEXT:    mffprwz r5, f6
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    xxswapd v3, vs4
+; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    mtvsrd v0, r5
+; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghh v5, v5, v0
+; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrglw v3, v5, v4
+; CHECK-P9-NEXT:    mffprwz r5, f2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mtfprd f6, r5
-; CHECK-P9-NEXT:    mffprwz r5, f7
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mffprwz r5, f1
 ; CHECK-P9-NEXT:    lxv vs1, 48(r4)
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs5
-; CHECK-P9-NEXT:    mtfprd f7, r5
-; CHECK-P9-NEXT:    mffprwz r5, f3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs6
-; CHECK-P9-NEXT:    xxswapd v5, vs7
-; CHECK-P9-NEXT:    mtfprd f3, r5
-; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    xxswapd v0, vs3
-; CHECK-P9-NEXT:    vmrglh v4, v5, v4
-; CHECK-P9-NEXT:    xxswapd v5, vs8
-; CHECK-P9-NEXT:    vmrglh v5, v5, v0
+; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    mtvsrd v1, r5
+; CHECK-P9-NEXT:    vmrghh v0, v1, v0
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    mtfprd f2, r4
-; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglw v3, v5, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xxmrgld vs2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v2, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r4
+; CHECK-P9-NEXT:    mffprwz r4, f0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs1, vs1, 3
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
+; CHECK-P9-NEXT:    vmrghh v2, v4, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrglw v2, v2, v0
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglh v2, v4, v2
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    mffprwz r5, f9
-; CHECK-P9-NEXT:    mtfprd f9, r5
-; CHECK-P9-NEXT:    mffprwz r5, f10
-; CHECK-P9-NEXT:    mtfprd f10, r5
-; CHECK-P9-NEXT:    xxswapd v0, vs9
-; CHECK-P9-NEXT:    xxswapd v1, vs10
-; CHECK-P9-NEXT:    vmrglh v0, v1, v0
-; CHECK-P9-NEXT:    vmrglw v2, v2, v0
-; CHECK-P9-NEXT:    stxv vs2, 0(r3)
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r4
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld vs0, v3, v2
 ; CHECK-P9-NEXT:    stxv vs0, 16(r3)
+; CHECK-P9-NEXT:    stxv vs2, 0(r3)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r4)
 ; CHECK-BE-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-BE-NEXT:    xscvspdpn f2, vs2
 ; CHECK-BE-NEXT:    xxswapd vs3, vs1

llvm/test/CodeGen/PowerPC/vec_conv_fp32_to_i8_elts.ll

 ; CHECK-P8-NEXT:    mtfprd f0, r3
 ; CHECK-P8-NEXT:    xxswapd v2, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    clrldi r3, r3, 48
 ; CHECK-P8-NEXT:    sth r3, -2(r1)
 ; CHECK-P8-NEXT:    lhz r3, -2(r1)
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtfprd f0, r3
 ; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    addi r3, r1, -2
-; CHECK-P9-NEXT:    xxswapd v2, vs1
-; CHECK-P9-NEXT:    xxswapd v3, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    vsldoi v2, v2, v2, 8
 ; CHECK-P9-NEXT:    stxsihx v2, 0, r3
 ; CHECK-P9-NEXT:    lhz r3, -2(r1)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtfprd f0, r3

 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f3, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs2
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
-; CHECK-P8-NEXT:    vmrglb v3, v4, v5
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    vmrghb v3, v4, v3
+; CHECK-P8-NEXT:    vmrghb v2, v2, v5
+; CHECK-P8-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v4, v2
+; CHECK-P9-NEXT:    vmrghb v2, v4, v2
 ; CHECK-P9-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-BE-NEXT:    xscvspdpn f0, vs0
entry:
   ret i32 %1
 }
 
 define i64 @test8elt(<8 x float>* nocapture readonly) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v2, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    lvx v5, r3, r4
-; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    lvx v3, r3, r4
 ; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xxsldwi vs2, v5, v5, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v5
-; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    xscvspdpn f2, v2
+; CHECK-P8-NEXT:    xxsldwi vs4, v2, v2, 1
+; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mffprwz r5, f0
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    mtfprd f0, r5
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglb v2, v4, v3
-; CHECK-P8-NEXT:    mtfprd f2, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v5, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    xxswapd vs0, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xxsldwi vs1, v3, v3, 1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    mffprwz r3, f2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f4
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvdpsxws f4, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghb v2, v4, v2
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v6, vs3
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    vmrglb v3, v3, v4
-; CHECK-P8-NEXT:    vmrglb v4, v0, v5
-; CHECK-P8-NEXT:    vmrglb v5, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghb v3, v3, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghb v5, v0, v5
+; CHECK-P8-NEXT:    mtvsrd v1, r3
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
+; CHECK-P8-NEXT:    vmrghb v4, v4, v1
+; CHECK-P8-NEXT:    vmrglh v3, v4, v5
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
entry:
   %1 = fptoui <8 x float> %a to <8 x i8>
   %2 = bitcast <8 x i8> %1 to i64
   ret i64 %2
 }
 
 define <16 x i8> @test16elt(<16 x float>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    lvx v2, 0, r3
+; CHECK-P8-NEXT:    lvx v4, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
+; CHECK-P8-NEXT:    li r5, 32
 ; CHECK-P8-NEXT:    lvx v3, r3, r4
-; CHECK-P8-NEXT:    li r4, 32
-; CHECK-P8-NEXT:    xscvspdpn f2, v2
-; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v3
-; CHECK-P8-NEXT:    xxswapd vs1, v2
-; CHECK-P8-NEXT:    xxsldwi vs3, v2, v2, 1
-; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
-; CHECK-P8-NEXT:    lvx v2, r3, r4
+; CHECK-P8-NEXT:    lvx v2, r3, r5
+; CHECK-P8-NEXT:    xxsldwi vs0, v4, v4, 3
+; CHECK-P8-NEXT:    xxswapd vs2, v4
+; CHECK-P8-NEXT:    xxsldwi vs4, v4, v4, 1
+; CHECK-P8-NEXT:    xscvspdpn f1, v4
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
+; CHECK-P8-NEXT:    xxsldwi vs6, v3, v3, 3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xxswapd vs6, v3
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xxsldwi vs7, v3, v3, 1
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xxsldwi vs8, v2, v2, 3
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxswapd vs9, v2
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    xxswapd vs7, v3
+; CHECK-P8-NEXT:    xscvspdpn f2, vs2
+; CHECK-P8-NEXT:    xxsldwi vs8, v3, v3, 1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
+; CHECK-P8-NEXT:    xxsldwi vs9, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f6, vs6
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r4, f2
 ; CHECK-P8-NEXT:    xscvspdpn f7, vs7
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    xscvdpsxws f4, f4
 ; CHECK-P8-NEXT:    xscvspdpn f8, vs8
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvdpsxws f0, f5
-; CHECK-P8-NEXT:    xxswapd v0, vs4
+; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    xscvspdpn f9, vs9
-; CHECK-P8-NEXT:    mtfprd f5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxswapd vs0, v2
+; CHECK-P8-NEXT:    mffprwz r5, f2
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    mtvsrd v4, r5
+; CHECK-P8-NEXT:    mffprwz r5, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f6
-; CHECK-P8-NEXT:    xxswapd v3, vs5
-; CHECK-P8-NEXT:    mtfprd f6, r4
-; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    vmrghb v3, v4, v3
+; CHECK-P8-NEXT:    mtvsrd v4, r5
+; CHECK-P8-NEXT:    mffprwz r5, f3
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f7
-; CHECK-P8-NEXT:    xxswapd v4, vs6
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvdpsxws f0, f8
-; CHECK-P8-NEXT:    xxswapd v5, vs7
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xscvdpsxws f1, f9
-; CHECK-P8-NEXT:    xxswapd v1, vs8
-; CHECK-P8-NEXT:    mtfprd f9, r4
-; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    vmrglb v3, v4, v3
-; CHECK-P8-NEXT:    xxswapd v4, vs2
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs9
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    xxswapd v7, vs3
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    vmrglb v4, v4, v5
-; CHECK-P8-NEXT:    xxswapd v5, vs5
-; CHECK-P8-NEXT:    mtfprd f1, r4
+; CHECK-P8-NEXT:    xscvdpsxws f4, f8
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    li r4, 48
-; CHECK-P8-NEXT:    lvx v9, r3, r4
-; CHECK-P8-NEXT:    vmrglb v1, v6, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs1
+; CHECK-P8-NEXT:    lvx v0, r3, r4
+; CHECK-P8-NEXT:    mffprwz r3, f1
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxsldwi vs2, v9, v9, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v9
-; CHECK-P8-NEXT:    xxswapd vs3, v9
-; CHECK-P8-NEXT:    xxsldwi vs5, v9, v9, 1
+; CHECK-P8-NEXT:    xscvspdpn f5, v2
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xxsldwi vs3, v0, v0, 3
+; CHECK-P8-NEXT:    mtvsrd v1, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    xxswapd vs4, v0
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
+; CHECK-P8-NEXT:    mtvsrd v7, r3
+; CHECK-P8-NEXT:    mffprwz r3, f0
+; CHECK-P8-NEXT:    xxsldwi vs0, v0, v0, 1
+; CHECK-P8-NEXT:    xscvspdpn f2, v0
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f6, f9
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvdpsxws f5, f5
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghb v2, v6, v1
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f5
+; CHECK-P8-NEXT:    mtvsrd v6, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghb v4, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v5, r5
+; CHECK-P8-NEXT:    vmrghb v0, v6, v1
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v9, vs4
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v6, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    vmrglb v2, v0, v7
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v7, vs2
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    vmrglb v5, v8, v5
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    xxswapd v10, vs3
-; CHECK-P8-NEXT:    vmrglb v0, v0, v6
+; CHECK-P8-NEXT:    vmrghb v5, v5, v7
+; CHECK-P8-NEXT:    vmrghb v1, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    mtvsrd v7, r3
+; CHECK-P8-NEXT:    mffprwz r3, f0
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mtvsrd v9, r3
+; CHECK-P8-NEXT:    vmrghb v7, v8, v7
+; CHECK-P8-NEXT:    vmrghb v6, v6, v9
 ; CHECK-P8-NEXT:    vmrglh v3, v4, v3
-; CHECK-P8-NEXT:    vmrglb v6, v8, v7
-; CHECK-P8-NEXT:    vmrglb v7, v9, v10
-; CHECK-P8-NEXT:    vmrglh v2, v2, v1
-; CHECK-P8-NEXT:    vmrglh v4, v0, v5
-; CHECK-P8-NEXT:    vmrglh v5, v7, v6
+; CHECK-P8-NEXT:    vmrglh v2, v5, v2
+; CHECK-P8-NEXT:    vmrglh v4, v1, v0
+; CHECK-P8-NEXT:    vmrglh v5, v6, v7
 ; CHECK-P8-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P8-NEXT:    vmrglw v3, v5, v4
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs2, 0(r3)
+; CHECK-P9-NEXT:    lxv vs3, 0(r3)
+; CHECK-P9-NEXT:    xxsldwi vs4, vs3, vs3, 3
+; CHECK-P9-NEXT:    xscvspdpn f4, vs4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    lxv vs0, 48(r3)
+; CHECK-P9-NEXT:    lxv vs1, 32(r3)
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    xxswapd vs4, vs3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    xscvspdpn f4, vs4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    xscvspdpn f4, vs3
+; CHECK-P9-NEXT:    xxsldwi vs3, vs3, vs3, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f3
 ; CHECK-P9-NEXT:    xxsldwi vs3, vs2, vs2, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    lxv vs0, 48(r3)
-; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs4, 16(r3)
+; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs2
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs2, vs2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xxsldwi vs2, vs4, vs4, 3
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    xxswapd vs2, vs4
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xscvspdpn f2, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xxsldwi vs2, vs4, vs4, 1
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs2
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v4, v5, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v4, v5, v4
-; CHECK-P9-NEXT:    xxswapd v5, vs1
-; CHECK-P9-NEXT:    xxswapd v0, vs0
-; CHECK-P9-NEXT:    vmrglb v5, v5, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r3
+; CHECK-P9-NEXT:    vmrghb v5, v5, v0
 ; CHECK-P9-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)

 ; CHECK-P8-NEXT:    mtfprd f0, r3
 ; CHECK-P8-NEXT:    xxswapd v2, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    clrldi r3, r3, 48
 ; CHECK-P8-NEXT:    sth r3, -2(r1)
 ; CHECK-P8-NEXT:    lhz r3, -2(r1)
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtfprd f0, r3
 ; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs1, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    addi r3, r1, -2
-; CHECK-P9-NEXT:    xxswapd v2, vs1
-; CHECK-P9-NEXT:    xxswapd v3, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    vsldoi v2, v2, v2, 8
 ; CHECK-P9-NEXT:    stxsihx v2, 0, r3
 ; CHECK-P9-NEXT:    lhz r3, -2(r1)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtfprd f0, r3

 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    mtfprd f3, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs2
-; CHECK-P8-NEXT:    xxswapd v5, vs3
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
-; CHECK-P8-NEXT:    vmrglb v3, v4, v5
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    vmrghb v3, v4, v3
+; CHECK-P8-NEXT:    vmrghb v2, v2, v5
+; CHECK-P8-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xscvspdpn f0, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, v2, v2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v4, v2
+; CHECK-P9-NEXT:    vmrghb v2, v4, v2
 ; CHECK-P9-NEXT:    vmrglh v2, v2, v3
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, v2, v2, 3
 ; CHECK-BE-NEXT:    xscvspdpn f0, vs0
entry:
   ret i32 %1
 }
 
 define i64 @test8elt_signed(<8 x float>* nocapture readonly) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    lvx v2, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    lvx v5, r3, r4
-; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    lvx v3, r3, r4
 ; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xxsldwi vs2, v5, v5, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v5
-; CHECK-P8-NEXT:    xxswapd vs3, v5
-; CHECK-P8-NEXT:    xxsldwi vs5, v5, v5, 1
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xxswapd vs1, v2
+; CHECK-P8-NEXT:    xscvspdpn f2, v2
+; CHECK-P8-NEXT:    xxsldwi vs4, v2, v2, 1
+; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
 ; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mffprwz r5, f0
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    mtfprd f0, r5
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v1, vs4
-; CHECK-P8-NEXT:    vmrglb v2, v4, v3
-; CHECK-P8-NEXT:    mtfprd f2, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v5, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mffprwz r3, f1
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    xxswapd vs0, v3
+; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    xxsldwi vs1, v3, v3, 1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    mffprwz r3, f2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f4
+; CHECK-P8-NEXT:    xscvspdpn f1, vs1
+; CHECK-P8-NEXT:    xscvdpsxws f4, f5
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghb v2, v4, v2
+; CHECK-P8-NEXT:    mffprwz r4, f2
+; CHECK-P8-NEXT:    xscvdpsxws f1, f1
+; CHECK-P8-NEXT:    mtvsrd v3, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v6, vs3
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    vmrglb v3, v3, v4
-; CHECK-P8-NEXT:    vmrglb v4, v0, v5
-; CHECK-P8-NEXT:    vmrglb v5, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghb v3, v3, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    mtvsrd v0, r4
+; CHECK-P8-NEXT:    mtvsrd v5, r3
+; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghb v5, v0, v5
+; CHECK-P8-NEXT:    mtvsrd v1, r3
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
+; CHECK-P8-NEXT:    vmrghb v4, v4, v1
+; CHECK-P8-NEXT:    vmrglh v3, v4, v5
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
entry:
   %1 = fptosi <8 x float> %a to <8 x i8>
   %2 = bitcast <8 x i8> %1 to i64
   ret i64 %2
 }
 
 define <16 x i8> @test16elt_signed(<16 x float>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    lvx v2, 0, r3
+; CHECK-P8-NEXT:    lvx v4, 0, r3
 ; CHECK-P8-NEXT:    li r4, 16
+; CHECK-P8-NEXT:    li r5, 32
 ; CHECK-P8-NEXT:    lvx v3, r3, r4
-; CHECK-P8-NEXT:    li r4, 32
-; CHECK-P8-NEXT:    xscvspdpn f2, v2
-; CHECK-P8-NEXT:    xxsldwi vs0, v2, v2, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v3
-; CHECK-P8-NEXT:    xxswapd vs1, v2
-; CHECK-P8-NEXT:    xxsldwi vs3, v2, v2, 1
-; CHECK-P8-NEXT:    xxsldwi vs5, v3, v3, 3
-; CHECK-P8-NEXT:    lvx v2, r3, r4
+; CHECK-P8-NEXT:    lvx v2, r3, r5
+; CHECK-P8-NEXT:    xxsldwi vs0, v4, v4, 3
+; CHECK-P8-NEXT:    xxswapd vs2, v4
+; CHECK-P8-NEXT:    xxsldwi vs4, v4, v4, 1
+; CHECK-P8-NEXT:    xscvspdpn f1, v4
+; CHECK-P8-NEXT:    xscvspdpn f3, v3
+; CHECK-P8-NEXT:    xxsldwi vs6, v3, v3, 3
 ; CHECK-P8-NEXT:    xscvspdpn f0, vs0
-; CHECK-P8-NEXT:    xxswapd vs6, v3
-; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xxsldwi vs7, v3, v3, 1
-; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xxsldwi vs8, v2, v2, 3
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxswapd vs9, v2
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
-; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    xxswapd vs7, v3
+; CHECK-P8-NEXT:    xscvspdpn f2, vs2
+; CHECK-P8-NEXT:    xxsldwi vs8, v3, v3, 1
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
+; CHECK-P8-NEXT:    xxsldwi vs9, v2, v2, 3
 ; CHECK-P8-NEXT:    xscvspdpn f6, vs6
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r4, f2
 ; CHECK-P8-NEXT:    xscvspdpn f7, vs7
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    xscvdpsxws f4, f4
 ; CHECK-P8-NEXT:    xscvspdpn f8, vs8
-; CHECK-P8-NEXT:    mtfprd f4, r4
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvdpsxws f0, f5
-; CHECK-P8-NEXT:    xxswapd v0, vs4
+; CHECK-P8-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P8-NEXT:    xscvspdpn f9, vs9
-; CHECK-P8-NEXT:    mtfprd f5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxswapd vs0, v2
+; CHECK-P8-NEXT:    mffprwz r5, f2
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f1
+; CHECK-P8-NEXT:    mtvsrd v4, r5
+; CHECK-P8-NEXT:    mffprwz r5, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f6
-; CHECK-P8-NEXT:    xxswapd v3, vs5
-; CHECK-P8-NEXT:    mtfprd f6, r4
-; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    vmrghb v3, v4, v3
+; CHECK-P8-NEXT:    mtvsrd v4, r5
+; CHECK-P8-NEXT:    mffprwz r5, f3
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f7
-; CHECK-P8-NEXT:    xxswapd v4, vs6
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvdpsxws f0, f8
-; CHECK-P8-NEXT:    xxswapd v5, vs7
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xscvdpsxws f1, f9
-; CHECK-P8-NEXT:    xxswapd v1, vs8
-; CHECK-P8-NEXT:    mtfprd f9, r4
-; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    vmrglb v3, v4, v3
-; CHECK-P8-NEXT:    xxswapd v4, vs2
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs9
-; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    xscvspdpn f0, v2
-; CHECK-P8-NEXT:    xxswapd v7, vs3
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    vmrglb v4, v4, v5
-; CHECK-P8-NEXT:    xxswapd v5, vs5
-; CHECK-P8-NEXT:    mtfprd f1, r4
+; CHECK-P8-NEXT:    xscvdpsxws f4, f8
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    li r4, 48
-; CHECK-P8-NEXT:    lvx v9, r3, r4
-; CHECK-P8-NEXT:    vmrglb v1, v6, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs1
+; CHECK-P8-NEXT:    lvx v0, r3, r4
+; CHECK-P8-NEXT:    mffprwz r3, f1
 ; CHECK-P8-NEXT:    xxsldwi vs1, v2, v2, 1
-; CHECK-P8-NEXT:    xxsldwi vs2, v9, v9, 3
-; CHECK-P8-NEXT:    xscvspdpn f4, v9
-; CHECK-P8-NEXT:    xxswapd vs3, v9
-; CHECK-P8-NEXT:    xxsldwi vs5, v9, v9, 1
+; CHECK-P8-NEXT:    xscvspdpn f5, v2
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xxsldwi vs3, v0, v0, 3
+; CHECK-P8-NEXT:    mtvsrd v1, r3
+; CHECK-P8-NEXT:    mffprwz r3, f4
+; CHECK-P8-NEXT:    xxswapd vs4, v0
 ; CHECK-P8-NEXT:    xscvspdpn f1, vs1
-; CHECK-P8-NEXT:    xscvspdpn f2, vs2
+; CHECK-P8-NEXT:    mtvsrd v7, r3
+; CHECK-P8-NEXT:    mffprwz r3, f0
+; CHECK-P8-NEXT:    xxsldwi vs0, v0, v0, 1
+; CHECK-P8-NEXT:    xscvspdpn f2, v0
 ; CHECK-P8-NEXT:    xscvspdpn f3, vs3
-; CHECK-P8-NEXT:    xscvspdpn f5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    xscvdpsxws f4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f6, f9
+; CHECK-P8-NEXT:    xscvspdpn f4, vs4
+; CHECK-P8-NEXT:    xscvspdpn f0, vs0
+; CHECK-P8-NEXT:    xscvdpsxws f5, f5
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f4, f4
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghb v2, v6, v1
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f5
+; CHECK-P8-NEXT:    mtvsrd v6, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    vmrghb v4, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v5, r5
+; CHECK-P8-NEXT:    vmrghb v0, v6, v1
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    xxswapd v9, vs4
-; CHECK-P8-NEXT:    mtfprd f1, r3
+; CHECK-P8-NEXT:    mtvsrd v6, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f3
-; CHECK-P8-NEXT:    mtfprd f2, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs1
-; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    vmrglb v2, v0, v7
-; CHECK-P8-NEXT:    xxswapd v0, vs0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v7, vs2
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    vmrglb v5, v8, v5
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    xxswapd v10, vs3
-; CHECK-P8-NEXT:    vmrglb v0, v0, v6
+; CHECK-P8-NEXT:    vmrghb v5, v5, v7
+; CHECK-P8-NEXT:    vmrghb v1, v1, v6
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    mtvsrd v7, r3
+; CHECK-P8-NEXT:    mffprwz r3, f0
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mtvsrd v9, r3
+; CHECK-P8-NEXT:    vmrghb v7, v8, v7
+; CHECK-P8-NEXT:    vmrghb v6, v6, v9
 ; CHECK-P8-NEXT:    vmrglh v3, v4, v3
-; CHECK-P8-NEXT:    vmrglb v6, v8, v7
-; CHECK-P8-NEXT:    vmrglb v7, v9, v10
-; CHECK-P8-NEXT:    vmrglh v2, v2, v1
-; CHECK-P8-NEXT:    vmrglh v4, v0, v5
-; CHECK-P8-NEXT:    vmrglh v5, v7, v6
+; CHECK-P8-NEXT:    vmrglh v2, v5, v2
+; CHECK-P8-NEXT:    vmrglh v4, v1, v0
+; CHECK-P8-NEXT:    vmrglh v5, v6, v7
 ; CHECK-P8-NEXT:    vmrglw v2, v2, v3
 ; CHECK-P8-NEXT:    vmrglw v3, v5, v4
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs2, 0(r3)
+; CHECK-P9-NEXT:    lxv vs3, 0(r3)
+; CHECK-P9-NEXT:    xxsldwi vs4, vs3, vs3, 3
+; CHECK-P9-NEXT:    xscvspdpn f4, vs4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    lxv vs0, 48(r3)
+; CHECK-P9-NEXT:    lxv vs1, 32(r3)
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    xxswapd vs4, vs3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    xscvspdpn f4, vs4
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    xscvspdpn f4, vs3
+; CHECK-P9-NEXT:    xxsldwi vs3, vs3, vs3, 1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    mffprwz r3, f4
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f3
 ; CHECK-P9-NEXT:    xxsldwi vs3, vs2, vs2, 3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs3
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    lxv vs0, 48(r3)
-; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs4, 16(r3)
+; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvspdpn f3, vs2
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs2, vs2, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xxsldwi vs2, vs4, vs4, 3
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    xxswapd vs2, vs4
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xscvspdpn f2, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    xxsldwi vs2, vs4, vs4, 1
-; CHECK-P9-NEXT:    xscvspdpn f2, vs2
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs2
 ; CHECK-P9-NEXT:    xxsldwi vs2, vs1, vs1, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvspdpn f2, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs1, vs1, 1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v3, v4, v3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xxsldwi vs1, vs0, vs0, 3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglb v3, v4, v3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xscvspdpn f1, vs0
 ; CHECK-P9-NEXT:    xxsldwi vs0, vs0, vs0, 1
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvspdpn f0, vs0
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v4, v5, v4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v4, v5, v4
-; CHECK-P9-NEXT:    xxswapd v5, vs1
-; CHECK-P9-NEXT:    xxswapd v0, vs0
-; CHECK-P9-NEXT:    vmrglb v5, v5, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r3
+; CHECK-P9-NEXT:    vmrghb v5, v5, v0
 ; CHECK-P9-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)

llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i16_elts.ll

 
 define i32 @test2elt(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglh v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpsxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpsxws f0, v2
 ; CHECK-BE-NEXT:    mffprwz r3, f0
 ; CHECK-BE-NEXT:    xxswapd vs0, v2

 ; CHECK-P8-NEXT:    xscvdpsxws f2, f0
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f2
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    mtfprd f3, r4
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    xxswapd v2, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs3
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xxswapd v5, vs1
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    vmrghh v2, v4, v2
+; CHECK-P8-NEXT:    vmrghh v3, v5, v3
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
-; CHECK-P9-NEXT:    xxswapd v4, vs0
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
+; CHECK-P9-NEXT:    mtvsrd v4, r3
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f2, f1

 ; CHECK-P8-NEXT:    xscvdpsxws f5, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f2
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    mtfprd f4, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs4
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f6, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs5
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs6
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v1, vs7
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v0, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs2
-; CHECK-P8-NEXT:    vmrglh v2, v5, v2
-; CHECK-P8-NEXT:    xxswapd v5, vs0
-; CHECK-P8-NEXT:    vmrglh v3, v0, v3
-; CHECK-P8-NEXT:    vmrglh v4, v6, v4
-; CHECK-P8-NEXT:    vmrglh v5, v5, v1
+; CHECK-P8-NEXT:    vmrghh v2, v0, v2
+; CHECK-P8-NEXT:    vmrghh v3, v1, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    vmrghh v4, v0, v4
+; CHECK-P8-NEXT:    vmrghh v5, v1, v5
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    vmrglw v3, v5, v4
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs3, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    xxswapd v2, vs4
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f4, f3
entry:
   %a = load <8 x double>, <8 x double>* %0, align 64
   %1 = fptoui <8 x double> %a to <8 x i16>
   ret <8 x i16> %1
 }
 
 define void @test16elt(<16 x i16>* noalias nocapture sret %agg.result, <16 x double>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r5, 16
+; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r6, 32
+; CHECK-P8-NEXT:    li r7, 48
 ; CHECK-P8-NEXT:    lxvd2x vs1, r4, r5
 ; CHECK-P8-NEXT:    lxvd2x vs2, r4, r6
-; CHECK-P8-NEXT:    li r6, 48
-; CHECK-P8-NEXT:    lxvd2x vs3, r4, r6
 ; CHECK-P8-NEXT:    li r6, 64
-; CHECK-P8-NEXT:    xscvdpsxws f4, f0
+; CHECK-P8-NEXT:    lxvd2x vs3, r4, r7
 ; CHECK-P8-NEXT:    lxvd2x vs5, r4, r6
-; CHECK-P8-NEXT:    li r6, 80
+; CHECK-P8-NEXT:    li r7, 80
+; CHECK-P8-NEXT:    li r6, 96
+; CHECK-P8-NEXT:    xscvdpsxws f4, f0
+; CHECK-P8-NEXT:    lxvd2x vs7, r4, r7
+; CHECK-P8-NEXT:    lxvd2x vs10, r4, r6
+; CHECK-P8-NEXT:    li r6, 112
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f1
-; CHECK-P8-NEXT:    lxvd2x vs7, r4, r6
-; CHECK-P8-NEXT:    li r6, 96
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f8, f2
-; CHECK-P8-NEXT:    lxvd2x vs9, r4, r6
-; CHECK-P8-NEXT:    li r6, 112
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
-; CHECK-P8-NEXT:    xscvdpsxws f10, f3
-; CHECK-P8-NEXT:    lxvd2x vs11, r4, r6
+; CHECK-P8-NEXT:    xscvdpsxws f9, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
-; CHECK-P8-NEXT:    xscvdpsxws f12, f5
+; CHECK-P8-NEXT:    xscvdpsxws f11, f5
 ; CHECK-P8-NEXT:    xxswapd vs5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f13, f7
+; CHECK-P8-NEXT:    xscvdpsxws f12, f7
 ; CHECK-P8-NEXT:    xxswapd vs7, vs7
-; CHECK-P8-NEXT:    xscvdpsxws v2, f9
-; CHECK-P8-NEXT:    xxswapd vs9, vs9
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws v3, f11
-; CHECK-P8-NEXT:    xxswapd vs11, vs11
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r6, f6
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    mffprwz r7, f4
+; CHECK-P8-NEXT:    lxvd2x vs4, r4, r6
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f13, f10
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f8
+; CHECK-P8-NEXT:    xscvdpsxws f6, f4
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f9
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f11
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs4
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    mtfprd f6, r6
-; CHECK-P8-NEXT:    mffprwz r6, f10
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs6
+; CHECK-P8-NEXT:    mtvsrd v0, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f12
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    xxswapd v0, vs8
-; CHECK-P8-NEXT:    mtfprd f10, r6
-; CHECK-P8-NEXT:    mffprwz r6, f13
-; CHECK-P8-NEXT:    mtfprd f12, r4
-; CHECK-P8-NEXT:    xxswapd v1, vs10
-; CHECK-P8-NEXT:    mfvsrwz r4, v2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f13
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    mtfprd f13, r6
-; CHECK-P8-NEXT:    mfvsrwz r6, v3
-; CHECK-P8-NEXT:    mtvsrd v2, r4
-; CHECK-P8-NEXT:    xxswapd v7, vs13
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xxswapd vs6, vs10
+; CHECK-P8-NEXT:    xscvdpsxws f5, f5
+; CHECK-P8-NEXT:    mtvsrd v7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxswapd vs0, vs4
+; CHECK-P8-NEXT:    mtvsrd v2, r7
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    xxswapd v2, v2
-; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    mtvsrd v3, r6
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v3, v3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    mtfprd f2, r4
+; CHECK-P8-NEXT:    xscvdpsxws f4, f6
+; CHECK-P8-NEXT:    vmrghh v2, v8, v2
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v3, v9, v3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v9, vs1
-; CHECK-P8-NEXT:    mffprwz r6, f3
-; CHECK-P8-NEXT:    xxswapd v10, vs2
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f9
-; CHECK-P8-NEXT:    mtfprd f3, r6
-; CHECK-P8-NEXT:    mffprwz r6, f7
-; CHECK-P8-NEXT:    mtfprd f9, r4
-; CHECK-P8-NEXT:    mffprwz r4, f11
-; CHECK-P8-NEXT:    vmrglh v4, v8, v4
-; CHECK-P8-NEXT:    xxswapd v8, vs3
-; CHECK-P8-NEXT:    vmrglh v5, v9, v5
-; CHECK-P8-NEXT:    xxswapd v9, vs5
-; CHECK-P8-NEXT:    mtfprd f7, r6
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    vmrglh v0, v10, v0
-; CHECK-P8-NEXT:    xxswapd v10, vs7
-; CHECK-P8-NEXT:    vmrglh v1, v8, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs9
-; CHECK-P8-NEXT:    vmrglh v6, v9, v6
-; CHECK-P8-NEXT:    xxswapd v9, vs0
-; CHECK-P8-NEXT:    vmrglh v7, v10, v7
-; CHECK-P8-NEXT:    vmrglh v2, v8, v2
-; CHECK-P8-NEXT:    vmrglh v3, v9, v3
-; CHECK-P8-NEXT:    vmrglw v4, v5, v4
-; CHECK-P8-NEXT:    vmrglw v5, v1, v0
-; CHECK-P8-NEXT:    vmrglw v0, v7, v6
+; CHECK-P8-NEXT:    vmrghh v4, v8, v4
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f7
+; CHECK-P8-NEXT:    vmrghh v5, v9, v5
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    vmrghh v0, v8, v0
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghh v1, v9, v1
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghh v6, v8, v6
+; CHECK-P8-NEXT:    vmrghh v7, v9, v7
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    vmrglw v3, v5, v4
+; CHECK-P8-NEXT:    vmrglw v4, v1, v0
+; CHECK-P8-NEXT:    vmrglw v5, v7, v6
+; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
+; CHECK-P8-NEXT:    stvx v2, 0, r3
 ; CHECK-P8-NEXT:    xxmrgld v3, v5, v4
-; CHECK-P8-NEXT:    stvx v3, 0, r3
-; CHECK-P8-NEXT:    xxmrgld v2, v2, v0
-; CHECK-P8-NEXT:    stvx v2, r3, r5
+; CHECK-P8-NEXT:    stvx v3, r3, r5
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs4, 0(r4)
-; CHECK-P9-NEXT:    lxv vs3, 16(r4)
-; CHECK-P9-NEXT:    lxv vs2, 32(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f5, f4
-; CHECK-P9-NEXT:    lxv vs1, 48(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f6, f3
-; CHECK-P9-NEXT:    lxv vs0, 64(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f7, f2
-; CHECK-P9-NEXT:    xscvdpsxws f8, f1
-; CHECK-P9-NEXT:    xxswapd vs4, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f4, f4
-; CHECK-P9-NEXT:    mffprwz r5, f5
-; CHECK-P9-NEXT:    xscvdpsxws f9, f0
+; CHECK-P9-NEXT:    lxv vs3, 0(r4)
+; CHECK-P9-NEXT:    lxv vs2, 16(r4)
+; CHECK-P9-NEXT:    lxv vs1, 32(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f4, f3
+; CHECK-P9-NEXT:    lxv vs0, 48(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f5, f2
+; CHECK-P9-NEXT:    xscvdpsxws f6, f1
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    xscvdpsxws f7, f0
+; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    mffprwz r5, f4
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    mtfprd f5, r5
-; CHECK-P9-NEXT:    mffprwz r5, f6
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mtfprd f6, r5
+; CHECK-P9-NEXT:    mtvsrd v2, r5
+; CHECK-P9-NEXT:    mffprwz r5, f5
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    mffprwz r5, f6
+; CHECK-P9-NEXT:    mtvsrd v4, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f7
-; CHECK-P9-NEXT:    mtfprd f7, r5
-; CHECK-P9-NEXT:    mffprwz r5, f8
-; CHECK-P9-NEXT:    mtfprd f8, r5
-; CHECK-P9-NEXT:    mffprwz r5, f9
-; CHECK-P9-NEXT:    mtfprd f9, r5
-; CHECK-P9-NEXT:    mffprwz r5, f4
-; CHECK-P9-NEXT:    mtfprd f4, r5
+; CHECK-P9-NEXT:    mtvsrd v5, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    lxv vs3, 64(r4)
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs5
-; CHECK-P9-NEXT:    xxswapd v5, vs8
-; CHECK-P9-NEXT:    xxswapd v0, vs9
-; CHECK-P9-NEXT:    mtfprd f3, r5
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f2
-; CHECK-P9-NEXT:    mtfprd f2, r5
-; CHECK-P9-NEXT:    xxswapd vs0, vs0
-; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    xxswapd v1, vs2
 ; CHECK-P9-NEXT:    lxv vs2, 80(r4)
-; CHECK-P9-NEXT:    xxswapd v3, vs4
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs6
-; CHECK-P9-NEXT:    xxswapd v4, vs3
-; CHECK-P9-NEXT:    xscvdpsxws f3, f2
-; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    vmrghh v2, v2, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f1
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs7
-; CHECK-P9-NEXT:    mtfprd f1, r5
+; CHECK-P9-NEXT:    lxv vs1, 96(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f4, f3
+; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    vmrghh v3, v3, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v1
-; CHECK-P9-NEXT:    xxswapd v1, vs1
-; CHECK-P9-NEXT:    mtfprd f0, r5
-; CHECK-P9-NEXT:    vmrglh v5, v5, v1
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    xxswapd v1, vs0
 ; CHECK-P9-NEXT:    lxv vs0, 112(r4)
-; CHECK-P9-NEXT:    lxv vs1, 96(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    vmrghh v5, v5, v0
+; CHECK-P9-NEXT:    mffprwz r4, f4
+; CHECK-P9-NEXT:    vmrglw v4, v5, v4
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f3
-; CHECK-P9-NEXT:    mtfprd f3, r4
+; CHECK-P9-NEXT:    xscvdpsxws f3, f2
+; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    xxmrgld vs4, v4, v2
+; CHECK-P9-NEXT:    mtvsrd v2, r4
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
+; CHECK-P9-NEXT:    stxv vs4, 0(r3)
+; CHECK-P9-NEXT:    mffprwz r4, f3
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    vmrglw v3, v5, v4
-; CHECK-P9-NEXT:    xxmrgld vs4, v3, v2
-; CHECK-P9-NEXT:    xxswapd v2, vs3
-; CHECK-P9-NEXT:    vmrglh v0, v0, v1
-; CHECK-P9-NEXT:    mtfprd f2, r4
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    mtfprd f2, r4
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f1
-; CHECK-P9-NEXT:    mtfprd f1, r4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r4, f1
-; CHECK-P9-NEXT:    mtfprd f1, r4
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r4
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld vs0, v3, v2
 ; CHECK-P9-NEXT:    stxv vs0, 16(r3)
-; CHECK-P9-NEXT:    stxv vs4, 0(r3)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs4, 48(r4)
 ; CHECK-BE-NEXT:    xscvdpsxws f5, f4
 ; CHECK-BE-NEXT:    xxswapd vs4, vs4
 ; CHECK-BE-NEXT:    lxv vs3, 32(r4)

 
 define i32 @test2elt_signed(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglh v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpsxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpsxws f0, v2
 ; CHECK-BE-NEXT:    mffprwz r3, f0
 ; CHECK-BE-NEXT:    xxswapd vs0, v2

 ; CHECK-P8-NEXT:    xscvdpsxws f2, f0
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f2
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    mtfprd f3, r4
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    xxswapd v2, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs3
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xxswapd v5, vs1
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglh v3, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    vmrghh v2, v4, v2
+; CHECK-P8-NEXT:    vmrghh v3, v5, v3
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
-; CHECK-P9-NEXT:    xxswapd v4, vs0
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
+; CHECK-P9-NEXT:    mtvsrd v4, r3
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f2, f1

 ; CHECK-P8-NEXT:    xscvdpsxws f5, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f2
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    mtfprd f4, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs4
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f6, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs5
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs6
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v1, vs7
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v0, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs2
-; CHECK-P8-NEXT:    vmrglh v2, v5, v2
-; CHECK-P8-NEXT:    xxswapd v5, vs0
-; CHECK-P8-NEXT:    vmrglh v3, v0, v3
-; CHECK-P8-NEXT:    vmrglh v4, v6, v4
-; CHECK-P8-NEXT:    vmrglh v5, v5, v1
+; CHECK-P8-NEXT:    vmrghh v2, v0, v2
+; CHECK-P8-NEXT:    vmrghh v3, v1, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    vmrghh v4, v0, v4
+; CHECK-P8-NEXT:    vmrghh v5, v1, v5
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    vmrglw v3, v5, v4
 ; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs3, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    xxswapd v2, vs4
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f4, f3
entry:
   %a = load <8 x double>, <8 x double>* %0, align 64
   %1 = fptosi <8 x double> %a to <8 x i16>
   ret <8 x i16> %1
 }
 
 define void @test16elt_signed(<16 x i16>* noalias nocapture sret %agg.result, <16 x double>* nocapture readonly) local_unnamed_addr #3 {
 ; CHECK-P8-LABEL: test16elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r5, 16
+; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r6, 32
+; CHECK-P8-NEXT:    li r7, 48
 ; CHECK-P8-NEXT:    lxvd2x vs1, r4, r5
 ; CHECK-P8-NEXT:    lxvd2x vs2, r4, r6
-; CHECK-P8-NEXT:    li r6, 48
-; CHECK-P8-NEXT:    lxvd2x vs3, r4, r6
 ; CHECK-P8-NEXT:    li r6, 64
-; CHECK-P8-NEXT:    xscvdpsxws f4, f0
+; CHECK-P8-NEXT:    lxvd2x vs3, r4, r7
 ; CHECK-P8-NEXT:    lxvd2x vs5, r4, r6
-; CHECK-P8-NEXT:    li r6, 80
+; CHECK-P8-NEXT:    li r7, 80
+; CHECK-P8-NEXT:    li r6, 96
+; CHECK-P8-NEXT:    xscvdpsxws f4, f0
+; CHECK-P8-NEXT:    lxvd2x vs7, r4, r7
+; CHECK-P8-NEXT:    lxvd2x vs10, r4, r6
+; CHECK-P8-NEXT:    li r6, 112
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f1
-; CHECK-P8-NEXT:    lxvd2x vs7, r4, r6
-; CHECK-P8-NEXT:    li r6, 96
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f8, f2
-; CHECK-P8-NEXT:    lxvd2x vs9, r4, r6
-; CHECK-P8-NEXT:    li r6, 112
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
-; CHECK-P8-NEXT:    xscvdpsxws f10, f3
-; CHECK-P8-NEXT:    lxvd2x vs11, r4, r6
+; CHECK-P8-NEXT:    xscvdpsxws f9, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
-; CHECK-P8-NEXT:    xscvdpsxws f12, f5
+; CHECK-P8-NEXT:    xscvdpsxws f11, f5
 ; CHECK-P8-NEXT:    xxswapd vs5, vs5
-; CHECK-P8-NEXT:    xscvdpsxws f13, f7
+; CHECK-P8-NEXT:    xscvdpsxws f12, f7
 ; CHECK-P8-NEXT:    xxswapd vs7, vs7
-; CHECK-P8-NEXT:    xscvdpsxws v2, f9
-; CHECK-P8-NEXT:    xxswapd vs9, vs9
-; CHECK-P8-NEXT:    mffprwz r4, f4
-; CHECK-P8-NEXT:    xscvdpsxws v3, f11
-; CHECK-P8-NEXT:    xxswapd vs11, vs11
-; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mffprwz r6, f6
-; CHECK-P8-NEXT:    mtfprd f4, r4
+; CHECK-P8-NEXT:    mffprwz r7, f4
+; CHECK-P8-NEXT:    lxvd2x vs4, r4, r6
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xscvdpsxws f13, f10
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f8
+; CHECK-P8-NEXT:    xscvdpsxws f6, f4
+; CHECK-P8-NEXT:    mtvsrd v4, r4
+; CHECK-P8-NEXT:    mffprwz r4, f9
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    mffprwz r4, f11
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs4
-; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    mtfprd f6, r6
-; CHECK-P8-NEXT:    mffprwz r6, f10
-; CHECK-P8-NEXT:    mtfprd f8, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs6
+; CHECK-P8-NEXT:    mtvsrd v0, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f12
-; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    xxswapd v0, vs8
-; CHECK-P8-NEXT:    mtfprd f10, r6
-; CHECK-P8-NEXT:    mffprwz r6, f13
-; CHECK-P8-NEXT:    mtfprd f12, r4
-; CHECK-P8-NEXT:    xxswapd v1, vs10
-; CHECK-P8-NEXT:    mfvsrwz r4, v2
+; CHECK-P8-NEXT:    xscvdpsxws f2, f2
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    mffprwz r4, f13
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    mtfprd f13, r6
-; CHECK-P8-NEXT:    mfvsrwz r6, v3
-; CHECK-P8-NEXT:    mtvsrd v2, r4
-; CHECK-P8-NEXT:    xxswapd v7, vs13
+; CHECK-P8-NEXT:    mtvsrd v6, r4
+; CHECK-P8-NEXT:    mffprwz r4, f6
+; CHECK-P8-NEXT:    xxswapd vs6, vs10
+; CHECK-P8-NEXT:    xscvdpsxws f5, f5
+; CHECK-P8-NEXT:    mtvsrd v7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    xxswapd vs0, vs4
+; CHECK-P8-NEXT:    mtvsrd v2, r7
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f1
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    xxswapd v2, v2
-; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    mtvsrd v3, r6
-; CHECK-P8-NEXT:    mffprwz r6, f1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v3, v3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f2
-; CHECK-P8-NEXT:    mtfprd f1, r6
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    mtfprd f2, r4
+; CHECK-P8-NEXT:    xscvdpsxws f4, f6
+; CHECK-P8-NEXT:    vmrghh v2, v8, v2
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f3
+; CHECK-P8-NEXT:    xscvdpsxws f0, f0
+; CHECK-P8-NEXT:    vmrghh v3, v9, v3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    xxswapd v9, vs1
-; CHECK-P8-NEXT:    mffprwz r6, f3
-; CHECK-P8-NEXT:    xxswapd v10, vs2
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    mffprwz r4, f9
-; CHECK-P8-NEXT:    mtfprd f3, r6
-; CHECK-P8-NEXT:    mffprwz r6, f7
-; CHECK-P8-NEXT:    mtfprd f9, r4
-; CHECK-P8-NEXT:    mffprwz r4, f11
-; CHECK-P8-NEXT:    vmrglh v4, v8, v4
-; CHECK-P8-NEXT:    xxswapd v8, vs3
-; CHECK-P8-NEXT:    vmrglh v5, v9, v5
-; CHECK-P8-NEXT:    xxswapd v9, vs5
-; CHECK-P8-NEXT:    mtfprd f7, r6
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    vmrglh v0, v10, v0
-; CHECK-P8-NEXT:    xxswapd v10, vs7
-; CHECK-P8-NEXT:    vmrglh v1, v8, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs9
-; CHECK-P8-NEXT:    vmrglh v6, v9, v6
-; CHECK-P8-NEXT:    xxswapd v9, vs0
-; CHECK-P8-NEXT:    vmrglh v7, v10, v7
-; CHECK-P8-NEXT:    vmrglh v2, v8, v2
-; CHECK-P8-NEXT:    vmrglh v3, v9, v3
-; CHECK-P8-NEXT:    vmrglw v4, v5, v4
-; CHECK-P8-NEXT:    vmrglw v5, v1, v0
-; CHECK-P8-NEXT:    vmrglw v0, v7, v6
+; CHECK-P8-NEXT:    vmrghh v4, v8, v4
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f7
+; CHECK-P8-NEXT:    vmrghh v5, v9, v5
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    mffprwz r4, f4
+; CHECK-P8-NEXT:    vmrghh v0, v8, v0
+; CHECK-P8-NEXT:    mtvsrd v8, r4
+; CHECK-P8-NEXT:    mffprwz r4, f0
+; CHECK-P8-NEXT:    vmrghh v1, v9, v1
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghh v6, v8, v6
+; CHECK-P8-NEXT:    vmrghh v7, v9, v7
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    vmrglw v3, v5, v4
+; CHECK-P8-NEXT:    vmrglw v4, v1, v0
+; CHECK-P8-NEXT:    vmrglw v5, v7, v6
+; CHECK-P8-NEXT:    xxmrgld v2, v3, v2
+; CHECK-P8-NEXT:    stvx v2, 0, r3
 ; CHECK-P8-NEXT:    xxmrgld v3, v5, v4
-; CHECK-P8-NEXT:    stvx v3, 0, r3
-; CHECK-P8-NEXT:    xxmrgld v2, v2, v0
-; CHECK-P8-NEXT:    stvx v2, r3, r5
+; CHECK-P8-NEXT:    stvx v3, r3, r5
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    lxv vs4, 0(r4)
-; CHECK-P9-NEXT:    lxv vs3, 16(r4)
-; CHECK-P9-NEXT:    lxv vs2, 32(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f5, f4
-; CHECK-P9-NEXT:    lxv vs1, 48(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f6, f3
-; CHECK-P9-NEXT:    lxv vs0, 64(r4)
-; CHECK-P9-NEXT:    xscvdpsxws f7, f2
-; CHECK-P9-NEXT:    xscvdpsxws f8, f1
-; CHECK-P9-NEXT:    xxswapd vs4, vs4
-; CHECK-P9-NEXT:    xscvdpsxws f4, f4
-; CHECK-P9-NEXT:    mffprwz r5, f5
-; CHECK-P9-NEXT:    xscvdpsxws f9, f0
+; CHECK-P9-NEXT:    lxv vs3, 0(r4)
+; CHECK-P9-NEXT:    lxv vs2, 16(r4)
+; CHECK-P9-NEXT:    lxv vs1, 32(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f4, f3
+; CHECK-P9-NEXT:    lxv vs0, 48(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f5, f2
+; CHECK-P9-NEXT:    xscvdpsxws f6, f1
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    xscvdpsxws f7, f0
+; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    mffprwz r5, f4
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    mtfprd f5, r5
-; CHECK-P9-NEXT:    mffprwz r5, f6
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    mtfprd f6, r5
+; CHECK-P9-NEXT:    mtvsrd v2, r5
+; CHECK-P9-NEXT:    mffprwz r5, f5
+; CHECK-P9-NEXT:    mtvsrd v3, r5
+; CHECK-P9-NEXT:    mffprwz r5, f6
+; CHECK-P9-NEXT:    mtvsrd v4, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f7
-; CHECK-P9-NEXT:    mtfprd f7, r5
-; CHECK-P9-NEXT:    mffprwz r5, f8
-; CHECK-P9-NEXT:    mtfprd f8, r5
-; CHECK-P9-NEXT:    mffprwz r5, f9
-; CHECK-P9-NEXT:    mtfprd f9, r5
-; CHECK-P9-NEXT:    mffprwz r5, f4
-; CHECK-P9-NEXT:    mtfprd f4, r5
+; CHECK-P9-NEXT:    mtvsrd v5, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f3
+; CHECK-P9-NEXT:    lxv vs3, 64(r4)
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs5
-; CHECK-P9-NEXT:    xxswapd v5, vs8
-; CHECK-P9-NEXT:    xxswapd v0, vs9
-; CHECK-P9-NEXT:    mtfprd f3, r5
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f2
-; CHECK-P9-NEXT:    mtfprd f2, r5
-; CHECK-P9-NEXT:    xxswapd vs0, vs0
-; CHECK-P9-NEXT:    xscvdpsxws f0, f0
-; CHECK-P9-NEXT:    xxswapd v1, vs2
 ; CHECK-P9-NEXT:    lxv vs2, 80(r4)
-; CHECK-P9-NEXT:    xxswapd v3, vs4
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs6
-; CHECK-P9-NEXT:    xxswapd v4, vs3
-; CHECK-P9-NEXT:    xscvdpsxws f3, f2
-; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    vmrghh v2, v2, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f1
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs7
-; CHECK-P9-NEXT:    mtfprd f1, r5
+; CHECK-P9-NEXT:    lxv vs1, 96(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f4, f3
+; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    vmrghh v3, v3, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
 ; CHECK-P9-NEXT:    mffprwz r5, f0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v1
-; CHECK-P9-NEXT:    xxswapd v1, vs1
-; CHECK-P9-NEXT:    mtfprd f0, r5
-; CHECK-P9-NEXT:    vmrglh v5, v5, v1
-; CHECK-P9-NEXT:    xscvdpsxws f2, f2
-; CHECK-P9-NEXT:    xxswapd v1, vs0
 ; CHECK-P9-NEXT:    lxv vs0, 112(r4)
-; CHECK-P9-NEXT:    lxv vs1, 96(r4)
+; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    vmrghh v4, v4, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r5
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    vmrghh v5, v5, v0
+; CHECK-P9-NEXT:    mffprwz r4, f4
+; CHECK-P9-NEXT:    vmrglw v4, v5, v4
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f3
-; CHECK-P9-NEXT:    mtfprd f3, r4
+; CHECK-P9-NEXT:    xscvdpsxws f3, f2
+; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    xxmrgld vs4, v4, v2
+; CHECK-P9-NEXT:    mtvsrd v2, r4
+; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
+; CHECK-P9-NEXT:    stxv vs4, 0(r3)
+; CHECK-P9-NEXT:    mffprwz r4, f3
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    vmrglw v3, v5, v4
-; CHECK-P9-NEXT:    xxmrgld vs4, v3, v2
-; CHECK-P9-NEXT:    xxswapd v2, vs3
-; CHECK-P9-NEXT:    vmrglh v0, v0, v1
-; CHECK-P9-NEXT:    mtfprd f2, r4
-; CHECK-P9-NEXT:    xxswapd v3, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r4, f2
-; CHECK-P9-NEXT:    mtfprd f2, r4
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f1
-; CHECK-P9-NEXT:    mtfprd f1, r4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghh v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r4, f1
-; CHECK-P9-NEXT:    mtfprd f1, r4
+; CHECK-P9-NEXT:    mtvsrd v4, r4
 ; CHECK-P9-NEXT:    mffprwz r4, f0
-; CHECK-P9-NEXT:    vmrglh v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    vmrglh v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v0
-; CHECK-P9-NEXT:    mtfprd f0, r4
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglh v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r4
+; CHECK-P9-NEXT:    vmrghh v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld vs0, v3, v2
 ; CHECK-P9-NEXT:    stxv vs0, 16(r3)
-; CHECK-P9-NEXT:    stxv vs4, 0(r3)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs4, 48(r4)
 ; CHECK-BE-NEXT:    xscvdpsxws f5, f4
 ; CHECK-BE-NEXT:    xxswapd vs4, vs4
 ; CHECK-BE-NEXT:    lxv vs3, 32(r4)

llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i32_elts.ll

 
 define i64 @test2elt(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpuxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpuxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrwz v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglw v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrwz v3, r4
+; CHECK-P8-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpuxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
 ; CHECK-P9-NEXT:    mtvsrws v3, r3
 ; CHECK-P9-NEXT:    xscvdpuxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpuxws f0, v2
 ; CHECK-BE-NEXT:    mffprwz r3, f0
 ; CHECK-BE-NEXT:    xxswapd vs0, v2

 
 define i64 @test2elt_signed(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrwz v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglw v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrwz v3, r4
+; CHECK-P8-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpsxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
 ; CHECK-P9-NEXT:    mtvsrws v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpsxws f0, v2
 ; CHECK-BE-NEXT:    mffprwz r3, f0
 ; CHECK-BE-NEXT:    xxswapd vs0, v2

llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i8_elts.ll

 
 define i16 @test2elt(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglb v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    clrldi r3, r3, 48
 ; CHECK-P8-NEXT:    sth r3, -2(r1)
 ; CHECK-P8-NEXT:    lhz r3, -2(r1)
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpsxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    addi r3, r1, -2
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    vsldoi v2, v2, v2, 8
 ; CHECK-P9-NEXT:    stxsihx v2, 0, r3
 ; CHECK-P9-NEXT:    lhz r3, -2(r1)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpsxws f0, v2

 ; CHECK-P8-NEXT:    xscvdpsxws f2, f0
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f2
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    mtfprd f3, r4
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    xxswapd v2, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs3
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xxswapd v5, vs1
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
-; CHECK-P8-NEXT:    vmrglb v3, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    vmrghb v2, v4, v2
+; CHECK-P8-NEXT:    vmrghb v3, v5, v3
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
-; CHECK-P9-NEXT:    xxswapd v4, vs0
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    li r3, 0
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
+; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f2, f1
 ; CHECK-BE-NEXT:    xxswapd vs1, vs1

 ; CHECK-P8-NEXT:    xscvdpsxws f5, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f2
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    mtfprd f4, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs4
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f6, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs5
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs6
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v1, vs7
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v0, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs2
-; CHECK-P8-NEXT:    vmrglb v2, v5, v2
-; CHECK-P8-NEXT:    xxswapd v5, vs0
-; CHECK-P8-NEXT:    vmrglb v3, v0, v3
-; CHECK-P8-NEXT:    vmrglb v4, v6, v4
-; CHECK-P8-NEXT:    vmrglb v5, v5, v1
+; CHECK-P8-NEXT:    vmrghb v2, v0, v2
+; CHECK-P8-NEXT:    vmrghb v3, v1, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    vmrghb v4, v0, v4
+; CHECK-P8-NEXT:    vmrghb v5, v1, v5
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P8-NEXT:    vmrglh v3, v5, v4
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs3, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    xxswapd v2, vs4
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)

 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    lxvd2x vs11, r3, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f12, f5
 ; CHECK-P8-NEXT:    xxswapd vs5, vs5
 ; CHECK-P8-NEXT:    xscvdpsxws f13, f7
 ; CHECK-P8-NEXT:    xxswapd vs7, vs7
 ; CHECK-P8-NEXT:    xscvdpsxws v2, f9
 ; CHECK-P8-NEXT:    xxswapd vs9, vs9
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws v3, f11
 ; CHECK-P8-NEXT:    xxswapd vs11, vs11
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f6
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mtfprd f4, r3
-; CHECK-P8-NEXT:    mffprwz r3, f8
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs4
-; CHECK-P8-NEXT:    mtfprd f6, r4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f8
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f10
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxswapd v5, vs6
-; CHECK-P8-NEXT:    mtfprd f8, r3
-; CHECK-P8-NEXT:    mffprwz r3, f12
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xxswapd v0, vs8
-; CHECK-P8-NEXT:    mtfprd f10, r4
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mffprwz r3, f12
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f13
 ; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    xxswapd v1, vs10
-; CHECK-P8-NEXT:    mtfprd f12, r3
-; CHECK-P8-NEXT:    mfvsrwz r3, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    mtfprd f13, r4
+; CHECK-P8-NEXT:    mtvsrd v6, r3
+; CHECK-P8-NEXT:    mfvsrwz r3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r4
 ; CHECK-P8-NEXT:    mfvsrwz r4, v3
-; CHECK-P8-NEXT:    mtvsrd v2, r3
-; CHECK-P8-NEXT:    xxswapd v7, vs13
-; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    xxswapd v2, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    mtvsrd v3, r3
+; CHECK-P8-NEXT:    mtvsrd v7, r4
+; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, v3
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v8, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v9, vs1
+; CHECK-P8-NEXT:    vmrghb v4, v8, v4
+; CHECK-P8-NEXT:    vmrghb v5, v9, v5
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f5
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v10, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f5, r3
+; CHECK-P8-NEXT:    vmrghb v0, v8, v0
+; CHECK-P8-NEXT:    vmrghb v1, v9, v1
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f9
-; CHECK-P8-NEXT:    mtfprd f7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f11
-; CHECK-P8-NEXT:    vmrglb v4, v8, v4
-; CHECK-P8-NEXT:    xxswapd v8, vs3
-; CHECK-P8-NEXT:    vmrglb v5, v9, v5
-; CHECK-P8-NEXT:    xxswapd v9, vs5
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    vmrglb v0, v10, v0
-; CHECK-P8-NEXT:    xxswapd v10, vs7
-; CHECK-P8-NEXT:    vmrglb v1, v8, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    vmrglb v6, v9, v6
-; CHECK-P8-NEXT:    xxswapd v9, vs1
-; CHECK-P8-NEXT:    vmrglb v7, v10, v7
-; CHECK-P8-NEXT:    vmrglb v2, v8, v2
-; CHECK-P8-NEXT:    vmrglb v3, v9, v3
+; CHECK-P8-NEXT:    vmrghb v6, v8, v6
+; CHECK-P8-NEXT:    vmrghb v2, v9, v2
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghb v3, v8, v3
+; CHECK-P8-NEXT:    vmrghb v7, v9, v7
 ; CHECK-P8-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P8-NEXT:    vmrglh v5, v1, v0
-; CHECK-P8-NEXT:    vmrglh v0, v7, v6
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
-; CHECK-P8-NEXT:    vmrglw v2, v2, v0
-; CHECK-P8-NEXT:    xxmrgld v2, v2, v3
+; CHECK-P8-NEXT:    vmrglh v2, v2, v6
+; CHECK-P8-NEXT:    vmrglh v3, v7, v3
+; CHECK-P8-NEXT:    vmrglw v4, v5, v4
+; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxmrgld v2, v2, v4
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs7, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f8, f7
 ; CHECK-P9-NEXT:    xxswapd vs7, vs7
 ; CHECK-P9-NEXT:    xscvdpsxws f7, f7
+; CHECK-P9-NEXT:    lxv vs6, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 112(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 96(r3)
 ; CHECK-P9-NEXT:    lxv vs2, 80(r3)
 ; CHECK-P9-NEXT:    lxv vs3, 64(r3)
 ; CHECK-P9-NEXT:    lxv vs4, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs5, 32(r3)
-; CHECK-P9-NEXT:    lxv vs6, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f8
-; CHECK-P9-NEXT:    mtfprd f8, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f7
-; CHECK-P9-NEXT:    xxswapd v2, vs8
-; CHECK-P9-NEXT:    mtfprd f7, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs7
 ; CHECK-P9-NEXT:    xscvdpsxws f7, f6
 ; CHECK-P9-NEXT:    xxswapd vs6, vs6
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f6, f6
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f7
-; CHECK-P9-NEXT:    mtfprd f7, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f6
-; CHECK-P9-NEXT:    mtfprd f6, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs6
 ; CHECK-P9-NEXT:    xscvdpsxws f6, f5
 ; CHECK-P9-NEXT:    xxswapd vs5, vs5
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f5, f5
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f6
-; CHECK-P9-NEXT:    mtfprd f6, r3
-; CHECK-P9-NEXT:    mffprwz r3, f5
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs7
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs6
-; CHECK-P9-NEXT:    mtfprd f5, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs5
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f5
 ; CHECK-P9-NEXT:    xscvdpsxws f5, f4
 ; CHECK-P9-NEXT:    xxswapd vs4, vs4
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f5
-; CHECK-P9-NEXT:    mtfprd f5, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs4
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs5
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs4
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs3
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    vmrglh v3, v4, v3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
-; CHECK-P9-NEXT:    xxswapd v5, vs1
-; CHECK-P9-NEXT:    xxswapd v0, vs0
-; CHECK-P9-NEXT:    vmrglb v5, v5, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r3
+; CHECK-P9-NEXT:    vmrghb v5, v5, v0
 ; CHECK-P9-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs7, 112(r3)

 
 define i16 @test2elt_signed(<2 x double> %a) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f1, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    mffprwz r3, f1
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r4, f0
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    xxswapd v3, vs1
-; CHECK-P8-NEXT:    vmrglb v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    clrldi r3, r3, 48
 ; CHECK-P8-NEXT:    sth r3, -2(r1)
 ; CHECK-P8-NEXT:    lhz r3, -2(r1)
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xscvdpsxws f0, v2
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs0
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    addi r3, r1, -2
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglb v2, v3, v2
+; CHECK-P9-NEXT:    vmrghb v2, v3, v2
 ; CHECK-P9-NEXT:    vsldoi v2, v2, v2, 8
 ; CHECK-P9-NEXT:    stxsihx v2, 0, r3
 ; CHECK-P9-NEXT:    lhz r3, -2(r1)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xscvdpsxws f0, v2

 ; CHECK-P8-NEXT:    xscvdpsxws f2, f0
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P8-NEXT:    mffprwz r3, f2
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    mtfprd f3, r4
+; CHECK-P8-NEXT:    mtvsrd v2, r3
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    xxswapd v2, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs3
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    xxswapd v5, vs1
-; CHECK-P8-NEXT:    vmrglb v2, v3, v2
-; CHECK-P8-NEXT:    vmrglb v3, v5, v4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
+; CHECK-P8-NEXT:    vmrghb v2, v4, v2
+; CHECK-P8-NEXT:    vmrghb v3, v5, v3
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs1, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
 ; CHECK-P9-NEXT:    lxv vs0, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v2, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs1
-; CHECK-P9-NEXT:    xxswapd v4, vs0
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    vmrglh v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    li r3, 0
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
+; CHECK-P9-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs1, 16(r3)
 ; CHECK-BE-NEXT:    xscvdpsxws f2, f1
 ; CHECK-BE-NEXT:    xxswapd vs1, vs1

 ; CHECK-P8-NEXT:    xscvdpsxws f5, f1
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    xscvdpsxws f6, f2
 ; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f3
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f5
-; CHECK-P8-NEXT:    mtfprd f4, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    mffprwz r3, f6
-; CHECK-P8-NEXT:    mtfprd f5, r4
-; CHECK-P8-NEXT:    xxswapd v2, vs4
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f6, r3
-; CHECK-P8-NEXT:    xxswapd v3, vs5
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f0
-; CHECK-P8-NEXT:    mtfprd f7, r4
-; CHECK-P8-NEXT:    xxswapd v4, vs6
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v1, vs7
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v5, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v0, vs1
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    xxswapd v6, vs2
-; CHECK-P8-NEXT:    vmrglb v2, v5, v2
-; CHECK-P8-NEXT:    xxswapd v5, vs0
-; CHECK-P8-NEXT:    vmrglb v3, v0, v3
-; CHECK-P8-NEXT:    vmrglb v4, v6, v4
-; CHECK-P8-NEXT:    vmrglb v5, v5, v1
+; CHECK-P8-NEXT:    vmrghb v2, v0, v2
+; CHECK-P8-NEXT:    vmrghb v3, v1, v3
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mtvsrd v1, r4
+; CHECK-P8-NEXT:    vmrghb v4, v0, v4
+; CHECK-P8-NEXT:    vmrghb v5, v1, v5
 ; CHECK-P8-NEXT:    vmrglh v2, v3, v2
 ; CHECK-P8-NEXT:    vmrglh v3, v5, v4
 ; CHECK-P8-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs3, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
+; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 32(r3)
-; CHECK-P9-NEXT:    lxv vs2, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    xxswapd v2, vs4
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs1
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs1
-; CHECK-P9-NEXT:    xxswapd v5, vs0
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    mtvsrd v5, r3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    vmrglw v2, v3, v2
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs3, 48(r3)

 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    lxvd2x vs11, r3, r4
 ; CHECK-P8-NEXT:    xscvdpsxws f12, f5
 ; CHECK-P8-NEXT:    xxswapd vs5, vs5
 ; CHECK-P8-NEXT:    xscvdpsxws f13, f7
 ; CHECK-P8-NEXT:    xxswapd vs7, vs7
 ; CHECK-P8-NEXT:    xscvdpsxws v2, f9
 ; CHECK-P8-NEXT:    xxswapd vs9, vs9
-; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    xscvdpsxws v3, f11
 ; CHECK-P8-NEXT:    xxswapd vs11, vs11
+; CHECK-P8-NEXT:    mffprwz r3, f4
 ; CHECK-P8-NEXT:    mffprwz r4, f6
 ; CHECK-P8-NEXT:    xscvdpsxws f0, f0
-; CHECK-P8-NEXT:    mtfprd f4, r3
-; CHECK-P8-NEXT:    mffprwz r3, f8
 ; CHECK-P8-NEXT:    xscvdpsxws f1, f1
-; CHECK-P8-NEXT:    xxswapd v4, vs4
-; CHECK-P8-NEXT:    mtfprd f6, r4
+; CHECK-P8-NEXT:    mtvsrd v4, r3
+; CHECK-P8-NEXT:    mffprwz r3, f8
+; CHECK-P8-NEXT:    mtvsrd v5, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f10
 ; CHECK-P8-NEXT:    xscvdpsxws f2, f2
-; CHECK-P8-NEXT:    xxswapd v5, vs6
-; CHECK-P8-NEXT:    mtfprd f8, r3
-; CHECK-P8-NEXT:    mffprwz r3, f12
 ; CHECK-P8-NEXT:    xscvdpsxws f3, f3
-; CHECK-P8-NEXT:    xxswapd v0, vs8
-; CHECK-P8-NEXT:    mtfprd f10, r4
+; CHECK-P8-NEXT:    mtvsrd v0, r3
+; CHECK-P8-NEXT:    mffprwz r3, f12
+; CHECK-P8-NEXT:    mtvsrd v1, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f13
 ; CHECK-P8-NEXT:    xscvdpsxws f5, f5
-; CHECK-P8-NEXT:    xxswapd v1, vs10
-; CHECK-P8-NEXT:    mtfprd f12, r3
-; CHECK-P8-NEXT:    mfvsrwz r3, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f7, f7
-; CHECK-P8-NEXT:    xxswapd v6, vs12
-; CHECK-P8-NEXT:    mtfprd f13, r4
+; CHECK-P8-NEXT:    mtvsrd v6, r3
+; CHECK-P8-NEXT:    mfvsrwz r3, v2
+; CHECK-P8-NEXT:    mtvsrd v2, r4
 ; CHECK-P8-NEXT:    mfvsrwz r4, v3
-; CHECK-P8-NEXT:    mtvsrd v2, r3
-; CHECK-P8-NEXT:    xxswapd v7, vs13
-; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    xscvdpsxws f9, f9
-; CHECK-P8-NEXT:    xxswapd v2, v2
 ; CHECK-P8-NEXT:    xscvdpsxws f11, f11
-; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    mtvsrd v3, r3
+; CHECK-P8-NEXT:    mtvsrd v7, r4
+; CHECK-P8-NEXT:    mffprwz r3, f0
 ; CHECK-P8-NEXT:    mffprwz r4, f1
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    xxswapd v3, v3
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f2
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    xxswapd v8, vs0
 ; CHECK-P8-NEXT:    mffprwz r4, f3
-; CHECK-P8-NEXT:    mtfprd f2, r3
-; CHECK-P8-NEXT:    xxswapd v9, vs1
+; CHECK-P8-NEXT:    vmrghb v4, v8, v4
+; CHECK-P8-NEXT:    vmrghb v5, v9, v5
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f5
-; CHECK-P8-NEXT:    mtfprd f3, r4
-; CHECK-P8-NEXT:    xxswapd v10, vs2
 ; CHECK-P8-NEXT:    mffprwz r4, f7
-; CHECK-P8-NEXT:    mtfprd f5, r3
+; CHECK-P8-NEXT:    vmrghb v0, v8, v0
+; CHECK-P8-NEXT:    vmrghb v1, v9, v1
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
 ; CHECK-P8-NEXT:    mffprwz r3, f9
-; CHECK-P8-NEXT:    mtfprd f7, r4
 ; CHECK-P8-NEXT:    mffprwz r4, f11
-; CHECK-P8-NEXT:    vmrglb v4, v8, v4
-; CHECK-P8-NEXT:    xxswapd v8, vs3
-; CHECK-P8-NEXT:    vmrglb v5, v9, v5
-; CHECK-P8-NEXT:    xxswapd v9, vs5
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    mtfprd f1, r4
-; CHECK-P8-NEXT:    vmrglb v0, v10, v0
-; CHECK-P8-NEXT:    xxswapd v10, vs7
-; CHECK-P8-NEXT:    vmrglb v1, v8, v1
-; CHECK-P8-NEXT:    xxswapd v8, vs0
-; CHECK-P8-NEXT:    vmrglb v6, v9, v6
-; CHECK-P8-NEXT:    xxswapd v9, vs1
-; CHECK-P8-NEXT:    vmrglb v7, v10, v7
-; CHECK-P8-NEXT:    vmrglb v2, v8, v2
-; CHECK-P8-NEXT:    vmrglb v3, v9, v3
+; CHECK-P8-NEXT:    vmrghb v6, v8, v6
+; CHECK-P8-NEXT:    vmrghb v2, v9, v2
+; CHECK-P8-NEXT:    mtvsrd v8, r3
+; CHECK-P8-NEXT:    mtvsrd v9, r4
+; CHECK-P8-NEXT:    vmrghb v3, v8, v3
+; CHECK-P8-NEXT:    vmrghb v7, v9, v7
 ; CHECK-P8-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P8-NEXT:    vmrglh v5, v1, v0
-; CHECK-P8-NEXT:    vmrglh v0, v7, v6
-; CHECK-P8-NEXT:    vmrglh v2, v3, v2
-; CHECK-P8-NEXT:    vmrglw v3, v5, v4
-; CHECK-P8-NEXT:    vmrglw v2, v2, v0
-; CHECK-P8-NEXT:    xxmrgld v2, v2, v3
+; CHECK-P8-NEXT:    vmrglh v2, v2, v6
+; CHECK-P8-NEXT:    vmrglh v3, v7, v3
+; CHECK-P8-NEXT:    vmrglw v4, v5, v4
+; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxmrgld v2, v2, v4
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test16elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    lxv vs7, 0(r3)
 ; CHECK-P9-NEXT:    xscvdpsxws f8, f7
 ; CHECK-P9-NEXT:    xxswapd vs7, vs7
 ; CHECK-P9-NEXT:    xscvdpsxws f7, f7
+; CHECK-P9-NEXT:    lxv vs6, 16(r3)
 ; CHECK-P9-NEXT:    lxv vs0, 112(r3)
 ; CHECK-P9-NEXT:    lxv vs1, 96(r3)
 ; CHECK-P9-NEXT:    lxv vs2, 80(r3)
 ; CHECK-P9-NEXT:    lxv vs3, 64(r3)
 ; CHECK-P9-NEXT:    lxv vs4, 48(r3)
 ; CHECK-P9-NEXT:    lxv vs5, 32(r3)
-; CHECK-P9-NEXT:    lxv vs6, 16(r3)
 ; CHECK-P9-NEXT:    mffprwz r3, f8
-; CHECK-P9-NEXT:    mtfprd f8, r3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f7
-; CHECK-P9-NEXT:    xxswapd v2, vs8
-; CHECK-P9-NEXT:    mtfprd f7, r3
-; CHECK-P9-NEXT:    xxswapd v3, vs7
 ; CHECK-P9-NEXT:    xscvdpsxws f7, f6
 ; CHECK-P9-NEXT:    xxswapd vs6, vs6
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f6, f6
+; CHECK-P9-NEXT:    vmrghb v2, v2, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f7
-; CHECK-P9-NEXT:    mtfprd f7, r3
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f6
-; CHECK-P9-NEXT:    mtfprd f6, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs6
 ; CHECK-P9-NEXT:    xscvdpsxws f6, f5
 ; CHECK-P9-NEXT:    xxswapd vs5, vs5
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f5, f5
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f6
-; CHECK-P9-NEXT:    mtfprd f6, r3
-; CHECK-P9-NEXT:    mffprwz r3, f5
-; CHECK-P9-NEXT:    vmrglb v2, v2, v3
-; CHECK-P9-NEXT:    xxswapd v3, vs7
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
 ; CHECK-P9-NEXT:    vmrglh v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs6
-; CHECK-P9-NEXT:    mtfprd f5, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs5
+; CHECK-P9-NEXT:    mtvsrd v3, r3
+; CHECK-P9-NEXT:    mffprwz r3, f5
 ; CHECK-P9-NEXT:    xscvdpsxws f5, f4
 ; CHECK-P9-NEXT:    xxswapd vs4, vs4
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f4
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f5
-; CHECK-P9-NEXT:    mtfprd f5, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs4
 ; CHECK-P9-NEXT:    xscvdpsxws f4, f3
 ; CHECK-P9-NEXT:    xxswapd vs3, vs3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f3
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs5
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    vmrglh v3, v4, v3
 ; CHECK-P9-NEXT:    mffprwz r3, f4
-; CHECK-P9-NEXT:    mtfprd f4, r3
+; CHECK-P9-NEXT:    vmrglw v2, v3, v2
+; CHECK-P9-NEXT:    mtvsrd v3, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
-; CHECK-P9-NEXT:    xxswapd v4, vs3
 ; CHECK-P9-NEXT:    xscvdpsxws f3, f2
 ; CHECK-P9-NEXT:    xxswapd vs2, vs2
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f2
+; CHECK-P9-NEXT:    vmrghb v3, v3, v4
 ; CHECK-P9-NEXT:    mffprwz r3, f3
-; CHECK-P9-NEXT:    mtfprd f3, r3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs2
 ; CHECK-P9-NEXT:    xscvdpsxws f2, f1
 ; CHECK-P9-NEXT:    xxswapd vs1, vs1
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f1
-; CHECK-P9-NEXT:    vmrglw v2, v3, v2
-; CHECK-P9-NEXT:    xxswapd v3, vs4
-; CHECK-P9-NEXT:    vmrglb v3, v3, v4
-; CHECK-P9-NEXT:    xxswapd v4, vs3
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
-; CHECK-P9-NEXT:    vmrglh v3, v4, v3
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    mffprwz r3, f2
-; CHECK-P9-NEXT:    mtfprd f2, r3
+; CHECK-P9-NEXT:    vmrglh v3, v4, v3
+; CHECK-P9-NEXT:    mtvsrd v4, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    xxswapd v4, vs2
-; CHECK-P9-NEXT:    mtfprd f1, r3
-; CHECK-P9-NEXT:    xxswapd v5, vs1
 ; CHECK-P9-NEXT:    xscvdpsxws f1, f0
 ; CHECK-P9-NEXT:    xxswapd vs0, vs0
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    xscvdpsxws f0, f0
+; CHECK-P9-NEXT:    vmrghb v4, v4, v5
 ; CHECK-P9-NEXT:    mffprwz r3, f1
-; CHECK-P9-NEXT:    mtfprd f1, r3
+; CHECK-P9-NEXT:    mtvsrd v5, r3
 ; CHECK-P9-NEXT:    mffprwz r3, f0
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    vmrglb v4, v4, v5
-; CHECK-P9-NEXT:    xxswapd v5, vs1
-; CHECK-P9-NEXT:    xxswapd v0, vs0
-; CHECK-P9-NEXT:    vmrglb v5, v5, v0
+; CHECK-P9-NEXT:    mtvsrd v0, r3
+; CHECK-P9-NEXT:    vmrghb v5, v5, v0
 ; CHECK-P9-NEXT:    vmrglh v4, v5, v4
 ; CHECK-P9-NEXT:    vmrglw v3, v4, v3
 ; CHECK-P9-NEXT:    xxmrgld v2, v3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test16elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    lxv vs7, 112(r3)

llvm/test/CodeGen/PowerPC/vec_conv_i16_to_fp32_elts.ll

 ; CHECK-P8-NEXT:    clrlwi r4, r4, 16
 ; CHECK-P8-NEXT:    clrlwi r3, r3, 16
 ; CHECK-P8-NEXT:    mtfprwz f0, r4
 ; CHECK-P8-NEXT:    mtfprwz f1, r3
 ; CHECK-P8-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvuxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
 ; CHECK-P9-NEXT:    vextuhrx r3, r3, v2
 ; CHECK-P9-NEXT:    clrlwi r3, r3, 16
 ; CHECK-P9-NEXT:    mtfprwz f0, r3
 ; CHECK-P9-NEXT:    li r3, 2
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P9-NEXT:    vextuhrx r3, r3, v2
 ; CHECK-P9-NEXT:    clrlwi r3, r3, 16
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    mtfprwz f0, r3
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrws v2, r3
 ; CHECK-BE-NEXT:    li r3, 2
 ; CHECK-BE-NEXT:    vextuhlx r3, r3, v2
entry:
   %1 = uitofp <2 x i16> %0 to <2 x float>
   %2 = bitcast <2 x float> %1 to i64
   ret i64 %2
 }
 
 define <4 x float> @test4elt(i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI1_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI1_0@toc@l
-; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v4, v2, v3
+; CHECK-P8-NEXT:    xxlxor v2, v2, v2
+; CHECK-P8-NEXT:    mtvsrd v3, r3
+; CHECK-P8-NEXT:    vmrghh v2, v2, v3
 ; CHECK-P8-NEXT:    xvcvuxwsp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
-; CHECK-P9-NEXT:    addi r3, r3, .LCPI1_0@toc@l
-; CHECK-P9-NEXT:    lxvx v3, 0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    xxlxor v4, v4, v4
-; CHECK-P9-NEXT:    vperm v2, v4, v2, v3
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    xxlxor v3, v3, v3
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
 ; CHECK-P9-NEXT:    xvcvuxwsp v2, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrd v2, r3
 ; CHECK-BE-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
 ; CHECK-BE-NEXT:    addi r3, r3, .LCPI1_0@toc@l
 ; CHECK-BE-NEXT:    lxvx v3, 0, r3
 ; CHECK-BE-NEXT:    xxlxor v4, v4, v4
 ; CHECK-BE-NEXT:    vperm v2, v2, v4, v3
 ; CHECK-BE-NEXT:    xvcvuxwsp v2, v2
 ; CHECK-BE-NEXT:    blr
 entry:
   %0 = bitcast i64 %a.coerce to <4 x i16>
   %1 = uitofp <4 x i16> %0 to <4 x float>
   ret <4 x float> %1
 }
 
 define void @test8elt(<8 x float>* noalias nocapture sret %agg.result, <8 x i16> %a) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
-; CHECK-P8-NEXT:    addis r5, r2, .LCPI2_1@toc@ha
-; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI2_0@toc@l
-; CHECK-P8-NEXT:    lvx v3, 0, r4
-; CHECK-P8-NEXT:    addi r4, r5, .LCPI2_1@toc@l
-; CHECK-P8-NEXT:    lvx v5, 0, r4
+; CHECK-P8-NEXT:    xxlxor v3, v3, v3
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    vperm v3, v4, v2, v3
-; CHECK-P8-NEXT:    vperm v2, v4, v2, v5
-; CHECK-P8-NEXT:    xvcvuxwsp v3, v3
+; CHECK-P8-NEXT:    vmrglh v4, v3, v2
+; CHECK-P8-NEXT:    vmrghh v2, v3, v2
+; CHECK-P8-NEXT:    xvcvuxwsp v3, v4
 ; CHECK-P8-NEXT:    xvcvuxwsp v2, v2
 ; CHECK-P8-NEXT:    stvx v3, 0, r3
 ; CHECK-P8-NEXT:    stvx v2, r3, r4
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
-; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_0@toc@l
-; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxlxor v4, v4, v4
-; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_1@toc@ha
-; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_1@toc@l
-; CHECK-P9-NEXT:    vperm v3, v4, v2, v3
-; CHECK-P9-NEXT:    xvcvuxwsp vs0, v3
-; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    vperm v2, v4, v2, v3
-; CHECK-P9-NEXT:    stxv vs0, 0(r3)
+; CHECK-P9-NEXT:    xxlxor v3, v3, v3
+; CHECK-P9-NEXT:    vmrglh v4, v3, v2
+; CHECK-P9-NEXT:    vmrghh v2, v3, v2
+; CHECK-P9-NEXT:    xvcvuxwsp vs0, v4
 ; CHECK-P9-NEXT:    xvcvuxwsp vs1, v2
 ; CHECK-P9-NEXT:    stxv vs1, 16(r3)
+; CHECK-P9-NEXT:    stxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test8elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
 ; CHECK-BE-NEXT:    addi r4, r4, .LCPI2_0@toc@l
 ; CHECK-BE-NEXT:    lxvx v3, 0, r4
 ; CHECK-BE-NEXT:    xxlxor v4, v4, v4

 ; CHECK-P8-NEXT:    extsh r4, r4
 ; CHECK-P8-NEXT:    extsh r3, r3
 ; CHECK-P8-NEXT:    mtfprwa f0, r4
 ; CHECK-P8-NEXT:    mtfprwa f1, r3
 ; CHECK-P8-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvsxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
 ; CHECK-P9-NEXT:    vextuhrx r3, r3, v2
 ; CHECK-P9-NEXT:    extsh r3, r3
 ; CHECK-P9-NEXT:    mtfprwa f0, r3
 ; CHECK-P9-NEXT:    li r3, 2
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P9-NEXT:    vextuhrx r3, r3, v2
 ; CHECK-P9-NEXT:    extsh r3, r3
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    mtfprwa f0, r3
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrws v2, r3
 ; CHECK-BE-NEXT:    li r3, 2
 ; CHECK-BE-NEXT:    vextuhlx r3, r3, v2
entry:
   %1 = sitofp <2 x i16> %0 to <2 x float>
   %2 = bitcast <2 x float> %1 to i64
   ret i64 %2
 }
 
 define <4 x float> @test4elt_signed(i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    mtfprd f0, r3
+; CHECK-P8-NEXT:    mtvsrd v2, r3
 ; CHECK-P8-NEXT:    vspltisw v3, 8
-; CHECK-P8-NEXT:    xxswapd v2, vs0
+; CHECK-P8-NEXT:    vmrghh v2, v2, v2
 ; CHECK-P8-NEXT:    vadduwm v3, v3, v3
-; CHECK-P8-NEXT:    vmrglh v2, v2, v2
 ; CHECK-P8-NEXT:    vslw v2, v2, v3
 ; CHECK-P8-NEXT:    vsraw v2, v2, v3
 ; CHECK-P8-NEXT:    xvcvsxwsp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r3
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vmrglh v2, v2, v2
+; CHECK-P9-NEXT:    mtvsrd v2, r3
+; CHECK-P9-NEXT:    vmrghh v2, v2, v2
 ; CHECK-P9-NEXT:    vextsh2w v2, v2
 ; CHECK-P9-NEXT:    xvcvsxwsp v2, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test4elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrd v2, r3
 ; CHECK-BE-NEXT:    vmrghh v2, v2, v2

llvm/test/CodeGen/PowerPC/vec_conv_i16_to_fp64_elts.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr8 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-P8
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr9 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-P9
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr9 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-BE
 
 define <2 x double> @test2elt(i32 %a.coerce) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI0_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI0_0@toc@l
+; CHECK-P8-NEXT:    mtvsrwz v2, r3
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI0_0@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
+; CHECK-P8-NEXT:    lvx v3, 0, r4
 ; CHECK-P8-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-P8-NEXT:    xvcvuxddp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI0_0@toc@ha
entry:
   %1 = uitofp <2 x i16> %0 to <2 x double>
   ret <2 x double> %1
 }
 
 define void @test4elt(<4 x double>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI1_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI1_1@toc@ha
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI1_1@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v2, r4
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI1_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI1_1@toc@l
+; CHECK-P8-NEXT:    addi r4, r6, .LCPI1_1@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
+; CHECK-P8-NEXT:    lvx v3, 0, r5
 ; CHECK-P8-NEXT:    lvx v5, 0, r4
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    vperm v2, v4, v3, v2
-; CHECK-P8-NEXT:    vperm v3, v4, v3, v5
-; CHECK-P8-NEXT:    xvcvuxddp vs0, v2
-; CHECK-P8-NEXT:    xvcvuxddp vs1, v3
+; CHECK-P8-NEXT:    vperm v3, v4, v2, v3
+; CHECK-P8-NEXT:    vperm v2, v4, v2, v5
+; CHECK-P8-NEXT:    xvcvuxddp vs0, v3
+; CHECK-P8-NEXT:    xvcvuxddp vs1, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    stxvd2x vs1, r3, r4
 ; CHECK-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI1_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI1_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xxlxor v4, v4, v4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI1_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI1_1@toc@l
 ; CHECK-P9-NEXT:    vperm v3, v4, v2, v3
 ; CHECK-P9-NEXT:    xvcvuxddp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-P9-NEXT:    stxv vs0, 0(r3)
entry:
   store <16 x double> %1, <16 x double>* %agg.result, align 128
   ret void
 }
 
 define <2 x double> @test2elt_signed(i32 %a.coerce) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI4_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI4_0@toc@l
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
+; CHECK-P8-NEXT:    mtvsrwz v3, r3
 ; CHECK-P8-NEXT:    addis r3, r2, .LCPI4_1@toc@ha
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI4_0@toc@l
 ; CHECK-P8-NEXT:    addi r3, r3, .LCPI4_1@toc@l
+; CHECK-P8-NEXT:    lvx v2, 0, r4
 ; CHECK-P8-NEXT:    lxvd2x vs0, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v2, v2, v3
+; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    xxswapd v3, vs0
 ; CHECK-P8-NEXT:    vsld v2, v2, v3
 ; CHECK-P8-NEXT:    vsrad v2, v2, v3
 ; CHECK-P8-NEXT:    xvcvsxddp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
entry:
   %1 = sitofp <2 x i16> %0 to <2 x double>
   ret <2 x double> %1
 }
 
 define void @test4elt_signed(<4 x double>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI5_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI5_2@toc@ha
-; CHECK-P8-NEXT:    addi r5, r5, .LCPI5_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI5_2@toc@l
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    lvx v4, 0, r4
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI5_2@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI5_1@toc@ha
+; CHECK-P8-NEXT:    addi r5, r5, .LCPI5_0@toc@l
 ; CHECK-P8-NEXT:    addi r4, r4, .LCPI5_1@toc@l
+; CHECK-P8-NEXT:    lvx v2, 0, r5
+; CHECK-P8-NEXT:    addi r5, r6, .LCPI5_2@toc@l
 ; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r4, 16
+; CHECK-P8-NEXT:    lvx v4, 0, r5
 ; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    vperm v3, v3, v3, v4
 ; CHECK-P8-NEXT:    xxswapd v4, vs0
 ; CHECK-P8-NEXT:    vsld v2, v2, v4
 ; CHECK-P8-NEXT:    vsld v3, v3, v4
 ; CHECK-P8-NEXT:    vsrad v2, v2, v4
 ; CHECK-P8-NEXT:    vsrad v3, v3, v4
 ; CHECK-P8-NEXT:    xvcvsxddp vs0, v2
 ; CHECK-P8-NEXT:    xvcvsxddp vs1, v3
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    stxvd2x vs1, r3, r4
 ; CHECK-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI5_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI5_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI5_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI5_1@toc@l
+; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    vextsh2d v3, v3
 ; CHECK-P9-NEXT:    xvcvsxddp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    vperm v2, v2, v2, v3
 ; CHECK-P9-NEXT:    stxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    vextsh2d v2, v2
 ; CHECK-P9-NEXT:    xvcvsxddp vs1, v2
 ; CHECK-P9-NEXT:    stxv vs1, 16(r3)

llvm/test/CodeGen/PowerPC/vec_conv_i64_to_fp32_elts.ll

 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xxlor vs1, v2, v2
 ; CHECK-P8-NEXT:    xscvuxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    xxlor vs0, v2, v2
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxswapd vs0, v2
 ; CHECK-BE-NEXT:    xxlor vs1, v2, v2
 ; CHECK-BE-NEXT:    xscvuxdsp f1, f1

 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    xxlor vs1, v2, v2
 ; CHECK-P8-NEXT:    xscvsxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    xxswapd vs0, v2
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    xxlor vs0, v2, v2
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxswapd vs0, v2
 ; CHECK-BE-NEXT:    xxlor vs1, v2, v2
 ; CHECK-BE-NEXT:    xscvsxdsp f1, f1

llvm/test/CodeGen/PowerPC/vec_conv_i8_to_fp32_elts.ll

 ; CHECK-P8-NEXT:    clrlwi r4, r4, 24
 ; CHECK-P8-NEXT:    clrlwi r3, r3, 24
 ; CHECK-P8-NEXT:    mtfprwz f0, r4
 ; CHECK-P8-NEXT:    mtfprwz f1, r3
 ; CHECK-P8-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvuxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
 ; CHECK-P9-NEXT:    vextubrx r3, r3, v2
 ; CHECK-P9-NEXT:    clrlwi r3, r3, 24
 ; CHECK-P9-NEXT:    mtfprwz f0, r3
 ; CHECK-P9-NEXT:    li r3, 1
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P9-NEXT:    vextubrx r3, r3, v2
 ; CHECK-P9-NEXT:    clrlwi r3, r3, 24
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    mtfprwz f0, r3
 ; CHECK-P9-NEXT:    xscvuxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrws v2, r3
 ; CHECK-BE-NEXT:    li r3, 1
 ; CHECK-BE-NEXT:    vextublx r3, r3, v2
entry:
   %2 = bitcast <2 x float> %1 to i64
   ret i64 %2
 }
 
 define <4 x float> @test4elt(i32 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI1_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI1_0@toc@l
+; CHECK-P8-NEXT:    mtvsrwz v2, r3
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI1_0@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
+; CHECK-P8-NEXT:    lvx v3, 0, r4
 ; CHECK-P8-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-P8-NEXT:    xvcvuxwsp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI1_0@toc@ha
entry:
   %1 = uitofp <4 x i8> %0 to <4 x float>
   ret <4 x float> %1
 }
 
 define void @test8elt(<8 x float>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI2_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI2_1@toc@ha
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI2_1@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v2, r4
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI2_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI2_1@toc@l
+; CHECK-P8-NEXT:    addi r4, r6, .LCPI2_1@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
+; CHECK-P8-NEXT:    lvx v3, 0, r5
 ; CHECK-P8-NEXT:    lvx v5, 0, r4
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    vperm v2, v4, v3, v2
-; CHECK-P8-NEXT:    vperm v3, v4, v3, v5
-; CHECK-P8-NEXT:    xvcvuxwsp v2, v2
+; CHECK-P8-NEXT:    vperm v3, v4, v2, v3
+; CHECK-P8-NEXT:    vperm v2, v4, v2, v5
 ; CHECK-P8-NEXT:    xvcvuxwsp v3, v3
-; CHECK-P8-NEXT:    stvx v2, 0, r3
-; CHECK-P8-NEXT:    stvx v3, r3, r4
+; CHECK-P8-NEXT:    xvcvuxwsp v2, v2
+; CHECK-P8-NEXT:    stvx v3, 0, r3
+; CHECK-P8-NEXT:    stvx v2, r3, r4
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xxlxor v4, v4, v4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_1@toc@l
 ; CHECK-P9-NEXT:    vperm v3, v4, v2, v3
 ; CHECK-P9-NEXT:    xvcvuxwsp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-P9-NEXT:    stxv vs0, 0(r3)

 ; CHECK-P8-NEXT:    extsb r4, r4
 ; CHECK-P8-NEXT:    extsb r3, r3
 ; CHECK-P8-NEXT:    mtfprwa f0, r4
 ; CHECK-P8-NEXT:    mtfprwa f1, r3
 ; CHECK-P8-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P8-NEXT:    xscvsxdsp f1, f1
 ; CHECK-P8-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P8-NEXT:    xscvdpspn vs1, f1
-; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-P8-NEXT:    vmrglw v2, v3, v2
+; CHECK-P8-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P8-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-P8-NEXT:    vmrghw v2, v3, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, v2
 ; CHECK-P8-NEXT:    mffprd r3, f0
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    li r3, 0
 ; CHECK-P9-NEXT:    vextubrx r3, r3, v2
 ; CHECK-P9-NEXT:    extsb r3, r3
 ; CHECK-P9-NEXT:    mtfprwa f0, r3
 ; CHECK-P9-NEXT:    li r3, 1
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
 ; CHECK-P9-NEXT:    vextubrx r3, r3, v2
 ; CHECK-P9-NEXT:    extsb r3, r3
-; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 1
+; CHECK-P9-NEXT:    xxsldwi v3, vs0, vs0, 3
 ; CHECK-P9-NEXT:    mtfprwa f0, r3
 ; CHECK-P9-NEXT:    xscvsxdsp f0, f0
 ; CHECK-P9-NEXT:    xscvdpspn vs0, f0
-; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-P9-NEXT:    vmrglw v2, v2, v3
+; CHECK-P9-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-P9-NEXT:    vmrghw v2, v2, v3
 ; CHECK-P9-NEXT:    mfvsrld r3, v2
 ; CHECK-P9-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: test2elt_signed:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    mtvsrws v2, r3
 ; CHECK-BE-NEXT:    li r3, 1
 ; CHECK-BE-NEXT:    vextublx r3, r3, v2
entry:
   %2 = bitcast <2 x float> %1 to i64
   ret i64 %2
 }
 
 define <4 x float> @test4elt_signed(i32 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI5_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI5_0@toc@l
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v2, v2, v3
+; CHECK-P8-NEXT:    mtvsrwz v3, r3
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI5_0@toc@l
+; CHECK-P8-NEXT:    lvx v2, 0, r4
+; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    vspltisw v3, 12
 ; CHECK-P8-NEXT:    vadduwm v3, v3, v3
 ; CHECK-P8-NEXT:    vslw v2, v2, v3
 ; CHECK-P8-NEXT:    vsraw v2, v2, v3
 ; CHECK-P8-NEXT:    xvcvsxwsp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt_signed:
entry:
   %1 = sitofp <4 x i8> %0 to <4 x float>
   ret <4 x float> %1
 }
 
 define void @test8elt_signed(<8 x float>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #2 {
 ; CHECK-P8-LABEL: test8elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI6_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI6_1@toc@ha
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI6_1@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v3, r4
 ; CHECK-P8-NEXT:    vspltisw v5, 12
+; CHECK-P8-NEXT:    li r4, 16
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI6_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI6_1@toc@l
 ; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    lvx v4, 0, r4
-; CHECK-P8-NEXT:    li r4, 16
+; CHECK-P8-NEXT:    addi r5, r6, .LCPI6_1@toc@l
+; CHECK-P8-NEXT:    lvx v4, 0, r5
 ; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    vperm v3, v3, v3, v4
 ; CHECK-P8-NEXT:    vadduwm v4, v5, v5
 ; CHECK-P8-NEXT:    vslw v2, v2, v4
 ; CHECK-P8-NEXT:    vslw v3, v3, v4
 ; CHECK-P8-NEXT:    vsraw v2, v2, v4
 ; CHECK-P8-NEXT:    vsraw v3, v3, v4
 ; CHECK-P8-NEXT:    xvcvsxwsp v2, v2
 ; CHECK-P8-NEXT:    xvcvsxwsp v3, v3
 ; CHECK-P8-NEXT:    stvx v2, 0, r3
 ; CHECK-P8-NEXT:    stvx v3, r3, r4
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI6_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI6_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI6_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI6_1@toc@l
+; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    vextsb2w v3, v3
 ; CHECK-P9-NEXT:    xvcvsxwsp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    vperm v2, v2, v2, v3
 ; CHECK-P9-NEXT:    stxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    vextsb2w v2, v2
 ; CHECK-P9-NEXT:    xvcvsxwsp vs1, v2
 ; CHECK-P9-NEXT:    stxv vs1, 16(r3)

llvm/test/CodeGen/PowerPC/vec_conv_i8_to_fp64_elts.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr8 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-P8
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr9 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-P9
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
 ; RUN:     -mcpu=pwr9 -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr < %s | \
 ; RUN: FileCheck %s --check-prefix=CHECK-BE
 
 define <2 x double> @test2elt(i16 %a.coerce) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI0_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI0_0@toc@l
+; CHECK-P8-NEXT:    mtvsrwz v2, r3
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI0_0@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
+; CHECK-P8-NEXT:    lvx v3, 0, r4
 ; CHECK-P8-NEXT:    vperm v2, v4, v2, v3
 ; CHECK-P8-NEXT:    xvcvuxddp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    mtvsrws v2, r3
 ; CHECK-P9-NEXT:    addis r3, r2, .LCPI0_0@toc@ha
entry:
   %1 = uitofp <2 x i8> %0 to <2 x double>
   ret <2 x double> %1
 }
 
 define void @test4elt(<4 x double>* noalias nocapture sret %agg.result, i32 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI1_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI1_1@toc@ha
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI1_1@toc@ha
+; CHECK-P8-NEXT:    mtvsrwz v2, r4
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI1_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI1_1@toc@l
+; CHECK-P8-NEXT:    addi r4, r6, .LCPI1_1@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
+; CHECK-P8-NEXT:    lvx v3, 0, r5
 ; CHECK-P8-NEXT:    lvx v5, 0, r4
 ; CHECK-P8-NEXT:    li r4, 16
-; CHECK-P8-NEXT:    vperm v2, v4, v3, v2
-; CHECK-P8-NEXT:    vperm v3, v4, v3, v5
-; CHECK-P8-NEXT:    xvcvuxddp vs0, v2
-; CHECK-P8-NEXT:    xvcvuxddp vs1, v3
+; CHECK-P8-NEXT:    vperm v3, v4, v2, v3
+; CHECK-P8-NEXT:    vperm v2, v4, v2, v5
+; CHECK-P8-NEXT:    xvcvuxddp vs0, v3
+; CHECK-P8-NEXT:    xvcvuxddp vs1, v2
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
 ; CHECK-P8-NEXT:    stxvd2x vs1, r3, r4
 ; CHECK-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test4elt:
 ; CHECK-P9:       # %bb.0: # %entry
entry:
   store <4 x double> %1, <4 x double>* %agg.result, align 32
   ret void
 }
 
 define void @test8elt(<8 x double>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test8elt:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI2_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI2_2@toc@ha
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI2_2@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v2, r4
+; CHECK-P8-NEXT:    addis r4, r2, .LCPI2_3@toc@ha
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI2_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI2_2@toc@l
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI2_3@toc@l
 ; CHECK-P8-NEXT:    xxlxor v4, v4, v4
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    addis r5, r2, .LCPI2_3@toc@ha
-; CHECK-P8-NEXT:    lvx v5, 0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI2_1@toc@ha
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    addi r5, r5, .LCPI2_3@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI2_1@toc@l
-; CHECK-P8-NEXT:    lvx v0, 0, r5
-; CHECK-P8-NEXT:    lvx v1, 0, r4
+; CHECK-P8-NEXT:    lvx v3, 0, r5
+; CHECK-P8-NEXT:    addi r5, r6, .LCPI2_2@toc@l
+; CHECK-P8-NEXT:    lvx v0, 0, r4
 ; CHECK-P8-NEXT:    li r4, 48
+; CHECK-P8-NEXT:    lvx v5, 0, r5
+; CHECK-P8-NEXT:    addis r5, r2, .LCPI2_1@toc@ha
+; CHECK-P8-NEXT:    addi r5, r5, .LCPI2_1@toc@l
+; CHECK-P8-NEXT:    lvx v1, 0, r5
+; CHECK-P8-NEXT:    vperm v0, v4, v2, v0
 ; CHECK-P8-NEXT:    li r5, 32
-; CHECK-P8-NEXT:    vperm v2, v4, v3, v2
-; CHECK-P8-NEXT:    vperm v5, v4, v3, v5
-; CHECK-P8-NEXT:    vperm v0, v4, v3, v0
-; CHECK-P8-NEXT:    vperm v3, v4, v3, v1
-; CHECK-P8-NEXT:    xvcvuxddp vs0, v2
-; CHECK-P8-NEXT:    xvcvuxddp vs1, v5
+; CHECK-P8-NEXT:    vperm v3, v4, v2, v3
+; CHECK-P8-NEXT:    vperm v5, v4, v2, v5
+; CHECK-P8-NEXT:    vperm v2, v4, v2, v1
 ; CHECK-P8-NEXT:    xvcvuxddp vs2, v0
-; CHECK-P8-NEXT:    xvcvuxddp vs3, v3
+; CHECK-P8-NEXT:    xvcvuxddp vs0, v3
+; CHECK-P8-NEXT:    xvcvuxddp vs1, v5
+; CHECK-P8-NEXT:    xvcvuxddp vs3, v2
+; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xxswapd vs0, vs0
 ; CHECK-P8-NEXT:    xxswapd vs1, vs1
-; CHECK-P8-NEXT:    xxswapd vs2, vs2
 ; CHECK-P8-NEXT:    xxswapd vs3, vs3
 ; CHECK-P8-NEXT:    stxvd2x vs2, r3, r4
 ; CHECK-P8-NEXT:    li r4, 16
 ; CHECK-P8-NEXT:    stxvd2x vs1, r3, r5
 ; CHECK-P8-NEXT:    stxvd2x vs3, r3, r4
 ; CHECK-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
 ; CHECK-P9-NEXT:    xxlxor v4, v4, v4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_1@toc@l
 ; CHECK-P9-NEXT:    vperm v3, v4, v2, v3
 ; CHECK-P9-NEXT:    xvcvuxddp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI2_2@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI2_2@toc@l
entry:
   store <16 x double> %0, <16 x double>* %agg.result, align 128
   ret void
 }
 
 define <2 x double> @test2elt_signed(i16 %a.coerce) local_unnamed_addr #0 {
 ; CHECK-P8-LABEL: test2elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI4_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r3
-; CHECK-P8-NEXT:    addi r3, r4, .LCPI4_0@toc@l
-; CHECK-P8-NEXT:    xxswapd v2, vs0
-; CHECK-P8-NEXT:    lvx v3, 0, r3
+; CHECK-P8-NEXT:    mtvsrwz v3, r3
 ; CHECK-P8-NEXT:    addis r3, r2, .LCPI4_1@toc@ha
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI4_0@toc@l
 ; CHECK-P8-NEXT:    addi r3, r3, .LCPI4_1@toc@l
+; CHECK-P8-NEXT:    lvx v2, 0, r4
 ; CHECK-P8-NEXT:    lxvd2x vs0, 0, r3
-; CHECK-P8-NEXT:    vperm v2, v2, v2, v3
+; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    xxswapd v3, vs0
 ; CHECK-P8-NEXT:    vsld v2, v2, v3
 ; CHECK-P8-NEXT:    vsrad v2, v2, v3
 ; CHECK-P8-NEXT:    xvcvsxddp v2, v2
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test2elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
entry:
   %1 = sitofp <2 x i8> %0 to <2 x double>
   ret <2 x double> %1
 }
 
 define void @test4elt_signed(<4 x double>* noalias nocapture sret %agg.result, i32 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test4elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI5_0@toc@ha
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI5_2@toc@ha
-; CHECK-P8-NEXT:    addi r5, r5, .LCPI5_0@toc@l
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI5_2@toc@l
-; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    lvx v4, 0, r4
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI5_2@toc@ha
+; CHECK-P8-NEXT:    mtvsrwz v3, r4
 ; CHECK-P8-NEXT:    addis r4, r2, .LCPI5_1@toc@ha
+; CHECK-P8-NEXT:    addi r5, r5, .LCPI5_0@toc@l
 ; CHECK-P8-NEXT:    addi r4, r4, .LCPI5_1@toc@l
+; CHECK-P8-NEXT:    lvx v2, 0, r5
+; CHECK-P8-NEXT:    addi r5, r6, .LCPI5_2@toc@l
 ; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r4, 16
+; CHECK-P8-NEXT:    lvx v4, 0, r5
 ; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
 ; CHECK-P8-NEXT:    vperm v3, v3, v3, v4
 ; CHECK-P8-NEXT:    xxswapd v4, vs0
 ; CHECK-P8-NEXT:    vsld v2, v2, v4
 ; CHECK-P8-NEXT:    vsld v3, v3, v4
 ; CHECK-P8-NEXT:    vsrad v2, v2, v4
 ; CHECK-P8-NEXT:    vsrad v3, v3, v4
 ; CHECK-P8-NEXT:    xvcvsxddp vs0, v2
entry:
   %1 = sitofp <4 x i8> %0 to <4 x double>
   store <4 x double> %1, <4 x double>* %agg.result, align 32
   ret void
 }
 
 define void @test8elt_signed(<8 x double>* noalias nocapture sret %agg.result, i64 %a.coerce) local_unnamed_addr #1 {
 ; CHECK-P8-LABEL: test8elt_signed:
 ; CHECK-P8:       # %bb.0: # %entry
-; CHECK-P8-NEXT:    mtfprd f0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI6_2@toc@ha
 ; CHECK-P8-NEXT:    addis r5, r2, .LCPI6_0@toc@ha
-; CHECK-P8-NEXT:    addis r6, r2, .LCPI6_3@toc@ha
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI6_2@toc@l
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI6_2@toc@ha
+; CHECK-P8-NEXT:    mtvsrd v3, r4
+; CHECK-P8-NEXT:    addis r4, r2, .LCPI6_1@toc@ha
 ; CHECK-P8-NEXT:    addi r5, r5, .LCPI6_0@toc@l
-; CHECK-P8-NEXT:    addi r6, r6, .LCPI6_3@toc@l
-; CHECK-P8-NEXT:    lvx v4, 0, r4
-; CHECK-P8-NEXT:    addis r4, r2, .LCPI6_4@toc@ha
+; CHECK-P8-NEXT:    addi r6, r6, .LCPI6_2@toc@l
+; CHECK-P8-NEXT:    addi r4, r4, .LCPI6_1@toc@l
 ; CHECK-P8-NEXT:    lvx v2, 0, r5
-; CHECK-P8-NEXT:    xxswapd v3, vs0
-; CHECK-P8-NEXT:    lvx v5, 0, r6
-; CHECK-P8-NEXT:    addis r5, r2, .LCPI6_1@toc@ha
-; CHECK-P8-NEXT:    addi r4, r4, .LCPI6_4@toc@l
-; CHECK-P8-NEXT:    addi r5, r5, .LCPI6_1@toc@l
-; CHECK-P8-NEXT:    lvx v0, 0, r4
-; CHECK-P8-NEXT:    lxvd2x vs0, 0, r5
+; CHECK-P8-NEXT:    addis r5, r2, .LCPI6_3@toc@ha
+; CHECK-P8-NEXT:    lvx v4, 0, r6
+; CHECK-P8-NEXT:    addis r6, r2, .LCPI6_4@toc@ha
+; CHECK-P8-NEXT:    lxvd2x vs0, 0, r4
 ; CHECK-P8-NEXT:    li r4, 48
-; CHECK-P8-NEXT:    li r5, 32
+; CHECK-P8-NEXT:    addi r5, r5, .LCPI6_3@toc@l
+; CHECK-P8-NEXT:    lvx v5, 0, r5
+; CHECK-P8-NEXT:    addi r5, r6, .LCPI6_4@toc@l
+; CHECK-P8-NEXT:    lvx v0, 0, r5
 ; CHECK-P8-NEXT:    vperm v2, v3, v3, v2
+; CHECK-P8-NEXT:    li r5, 32
 ; CHECK-P8-NEXT:    vperm v4, v3, v3, v4
 ; CHECK-P8-NEXT:    vperm v5, v3, v3, v5
 ; CHECK-P8-NEXT:    vperm v3, v3, v3, v0
 ; CHECK-P8-NEXT:    xxswapd v0, vs0
 ; CHECK-P8-NEXT:    vsld v2, v2, v0
 ; CHECK-P8-NEXT:    vsld v4, v4, v0
 ; CHECK-P8-NEXT:    vsld v5, v5, v0
 ; CHECK-P8-NEXT:    vsld v3, v3, v0

 ; CHECK-P8-NEXT:    li r4, 16
 ; CHECK-P8-NEXT:    stxvd2x vs1, r3, r5
 ; CHECK-P8-NEXT:    stxvd2x vs3, r3, r4
 ; CHECK-P8-NEXT:    stxvd2x vs0, 0, r3
 ; CHECK-P8-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: test8elt_signed:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mtfprd f0, r4
+; CHECK-P9-NEXT:    mtvsrd v2, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI6_0@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI6_0@toc@l
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
-; CHECK-P9-NEXT:    xxswapd v2, vs0
-; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI6_1@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI6_1@toc@l
+; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    vextsb2d v3, v3
 ; CHECK-P9-NEXT:    xvcvsxddp vs0, v3
 ; CHECK-P9-NEXT:    lxvx v3, 0, r4
 ; CHECK-P9-NEXT:    addis r4, r2, .LCPI6_2@toc@ha
 ; CHECK-P9-NEXT:    addi r4, r4, .LCPI6_2@toc@l
 ; CHECK-P9-NEXT:    vperm v3, v2, v2, v3
 ; CHECK-P9-NEXT:    stxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    vextsb2d v3, v3

llvm/test/CodeGen/PowerPC/vector-constrained-fp-intrinsics.ll

 ; PC64LE-NEXT:    addis 3, 2, .LCPI2_4@toc@ha
 ; PC64LE-NEXT:    xsdivsp 2, 2, 0
 ; PC64LE-NEXT:    addi 3, 3, .LCPI2_4@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
 ; PC64LE-NEXT:    xsdivsp 0, 3, 0
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    xscvdpspn 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
-; PC64LE-NEXT:    xxsldwi 34, 1, 1, 1
-; PC64LE-NEXT:    xxsldwi 35, 2, 2, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 1, 1, 3
+; PC64LE-NEXT:    xxsldwi 35, 2, 2, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fdiv_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI2_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI2_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI2_1@toc@ha
 ; PC64LE9-NEXT:    lfs 1, .LCPI2_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI2_2@toc@ha
 ; PC64LE9-NEXT:    xsdivsp 1, 1, 0
 ; PC64LE9-NEXT:    lfs 2, .LCPI2_2@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI2_3@toc@ha
 ; PC64LE9-NEXT:    lfs 3, .LCPI2_3@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI2_4@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI2_4@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
 ; PC64LE9-NEXT:    xsdivsp 2, 2, 0
 ; PC64LE9-NEXT:    xsdivsp 0, 3, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    blr
 entry:
   %div = call <3 x float> @llvm.experimental.constrained.fdiv.v3f32(
            <3 x float> <float 1.000000e+00, float 2.000000e+00, float 3.000000e+00>,
            <3 x float> <float 1.000000e+01, float 1.000000e+01, float 1.000000e+01>,
            metadata !"round.dynamic",
            metadata !"fpexcept.strict") #1

 ; PC64LE-NEXT:    lfs 1, .LCPI7_3@toc@l(3)
 ; PC64LE-NEXT:    bl fmodf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 29
 ; PC64LE-NEXT:    addis 3, 2, .LCPI7_4@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI7_4@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 30
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    addi 1, 1, 64
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr

 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI7_3@toc@ha
 ; PC64LE9-NEXT:    fmr 29, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI7_3@toc@l(3)
 ; PC64LE9-NEXT:    fmr 2, 31
 ; PC64LE9-NEXT:    bl fmodf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 29
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI7_4@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI7_4@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    addi 1, 1, 64
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    blr

 ; PC64LE-NEXT:    addis 3, 2, .LCPI12_4@toc@ha
 ; PC64LE-NEXT:    xsmulsp 2, 0, 2
 ; PC64LE-NEXT:    addi 3, 3, .LCPI12_4@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
 ; PC64LE-NEXT:    xsmulsp 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    xscvdpspn 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
-; PC64LE-NEXT:    xxsldwi 34, 1, 1, 1
-; PC64LE-NEXT:    xxsldwi 35, 2, 2, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 1, 1, 3
+; PC64LE-NEXT:    xxsldwi 35, 2, 2, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fmul_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI12_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI12_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI12_1@toc@ha
 ; PC64LE9-NEXT:    lfs 1, .LCPI12_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI12_2@toc@ha
 ; PC64LE9-NEXT:    xsmulsp 0, 1, 0
 ; PC64LE9-NEXT:    lfs 2, .LCPI12_2@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI12_3@toc@ha
 ; PC64LE9-NEXT:    lfs 3, .LCPI12_3@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI12_4@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI12_4@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
 ; PC64LE9-NEXT:    xsmulsp 2, 1, 2
 ; PC64LE9-NEXT:    xsmulsp 1, 1, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 1, 1
-; PC64LE9-NEXT:    xxsldwi 34, 1, 1, 1
+; PC64LE9-NEXT:    xxsldwi 34, 1, 1, 3
 ; PC64LE9-NEXT:    xscvdpspn 1, 2
-; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    blr
 entry:
   %mul = call <3 x float> @llvm.experimental.constrained.fmul.v3f32(
            <3 x float> <float 0x7FF0000000000000, float 0x7FF0000000000000,
                         float 0x7FF0000000000000>,
            <3 x float> <float 1.000000e+00, float 1.000000e+01, float 1.000000e+02>,
            metadata !"round.dynamic",

 ; PC64LE-NEXT:    addi 3, 3, .LCPI17_3@toc@l
 ; PC64LE-NEXT:    xsaddsp 1, 0, 1
 ; PC64LE-NEXT:    lvx 4, 0, 3
 ; PC64LE-NEXT:    xsaddsp 2, 0, 2
 ; PC64LE-NEXT:    xsaddsp 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    xscvdpspn 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
-; PC64LE-NEXT:    xxsldwi 34, 1, 1, 1
-; PC64LE-NEXT:    xxsldwi 35, 2, 2, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 1, 1, 3
+; PC64LE-NEXT:    xxsldwi 35, 2, 2, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fadd_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI17_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI17_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI17_1@toc@ha
 ; PC64LE9-NEXT:    lfs 2, .LCPI17_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI17_2@toc@ha
 ; PC64LE9-NEXT:    xsaddsp 2, 0, 2
 ; PC64LE9-NEXT:    lfs 3, .LCPI17_2@toc@l(3)
 ; PC64LE9-NEXT:    xxlxor 1, 1, 1
 ; PC64LE9-NEXT:    xsaddsp 1, 0, 1
 ; PC64LE9-NEXT:    xsaddsp 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI17_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI17_3@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    blr
 entry:
   %add = call <3 x float> @llvm.experimental.constrained.fadd.v3f32(
            <3 x float> <float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000,
                         float 0xFFFFFFFFE0000000>,
            <3 x float> <float 2.0, float 1.0, float 0.0>,
            metadata !"round.dynamic",

 ; PC64LE-NEXT:    addi 3, 3, .LCPI22_3@toc@l
 ; PC64LE-NEXT:    xssubsp 1, 0, 1
 ; PC64LE-NEXT:    lvx 4, 0, 3
 ; PC64LE-NEXT:    xssubsp 2, 0, 2
 ; PC64LE-NEXT:    xssubsp 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    xscvdpspn 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
-; PC64LE-NEXT:    xxsldwi 34, 1, 1, 1
-; PC64LE-NEXT:    xxsldwi 35, 2, 2, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 1, 1, 3
+; PC64LE-NEXT:    xxsldwi 35, 2, 2, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fsub_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI22_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI22_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI22_1@toc@ha
 ; PC64LE9-NEXT:    lfs 2, .LCPI22_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI22_2@toc@ha
 ; PC64LE9-NEXT:    xssubsp 2, 0, 2
 ; PC64LE9-NEXT:    lfs 3, .LCPI22_2@toc@l(3)
 ; PC64LE9-NEXT:    xxlxor 1, 1, 1
 ; PC64LE9-NEXT:    xssubsp 1, 0, 1
 ; PC64LE9-NEXT:    xssubsp 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI22_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI22_3@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    blr
 entry:
   %sub = call <3 x float> @llvm.experimental.constrained.fsub.v3f32(
            <3 x float> <float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000,
                         float 0xFFFFFFFFE0000000>,
            <3 x float> <float 2.0, float 1.0, float 0.0>,
            metadata !"round.dynamic",

 ; PC64LE-NEXT:    xssqrtsp 0, 0
 ; PC64LE-NEXT:    lfs 2, .LCPI27_0@toc@l(3)
 ; PC64LE-NEXT:    addis 3, 2, .LCPI27_3@toc@ha
 ; PC64LE-NEXT:    xssqrtsp 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI27_3@toc@l
 ; PC64LE-NEXT:    xssqrtsp 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
 ; PC64LE-NEXT:    xscvdpspn 1, 1
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 2
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_sqrt_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI27_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI27_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI27_1@toc@ha
 ; PC64LE9-NEXT:    lfs 1, .LCPI27_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI27_2@toc@ha
 ; PC64LE9-NEXT:    xssqrtsp 0, 0
 ; PC64LE9-NEXT:    lfs 2, .LCPI27_2@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI27_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI27_3@toc@l
 ; PC64LE9-NEXT:    xssqrtsp 1, 1
 ; PC64LE9-NEXT:    xssqrtsp 2, 2
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 1, 1
 ; PC64LE9-NEXT:    xscvdpspn 2, 2
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
-; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE9-NEXT:    xxsldwi 34, 2, 2, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
+; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE9-NEXT:    xxsldwi 34, 2, 2, 3
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    blr
 entry:
   %sqrt = call <3 x float> @llvm.experimental.constrained.sqrt.v3f32(
                               <3 x float> <float 42.0, float 43.0, float 44.0>,
                               metadata !"round.dynamic",
                               metadata !"fpexcept.strict") #1

 ; PC64LE-NEXT:    lfs 1, .LCPI32_3@toc@l(3)
 ; PC64LE-NEXT:    bl powf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 29
 ; PC64LE-NEXT:    addis 3, 2, .LCPI32_4@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI32_4@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 30
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    addi 1, 1, 64
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr

 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI32_3@toc@ha
 ; PC64LE9-NEXT:    fmr 29, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI32_3@toc@l(3)
 ; PC64LE9-NEXT:    fmr 2, 31
 ; PC64LE9-NEXT:    bl powf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 29
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI32_4@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI32_4@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    addi 1, 1, 64
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    blr

 ; PC64LE-NEXT:    lfs 1, .LCPI37_2@toc@l(3)
 ; PC64LE-NEXT:    bl __powisf2
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI37_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI37_3@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI37_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI37_2@toc@l(3)
 ; PC64LE9-NEXT:    li 4, 3
 ; PC64LE9-NEXT:    bl __powisf2
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI37_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI37_3@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI42_2@toc@l(3)
 ; PC64LE-NEXT:    bl sinf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI42_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI42_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl sinf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI42_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI42_2@toc@l(3)
 ; PC64LE9-NEXT:    bl sinf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI42_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI42_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI47_2@toc@l(3)
 ; PC64LE-NEXT:    bl cosf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI47_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI47_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl cosf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI47_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI47_2@toc@l(3)
 ; PC64LE9-NEXT:    bl cosf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI47_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI47_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI52_2@toc@l(3)
 ; PC64LE-NEXT:    bl expf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI52_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI52_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl expf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI52_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI52_2@toc@l(3)
 ; PC64LE9-NEXT:    bl expf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI52_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI52_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI57_2@toc@l(3)
 ; PC64LE-NEXT:    bl exp2f
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI57_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI57_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl exp2f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI57_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI57_2@toc@l(3)
 ; PC64LE9-NEXT:    bl exp2f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI57_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI57_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI62_2@toc@l(3)
 ; PC64LE-NEXT:    bl logf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI62_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI62_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl logf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI62_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI62_2@toc@l(3)
 ; PC64LE9-NEXT:    bl logf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI62_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI62_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI67_2@toc@l(3)
 ; PC64LE-NEXT:    bl log10f
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI67_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI67_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl log10f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI67_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI67_2@toc@l(3)
 ; PC64LE9-NEXT:    bl log10f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI67_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI67_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI72_2@toc@l(3)
 ; PC64LE-NEXT:    bl log2f
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI72_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI72_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl log2f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI72_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI72_2@toc@l(3)
 ; PC64LE9-NEXT:    bl log2f
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI72_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI72_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    xsrdpic 0, 0
 ; PC64LE-NEXT:    lfs 2, .LCPI77_0@toc@l(3)
 ; PC64LE-NEXT:    addis 3, 2, .LCPI77_3@toc@ha
 ; PC64LE-NEXT:    xsrdpic 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI77_3@toc@l
 ; PC64LE-NEXT:    xsrdpic 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
 ; PC64LE-NEXT:    xscvdpspn 1, 1
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 2
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_rint_v3f32:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI77_0@toc@ha
 ; PC64LE9-NEXT:    lfs 0, .LCPI77_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI77_1@toc@ha
 ; PC64LE9-NEXT:    lfs 1, .LCPI77_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI77_2@toc@ha
 ; PC64LE9-NEXT:    xsrdpic 0, 0
 ; PC64LE9-NEXT:    lfs 2, .LCPI77_2@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI77_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI77_3@toc@l
 ; PC64LE9-NEXT:    xsrdpic 1, 1
 ; PC64LE9-NEXT:    xsrdpic 2, 2
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 1, 1
 ; PC64LE9-NEXT:    xscvdpspn 2, 2
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
-; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE9-NEXT:    xxsldwi 34, 2, 2, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
+; PC64LE9-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE9-NEXT:    xxsldwi 34, 2, 2, 3
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    blr
  entry:
   %rint = call <3 x float> @llvm.experimental.constrained.rint.v3f32(
                               <3 x float> <float 42.0, float 43.0, float 44.0>,
                               metadata !"round.dynamic",
                               metadata !"fpexcept.strict") #1

 ; PC64LE-NEXT:    fmr 30, 1
 ; PC64LE-NEXT:    lfs 1, .LCPI82_2@toc@l(3)
 ; PC64LE-NEXT:    bl nearbyintf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 30
 ; PC64LE-NEXT:    addis 3, 2, .LCPI82_3@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI82_3@toc@l
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 31
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    addi 1, 1, 48
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr
 ;

 ; PC64LE9-NEXT:    bl nearbyintf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI82_2@toc@ha
 ; PC64LE9-NEXT:    fmr 30, 1
 ; PC64LE9-NEXT:    lfs 1, .LCPI82_2@toc@l(3)
 ; PC64LE9-NEXT:    bl nearbyintf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 31
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI82_3@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI82_3@toc@l
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    addi 1, 1, 48
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    blr
 entry:

 ; PC64LE-NEXT:    lfs 2, .LCPI87_4@toc@l(3)
 ; PC64LE-NEXT:    bl fmaxf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 29
 ; PC64LE-NEXT:    addis 3, 2, .LCPI87_5@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI87_5@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 30
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    addi 1, 1, 64
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr

 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI87_4@toc@ha
 ; PC64LE9-NEXT:    lfs 2, .LCPI87_4@toc@l(3)
 ; PC64LE9-NEXT:    fmr 29, 1
 ; PC64LE9-NEXT:    fmr 1, 31
 ; PC64LE9-NEXT:    bl fmaxf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 29
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI87_5@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI87_5@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    addi 1, 1, 64
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    blr

 ; PC64LE-NEXT:    lfs 2, .LCPI92_4@toc@l(3)
 ; PC64LE-NEXT:    bl fminf
 ; PC64LE-NEXT:    nop
 ; PC64LE-NEXT:    xscvdpspn 0, 29
 ; PC64LE-NEXT:    addis 3, 2, .LCPI92_5@toc@ha
 ; PC64LE-NEXT:    xscvdpspn 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI92_5@toc@l
 ; PC64LE-NEXT:    lvx 4, 0, 3
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 30
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 2, 3
-; PC64LE-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 2, 3
+; PC64LE-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE-NEXT:    addi 1, 1, 64
 ; PC64LE-NEXT:    ld 0, 16(1)
 ; PC64LE-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE-NEXT:    mtlr 0
 ; PC64LE-NEXT:    blr

 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI92_4@toc@ha
 ; PC64LE9-NEXT:    lfs 2, .LCPI92_4@toc@l(3)
 ; PC64LE9-NEXT:    fmr 29, 1
 ; PC64LE9-NEXT:    fmr 1, 31
 ; PC64LE9-NEXT:    bl fminf
 ; PC64LE9-NEXT:    nop
 ; PC64LE9-NEXT:    xscvdpspn 0, 1
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 29
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    xscvdpspn 0, 30
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI92_5@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI92_5@toc@l
 ; PC64LE9-NEXT:    lxvx 36, 0, 3
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 3, 2, 4
 ; PC64LE9-NEXT:    addi 1, 1, 64
 ; PC64LE9-NEXT:    ld 0, 16(1)
 ; PC64LE9-NEXT:    lfd 31, -8(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    lfd 30, -16(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    mtlr 0
 ; PC64LE9-NEXT:    lfd 29, -24(1) # 8-byte Folded Reload
 ; PC64LE9-NEXT:    blr

 ; PC64LE-NEXT:    addis 3, 2, .LCPI96_0@toc@ha
 ; PC64LE-NEXT:    addis 4, 2, .LCPI96_1@toc@ha
 ; PC64LE-NEXT:    lfd 0, .LCPI96_0@toc@l(3)
 ; PC64LE-NEXT:    lfd 1, .LCPI96_1@toc@l(4)
 ; PC64LE-NEXT:    xsrsp 0, 0
 ; PC64LE-NEXT:    xsrsp 1, 1
 ; PC64LE-NEXT:    xscvdpspn 0, 0
 ; PC64LE-NEXT:    xscvdpspn 1, 1
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fptrunc_v2f64:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI96_0@toc@ha
 ; PC64LE9-NEXT:    lfd 0, .LCPI96_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI96_1@toc@ha
 ; PC64LE9-NEXT:    xsrsp 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    lfd 0, .LCPI96_1@toc@l(3)
 ; PC64LE9-NEXT:    xsrsp 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    blr
 entry:
   %result = call <2 x float> @llvm.experimental.constrained.fptrunc.v2f32.v2f64(
                                 <2 x double><double 42.1, double 42.2>,
                                 metadata !"round.dynamic",
                                 metadata !"fpexcept.strict") #1
   ret <2 x float> %result
 }

 ; PC64LE-NEXT:    xsrsp 0, 0
 ; PC64LE-NEXT:    lfd 2, .LCPI97_3@toc@l(3)
 ; PC64LE-NEXT:    addis 3, 2, .LCPI97_2@toc@ha
 ; PC64LE-NEXT:    xsrsp 1, 1
 ; PC64LE-NEXT:    addi 3, 3, .LCPI97_2@toc@l
 ; PC64LE-NEXT:    xsrsp 2, 2
 ; PC64LE-NEXT:    xscvdpspn 0, 0
 ; PC64LE-NEXT:    xscvdpspn 1, 1
-; PC64LE-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE-NEXT:    xscvdpspn 0, 2
-; PC64LE-NEXT:    xxsldwi 35, 1, 1, 1
-; PC64LE-NEXT:    vmrglw 2, 3, 2
+; PC64LE-NEXT:    xxsldwi 35, 1, 1, 3
+; PC64LE-NEXT:    vmrghw 2, 3, 2
 ; PC64LE-NEXT:    lvx 3, 0, 3
-; PC64LE-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE-NEXT:    blr
 ;
 ; PC64LE9-LABEL: constrained_vector_fptrunc_v3f64:
 ; PC64LE9:       # %bb.0: # %entry
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI97_0@toc@ha
 ; PC64LE9-NEXT:    lfd 0, .LCPI97_0@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI97_1@toc@ha
 ; PC64LE9-NEXT:    xsrsp 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 34, 0, 0, 3
 ; PC64LE9-NEXT:    lfd 0, .LCPI97_1@toc@l(3)
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI97_2@toc@ha
 ; PC64LE9-NEXT:    addi 3, 3, .LCPI97_2@toc@l
 ; PC64LE9-NEXT:    xsrsp 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 1
-; PC64LE9-NEXT:    vmrglw 2, 3, 2
+; PC64LE9-NEXT:    xxsldwi 35, 0, 0, 3
+; PC64LE9-NEXT:    vmrghw 2, 3, 2
 ; PC64LE9-NEXT:    lxvx 35, 0, 3
 ; PC64LE9-NEXT:    addis 3, 2, .LCPI97_3@toc@ha
 ; PC64LE9-NEXT:    lfd 0, .LCPI97_3@toc@l(3)
 ; PC64LE9-NEXT:    xsrsp 0, 0
 ; PC64LE9-NEXT:    xscvdpspn 0, 0
-; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 1
+; PC64LE9-NEXT:    xxsldwi 36, 0, 0, 3
 ; PC64LE9-NEXT:    vperm 2, 4, 2, 3
 ; PC64LE9-NEXT:    blr
 entry:
   %result = call <3 x float> @llvm.experimental.constrained.fptrunc.v3f32.v3f64(
                                 <3 x double><double 42.1, double 42.2,
                                              double 42.3>,
                                 metadata !"round.dynamic",
                                 metadata !"fpexcept.strict") #1

llvm/test/CodeGen/PowerPC/vsx.ll

 ; CHECK-LE-LABEL: test44:
 ; CHECK-LE:       # %bb.0:
 ; CHECK-LE-NEXT:    xxswapd vs0, v2
 ; CHECK-LE-NEXT:    xxlor vs1, v2, v2
 ; CHECK-LE-NEXT:    xscvuxdsp f1, f1
 ; CHECK-LE-NEXT:    xscvuxdsp f0, f0
 ; CHECK-LE-NEXT:    xscvdpspn vs1, f1
 ; CHECK-LE-NEXT:    xscvdpspn vs0, f0
-; CHECK-LE-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-LE-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-LE-NEXT:    vmrglw v2, v3, v2
+; CHECK-LE-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-LE-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-LE-NEXT:    vmrghw v2, v3, v2
 ; CHECK-LE-NEXT:    blr
   %v = uitofp <2 x i64> %a to <2 x float>
   ret <2 x float> %v
 
 ; FIXME: The code quality here looks pretty bad.
 }
 
 define <2 x float> @test45(<2 x i64> %a) {

 ; CHECK-LE-LABEL: test45:
 ; CHECK-LE:       # %bb.0:
 ; CHECK-LE-NEXT:    xxswapd vs0, v2
 ; CHECK-LE-NEXT:    xxlor vs1, v2, v2
 ; CHECK-LE-NEXT:    xscvsxdsp f1, f1
 ; CHECK-LE-NEXT:    xscvsxdsp f0, f0
 ; CHECK-LE-NEXT:    xscvdpspn vs1, f1
 ; CHECK-LE-NEXT:    xscvdpspn vs0, f0
-; CHECK-LE-NEXT:    xxsldwi v3, vs1, vs1, 1
-; CHECK-LE-NEXT:    xxsldwi v2, vs0, vs0, 1
-; CHECK-LE-NEXT:    vmrglw v2, v3, v2
+; CHECK-LE-NEXT:    xxsldwi v3, vs1, vs1, 3
+; CHECK-LE-NEXT:    xxsldwi v2, vs0, vs0, 3
+; CHECK-LE-NEXT:    vmrghw v2, v3, v2
 ; CHECK-LE-NEXT:    blr
   %v = sitofp <2 x i64> %a to <2 x float>
   ret <2 x float> %v
 
 ; FIXME: The code quality here looks pretty bad.
 }
 
 define <2 x i64> @test46(<2 x float> %a) {

 ; CHECK-FISL-NEXT:    addis r3, r2, .LCPI65_0@toc@ha
 ; CHECK-FISL-NEXT:    addi r3, r3, .LCPI65_0@toc@l
 ; CHECK-FISL-NEXT:    lxvw4x v3, 0, r3
 ; CHECK-FISL-NEXT:    vadduwm v2, v2, v3
 ; CHECK-FISL-NEXT:    blr
 ;
 ; CHECK-LE-LABEL: test80:
 ; CHECK-LE:       # %bb.0:
-; CHECK-LE-NEXT:    mtfprd f0, r3
+; CHECK-LE-NEXT:    mtfprwz f0, r3
 ; CHECK-LE-NEXT:    addis r4, r2, .LCPI65_0@toc@ha
 ; CHECK-LE-NEXT:    addi r3, r4, .LCPI65_0@toc@l
-; CHECK-LE-NEXT:    xxswapd vs0, vs0
+; CHECK-LE-NEXT:    xxspltw v2, vs0, 1
 ; CHECK-LE-NEXT:    lvx v3, 0, r3
-; CHECK-LE-NEXT:    xxspltw v2, vs0, 3
 ; CHECK-LE-NEXT:    vadduwm v2, v2, v3
 ; CHECK-LE-NEXT:    blr
   %b1 = insertelement <2 x i32> undef, i32 %v, i32 0
   %b2 = shufflevector <2 x i32> %b1, <2 x i32> undef, <2 x i32> zeroinitializer
   %i = add <2 x i32> %b2, <i32 2, i32 3>
   ret <2 x i32> %i
 
 

llvm/test/CodeGen/PowerPC/vsx_insert_extract_le.ll

 ; RUN:   --check-prefix=CHECK-P9 --implicit-check-not xxswapd
 
 define <2 x double> @testi0(<2 x double>* %p1, double* %p2) {
 ; CHECK-LABEL: testi0:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    lxvd2x vs0, 0, r3
 ; CHECK-NEXT:    lfdx f1, 0, r4
 ; CHECK-NEXT:    xxswapd vs0, vs0
-; CHECK-NEXT:    xxspltd vs1, vs1, 0
-; CHECK-NEXT:    xxpermdi v2, vs0, vs1, 1
+; CHECK-NEXT:    xxmrghd v2, vs0, vs1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-VECTOR-LABEL: testi0:
 ; CHECK-P9-VECTOR:       # %bb.0:
 ; CHECK-P9-VECTOR-NEXT:    lxvd2x vs0, 0, r3
 ; CHECK-P9-VECTOR-NEXT:    lfdx f1, 0, r4
-; CHECK-P9-VECTOR-NEXT:    xxspltd vs1, vs1, 0
 ; CHECK-P9-VECTOR-NEXT:    xxswapd vs0, vs0
-; CHECK-P9-VECTOR-NEXT:    xxpermdi v2, vs0, vs1, 1
+; CHECK-P9-VECTOR-NEXT:    xxmrghd v2, vs0, vs1
 ; CHECK-P9-VECTOR-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi0:
 ; CHECK-P9:       # %bb.0:
 ; CHECK-P9-NEXT:    lfd f1, 0(r4)
 ; CHECK-P9-NEXT:    lxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    xxswapd vs1, f1
 ; CHECK-P9-NEXT:    xxpermdi v2, vs0, vs1, 1
 ; CHECK-P9-NEXT:    blr
   %v = load <2 x double>, <2 x double>* %p1
   %s = load double, double* %p2
   %r = insertelement <2 x double> %v, double %s, i32 0
   ret <2 x double> %r
 
 
 }
 
 define <2 x double> @testi1(<2 x double>* %p1, double* %p2) {
 ; CHECK-LABEL: testi1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    lxvd2x vs0, 0, r3
 ; CHECK-NEXT:    lfdx f1, 0, r4
 ; CHECK-NEXT:    xxswapd vs0, vs0
-; CHECK-NEXT:    xxspltd vs1, vs1, 0
-; CHECK-NEXT:    xxmrgld v2, vs1, vs0
+; CHECK-NEXT:    xxpermdi v2, vs1, vs0, 1
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-P9-VECTOR-LABEL: testi1:
 ; CHECK-P9-VECTOR:       # %bb.0:
 ; CHECK-P9-VECTOR-NEXT:    lxvd2x vs0, 0, r3
 ; CHECK-P9-VECTOR-NEXT:    lfdx f1, 0, r4
-; CHECK-P9-VECTOR-NEXT:    xxspltd vs1, vs1, 0
 ; CHECK-P9-VECTOR-NEXT:    xxswapd vs0, vs0
-; CHECK-P9-VECTOR-NEXT:    xxmrgld v2, vs1, vs0
+; CHECK-P9-VECTOR-NEXT:    xxpermdi v2, vs1, vs0, 1
 ; CHECK-P9-VECTOR-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi1:
 ; CHECK-P9:       # %bb.0:
 ; CHECK-P9-NEXT:    lfd f1, 0(r4)
 ; CHECK-P9-NEXT:    lxv vs0, 0(r3)
 ; CHECK-P9-NEXT:    xxswapd vs1, f1
 ; CHECK-P9-NEXT:    xxmrgld v2, vs1, vs0