Power9 - Part-word VSX integer scalar loads/stores and sign extend instructions

[nemanjai]

Summary

This patch uses a portion of https://reviews.llvm.org/D20310 that removes the VSHRC register class. The reason for this is that these instructions have a similar need for the changes to the register classes and the other revision is on hold for now. When this revision is committed, the other one will simply be an addition of the patterns for the instructions.

The new instructions are useful for a few purposes:

• Int to Fp conversions of 1 or 2-byte values loaded from memory
• Building vectors of 1 or 2-byte integers with values loaded from memory
• Storing individual 1 or 2-byte elements from integer vectors

This patch implements all of those uses.

[cycheng]

Add Tom.

Hi Nemanja,
Thank you for working on it :D
CY

[kbarton]

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

476

What does this comment mean?
Does this need to be done? If not, can it be removed?

lib/Target/PowerPC/MCTargetDesc/PPCMCCodeEmitter.cpp

354

This routine is defined as static in at least two files.
Can we add it to a more general location and just define it once?

lib/Target/PowerPC/PPCAsmPrinter.cpp

165

Same comment as above. Can these go into a more general location?

177

Extraneous comment?

lib/Target/PowerPC/PPCISelLowering.cpp

10559

Was this new section formatted with clang-format?
The indentation looks strange to me, but this might be what it does when you have these types of long lines.

[cycheng]

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

476

Because ISA3.0 add new VSX instruction formats, so couple of new VSX instructions (xsabsqp, xsnabsqp, ...) uses V registers.

When we want to print operand for VSX instructions, we have to know which register class it is.

if ((MII.get(MI->getOpcode()).TSFlags & PPCII::UseVSXReg)) {

if PPCII::UseVSXReg is true, then it means the VSX instruction use VSX registers (without any V registers.), so the following code will treat these register as VSX register.

if (isVRRegister(Reg))
  Reg = PPC::VSX32 + (Reg - PPC::V0);

But because we use v0-v31 to represent vs32-vs63, so you will see the Reg is PPC::V0 - PPC::V31, so we get the offset value, and add that offset back to base PPC::VSX32.

The purpose here is, we want to translate v0-v31 back to vs32-vs63.

lib/Target/PowerPC/PPCAsmPrinter.cpp

177

Same logic as PPCInstPrinter::printOperand, maybe we can replace this comment with short one.

[nemanjai]

lib/Target/PowerPC/PPCISelLowering.cpp

10559

Yeah. Well I use Vim and the provided .vimrc which indents things according to the policy. It will line up the conditions when they're parenthesized (i.e. in an if statement). But for an assignment/initialization, it indents the continued lines 1 tab stop (2 spaces) from the indentation of the declarator.

[nemanjai]

Moved the definition of the static functions that were defined in 3 places to the class definition.
Clarified the comment regarding the handling of VMX (Altivec) registers.
Rebased the patch on a more recent revision (r282478) since a lot of files and behaviour changed.

Please note: there are a couple of FIXME's in the test cases. The reason for this is that there are a number of instructions we exploited recently that can match BUILD_VECTOR nodes. Some of the previously committed patches make the code in this patch unable to match a BUILD_VECTOR node under very specific conditions. I have a subsequent patch coming up which will take all the recent changes into account and produce optimal code for BUILD_VECTOR nodes for both Power8 and Power9.

[nemanjai]

@hfinkel Hi Hal, this is the patch I mentioned that I'd like you to have a look at since it involves that infrastructural change that CY started working on - removing the VSHRC register class.

[kbarton]

Aside from minor comments this LGTM.
I think it would be good for @hfinkel take a quick look too.

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

474

This should change to C++-style comments (i.e., //) as per the comment-formatting guidelines: http://llvm.org/docs/CodingStandards.html#comment-formatting

lib/Target/PowerPC/PPCAsmPrinter.cpp

174

Reformat comments

lib/Target/PowerPC/PPCCallingConv.td

71

This is very minor, but this used to be sorted by integer types (in decending order of vector elements), followed by floats sorted in the same way. To maintain this, v2f64 should be moved to the end after v4f32 unless there is a specific reason it has been put where it is.

122

Same comment about placement of v2f64.

192

Same comment about v2f64

lib/Target/PowerPC/PPCISelLowering.cpp

7079

Could this condition be combined with the one below?

10577

Is FP really needed? Can you just return on the lines where FP is being assigned?

10828

ByteWidth might not be the best name here, since it shouldn't vary ;)

10829

I don't see ByteWidth used anywhere else, so you can probably do away with it by just copying the logic to assign ByteWidth into this constructor call.

lib/Target/PowerPC/PPCInstrInfo.cpp

1063

This is duplicated in both storeRegToStackSlot and loadRegFromStackSlot.
This should be refactored into a separate method to get the TargetRegisterClass and called from both these methods instead, to ensure consistency.

lib/Target/PowerPC/PPCMIPeephole.cpp

198

Is it possible for MyOpcode to be VSPLTBs also? I don't see this case handled anywhere and it's not clear that this situation could not be symmetric (i.e., MyOpcode == PPC::VSPLTBs and DefOpcode == PPC::VSPLTB).

lib/Target/PowerPC/PPCRegisterInfo.td

286

Same comment about ordering.

[hfinkel]

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

479

We don't need to mention how it used to work, that won't help anyone looking at the current code, and might be confusing.

lib/Target/PowerPC/PPCAsmPrinter.cpp

179

Same here; we don't need to mention how it used to work.

lib/Target/PowerPC/PPCInstrInfo.cpp

1058

We should make this comment more specific, and explain that the difference is that the lanes are stored in a different order.

1061

It would be better if we could explain here why this happens. It is not obvious because the live range that was split via the spill/restore would have had some particular register class.

lib/Target/PowerPC/PPCInstrVSX.td

96

I don't really understand what this is saying and why using the same register would matter.

1125

Why don't you set UseVSXReg = 1 in an outer block (essentially over all definitions), and then override by setting UseVSXReg = 0 for those instructions that shouldn't have it. I'm under the impression that this is a smaller set.

[nemanjai]

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

474

Will do.

479

OK, I'll remove the last sentence.

lib/Target/PowerPC/PPCAsmPrinter.cpp

174

Will do.

179

OK, will remove the last sentence.

lib/Target/PowerPC/PPCCallingConv.td

71

I don't think the order implies anything in this list. I'll move v2f64 to the end (after v4f32).

122

Yup, will move it.

192

Yup.

lib/Target/PowerPC/PPCISelLowering.cpp

7079

Yeah, the one below was an afterthought and I guess I just missed adding it to this one. I'll combine them.

10577

Yup, I'll rewrite it that way.

10829

Yeah, I'll combine it.

lib/Target/PowerPC/PPCInstrInfo.cpp

1058

I'll rewrite it as:
We need to avoid a situation in which the value from a VRRC register is spilled using an Altivec instruction and reloaded into
a VSRC register using a VSX instruction. The issue with this is that the VSX load/store instructions swap the doublewords
in the vector and the Altivec ones don't. The register classes on the spill/reload may be different if the register is defined using
an Altivec instruction and is then used by a VSX instruction.

lib/Target/PowerPC/PPCInstrVSX.td

96

If you look at the instruction where I use this for an example:

def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi16i64)),
           (v2i64 (XXPERMDIs (LXSIHZX xoaddr:$src), 0))>;

If I were to replace that with

def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi16i64)),
           (v2i64 (XXPERMDI (LXSIHZX xoaddr:$src), (LXSIHZX xoaddr:$src), 0))>;

The output DAG actually produces code that will perform both loads into different registers (i.e. two separate LXSIHZX instructions) and will do the XXPERMDI on the result registers. I just found using a version of the instruction that has a single input register operand to be an easy way to avoid this.

1125

I think that we'd have to interrupt this block for definitions that do not inherit from Instruction such as MovesToVSR, VectorExtractions, etc.
In any case, I think that over time, our use of these blocks has introduced some subtle issues that may or may not ever manifest themselves.
For example, we may have a

let Predicates = [ListA] in {
  let Predicates = [ListB] in {
  }
}

where the content of the inner block should really have

let Predicates = [ListA, ListB] in {
}

but of course, it does not since the inner assignment overrides the Predicates rather than appending tothem.
I plan to go through the file and clean this up a bit but I'd prefer to do this as a separate patch.

lib/Target/PowerPC/PPCMIPeephole.cpp

198

We don't currently have any patterns that will produce such a combination. Namely, the versions suffixed with "s" are instructions that splat a scalar (actually loaded as a scalar) into the vector register. Without this scalar load, we should not be using these scalar splats. So a VSPLTBs can only be fed by an LXSIBZX (or I suppose also an MTVSR[DW]) but never a VSPLTB.

lib/Target/PowerPC/PPCRegisterInfo.td

286

OK, I'll reorder this as well.

[nemanjai]

@hfinkel @kbarton Do you guys want me to post an updated patch or just address the comments in the commit? I'm sorry to be spamming you, but I'd like to commit this patch so that I can get the other one in (as well as having clear direction on the one I'm working on now).

[hfinkel]

In D23155#559464, @nemanjai wrote:

@hfinkel @kbarton Do you guys want me to post an updated patch or just address the comments in the commit? I'm sorry to be spamming you, but I'd like to commit this patch so that I can get the other one in (as well as having clear direction on the one I'm working on now).

Based on your responses, I'm find with you addressing the comments and committing.

[nemanjai]

Committed revision 283190.

[shawnl]

I think the essential functionality of this patch should be in LLVM and not Clang, so that all front-ends can benefit. To many generally useful things are in Clang when they should be in LLVM (e.g. C ABI for ARM and x86; ranged switch statements). I opened an upstream bug to discuss this. https://github.com/clang-randstruct/llvm-project/issues/52

[shawnl]

Oh sorry, wrong bug.

lib/Target/PowerPC/AsmParser/PPCAsmParser.cpp

static const MCPhysReg FRegs[32] = {
   PPC::F4,  PPC::F5,  PPC::F6,  PPC::F7,
   PPC::F8,  PPC::F9,  PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
   PPC::F24, PPC::F25, PPC::F26, PPC::F27,
   PPC::F28, PPC::F29, PPC::F30, PPC::F31
 };
+static const MCPhysReg VFRegs[32] = {
+  PPC::VF0,  PPC::VF1,  PPC::VF2,  PPC::VF3,
+  PPC::VF4,  PPC::VF5,  PPC::VF6,  PPC::VF7,
+  PPC::VF8,  PPC::VF9,  PPC::VF10, PPC::VF11,
+  PPC::VF12, PPC::VF13, PPC::VF14, PPC::VF15,
+  PPC::VF16, PPC::VF17, PPC::VF18, PPC::VF19,
+  PPC::VF20, PPC::VF21, PPC::VF22, PPC::VF23,
+  PPC::VF24, PPC::VF25, PPC::VF26, PPC::VF27,
+  PPC::VF28, PPC::VF29, PPC::VF30, PPC::VF31
+};
 static const MCPhysReg VRegs[32] = {
   PPC::V0,  PPC::V1,  PPC::V2,  PPC::V3,
   PPC::V4,  PPC::V5,  PPC::V6,  PPC::V7,
   PPC::V8,  PPC::V9,  PPC::V10, PPC::V11,
   PPC::V12, PPC::V13, PPC::V14, PPC::V15,
   PPC::V16, PPC::V17, PPC::V18, PPC::V19,
   PPC::V20, PPC::V21, PPC::V22, PPC::V23,
   PPC::V24, PPC::V25, PPC::V26, PPC::V27,
   PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 static const MCPhysReg VSRegs[64] = {
   PPC::VSL0,  PPC::VSL1,  PPC::VSL2,  PPC::VSL3,
   PPC::VSL4,  PPC::VSL5,  PPC::VSL6,  PPC::VSL7,
   PPC::VSL8,  PPC::VSL9,  PPC::VSL10, PPC::VSL11,
   PPC::VSL12, PPC::VSL13, PPC::VSL14, PPC::VSL15,
   PPC::VSL16, PPC::VSL17, PPC::VSL18, PPC::VSL19,
   PPC::VSL20, PPC::VSL21, PPC::VSL22, PPC::VSL23,
   PPC::VSL24, PPC::VSL25, PPC::VSL26, PPC::VSL27,
   PPC::VSL28, PPC::VSL29, PPC::VSL30, PPC::VSL31,
 
-  PPC::VSH0,  PPC::VSH1,  PPC::VSH2,  PPC::VSH3,
-  PPC::VSH4,  PPC::VSH5,  PPC::VSH6,  PPC::VSH7,
-  PPC::VSH8,  PPC::VSH9,  PPC::VSH10, PPC::VSH11,
-  PPC::VSH12, PPC::VSH13, PPC::VSH14, PPC::VSH15,
-  PPC::VSH16, PPC::VSH17, PPC::VSH18, PPC::VSH19,
-  PPC::VSH20, PPC::VSH21, PPC::VSH22, PPC::VSH23,
-  PPC::VSH24, PPC::VSH25, PPC::VSH26, PPC::VSH27,
-  PPC::VSH28, PPC::VSH29, PPC::VSH30, PPC::VSH31
+  PPC::V0,  PPC::V1,  PPC::V2,  PPC::V3,
+  PPC::V4,  PPC::V5,  PPC::V6,  PPC::V7,
+  PPC::V8,  PPC::V9,  PPC::V10, PPC::V11,
+  PPC::V12, PPC::V13, PPC::V14, PPC::V15,
+  PPC::V16, PPC::V17, PPC::V18, PPC::V19,
+  PPC::V20, PPC::V21, PPC::V22, PPC::V23,
+  PPC::V24, PPC::V25, PPC::V26, PPC::V27,
+  PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 static const MCPhysReg VSFRegs[64] = {
   PPC::F0,  PPC::F1,  PPC::F2,  PPC::F3,
   PPC::F4,  PPC::F5,  PPC::F6,  PPC::F7,
   PPC::F8,  PPC::F9,  PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
void addRegF4RCOperands(MCInst &Inst, unsigned N) const {
     Inst.addOperand(MCOperand::createReg(FRegs[getReg()]));
   }
 
   void addRegF8RCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(FRegs[getReg()]));
   }
 
+  void addRegVFRCOperands(MCInst &Inst, unsigned N) const {
+    assert(N == 1 && "Invalid number of operands!");
+    Inst.addOperand(MCOperand::createReg(VFRegs[getReg()]));
+  }
+
   void addRegVRRCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(VRegs[getReg()]));
   }
 
   void addRegVSRCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(VSRegs[getVSReg()]));

lib/Target/PowerPC/Disassembler/PPCDisassembler.cpp

static const unsigned FRegs[] = {
   PPC::F8, PPC::F9, PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
   PPC::F24, PPC::F25, PPC::F26, PPC::F27,
   PPC::F28, PPC::F29, PPC::F30, PPC::F31
 };
 
+static const unsigned VFRegs[] = {
+  PPC::VF0, PPC::VF1, PPC::VF2, PPC::VF3,
+  PPC::VF4, PPC::VF5, PPC::VF6, PPC::VF7,
+  PPC::VF8, PPC::VF9, PPC::VF10, PPC::VF11,
+  PPC::VF12, PPC::VF13, PPC::VF14, PPC::VF15,
+  PPC::VF16, PPC::VF17, PPC::VF18, PPC::VF19,
+  PPC::VF20, PPC::VF21, PPC::VF22, PPC::VF23,
+  PPC::VF24, PPC::VF25, PPC::VF26, PPC::VF27,
+  PPC::VF28, PPC::VF29, PPC::VF30, PPC::VF31
+};
+
 static const unsigned VRegs[] = {
   PPC::V0, PPC::V1, PPC::V2, PPC::V3,
   PPC::V4, PPC::V5, PPC::V6, PPC::V7,
   PPC::V8, PPC::V9, PPC::V10, PPC::V11,
   PPC::V12, PPC::V13, PPC::V14, PPC::V15,
   PPC::V16, PPC::V17, PPC::V18, PPC::V19,
   PPC::V20, PPC::V21, PPC::V22, PPC::V23,
   PPC::V24, PPC::V25, PPC::V26, PPC::V27,
   PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 
 static const unsigned VSRegs[] = {
   PPC::VSL0, PPC::VSL1, PPC::VSL2, PPC::VSL3,
   PPC::VSL4, PPC::VSL5, PPC::VSL6, PPC::VSL7,
   PPC::VSL8, PPC::VSL9, PPC::VSL10, PPC::VSL11,
   PPC::VSL12, PPC::VSL13, PPC::VSL14, PPC::VSL15,
   PPC::VSL16, PPC::VSL17, PPC::VSL18, PPC::VSL19,
   PPC::VSL20, PPC::VSL21, PPC::VSL22, PPC::VSL23,
   PPC::VSL24, PPC::VSL25, PPC::VSL26, PPC::VSL27,
   PPC::VSL28, PPC::VSL29, PPC::VSL30, PPC::VSL31,
 
-  PPC::VSH0, PPC::VSH1, PPC::VSH2, PPC::VSH3,
-  PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7,
-  PPC::VSH8, PPC::VSH9, PPC::VSH10, PPC::VSH11,
-  PPC::VSH12, PPC::VSH13, PPC::VSH14, PPC::VSH15,
-  PPC::VSH16, PPC::VSH17, PPC::VSH18, PPC::VSH19,
-  PPC::VSH20, PPC::VSH21, PPC::VSH22, PPC::VSH23,
-  PPC::VSH24, PPC::VSH25, PPC::VSH26, PPC::VSH27,
-  PPC::VSH28, PPC::VSH29, PPC::VSH30, PPC::VSH31
+  PPC::V0, PPC::V1, PPC::V2, PPC::V3,
+  PPC::V4, PPC::V5, PPC::V6, PPC::V7,
+  PPC::V8, PPC::V9, PPC::V10, PPC::V11,
+  PPC::V12, PPC::V13, PPC::V14, PPC::V15,
+  PPC::V16, PPC::V17, PPC::V18, PPC::V19,
+  PPC::V20, PPC::V21, PPC::V22, PPC::V23,
+  PPC::V24, PPC::V25, PPC::V26, PPC::V27,
+  PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 
 static const unsigned VSFRegs[] = {
   PPC::F0, PPC::F1, PPC::F2, PPC::F3,
   PPC::F4, PPC::F5, PPC::F6, PPC::F7,
   PPC::F8, PPC::F9, PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,

 }
 
 static DecodeStatus DecodeF8RCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,
                                             const void *Decoder) {
   return decodeRegisterClass(Inst, RegNo, FRegs);
 }
 
+static DecodeStatus DecodeVFRCRegisterClass(MCInst &Inst, uint64_t RegNo,
+                                            uint64_t Address,
+                                            const void *Decoder) {
+  return decodeRegisterClass(Inst, RegNo, VFRegs);
+}
+
 static DecodeStatus DecodeVRRCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,
                                             const void *Decoder) {
   return decodeRegisterClass(Inst, RegNo, VRegs);
 }
 
 static DecodeStatus DecodeVSRCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

 //===-- PPCInstPrinter.cpp - Convert PPC MCInst to assembly syntax --------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This class prints an PPC MCInst to a .s file.
 //
 //===----------------------------------------------------------------------===//
 
 #include "PPCInstPrinter.h"
+#include "PPCInstrInfo.h"
 #include "MCTargetDesc/PPCMCTargetDesc.h"
 #include "MCTargetDesc/PPCPredicates.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
void PPCInstPrinter::printTLSCall(const MCInst *MI, unsigned OpNo,
   if (refExp.getKind() != MCSymbolRefExpr::VK_None)
     O << '@' << MCSymbolRefExpr::getVariantKindName(refExp.getKind());
 }
 
 
 /// stripRegisterPrefix - This method strips the character prefix from a
 /// register name so that only the number is left.  Used by for linux asm.
 static const char *stripRegisterPrefix(const char *RegName) {
-  if (FullRegNames)
+  if (FullRegNames || ShowVSRNumsAsVR)
     return RegName;
 
   switch (RegName[0]) {
   case 'r':
   case 'f':
   case 'q': // for QPX
   case 'v':
     if (RegName[1] == 's')
       return RegName + 2;
     return RegName + 1;
   case 'c': if (RegName[1] == 'r') return RegName + 2;
   }
   
   return RegName;
 }
 
 void PPCInstPrinter::printOperand(const MCInst *MI, unsigned OpNo,
                                   raw_ostream &O) {
   const MCOperand &Op = MI->getOperand(OpNo);
   if (Op.isReg()) {
-    const char *RegName = getRegisterName(Op.getReg());
-    if (ShowVSRNumsAsVR) {
-      unsigned RegNum = Op.getReg();
-      if (RegNum >= PPC::VSH0 && RegNum <= PPC::VSH31)
-        O << 'v' << RegNum - PPC::VSH0;
-      else
-        O << RegName;
-      return;
+    unsigned Reg = Op.getReg();
+
+    /*  There are VSX instructions that use VSX register numbering (vs0 - vs63)
+        as well as those that use VMX register numbering (v0 - v31 which
+        correspond to vs32 - vs63). If we have an instruction that uses VSX
+        numbering, we need to convert the VMX registers to VSX registers.
+        Namely, we print 32-63 when the instruction operates on one of the
+        VMX registers. We previously had the VSHRC register class for the
+        upper 32 VSX registers.
+        (Please synchronize with PPCAsmPrinter::printOperand)
+    */
+    if ((MII.get(MI->getOpcode()).TSFlags & PPCII::UseVSXReg) &&
+        !ShowVSRNumsAsVR) {
+      if (PPCInstrInfo::isVRRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::V0);
+      else if (PPCInstrInfo::isVFRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::VF0);
     }
+
+    const char *RegName = getRegisterName(Reg);
     // The linux and AIX assembler does not take register prefixes.
     if (!isDarwinSyntax())
       RegName = stripRegisterPrefix(RegName);
     
     O << RegName;
     return;
   }
   

lib/Target/PowerPC/MCTargetDesc/PPCMCCodeEmitter.cpp

 //===-- PPCMCCodeEmitter.cpp - Convert PPC code to machine code -----------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the PPCMCCodeEmitter class.
 //
 //===----------------------------------------------------------------------===//
 
+#include "PPCInstrInfo.h"
 #include "MCTargetDesc/PPCMCTargetDesc.h"
 #include "MCTargetDesc/PPCFixupKinds.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCCodeEmitter.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
get_crbitm_encoding(const MCInst &MI, unsigned OpNo,
                     const MCSubtargetInfo &STI) const {
   const MCOperand &MO = MI.getOperand(OpNo);
   assert((MI.getOpcode() == PPC::MTOCRF || MI.getOpcode() == PPC::MTOCRF8 ||
           MI.getOpcode() == PPC::MFOCRF || MI.getOpcode() == PPC::MFOCRF8) &&
          (MO.getReg() >= PPC::CR0 && MO.getReg() <= PPC::CR7));
   return 0x80 >> CTX.getRegisterInfo()->getEncodingValue(MO.getReg());
 }
 
-
 unsigned PPCMCCodeEmitter::
 getMachineOpValue(const MCInst &MI, const MCOperand &MO,
                   SmallVectorImpl<MCFixup> &Fixups,
                   const MCSubtargetInfo &STI) const {
   if (MO.isReg()) {
     // MTOCRF/MFOCRF should go through get_crbitm_encoding for the CR operand.
     // The GPR operand should come through here though.
     assert((MI.getOpcode() != PPC::MTOCRF && MI.getOpcode() != PPC::MTOCRF8 &&
             MI.getOpcode() != PPC::MFOCRF && MI.getOpcode() != PPC::MFOCRF8) ||
            MO.getReg() < PPC::CR0 || MO.getReg() > PPC::CR7);
-    return CTX.getRegisterInfo()->getEncodingValue(MO.getReg());
+    unsigned Reg = MO.getReg();
+    unsigned Encode = CTX.getRegisterInfo()->getEncodingValue(Reg);
+
+    if ((MCII.get(MI.getOpcode()).TSFlags & PPCII::UseVSXReg))
+      if (PPCInstrInfo::isVRRegister(Reg))
+        Encode += 32;
+
+    return Encode;
   }
   
   assert(MO.isImm() &&
          "Relocation required in an instruction that we cannot encode!");
   return MO.getImm();
 }
 
 
 #include "PPCGenMCCodeEmitter.inc"

lib/Target/PowerPC/PPCAsmPrinter.cpp

case 'v':
         return RegName + 2;
       return RegName + 1;
     case 'c': if (RegName[1] == 'r') return RegName + 2;
   }
 
   return RegName;
 }
 
 void PPCAsmPrinter::printOperand(const MachineInstr *MI, unsigned OpNo,
                                  raw_ostream &O) {
   const DataLayout &DL = getDataLayout();
   const MachineOperand &MO = MI->getOperand(OpNo);
 
   switch (MO.getType()) {
   case MachineOperand::MO_Register: {
-    const char *RegName = PPCInstPrinter::getRegisterName(MO.getReg());
+    unsigned Reg = MO.getReg();
+
+    /*  There are VSX instructions that use VSX register numbering (vs0 - vs63)
+        as well as those that use VMX register numbering (v0 - v31 which
+        correspond to vs32 - vs63). If we have an instruction that uses VSX
+        numbering, we need to convert the VMX registers to VSX registers.
+        Namely, we print 32-63 when the instruction operates on one of the
+        VMX registers. We previously had the VSHRC register class for the
+        upper 32 VSX registers.
+        (Please synchronize with PPCInstPrinter::printOperand)
+    */
+    if (MI->getDesc().TSFlags & PPCII::UseVSXReg) {
+      if (PPCInstrInfo::isVRRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::V0);
+      else if (PPCInstrInfo::isVFRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::VF0);
+    }
+    const char *RegName = PPCInstPrinter::getRegisterName(Reg);
+
     // Linux assembler (Others?) does not take register mnemonics.
     // FIXME - What about special registers used in mfspr/mtspr?
     if (!Subtarget->isDarwin())
       RegName = stripRegisterPrefix(RegName);
     O << RegName;
     return;
   }
   case MachineOperand::MO_Immediate:

lib/Target/PowerPC/PPCCallingConv.td

def RetCC_PPC : CallingConv<[
   CCIfType<[f64], CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
 
   // QPX vectors are returned in QF1 and QF2. 
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
  
   // Vector types returned as "direct" go into V2 .. V9; note that only the
   // ELFv2 ABI fully utilizes all these registers.
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()",
-           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
+           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>
 ]>;
 
 // No explicit register is specified for the AnyReg calling convention. The
 // register allocator may assign the arguments to any free register.
 //
 // This calling convention is currently only supported by the stackmap and
 // patchpoint intrinsics. All other uses will result in an assert on Debug
 // builds. On Release builds we fallback to the PPC C calling convention.
def RetCC_PPC64_ELF_FIS : CallingConv<[
   CCIfType<[i16],  CCPromoteToType<i64>>,
   CCIfType<[i32],  CCPromoteToType<i64>>,
   CCIfType<[i64],  CCAssignToReg<[X3, X4, X5, X6]>>,
   CCIfType<[i128], CCAssignToReg<[X3, X4, X5, X6]>>,
   CCIfType<[f32],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[f64],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()",
-           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
+           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>
 ]>;
 
 //===----------------------------------------------------------------------===//
 // PowerPC System V Release 4 32-bit ABI
 //===----------------------------------------------------------------------===//
 
 def CC_PPC32_SVR4_Common : CallingConv<[
   CCIfType<[i1], CCPromoteToType<i32>>,

 // In contrast to CC_PPC32_SVR4_VarArg, this calling convention first tries to
 // put vector arguments in vector registers before putting them on the stack.
 def CC_PPC32_SVR4 : CallingConv<[
   // QPX vectors mirror the scalar FP convention.
   CCIfType<[v4f64, v4f32, v4i1], CCIfSubtarget<"hasQPX()",
     CCAssignToReg<[QF1, QF2, QF3, QF4, QF5, QF6, QF7, QF8]>>>,
 
   // The first 12 Vector arguments are passed in AltiVec registers.
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()", CCAssignToReg<[V2, V3, V4, V5, V6, V7,
                           V8, V9, V10, V11, V12, V13]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9,
-                          VSH10, VSH11, VSH12, VSH13]>>>,
            
   CCDelegateTo<CC_PPC32_SVR4_Common>
 ]>;  
 
 // Helper "calling convention" to handle aggregate by value arguments.
 // Aggregate by value arguments are always placed in the local variable space
 // of the caller. This calling convention is only used to assign those stack
 // offsets in the callers stack frame.
def CSR_64_AllRegs: CalleeSavedRegs<(add X0, (sequence "X%u", 3, 10),
                                              (sequence "X%u", 14, 31),
                                              (sequence "F%u", 0, 31),
                                              (sequence "CR%u", 0, 7))>;
 
 def CSR_64_AllRegs_Altivec : CalleeSavedRegs<(add CSR_64_AllRegs,
                                              (sequence "V%u", 0, 31))>;
 
 def CSR_64_AllRegs_VSX : CalleeSavedRegs<(add CSR_64_AllRegs_Altivec,
-                                         (sequence "VSL%u", 0, 31),
-                                         (sequence "VSH%u", 0, 31))>;
+                                         (sequence "VSL%u", 0, 31))>;
 

lib/Target/PowerPC/PPCISelLowering.h

enum NodeType : unsigned {
       /// operand, producing an f64 value containing the integer representation
       /// of that FP value.
       FCTIDZ, FCTIWZ,
 
       /// Newer FCTI[D,W]UZ floating-point-to-integer conversion instructions for
       /// unsigned integers.
       FCTIDUZ, FCTIWUZ,
 
+      /// VEXTS, ByteWidth - takes an input in VSFRC and produces an output in
+      /// VSFRC that is sign-extended from ByteWidth to a 64-byte integer.
+      VEXTS,
+
       /// Reciprocal estimate instructions (unary FP ops).
       FRE, FRSQRTE,
 
       // VMADDFP, VNMSUBFP - The VMADDFP and VNMSUBFP instructions, taking
       // three v4f32 operands and producing a v4f32 result.
       VMADDFP, VNMSUBFP,
 
       /// VPERM - The PPC VPERM Instruction.
enum NodeType : unsigned {
       /// destination 64-bit register.
       LFIWAX,
 
       /// GPRC, CHAIN = LFIWZX CHAIN, Ptr - This is a floating-point
       /// load which zero-extends from a 32-bit integer value into the
       /// destination 64-bit register.
       LFIWZX,
 
+      /// GPRC, CHAIN = LXSIZX, CHAIN, Ptr, ByteWidth - This is a load of an
+      /// integer smaller than 64 bits into a VSR. The integer is zero-extended.
+      /// This can be used for converting loaded integers to floating point.
+      LXSIZX,
+
+      /// STXSIX - The STXSI[bh]X instruction. The first operand is an input
+      /// chain, then an f64 value to store, then an address to store it to,
+      /// followed by a byte-width for the store.
+      STXSIX,
+
       /// VSRC, CHAIN = LXVD2X_LE CHAIN, Ptr - Occurs only for little endian.
       /// Maps directly to an lxvd2x instruction that will be followed by
       /// an xxswapd.
       LXVD2X,
 
       /// CHAIN = STXVD2X CHAIN, VSRC, Ptr - Occurs only for little endian.
       /// Maps directly to an stxvd2x instruction that will be preceded by
       /// an xxswapd.

lib/Target/PowerPC/PPCISelLowering.cpp

if (Subtarget.hasAltivec()) {
     }
 
     if (Subtarget.hasP9Vector()) {
       setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4i32, Custom);
       setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4f32, Custom);
     }
 
     if (Subtarget.isISA3_0() && Subtarget.hasDirectMove())
-      setOperationAction(ISD::BUILD_VECTOR, MVT::v2i64, Legal);
+      setOperationAction(ISD::BUILD_VECTOR, MVT::v2i64, Custom);
   }
 
   if (Subtarget.hasQPX()) {
     setOperationAction(ISD::FADD, MVT::v4f64, Legal);
     setOperationAction(ISD::FSUB, MVT::v4f64, Legal);
     setOperationAction(ISD::FMUL, MVT::v4f64, Legal);
     setOperationAction(ISD::FREM, MVT::v4f64, Expand);
 
const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   case PPCISD::ANDIo_1_EQ_BIT:  return "PPCISD::ANDIo_1_EQ_BIT";
   case PPCISD::ANDIo_1_GT_BIT:  return "PPCISD::ANDIo_1_GT_BIT";
   case PPCISD::VCMP:            return "PPCISD::VCMP";
   case PPCISD::VCMPo:           return "PPCISD::VCMPo";
   case PPCISD::LBRX:            return "PPCISD::LBRX";
   case PPCISD::STBRX:           return "PPCISD::STBRX";
   case PPCISD::LFIWAX:          return "PPCISD::LFIWAX";
   case PPCISD::LFIWZX:          return "PPCISD::LFIWZX";
+  case PPCISD::LXSIZX:          return "PPCISD::LXSIZX";
+  case PPCISD::STXSIX:          return "PPCISD::STXSIX";
+  case PPCISD::VEXTS:           return "PPCISD::VEXTS";
   case PPCISD::LXVD2X:          return "PPCISD::LXVD2X";
   case PPCISD::STXVD2X:         return "PPCISD::STXVD2X";
   case PPCISD::COND_BRANCH:     return "PPCISD::COND_BRANCH";
   case PPCISD::BDNZ:            return "PPCISD::BDNZ";
   case PPCISD::BDZ:             return "PPCISD::BDZ";
   case PPCISD::MFFS:            return "PPCISD::MFFS";
   case PPCISD::FADDRTZ:         return "PPCISD::FADDRTZ";
   case PPCISD::TC_RETURN:       return "PPCISD::TC_RETURN";
if (VA.isRegLoc()) {
         case MVT::v4i32:
           RC = &PPC::VRRCRegClass;
           break;
         case MVT::v4f32:
           RC = Subtarget.hasQPX() ? &PPC::QSRCRegClass : &PPC::VRRCRegClass;
           break;
         case MVT::v2f64:
         case MVT::v2i64:
-          RC = &PPC::VSHRCRegClass;
+          RC = &PPC::VRRCRegClass;
           break;
         case MVT::v4f64:
           RC = &PPC::QFRCRegClass;
           break;
         case MVT::v4i1:
           RC = &PPC::QBRCRegClass;
           break;
       }
SDValue PPCTargetLowering::LowerFormalArguments_64SVR4(
   static const MCPhysReg GPR[] = {
     PPC::X3, PPC::X4, PPC::X5, PPC::X6,
     PPC::X7, PPC::X8, PPC::X9, PPC::X10,
   };
   static const MCPhysReg VR[] = {
     PPC::V2, PPC::V3, PPC::V4, PPC::V5, PPC::V6, PPC::V7, PPC::V8,
     PPC::V9, PPC::V10, PPC::V11, PPC::V12, PPC::V13
   };
-  static const MCPhysReg VSRH[] = {
-    PPC::VSH2, PPC::VSH3, PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7, PPC::VSH8,
-    PPC::VSH9, PPC::VSH10, PPC::VSH11, PPC::VSH12, PPC::VSH13
-  };
 
   const unsigned Num_GPR_Regs = array_lengthof(GPR);
   const unsigned Num_FPR_Regs = 13;
   const unsigned Num_VR_Regs  = array_lengthof(VR);
   const unsigned Num_QFPR_Regs = Num_FPR_Regs;
 
   // Do a first pass over the arguments to determine whether the ABI
   // guarantees that our caller has allocated the parameter save area
for (unsigned ArgNo = 0, e = Ins.size(); ArgNo != e; ++ArgNo) {
     case MVT::v2f64:
     case MVT::v2i64:
     case MVT::v1i128:
       if (!Subtarget.hasQPX()) {
       // These can be scalar arguments or elements of a vector array type
       // passed directly.  The latter are used to implement ELFv2 homogenous
       // vector aggregates.
       if (VR_idx != Num_VR_Regs) {
-        unsigned VReg = (ObjectVT == MVT::v2f64 || ObjectVT == MVT::v2i64) ?
-                        MF.addLiveIn(VSRH[VR_idx], &PPC::VSHRCRegClass) :
-                        MF.addLiveIn(VR[VR_idx], &PPC::VRRCRegClass);
+        unsigned VReg = MF.addLiveIn(VR[VR_idx], &PPC::VRRCRegClass);
         ArgVal = DAG.getCopyFromReg(Chain, dl, VReg, ObjectVT);
         ++VR_idx;
       } else {
         if (CallConv == CallingConv::Fast)
           ComputeArgOffset();
 
         needsLoad = true;
       }
SDValue PPCTargetLowering::LowerCall_64SVR4(
   static const MCPhysReg GPR[] = {
     PPC::X3, PPC::X4, PPC::X5, PPC::X6,
     PPC::X7, PPC::X8, PPC::X9, PPC::X10,
   };
   static const MCPhysReg VR[] = {
     PPC::V2, PPC::V3, PPC::V4, PPC::V5, PPC::V6, PPC::V7, PPC::V8,
     PPC::V9, PPC::V10, PPC::V11, PPC::V12, PPC::V13
   };
-  static const MCPhysReg VSRH[] = {
-    PPC::VSH2, PPC::VSH3, PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7, PPC::VSH8,
-    PPC::VSH9, PPC::VSH10, PPC::VSH11, PPC::VSH12, PPC::VSH13
-  };
 
   const unsigned NumGPRs = array_lengthof(GPR);
   const unsigned NumFPRs = 13;
   const unsigned NumVRs  = array_lengthof(VR);
   const unsigned NumQFPRs = NumFPRs;
 
   // When using the fast calling convention, we don't provide backing for
   // arguments that will be in registers.
case MVT::v1i128:
         // entirely in R registers.  Maybe later.
         SDValue Store =
             DAG.getStore(Chain, dl, Arg, PtrOff, MachinePointerInfo());
         MemOpChains.push_back(Store);
         if (VR_idx != NumVRs) {
           SDValue Load =
               DAG.getLoad(MVT::v4f32, dl, Store, PtrOff, MachinePointerInfo());
           MemOpChains.push_back(Load.getValue(1));
-
-          unsigned VReg = (Arg.getSimpleValueType() == MVT::v2f64 ||
-                           Arg.getSimpleValueType() == MVT::v2i64) ?
-                          VSRH[VR_idx] : VR[VR_idx];
-          ++VR_idx;
-
-          RegsToPass.push_back(std::make_pair(VReg, Load));
+          RegsToPass.push_back(std::make_pair(VR[VR_idx++], Load));
         }
         ArgOffset += 16;
         for (unsigned i=0; i<16; i+=PtrByteSize) {
           if (GPR_idx == NumGPRs)
             break;
           SDValue Ix = DAG.getNode(ISD::ADD, dl, PtrVT, PtrOff,
                                    DAG.getConstant(i, dl, PtrVT));
           SDValue Load =
               DAG.getLoad(PtrVT, dl, Store, Ix, MachinePointerInfo());
           MemOpChains.push_back(Load.getValue(1));
           RegsToPass.push_back(std::make_pair(GPR[GPR_idx++], Load));
         }
         break;
       }
 
       // Non-varargs Altivec params go into VRs or on the stack.
       if (VR_idx != NumVRs) {
-        unsigned VReg = (Arg.getSimpleValueType() == MVT::v2f64 ||
-                         Arg.getSimpleValueType() == MVT::v2i64) ?
-                        VSRH[VR_idx] : VR[VR_idx];
-        ++VR_idx;
-
-        RegsToPass.push_back(std::make_pair(VReg, Arg));
+        RegsToPass.push_back(std::make_pair(VR[VR_idx++], Arg));
       } else {
         if (CallConv == CallingConv::Fast)
           ComputePtrOff();
 
         LowerMemOpCallTo(DAG, MF, Chain, Arg, PtrOff, SPDiff, ArgOffset,
                          true, isTailCall, true, MemOpChains,
                          TailCallArguments, dl);
         if (CallConv == CallingConv::Fast)
static SDValue BuildVSLDOI(SDValue LHS, SDValue RHS, unsigned Amt, EVT VT,
   int Ops[16];
   for (unsigned i = 0; i != 16; ++i)
     Ops[i] = i + Amt;
   SDValue T = DAG.getVectorShuffle(MVT::v16i8, dl, LHS, RHS, Ops);
   return DAG.getNode(ISD::BITCAST, dl, VT, T);
 }
 
 static bool isNonConstSplatBV(BuildVectorSDNode *BVN, EVT Type) {
+  if (BVN->isConstant())
+    return false;
   if (BVN->getValueType(0) != Type)
     return false;
   auto OpZero = BVN->getOperand(0);
   for (int i = 1, e = BVN->getNumOperands(); i < e; i++)
     if (BVN->getOperand(i) != OpZero)
       return false;
   return true;
 }
SDValue PPCTargetLowering::LowerBUILD_VECTOR(SDValue Op,
   if (! BVN->isConstantSplat(APSplatBits, APSplatUndef, SplatBitSize,
                              HasAnyUndefs, 0, !Subtarget.isLittleEndian()) ||
       SplatBitSize > 32) {
     // We can splat a non-const value on CPU's that implement ISA 3.0
     // in two ways: LXVWSX (load and splat) and MTVSRWS(move and splat).
     auto OpZero = BVN->getOperand(0);
     bool CanLoadAndSplat = OpZero.getOpcode() == ISD::LOAD &&
       BVN->isOnlyUserOf(OpZero.getNode());
-    if (Subtarget.isISA3_0() &&
-        isNonConstSplatBV(BVN, MVT::v4i32) && !CanLoadAndSplat)
+    if (Subtarget.isISA3_0() && !CanLoadAndSplat &&
+        (isNonConstSplatBV(BVN, MVT::v4i32) ||
+         isNonConstSplatBV(BVN, MVT::v2i64)))
       return Op;
     return SDValue();
   }
 
   unsigned SplatBits = APSplatBits.getZExtValue();
   unsigned SplatUndef = APSplatUndef.getZExtValue();
   unsigned SplatSize = SplatBitSize / 8;
 
SDValue PPCTargetLowering::combineFPToIntToFP(SDNode *N,
 
   if (!Subtarget.has64BitSupport())
     return SDValue();
 
   SelectionDAG &DAG = DCI.DAG;
   SDLoc dl(N);
   SDValue Op(N, 0);
 
+  SDValue FirstOperand(Op.getOperand(0));
+  bool SubWordLoad = FirstOperand.getOpcode() == ISD::LOAD &&
+    (FirstOperand.getValueType() == MVT::i8 ||
+     FirstOperand.getValueType() == MVT::i16);
+  if (Subtarget.hasP9Vector() && Subtarget.hasP9Altivec() && SubWordLoad) {
+    int ByteWidth = FirstOperand.getValueType() == MVT::i8 ? 1 : 2;
+    bool Signed = N->getOpcode() == ISD::SINT_TO_FP;
+    bool DstDouble = Op.getValueType() == MVT::f64;
+    unsigned ConvOp = Signed ?
+      (DstDouble ? PPCISD::FCFID  : PPCISD::FCFIDS) :
+      (DstDouble ? PPCISD::FCFIDU : PPCISD::FCFIDUS);
+    SDValue WidthConst = DAG.getIntPtrConstant(ByteWidth, dl, false);
+    LoadSDNode *LDN = cast<LoadSDNode>(FirstOperand.getNode());
+    SDValue Ops[] = { LDN->getChain(), LDN->getBasePtr(), WidthConst };
+    SDValue Ld = DAG.getMemIntrinsicNode(PPCISD::LXSIZX, dl,
+                                         DAG.getVTList(MVT::f64, MVT::Other),
+                                         Ops, MVT::i8, LDN->getMemOperand());
+
+    SDValue FP;
+    // For signed conversion, we need to sign-extend the value in the VSR
+    if (Signed) {
+      SDValue ExtOps[] = { Ld, WidthConst };
+      SDValue Ext = DAG.getNode(PPCISD::VEXTS, dl, MVT::f64, ExtOps);
+      FP = DAG.getNode(ConvOp, dl, DstDouble ? MVT::f64 : MVT::f32, Ext);
+    } else
+      FP = DAG.getNode(ConvOp, dl, DstDouble ? MVT::f64 : MVT::f32, Ld);
+    return FP;
+  }
+
   // Don't handle ppc_fp128 here or i1 conversions.
   if (Op.getValueType() != MVT::f32 && Op.getValueType() != MVT::f64)
     return SDValue();
   if (Op.getOperand(0).getValueType() == MVT::i1)
     return SDValue();
 
   // For i32 intermediate values, unfortunately, the conversion functions
   // leave the upper 32 bits of the value are undefined. Within the set of
SDValue PPCTargetLowering::PerformDAGCombine(SDNode *N,
   case ISD::TRUNCATE:
   case ISD::SETCC:
   case ISD::SELECT_CC:
     return DAGCombineTruncBoolExt(N, DCI);
   case ISD::SINT_TO_FP:
   case ISD::UINT_TO_FP:
     return combineFPToIntToFP(N, DCI);
   case ISD::STORE: {
+    EVT Op1VT = N->getOperand(1).getValueType();
+    bool ValidTypeForStoreFltAsInt = (Op1VT == MVT::i32) ||
+      (Subtarget.hasP9Vector() && (Op1VT == MVT::i8 || Op1VT == MVT::i16));
+
     // Turn STORE (FP_TO_SINT F) -> STFIWX(FCTIWZ(F)).
     if (Subtarget.hasSTFIWX() && !cast<StoreSDNode>(N)->isTruncatingStore() &&
         N->getOperand(1).getOpcode() == ISD::FP_TO_SINT &&
-        N->getOperand(1).getValueType() == MVT::i32 &&
+        ValidTypeForStoreFltAsInt &&
         N->getOperand(1).getOperand(0).getValueType() != MVT::ppcf128) {
       SDValue Val = N->getOperand(1).getOperand(0);
       if (Val.getValueType() == MVT::f32) {
         Val = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f64, Val);
         DCI.AddToWorklist(Val.getNode());
       }
       Val = DAG.getNode(PPCISD::FCTIWZ, dl, MVT::f64, Val);
       DCI.AddToWorklist(Val.getNode());
 
+      if (Op1VT == MVT::i32) {
         SDValue Ops[] = {
           N->getOperand(0), Val, N->getOperand(2),
           DAG.getValueType(N->getOperand(1).getValueType())
         };
 
         Val = DAG.getMemIntrinsicNode(PPCISD::STFIWX, dl,
                 DAG.getVTList(MVT::Other), Ops,
                 cast<StoreSDNode>(N)->getMemoryVT(),
                 cast<StoreSDNode>(N)->getMemOperand());
+      } else {
+        unsigned ByteWidth = N->getOperand(1).getValueType() == MVT::i8 ? 1 : 2;
+        SDValue WidthConst = DAG.getIntPtrConstant(ByteWidth, dl, false);
+
+        SDValue Ops[] = {
+          N->getOperand(0), Val, N->getOperand(2), WidthConst,
+          DAG.getValueType(N->getOperand(1).getValueType())
+        };
+        Val = DAG.getMemIntrinsicNode(PPCISD::STXSIX, dl,
+                                      DAG.getVTList(MVT::Other), Ops,
+                                      cast<StoreSDNode>(N)->getMemoryVT(),
+                                      cast<StoreSDNode>(N)->getMemOperand());
+      }
+
       DCI.AddToWorklist(Val.getNode());
       return Val;
     }
 
     // Turn STORE (BSWAP) -> sthbrx/stwbrx.
     if (cast<StoreSDNode>(N)->isUnindexed() &&
         N->getOperand(1).getOpcode() == ISD::BSWAP &&
         N->getOperand(1).getNode()->hasOneUse() &&

lib/Target/PowerPC/PPCInstrAltivec.td

 def VSPLTH : VXForm_1<588, (outs vrrc:$vD), (ins u5imm:$UIMM, vrrc:$vB),
                       "vsplth $vD, $vB, $UIMM", IIC_VecPerm,
                       [(set v16i8:$vD,
                         (vsplth_shuffle:$UIMM v16i8:$vB, (undef)))]>;
 def VSPLTW : VXForm_1<652, (outs vrrc:$vD), (ins u5imm:$UIMM, vrrc:$vB),
                       "vspltw $vD, $vB, $UIMM", IIC_VecPerm,
                       [(set v16i8:$vD, 
                         (vspltw_shuffle:$UIMM v16i8:$vB, (undef)))]>;
+let isCodeGenOnly = 1 in {
+  def VSPLTBs : VXForm_1<524, (outs vrrc:$vD), (ins u5imm:$UIMM, vfrc:$vB),
+                         "vspltb $vD, $vB, $UIMM", IIC_VecPerm, []>;
+  def VSPLTHs : VXForm_1<588, (outs vrrc:$vD), (ins u5imm:$UIMM, vfrc:$vB),
+                         "vsplth $vD, $vB, $UIMM", IIC_VecPerm, []>;
+}
 
 def VSR    : VX1_Int_Ty< 708, "vsr"  , int_ppc_altivec_vsr,  v4i32>;
 def VSRO   : VX1_Int_Ty<1100, "vsro" , int_ppc_altivec_vsro, v4i32>;
 
 def VSRAB  : VX1_Int_Ty< 772, "vsrab", int_ppc_altivec_vsrab, v16i8>;
 def VSRAH  : VX1_Int_Ty< 836, "vsrah", int_ppc_altivec_vsrah, v8i16>;
 def VSRAW  : VX1_Int_Ty< 900, "vsraw", int_ppc_altivec_vsraw, v4i32>;
 def VSRB   : VX1_Int_Ty< 516, "vsrb" , int_ppc_altivec_vsrb , v16i8>;

 def VINSERTB : VX1_VT5_UIM5_VB5<781, "vinsertb", []>;
 def VINSERTH : VX1_VT5_UIM5_VB5<845, "vinserth", []>;
 def VINSERTW : VX1_VT5_UIM5_VB5<909, "vinsertw", []>;
 def VINSERTD : VX1_VT5_UIM5_VB5<973, "vinsertd", []>;
 
 class VX_VT5_EO5_VB5<bits<11> xo, bits<5> eo, string opc, list<dag> pattern>
   : VXForm_RD5_XO5_RS5<xo, eo, (outs vrrc:$vD), (ins vrrc:$vB),
                        !strconcat(opc, " $vD, $vB"), IIC_VecGeneral, pattern>;
+class VX_VT5_EO5_VB5s<bits<11> xo, bits<5> eo, string opc, list<dag> pattern>
+  : VXForm_RD5_XO5_RS5<xo, eo, (outs vfrc:$vD), (ins vfrc:$vB),
+                       !strconcat(opc, " $vD, $vB"), IIC_VecGeneral, pattern>;
 
 // Vector Count Leading/Trailing Zero LSB. Result is placed into GPR[rD]
 def VCLZLSBB : VXForm_RD5_XO5_RS5<1538, 0, (outs g8rc:$rD), (ins vrrc:$vB),
                                   "vclzlsbb $rD, $vB", IIC_VecGeneral, []>;
 def VCTZLSBB : VXForm_RD5_XO5_RS5<1538, 1, (outs g8rc:$rD), (ins vrrc:$vB),
                                   "vctzlsbb $rD, $vB", IIC_VecGeneral, []>;
 // Vector Count Trailing Zeros
 def VCTZB : VX_VT5_EO5_VB5<1538, 28, "vctzb",
                            [(set v16i8:$vD, (cttz v16i8:$vB))]>;
 def VCTZH : VX_VT5_EO5_VB5<1538, 29, "vctzh",
                            [(set v8i16:$vD, (cttz v8i16:$vB))]>;
 def VCTZW : VX_VT5_EO5_VB5<1538, 30, "vctzw",
                            [(set v4i32:$vD, (cttz v4i32:$vB))]>;
 def VCTZD : VX_VT5_EO5_VB5<1538, 31, "vctzd",
                            [(set v2i64:$vD, (cttz v2i64:$vB))]>;
 
 // Vector Extend Sign
 def VEXTSB2W : VX_VT5_EO5_VB5<1538, 16, "vextsb2w", []>;
 def VEXTSH2W : VX_VT5_EO5_VB5<1538, 17, "vextsh2w", []>;
 def VEXTSB2D : VX_VT5_EO5_VB5<1538, 24, "vextsb2d", []>;
 def VEXTSH2D : VX_VT5_EO5_VB5<1538, 25, "vextsh2d", []>;
 def VEXTSW2D : VX_VT5_EO5_VB5<1538, 26, "vextsw2d", []>;
+let isCodeGenOnly = 1 in {
+  def VEXTSB2Ws : VX_VT5_EO5_VB5s<1538, 16, "vextsb2w", []>;
+  def VEXTSH2Ws : VX_VT5_EO5_VB5s<1538, 17, "vextsh2w", []>;
+  def VEXTSB2Ds : VX_VT5_EO5_VB5s<1538, 24, "vextsb2d", []>;
+  def VEXTSH2Ds : VX_VT5_EO5_VB5s<1538, 25, "vextsh2d", []>;
+  def VEXTSW2Ds : VX_VT5_EO5_VB5s<1538, 26, "vextsw2d", []>;
+}
 
 // Vector Integer Negate
 def VNEGW : VX_VT5_EO5_VB5<1538, 6, "vnegw", []>;
 def VNEGD : VX_VT5_EO5_VB5<1538, 7, "vnegd", []>;
 
 // Vector Parity Byte
 def VPRTYBW : VX_VT5_EO5_VB5<1538,  8, "vprtybw", []>;
 def VPRTYBD : VX_VT5_EO5_VB5<1538,  9, "vprtybd", []>;

lib/Target/PowerPC/PPCInstrFormats.td

class I<bits<6> opcode, dag OOL, dag IOL, string asmstr, InstrItinClass itin>
 
   /// These fields correspond to the fields in PPCInstrInfo.h.  Any changes to
   /// these must be reflected there!  See comments there for what these are.
   let TSFlags{0}   = PPC970_First;
   let TSFlags{1}   = PPC970_Single;
   let TSFlags{2}   = PPC970_Cracked;
   let TSFlags{5-3} = PPC970_Unit;
 
+  /// Indicate that the VSX instruction is to use VSX numbering/encoding.
+  /// Since ISA 3.0, there are scalar instructions that use the upper
+  /// half of the VSX register set only. Rather than adding further complexity
+  /// to the register class set, the VSX registers just include the Altivec
+  /// registers and this flag decides the numbering to be used for them.
+  bits<1> UseVSXReg = 0;
+  let TSFlags{6}   = UseVSXReg;
+
   // Fields used for relation models.
   string BaseName = "";
 
   // For cases where multiple instruction definitions really represent the
   // same underlying instruction but with one definition for 64-bit arguments
   // and one for 32-bit arguments, this bit breaks the degeneracy between
   // the two forms and allows TableGen to generate mapping tables.
   bit Interpretation64Bit = 0;
 }
 
 class PPC970_DGroup_First   { bits<1> PPC970_First = 1;  }
 class PPC970_DGroup_Single  { bits<1> PPC970_Single = 1; }
 class PPC970_DGroup_Cracked { bits<1> PPC970_Cracked = 1; }
 class PPC970_MicroCode;
 
 class PPC970_Unit_Pseudo   { bits<3> PPC970_Unit = 0;   }
 class PPC970_Unit_FXU      { bits<3> PPC970_Unit = 1;   }
 class PPC970_Unit_LSU      { bits<3> PPC970_Unit = 2;   }
 class PPC970_Unit_FPU      { bits<3> PPC970_Unit = 3;   }
 class PPC970_Unit_CRU      { bits<3> PPC970_Unit = 4;   }
 class PPC970_Unit_VALU     { bits<3> PPC970_Unit = 5;   }
 class PPC970_Unit_VPERM    { bits<3> PPC970_Unit = 6;   }
 class PPC970_Unit_BRU      { bits<3> PPC970_Unit = 7;   }
 
+class UseVSXReg { bits<1> UseVSXReg = 1; }
+
 // Two joined instructions; used to emit two adjacent instructions as one.
 // The itinerary from the first instruction is used for scheduling and
 // classification.
 class I2<bits<6> opcode1, bits<6> opcode2, dag OOL, dag IOL, string asmstr,
          InstrItinClass itin>
         : Instruction {
   field bits<64> Inst;
   field bits<64> SoftFail = 0;

lib/Target/PowerPC/PPCInstrInfo.h

enum PPC970_Unit {
   PPC970_FXU    = 1 << PPC970_Shift,   // Fixed Point (aka Integer/ALU) Unit
   PPC970_LSU    = 2 << PPC970_Shift,   // Load Store Unit
   PPC970_FPU    = 3 << PPC970_Shift,   // Floating Point Unit
   PPC970_CRU    = 4 << PPC970_Shift,   // Control Register Unit
   PPC970_VALU   = 5 << PPC970_Shift,   // Vector ALU
   PPC970_VPERM  = 6 << PPC970_Shift,   // Vector Permute Unit
   PPC970_BRU    = 7 << PPC970_Shift    // Branch Unit
 };
+
+enum {
+  /// Shift count to bypass PPC970 flags
+  NewDef_Shift = 6,
+
+  /// The VSX instruction that uses VSX register (vs0-vs63), instead of VMX
+  /// register (v0-v31).
+  UseVSXReg = 0x1 << NewDef_Shift
+};
 } // end namespace PPCII
 
 class PPCSubtarget;
 class PPCInstrInfo : public PPCGenInstrInfo {
   PPCSubtarget &Subtarget;
   const PPCRegisterInfo RI;
 
   bool StoreRegToStackSlot(MachineFunction &MF,
public:
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableDirectMachineOperandTargetFlags() const override;
 
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableBitmaskMachineOperandTargetFlags() const override;
 
   // Lower pseudo instructions after register allocation.
   bool expandPostRAPseudo(MachineInstr &MI) const override;
+
+  static bool isVFRegister(unsigned Reg) {
+    return Reg >= PPC::VF0 && Reg <= PPC::VF31;
+  }
+  static bool isVRRegister(unsigned Reg) {
+    return Reg >= PPC::V0 && Reg <= PPC::V31;
+  }
 };
 
 }
 
 #endif

lib/Target/PowerPC/PPCInstrInfo.cpp

if (PPC::F8RCRegClass.contains(DestReg) &&
       PPC::VSRCRegClass.contains(SrcReg)) {
     unsigned SuperReg =
       TRI->getMatchingSuperReg(DestReg, PPC::sub_64, &PPC::VSRCRegClass);
 
     if (VSXSelfCopyCrash && SrcReg == SuperReg)
       llvm_unreachable("nop VSX copy");
 
     DestReg = SuperReg;
-  } else if (PPC::VRRCRegClass.contains(DestReg) &&
-             PPC::VSRCRegClass.contains(SrcReg)) {
-    unsigned SuperReg =
-      TRI->getMatchingSuperReg(DestReg, PPC::sub_128, &PPC::VSRCRegClass);
-
-    if (VSXSelfCopyCrash && SrcReg == SuperReg)
-      llvm_unreachable("nop VSX copy");
-
-    DestReg = SuperReg;
   } else if (PPC::F8RCRegClass.contains(SrcReg) &&
              PPC::VSRCRegClass.contains(DestReg)) {
     unsigned SuperReg =
       TRI->getMatchingSuperReg(SrcReg, PPC::sub_64, &PPC::VSRCRegClass);
 
     if (VSXSelfCopyCrash && DestReg == SuperReg)
       llvm_unreachable("nop VSX copy");
 
     SrcReg = SuperReg;
-  } else if (PPC::VRRCRegClass.contains(SrcReg) &&
-             PPC::VSRCRegClass.contains(DestReg)) {
-    unsigned SuperReg =
-      TRI->getMatchingSuperReg(SrcReg, PPC::sub_128, &PPC::VSRCRegClass);
-
-    if (VSXSelfCopyCrash && DestReg == SuperReg)
-      llvm_unreachable("nop VSX copy");
-
-    SrcReg = SuperReg;
   }
 
   // Different class register copy
   if (PPC::CRBITRCRegClass.contains(SrcReg) &&
       PPC::GPRCRegClass.contains(DestReg)) {
     unsigned CRReg = getCRFromCRBit(SrcReg);
     BuildMI(MBB, I, DL, get(PPC::MFOCRF), DestReg).addReg(CRReg);
     getKillRegState(KillSrc);
PPCInstrInfo::storeRegToStackSlot(MachineBasicBlock &MBB,
                                   const TargetRegisterClass *RC,
                                   const TargetRegisterInfo *TRI) const {
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr*, 4> NewMIs;
 
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
   FuncInfo->setHasSpills();
 
+  // Because VRRC and VSRC use incompatible store/load instruction, and backend
+  // can spill VRRC to a frame then reload it by using VSRC, we have to make
+  // sure this is not going to happen, so when target has VSX, we just
+  // spill/reload VRRC by VSRC.
+  if (Subtarget.hasVSX() && RC == &PPC::VRRCRegClass)
+    RC = &PPC::VSRCRegClass;
+
   bool NonRI = false, SpillsVRS = false;
   if (StoreRegToStackSlot(MF, SrcReg, isKill, FrameIdx, RC, NewMIs,
                           NonRI, SpillsVRS))
     FuncInfo->setSpillsCR();
 
   if (SpillsVRS)
     FuncInfo->setSpillsVRSAVE();
 
PPCInstrInfo::loadRegFromStackSlot(MachineBasicBlock &MBB,
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr*, 4> NewMIs;
   DebugLoc DL;
   if (MI != MBB.end()) DL = MI->getDebugLoc();
 
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
   FuncInfo->setHasSpills();
 
+  // Because VRRC and VSRC use incompatible store/load instruction, and backend
+  // can spill VRRC to a frame then reload it by using VSRC, we have to make
+  // sure this is not going to happen, so when target has VSX, we just
+  // spill/reload VRRC by VSRC.
+  if (Subtarget.hasVSX() && RC == &PPC::VRRCRegClass)
+    RC = &PPC::VSRCRegClass;
+
   bool NonRI = false, SpillsVRS = false;
   if (LoadRegFromStackSlot(MF, DL, DestReg, FrameIdx, RC, NewMIs,
                            NonRI, SpillsVRS))
     FuncInfo->setSpillsCR();
 
   if (SpillsVRS)
     FuncInfo->setSpillsVRSAVE();
 

lib/Target/PowerPC/PPCInstrInfo.td

 // PowerPC specific type constraints.
 //
 def SDT_PPCstfiwx : SDTypeProfile<0, 2, [ // stfiwx
   SDTCisVT<0, f64>, SDTCisPtrTy<1>
 ]>;
 def SDT_PPClfiwx : SDTypeProfile<1, 1, [ // lfiw[az]x
   SDTCisVT<0, f64>, SDTCisPtrTy<1>
 ]>;
+def SDT_PPCLxsizx : SDTypeProfile<1, 2, [
+  SDTCisVT<0, f64>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
+]>;
+def SDT_PPCstxsix : SDTypeProfile<0, 3, [
+  SDTCisVT<0, f64>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
+]>;
+def SDT_PPCVexts  : SDTypeProfile<1, 2, [
+  SDTCisVT<0, f64>, SDTCisVT<1, f64>, SDTCisPtrTy<2>
+]>;
 
 def SDT_PPCCallSeqStart : SDCallSeqStart<[ SDTCisVT<0, i32> ]>;
 def SDT_PPCCallSeqEnd   : SDCallSeqEnd<[ SDTCisVT<0, i32>,
                                          SDTCisVT<1, i32> ]>;
 def SDT_PPCvperm   : SDTypeProfile<1, 3, [
   SDTCisVT<3, v16i8>, SDTCisSameAs<0, 1>, SDTCisSameAs<0, 2>
 ]>;
 

 def PPCfctiduz: SDNode<"PPCISD::FCTIDUZ",SDTFPUnaryOp, []>;
 def PPCfctiwuz: SDNode<"PPCISD::FCTIWUZ",SDTFPUnaryOp, []>;
 def PPCstfiwx : SDNode<"PPCISD::STFIWX", SDT_PPCstfiwx,
                        [SDNPHasChain, SDNPMayStore]>;
 def PPClfiwax : SDNode<"PPCISD::LFIWAX", SDT_PPClfiwx,
                        [SDNPHasChain, SDNPMayLoad]>;
 def PPClfiwzx : SDNode<"PPCISD::LFIWZX", SDT_PPClfiwx,
                        [SDNPHasChain, SDNPMayLoad]>;
+def PPClxsizx : SDNode<"PPCISD::LXSIZX", SDT_PPCLxsizx,
+                       [SDNPHasChain, SDNPMayLoad]>;
+def PPCstxsix : SDNode<"PPCISD::STXSIX", SDT_PPCstxsix,
+                       [SDNPHasChain, SDNPMayStore]>;
+def PPCVexts  : SDNode<"PPCISD::VEXTS", SDT_PPCVexts, []>;
 
 // Extract FPSCR (not modeled at the DAG level).
 def PPCmffs   : SDNode<"PPCISD::MFFS",
                        SDTypeProfile<1, 0, [SDTCisVT<0, f64>]>, []>;
 
 // Perform FADD in round-to-zero mode.
 def PPCfaddrtz: SDNode<"PPCISD::FADDRTZ", SDTFPBinOp, []>;
 
def f4rc : RegisterOperand<F4RC> {
   let ParserMatchClass = PPCRegF4RCAsmOperand;
 }
 def PPCRegVRRCAsmOperand : AsmOperandClass {
   let Name = "RegVRRC"; let PredicateMethod = "isRegNumber";
 }
 def vrrc : RegisterOperand<VRRC> {
   let ParserMatchClass = PPCRegVRRCAsmOperand;
 }
+def PPCRegVFRCAsmOperand : AsmOperandClass {
+  let Name = "RegVFRC"; let PredicateMethod = "isRegNumber";
+}
+def vfrc : RegisterOperand<VFRC> {
+  let ParserMatchClass = PPCRegVFRCAsmOperand;
+}
 def PPCRegCRBITRCAsmOperand : AsmOperandClass {
   let Name = "RegCRBITRC"; let PredicateMethod = "isCRBitNumber";
 }
 def crbitrc : RegisterOperand<CRBITRC> {
   let ParserMatchClass = PPCRegCRBITRCAsmOperand;
 }
 def PPCRegCRRCAsmOperand : AsmOperandClass {
   let Name = "RegCRRC"; let PredicateMethod = "isCCRegNumber";

lib/Target/PowerPC/PPCInstrVSX.td

let BaseName = asmbase in {
     def o    : XX3Form_Rc<opcode, xo, (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                        !strconcat(asmbase, !strconcat(". ", asmstr)), itin,
                        [(set InTy:$XT,
                                 (InTy (PPCvcmp_o InTy:$XA, InTy:$XB, xo)))]>,
                        isDOT;
   }
 }
 
+// Instruction form with a single input register for instructions such as
+// XXPERMDI. The reason for defining this is that specifying multiple chained
+// operands (such as loads) to an instruction will perform both chained
+// operations rather than coalescing them into a single register. This simply
+// forces the instruction to use the same register for both inputs.
+class XX3Form_2s<bits<6> opcode, bits<5> xo, dag OOL, dag IOL, string asmstr,
+                 InstrItinClass itin, list<dag> pattern>
+  : XX3Form_2<opcode, xo, OOL, IOL, asmstr, itin, pattern> {
+    let XB = XA;
+}
+
 def HasVSX : Predicate<"PPCSubTarget->hasVSX()">;
 def IsLittleEndian : Predicate<"PPCSubTarget->isLittleEndian()">;
 def IsBigEndian : Predicate<"!PPCSubTarget->isLittleEndian()">;
 def HasOnlySwappingMemOps : Predicate<"!PPCSubTarget->hasP9Vector()">;
 
 let Predicates = [HasVSX] in {
 let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
+let UseVSXReg = 1 in {
 let hasSideEffects = 0 in { // VSX instructions don't have side effects.
 let Uses = [RM] in {
 
   // Load indexed instructions
   let mayLoad = 1 in {
     def LXSDX : XX1Form<31, 588,
                         (outs vsfrc:$XT), (ins memrr:$src),
                         "lxsdx $XT, $src", IIC_LdStLFD,
def XXMRGHW : XX3Form<60, 18,
                        "xxmrghw $XT, $XA, $XB", IIC_VecPerm, []>;
   def XXMRGLW : XX3Form<60, 50,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                        "xxmrglw $XT, $XA, $XB", IIC_VecPerm, []>;
 
   def XXPERMDI : XX3Form_2<60, 10,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB, u2imm:$DM),
                        "xxpermdi $XT, $XA, $XB, $DM", IIC_VecPerm, []>;
+  let isCodeGenOnly = 1 in
+  def XXPERMDIs : XX3Form_2s<60, 10, (outs vsrc:$XT), (ins vfrc:$XA, u2imm:$DM),
+                             "xxpermdi $XT, $XA, $XA, $DM", IIC_VecPerm, []>;
   def XXSEL : XX4Form<60, 3,
                       (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB, vsrc:$XC),
                       "xxsel $XT, $XA, $XB, $XC", IIC_VecPerm, []>;
 
   def XXSLDWI : XX3Form_2<60, 2,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB, u2imm:$SHW),
                        "xxsldwi $XT, $XA, $XB, $SHW", IIC_VecPerm,
                        [(set v4i32:$XT, (PPCvecshl v4i32:$XA, v4i32:$XB,
                                                   imm32SExt16:$SHW))]>;
   def XXSPLTW : XX2Form_2<60, 164,
                        (outs vsrc:$XT), (ins vsrc:$XB, u2imm:$UIM),
                        "xxspltw $XT, $XB, $UIM", IIC_VecPerm,
                        [(set v4i32:$XT,
                              (PPCxxsplt v4i32:$XB, imm32SExt16:$UIM))]>;
+  let isCodeGenOnly = 1 in
+  def XXSPLTWs : XX2Form_2<60, 164,
+                       (outs vsrc:$XT), (ins vfrc:$XB, u2imm:$UIM),
+                       "xxspltw $XT, $XB, $UIM", IIC_VecPerm, []>;
 } // hasSideEffects
+} // UseVSXReg = 1
 
 // SELECT_CC_* - Used to implement the SELECT_CC DAG operation.  Expanded after
 // instruction selection into a branch sequence.
 let usesCustomInserter = 1,    // Expanded after instruction selection.
     PPC970_Single = 1 in {
 
   def SELECT_CC_VSRC: Pseudo<(outs vsrc:$dst),
                              (ins crrc:$cond, vsrc:$T, vsrc:$F, i32imm:$BROPC),

 def : InstAlias<"xxspltd $XT, $XB, 1",
                 (XXPERMDI vsrc:$XT, vsrc:$XB, vsrc:$XB, 3)>;
 def : InstAlias<"xxmrghd $XT, $XA, $XB",
                 (XXPERMDI vsrc:$XT, vsrc:$XA, vsrc:$XB, 0)>;
 def : InstAlias<"xxmrgld $XT, $XA, $XB",
                 (XXPERMDI vsrc:$XT, vsrc:$XA, vsrc:$XB, 3)>;
 def : InstAlias<"xxswapd $XT, $XB",
                 (XXPERMDI vsrc:$XT, vsrc:$XB, vsrc:$XB, 2)>;
+def : InstAlias<"xxspltd $XT, $XB, 0",
+                (XXPERMDIs vsrc:$XT, vfrc:$XB, 0)>;
+def : InstAlias<"xxspltd $XT, $XB, 1",
+                (XXPERMDIs vsrc:$XT, vfrc:$XB, 3)>;
+def : InstAlias<"xxswapd $XT, $XB",
+                (XXPERMDIs vsrc:$XT, vfrc:$XB, 2)>;
 
 let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
 
 def : Pat<(v4i32 (vnot_ppc v4i32:$A)),
           (v4i32 (XXLNOR $A, $A))>;
 let Predicates = [IsBigEndian] in {
 def : Pat<(v2f64 (scalar_to_vector f64:$A)),
           (v2f64 (SUBREG_TO_REG (i64 1), $A, sub_64))>;
def : Pat<(f64 (PPCfcfidu (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
           (f64 (XSCVUXDDP (COPY_TO_REGCLASS $S, VSFRC)))>;
 def : Pat<(f64 (PPCfcfidu (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
           (f64 (XSCVUXDDP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
 } // IsBigEndian
 
 } // AddedComplexity
 } // HasVSX
 
+def ScalarLoads {
+  dag Li8 =       (i32 (extloadi8 xoaddr:$src));
+  dag ZELi8 =     (i32 (zextloadi8 xoaddr:$src));
+  dag ZELi8i64 =  (i64 (zextloadi8 xoaddr:$src));
+  dag SELi8 =     (i32 (sext_inreg (extloadi8 xoaddr:$src), i8));
+  dag SELi8i64 =  (i64 (sext_inreg (extloadi8 xoaddr:$src), i8));
+
+  dag Li16 =      (i32 (extloadi16 xoaddr:$src));
+  dag ZELi16 =    (i32 (zextloadi16 xoaddr:$src));
+  dag ZELi16i64 = (i64 (zextloadi16 xoaddr:$src));
+  dag SELi16 =    (i32 (sextloadi16 xoaddr:$src));
+  dag SELi16i64 = (i64 (sextloadi16 xoaddr:$src));
+
+  dag Li32 = (i32 (load xoaddr:$src));
+}
+
 // The following VSX instructions were introduced in Power ISA 2.07
 /* FIXME: if the operands are v2i64, these patterns will not match.
    we should define new patterns or otherwise match the same patterns
    when the elements are larger than i32.
 */
 def HasP8Vector : Predicate<"PPCSubTarget->hasP8Vector()">;
 def HasDirectMove : Predicate<"PPCSubTarget->hasDirectMove()">;
 let Predicates = [HasP8Vector] in {
 let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
-  let isCommutable = 1 in {
+  let isCommutable = 1, UseVSXReg = 1 in {
     def XXLEQV : XX3Form<60, 186,
                          (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                          "xxleqv $XT, $XA, $XB", IIC_VecGeneral,
                          [(set v4i32:$XT, (vnot_ppc (xor v4i32:$XA, v4i32:$XB)))]>;
     def XXLNAND : XX3Form<60, 178,
                           (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                           "xxlnand $XT, $XA, $XB", IIC_VecGeneral,
                           [(set v4i32:$XT, (vnot_ppc (and v4i32:$XA,
                                                     v4i32:$XB)))]>;
-  } // isCommutable
+  } // isCommutable, UseVSXReg
 
   def : Pat<(int_ppc_vsx_xxleqv v4i32:$A, v4i32:$B),
             (XXLEQV $A, $B)>;
 
+  let UseVSXReg = 1 in {
   def XXLORC : XX3Form<60, 170,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                        "xxlorc $XT, $XA, $XB", IIC_VecGeneral,
                        [(set v4i32:$XT, (or v4i32:$XA, (vnot_ppc v4i32:$XB)))]>;
 
   // VSX scalar loads introduced in ISA 2.07
   let mayLoad = 1 in {
     def LXSSPX : XX1Form<31, 524, (outs vssrc:$XT), (ins memrr:$src),
let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
   let mayStore = 1 in {
     def STXSSPX : XX1Form<31, 652, (outs), (ins vssrc:$XT, memrr:$dst),
                           "stxsspx $XT, $dst", IIC_LdStSTFD,
                           [(store f32:$XT, xoaddr:$dst)]>;
     def STXSIWX : XX1Form<31, 140, (outs), (ins vsfrc:$XT, memrr:$dst),
                           "stxsiwx $XT, $dst", IIC_LdStSTFD,
                           [(PPCstfiwx f64:$XT, xoaddr:$dst)]>;
   } // mayStore
+  } // UseVSXReg = 1
 
   def : Pat<(f64 (extloadf32 xoaddr:$src)),
             (COPY_TO_REGCLASS (LXSSPX xoaddr:$src), VSFRC)>;
   def : Pat<(f64 (fpextend f32:$src)),
             (COPY_TO_REGCLASS $src, VSFRC)>;
 
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLT)),
             (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGE)),
             (SELECT_VSSRC (CRORC  $lhs, $rhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGT)),
             (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGT)),
             (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETNE)),
             (SELECT_VSSRC (CRXOR $lhs, $rhs), $tval, $fval)>;
 
+  let UseVSXReg = 1 in {
   // VSX Elementary Scalar FP arithmetic (SP)
   let isCommutable = 1 in {
     def XSADDSP : XX3Form<60, 0,
                           (outs vssrc:$XT), (ins vssrc:$XA, vssrc:$XB),
                           "xsaddsp $XT, $XA, $XB", IIC_VecFP,
                           [(set f32:$XT, (fadd f32:$XA, f32:$XB))]>;
     def XSMULSP : XX3Form<60, 16,
                           (outs vssrc:$XT), (ins vssrc:$XA, vssrc:$XB),
def XSCVUXDSP : XX2Form<60, 296,
                       "xscvuxdsp $XT, $XB", IIC_VecFP,
                       [(set f32:$XT, (PPCfcfidus f64:$XB))]>;
 
   // Conversions between vector and scalar single precision
   def XSCVDPSPN : XX2Form<60, 267, (outs vsrc:$XT), (ins vssrc:$XB),
                           "xscvdpspn $XT, $XB", IIC_VecFP, []>;
   def XSCVSPDPN : XX2Form<60, 331, (outs vssrc:$XT), (ins vsrc:$XB),
                           "xscvspdpn $XT, $XB", IIC_VecFP, []>;
+  } // UseVSXReg = 1
 
   let Predicates = [IsLittleEndian] in {
   def : Pat<(f32 (PPCfcfids (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfids (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (f64 (COPY_TO_REGCLASS $S, VSRC)), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (f64 (COPY_TO_REGCLASS $S, VSRC)), VSFRC)))>;
   }
 
   let Predicates = [IsBigEndian] in {
   def : Pat<(f32 (PPCfcfids (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS $S, VSFRC)))>;
   def : Pat<(f32 (PPCfcfids (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS $S, VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   }
+  def : Pat<(v4i32 (scalar_to_vector ScalarLoads.Li32)),
+            (v4i32 (XXSPLTWs (LXSIWAX xoaddr:$src), 1))>;
 } // AddedComplexity = 400
 } // HasP8Vector
 
+let UseVSXReg = 1 in {
 let Predicates = [HasDirectMove] in {
   // VSX direct move instructions
   def MFVSRD : XX1_RS6_RD5_XO<31, 51, (outs g8rc:$rA), (ins vsfrc:$XT),
                               "mfvsrd $rA, $XT", IIC_VecGeneral,
                               [(set i64:$rA, (PPCmfvsr f64:$XT))]>,
       Requires<[In64BitMode]>;
   def MFVSRWZ : XX1_RS6_RD5_XO<31, 115, (outs gprc:$rA), (ins vsfrc:$XT),
                                "mfvsrwz $rA, $XT", IIC_VecGeneral,
def MTVSRDD: XX1Form<31, 435, (outs vsrc:$XT), (ins g8rc:$rA, g8rc:$rB),
                        "mtvsrdd $XT, $rA, $rB", IIC_VecGeneral,
                        []>, Requires<[In64BitMode]>;
 
   def MFVSRLD: XX1_RS6_RD5_XO<31, 307, (outs g8rc:$rA), (ins vsrc:$XT),
                               "mfvsrld $rA, $XT", IIC_VecGeneral,
                               []>, Requires<[In64BitMode]>;
 
 } // IsISA3_0, HasDirectMove
+} // UseVSXReg = 1
 
 /*  Direct moves of various widths from GPR's into VSR's. Each move lines
     the value up into element 0 (both BE and LE). Namely, entities smaller than
     a doubleword are shifted left and moved for BE. For LE, they're moved, then
     swapped to go into the least significant element of the VSR.
 */
 def MovesToVSR {
   dag BE_BYTE_0 =
let AddedComplexity = 400, Predicates = [HasP9Vector] in {
 
   // [PO VRT XO VRB XO /], but the VRB is only used the left 64 bits (or less),
   // So we use different operand class for VRB
   class X_VT5_XO5_VB5_TyVB<bits<6> opcode, bits<5> xo2, bits<10> xo, string opc,
                            RegisterOperand vbtype, list<dag> pattern>
     : X_RD5_XO5_RS5<opcode, xo2, xo, (outs vrrc:$vT), (ins vbtype:$vB),
                     !strconcat(opc, " $vT, $vB"), IIC_VecFP, pattern>;
 
+  let UseVSXReg = 1 in {
   // [PO T XO B XO BX /]
   class XX2_RT5_XO5_XB6<bits<6> opcode, bits<5> xo2, bits<9> xo, string opc,
                         list<dag> pattern>
     : XX2_RD5_XO5_RS6<opcode, xo2, xo, (outs g8rc:$rT), (ins vsfrc:$XB),
                       !strconcat(opc, " $rT, $XB"), IIC_VecFP, pattern>;
 
   // [PO T XO B XO BX TX]
   class XX2_XT6_XO5_XB6<bits<6> opcode, bits<5> xo2, bits<9> xo, string opc,
                         RegisterOperand vtype, list<dag> pattern>
     : XX2_RD6_XO5_RS6<opcode, xo2, xo, (outs vtype:$XT), (ins vtype:$XB),
                       !strconcat(opc, " $XT, $XB"), IIC_VecFP, pattern>;
 
   // [PO T A B XO AX BX TX], src and dest register use different operand class
   class XX3_XT5_XA5_XB5<bits<6> opcode, bits<8> xo, string opc,
                   RegisterOperand xty, RegisterOperand aty, RegisterOperand bty,
                   InstrItinClass itin, list<dag> pattern>
     : XX3Form<opcode, xo, (outs xty:$XT), (ins aty:$XA, bty:$XB),
               !strconcat(opc, " $XT, $XA, $XB"), itin, pattern>;
+  } // UseVSXReg = 1
 
   // [PO VRT VRA VRB XO /]
   class X_VT5_VA5_VB5<bits<6> opcode, bits<10> xo, string opc,
                       list<dag> pattern>
     : XForm_1<opcode, xo, (outs vrrc:$vT), (ins vrrc:$vA, vrrc:$vB),
               !strconcat(opc, " $vT, $vA, $vB"), IIC_VecFP, pattern>;
 
   // [PO VRT VRA VRB XO RO], Round to Odd version of [PO VRT VRA VRB XO /]
let AddedComplexity = 400, Predicates = [HasP9Vector] in {
 
   // QP Compare Ordered/Unordered
   def XSCMPOQP : X_BF3_VA5_VB5<63, 132, "xscmpoqp", []>;
   def XSCMPUQP : X_BF3_VA5_VB5<63, 644, "xscmpuqp", []>;
 
   // DP/QP Compare Exponents
   def XSCMPEXPDP : XX3Form_1<60, 59,
                              (outs crrc:$crD), (ins vsfrc:$XA, vsfrc:$XB),
-                             "xscmpexpdp $crD, $XA, $XB", IIC_FPCompare, []>;
+                             "xscmpexpdp $crD, $XA, $XB", IIC_FPCompare, []>,
+                   UseVSXReg;
   def XSCMPEXPQP : X_BF3_VA5_VB5<63, 164, "xscmpexpqp", []>;
 
   // DP Compare ==, >=, >, !=
   // Use vsrc for XT, because the entire register of XT is set.
   // XT.dword[1] = 0x0000_0000_0000_0000
   def XSCMPEQDP : XX3_XT5_XA5_XB5<60,  3, "xscmpeqdp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPGEDP : XX3_XT5_XA5_XB5<60, 19, "xscmpgedp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPGTDP : XX3_XT5_XA5_XB5<60, 11, "xscmpgtdp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPNEDP : XX3_XT5_XA5_XB5<60, 27, "xscmpnedp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
+  let UseVSXReg = 1 in {
   // Vector Compare Not Equal
   def XVCMPNEDP  : XX3Form_Rc<60, 123,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnedp  $XT, $XA, $XB", IIC_VecFPCompare, []>;
   let Defs = [CR6] in
   def XVCMPNEDPo : XX3Form_Rc<60, 123,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnedp. $XT, $XA, $XB", IIC_VecFPCompare, []>,
                               isDOT;
   def XVCMPNESP  : XX3Form_Rc<60,  91,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnesp  $XT, $XA, $XB", IIC_VecFPCompare, []>;
   let Defs = [CR6] in
   def XVCMPNESPo : XX3Form_Rc<60,  91,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnesp. $XT, $XA, $XB", IIC_VecFPCompare, []>,
                               isDOT;
+  } // UseVSXReg = 1
 
   //===--------------------------------------------------------------------===//
   // Quad-Precision Floating-Point Conversion Instructions:
 
   // Convert DP -> QP
-  def XSCVDPQP  : X_VT5_XO5_VB5_TyVB<63, 22, 836, "xscvdpqp", vsfrc, []>;
+  def XSCVDPQP  : X_VT5_XO5_VB5_TyVB<63, 22, 836, "xscvdpqp", vfrc, []>;
 
   // Round & Convert QP -> DP (dword[1] is set to zero)
   def XSCVQPDP  : X_VT5_XO5_VB5   <63, 20, 836, "xscvqpdp" , []>;
   def XSCVQPDPO : X_VT5_XO5_VB5_Ro<63, 20, 836, "xscvqpdpo", []>;
 
   // Truncate & Convert QP -> (Un)Signed (D)Word (dword[1] is set to zero)
   def XSCVQPSDZ : X_VT5_XO5_VB5<63, 25, 836, "xscvqpsdz", []>;
   def XSCVQPSWZ : X_VT5_XO5_VB5<63,  9, 836, "xscvqpswz", []>;
   def XSCVQPUDZ : X_VT5_XO5_VB5<63, 17, 836, "xscvqpudz", []>;
   def XSCVQPUWZ : X_VT5_XO5_VB5<63,  1, 836, "xscvqpuwz", []>;
 
   // Convert (Un)Signed DWord -> QP
-  def XSCVSDQP  : X_VT5_XO5_VB5_TyVB<63, 10, 836, "xscvsdqp", vsfrc, []>;
-  def XSCVUDQP  : X_VT5_XO5_VB5_TyVB<63,  2, 836, "xscvudqp", vsfrc, []>;
+  def XSCVSDQP  : X_VT5_XO5_VB5_TyVB<63, 10, 836, "xscvsdqp", vfrc, []>;
+  def XSCVUDQP  : X_VT5_XO5_VB5_TyVB<63,  2, 836, "xscvudqp", vfrc, []>;
 
   //===--------------------------------------------------------------------===//
   // Round to Floating-Point Integer Instructions
 
   // (Round &) Convert DP <-> HP
   // Note! xscvdphp's src and dest register both use the left 64 bits, so we use
   // vsfrc for src and dest register. xscvhpdp's src only use the left 16 bits,
   // but we still use vsfrc for it.
let AddedComplexity = 400, Predicates = [HasP9Vector] in {
   def XSRQPXP  : Z23_VT5_R1_VB5_RMC2_EX1<63, 37, 0, "xsrqpxp", []>;
 
   //===--------------------------------------------------------------------===//
   // Insert/Extract Instructions
 
   // Insert Exponent DP/QP
   // XT NOTE: XT.dword[1] = 0xUUUU_UUUU_UUUU_UUUU
   def XSIEXPDP : XX1Form <60, 918, (outs vsrc:$XT), (ins g8rc:$rA, g8rc:$rB),
-                          "xsiexpdp $XT, $rA, $rB", IIC_VecFP, []>;
+                          "xsiexpdp $XT, $rA, $rB", IIC_VecFP, []>, UseVSXReg;
   // vB NOTE: only vB.dword[0] is used, that's why we don't use
   //          X_VT5_VA5_VB5 form
   def XSIEXPQP : XForm_18<63, 868, (outs vrrc:$vT), (ins vrrc:$vA, vsfrc:$vB),
                           "xsiexpqp $vT, $vA, $vB", IIC_VecFP, []>;
 
   // Extract Exponent/Significand DP/QP
   def XSXEXPDP : XX2_RT5_XO5_XB6<60,  0, 347, "xsxexpdp", []>;
   def XSXSIGDP : XX2_RT5_XO5_XB6<60,  1, 347, "xsxsigdp", []>;
+
   def XSXEXPQP : X_VT5_XO5_VB5  <63,  2, 804, "xsxexpqp", []>;
   def XSXSIGQP : X_VT5_XO5_VB5  <63, 18, 804, "xsxsigqp", []>;
 
   // Vector Insert Word
+  let UseVSXReg = 1 in {
   // XB NOTE: Only XB.dword[1] is used, but we use vsrc on XB.
   def XXINSERTW   :
     XX2_RD6_UIM5_RS6<60, 181, (outs vsrc:$XT),
                      (ins vsrc:$XTi, vsrc:$XB, u4imm:$UIM),
                      "xxinsertw $XT, $XB, $UIM", IIC_VecFP,
                      [(set v4i32:$XT, (PPCxxinsert v4i32:$XTi, v4i32:$XB,
                                                    imm32SExt16:$UIM))]>,
                      RegConstraint<"$XTi = $XT">, NoEncode<"$XTi">;
 
   // Vector Extract Unsigned Word
   def XXEXTRACTUW : XX2_RD6_UIM5_RS6<60, 165,
                                   (outs vsfrc:$XT), (ins vsrc:$XB, u4imm:$UIMM),
                                   "xxextractuw $XT, $XB, $UIMM", IIC_VecFP, []>;
+  } // UseVSXReg = 1
 
   // Vector Insert Exponent DP/SP
   def XVIEXPDP : XX3_XT5_XA5_XB5<60, 248, "xviexpdp", vsrc, vsrc, vsrc,
                                  IIC_VecFP, []>;
   def XVIEXPSP : XX3_XT5_XA5_XB5<60, 216, "xviexpsp", vsrc, vsrc, vsrc,
                                  IIC_VecFP, []>;
 
   // Vector Extract Exponent/Significand DP/SP
   def XVXEXPDP : XX2_XT6_XO5_XB6<60,  0, 475, "xvxexpdp", vsrc, []>;
   def XVXEXPSP : XX2_XT6_XO5_XB6<60,  8, 475, "xvxexpsp", vsrc, []>;
   def XVXSIGDP : XX2_XT6_XO5_XB6<60,  1, 475, "xvxsigdp", vsrc, []>;
   def XVXSIGSP : XX2_XT6_XO5_XB6<60,  9, 475, "xvxsigsp", vsrc, []>;
 
   //===--------------------------------------------------------------------===//
 
   // Test Data Class SP/DP/QP
+  let UseVSXReg = 1 in {
   def XSTSTDCSP : XX2_BF3_DCMX7_RS6<60, 298,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vsfrc:$XB),
                               "xststdcsp $BF, $XB, $DCMX", IIC_VecFP, []>;
   def XSTSTDCDP : XX2_BF3_DCMX7_RS6<60, 362,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vsfrc:$XB),
                               "xststdcdp $BF, $XB, $DCMX", IIC_VecFP, []>;
+  } // UseVSXReg = 1
   def XSTSTDCQP : X_BF3_DCMX7_RS5  <63, 708,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vrrc:$vB),
                               "xststdcqp $BF, $vB, $DCMX", IIC_VecFP, []>;
 
   // Vector Test Data Class SP/DP
+  let UseVSXReg = 1 in {
   def XVTSTDCSP : XX2_RD6_DCMX7_RS6<60, 13, 5,
                               (outs vsrc:$XT), (ins u7imm:$DCMX, vsrc:$XB),
                               "xvtstdcsp $XT, $XB, $DCMX", IIC_VecFP, []>;
   def XVTSTDCDP : XX2_RD6_DCMX7_RS6<60, 15, 5,
                               (outs vsrc:$XT), (ins u7imm:$DCMX, vsrc:$XB),
                               "xvtstdcdp $XT, $XB, $DCMX", IIC_VecFP, []>;
+  } // UseVSXReg = 1
 
   //===--------------------------------------------------------------------===//
 
   // Maximum/Minimum Type-C/Type-J DP
   // XT.dword[1] = 0xUUUU_UUUU_UUUU_UUUU, so we use vsrc for XT
   def XSMAXCDP : XX3_XT5_XA5_XB5<60, 128, "xsmaxcdp", vsrc, vsfrc, vsfrc,
                                  IIC_VecFP, []>;
   def XSMAXJDP : XX3_XT5_XA5_XB5<60, 144, "xsmaxjdp", vsrc, vsfrc, vsfrc,
let AddedComplexity = 400, Predicates = [HasP9Vector] in {
   // Vector Permute
   def XXPERM  : XX3_XT5_XA5_XB5<60, 26, "xxperm" , vsrc, vsrc, vsrc,
                                 IIC_VecPerm, []>;
   def XXPERMR : XX3_XT5_XA5_XB5<60, 58, "xxpermr", vsrc, vsrc, vsrc,
                                 IIC_VecPerm, []>;
 
   // Vector Splat Immediate Byte
   def XXSPLTIB : X_RD6_IMM8<60, 360, (outs vsrc:$XT), (ins u8imm:$IMM8),
-                            "xxspltib $XT, $IMM8", IIC_VecPerm, []>;
+                            "xxspltib $XT, $IMM8", IIC_VecPerm, []>, UseVSXReg;
 
   //===--------------------------------------------------------------------===//
   // Vector/Scalar Load/Store Instructions
 
   let mayLoad = 1 in {
   // Load Vector
   def LXV : DQ_RD6_RS5_DQ12<61, 1, (outs vsrc:$XT), (ins memrix16:$src),
-                            "lxv $XT, $src", IIC_LdStLFD, []>;
+                            "lxv $XT, $src", IIC_LdStLFD, []>, UseVSXReg;
   // Load DWord
-  def LXSD  : DSForm_1<57, 2, (outs vrrc:$vD), (ins memrix:$src),
+  def LXSD  : DSForm_1<57, 2, (outs vfrc:$vD), (ins memrix:$src),
                        "lxsd $vD, $src", IIC_LdStLFD, []>;
   // Load SP from src, convert it to DP, and place in dword[0]
-  def LXSSP : DSForm_1<57, 3, (outs vrrc:$vD), (ins memrix:$src),
+  def LXSSP : DSForm_1<57, 3, (outs vfrc:$vD), (ins memrix:$src),
                        "lxssp $vD, $src", IIC_LdStLFD, []>;
 
   // [PO T RA RB XO TX] almost equal to [PO S RA RB XO SX], but has different
   // "out" and "in" dag
   class X_XT6_RA5_RB5<bits<6> opcode, bits<10> xo, string opc,
                       RegisterOperand vtype, list<dag> pattern>
     : XX1Form<opcode, xo, (outs vtype:$XT), (ins memrr:$src),
-              !strconcat(opc, " $XT, $src"), IIC_LdStLFD, pattern>;
+              !strconcat(opc, " $XT, $src"), IIC_LdStLFD, pattern>, UseVSXReg;
 
   // Load as Integer Byte/Halfword & Zero Indexed
-  def LXSIBZX : X_XT6_RA5_RB5<31, 781, "lxsibzx", vsfrc, []>;
-  def LXSIHZX : X_XT6_RA5_RB5<31, 813, "lxsihzx", vsfrc, []>;
+  def LXSIBZX : X_XT6_RA5_RB5<31, 781, "lxsibzx", vsfrc,
+                              [(set f64:$XT, (PPClxsizx xoaddr:$src, 1))]>;
+  def LXSIHZX : X_XT6_RA5_RB5<31, 813, "lxsihzx", vsfrc,
+                              [(set f64:$XT, (PPClxsizx xoaddr:$src, 2))]>;
 
   // Load Vector Halfword*8/Byte*16 Indexed
   def LXVH8X  : X_XT6_RA5_RB5<31, 812, "lxvh8x" , vsrc, []>;
   def LXVB16X : X_XT6_RA5_RB5<31, 876, "lxvb16x", vsrc, []>;
 
   // Load Vector Indexed
   def LXVX    : X_XT6_RA5_RB5<31, 268, "lxvx"   , vsrc,
                 [(set v2f64:$XT, (load xoaddr:$src))]>;
 
   // Load Vector (Left-justified) with Length
   def LXVL    : X_XT6_RA5_RB5<31, 269, "lxvl"   , vsrc, []>;
   def LXVLL   : X_XT6_RA5_RB5<31, 301, "lxvll"  , vsrc, []>;
 
   // Load Vector Word & Splat Indexed
   def LXVWSX  : X_XT6_RA5_RB5<31, 364, "lxvwsx" , vsrc, []>;
-  } // end mayLoad
+  } // mayLoad
 
   let mayStore = 1 in {
   // Store Vector
   def STXV : DQ_RD6_RS5_DQ12<61, 5, (outs), (ins vsrc:$XT, memrix16:$dst),
-                             "stxv $XT, $dst", IIC_LdStSTFD, []>;
+                             "stxv $XT, $dst", IIC_LdStSTFD, []>, UseVSXReg;
   // Store DWord
-  def STXSD  : DSForm_1<61, 2, (outs), (ins vrrc:$vS, memrix:$dst),
+  def STXSD  : DSForm_1<61, 2, (outs), (ins vfrc:$vS, memrix:$dst),
                         "stxsd $vS, $dst", IIC_LdStSTFD, []>;
   // Convert DP of dword[0] to SP, and Store to dst
-  def STXSSP : DSForm_1<61, 3, (outs), (ins vrrc:$vS, memrix:$dst),
+  def STXSSP : DSForm_1<61, 3, (outs), (ins vfrc:$vS, memrix:$dst),
                         "stxssp $vS, $dst", IIC_LdStSTFD, []>;
 
   // [PO S RA RB XO SX]
   class X_XS6_RA5_RB5<bits<6> opcode, bits<10> xo, string opc,
                       RegisterOperand vtype, list<dag> pattern>
     : XX1Form<opcode, xo, (outs), (ins vtype:$XT, memrr:$dst),
-              !strconcat(opc, " $XT, $dst"), IIC_LdStSTFD, pattern>;
+              !strconcat(opc, " $XT, $dst"), IIC_LdStSTFD, pattern>, UseVSXReg;
 
   // Store as Integer Byte/Halfword Indexed
-  def STXSIBX  : X_XS6_RA5_RB5<31,  909, "stxsibx" , vsfrc, []>;
-  def STXSIHX  : X_XS6_RA5_RB5<31,  941, "stxsihx" , vsfrc, []>;
+  def STXSIBX  : X_XS6_RA5_RB5<31,  909, "stxsibx" , vsfrc,
+                               [(PPCstxsix f64:$XT, xoaddr:$dst, 1)]>;
+  def STXSIHX  : X_XS6_RA5_RB5<31,  941, "stxsihx" , vsfrc,
+                               [(PPCstxsix f64:$XT, xoaddr:$dst, 2)]>;
+  let isCodeGenOnly = 1 in {
+    def STXSIBXv  : X_XS6_RA5_RB5<31,  909, "stxsibx" , vrrc, []>;
+    def STXSIHXv  : X_XS6_RA5_RB5<31,  941, "stxsihx" , vrrc, []>;
+  }
 
   // Store Vector Halfword*8/Byte*16 Indexed
   def STXVH8X  : X_XS6_RA5_RB5<31,  940, "stxvh8x" , vsrc, []>;
   def STXVB16X : X_XS6_RA5_RB5<31, 1004, "stxvb16x", vsrc, []>;
 
   // Store Vector Indexed
   def STXVX    : X_XS6_RA5_RB5<31,  396, "stxvx"   , vsrc,
                  [(store v2f64:$XT, xoaddr:$dst)]>;
 
   // Store Vector (Left-justified) with Length
   def STXVL    : X_XS6_RA5_RB5<31,  397, "stxvl"   , vsrc, []>;
   def STXVLL   : X_XS6_RA5_RB5<31,  429, "stxvll"  , vsrc, []>;
-  } // end mayStore
+  } // mayStore
 
   // Patterns for which instructions from ISA 3.0 are a better match
   let Predicates = [IsLittleEndian, HasP9Vector] in {
   def : Pat<(f32 (PPCfcfidus (PPCmtvsrz (i32 (extractelt v4i32:$A, 0))))),
             (f32 (XSCVUXDSP (XXEXTRACTUW $A, 12)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsrz (i32 (extractelt v4i32:$A, 1))))),
             (f32 (XSCVUXDSP (XXEXTRACTUW $A, 8)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsrz (i32 (extractelt v4i32:$A, 2))))),
let AddedComplexity = 400, Predicates = [HasP9Vector] in {
   def : Pat<(v16i8 immAllOnesV),
             (v16i8 (COPY_TO_REGCLASS (XXSPLTIB 255), VSRC))>;
   def : Pat<(v8i16 immAllOnesV),
             (v8i16 (COPY_TO_REGCLASS (XXSPLTIB 255), VSRC))>;
   def : Pat<(v4i32 immAllOnesV),
             (v4i32 (XXSPLTIB 255))>;
   def : Pat<(v2i64 immAllOnesV),
             (v2i64 (XXSPLTIB 255))>;
+
+  // Build vectors from i8 loads
+  def : Pat<(v16i8 (scalar_to_vector ScalarLoads.Li8)),
+            (v16i8 (VSPLTBs 7, (LXSIBZX xoaddr:$src)))>;
+  def : Pat<(v8i16 (scalar_to_vector ScalarLoads.ZELi8)),
+            (v8i16 (VSPLTHs 3, (LXSIBZX xoaddr:$src)))>;
+  def : Pat<(v4i32 (scalar_to_vector ScalarLoads.ZELi8)),
+           (v4i32 (XXSPLTWs (LXSIBZX xoaddr:$src), 1))>;
+  def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi8i64)),
+            (v2i64 (XXPERMDIs (LXSIBZX xoaddr:$src), 0))>;
+  def : Pat<(v4i32 (scalar_to_vector ScalarLoads.SELi8)),
+            (v4i32 (XXSPLTWs (VEXTSB2Ws (LXSIBZX xoaddr:$src)), 1))>;
+  def : Pat<(v2i64 (scalar_to_vector ScalarLoads.SELi8i64)),
+            (v2i64 (XXPERMDIs (VEXTSB2Ds (LXSIBZX xoaddr:$src)), 0))>;
+
+  // Build vectors from i16 loads
+  def : Pat<(v8i16 (scalar_to_vector ScalarLoads.Li16)),
+            (v8i16 (VSPLTHs 3, (LXSIHZX xoaddr:$src)))>;
+  def : Pat<(v4i32 (scalar_to_vector ScalarLoads.ZELi16)),
+            (v4i32 (XXSPLTWs (LXSIHZX xoaddr:$src), 1))>;
+  def : Pat<(v2i64 (scalar_to_vector ScalarLoads.ZELi16i64)),
+           (v2i64 (XXPERMDIs (LXSIHZX xoaddr:$src), 0))>;
+  def : Pat<(v4i32 (scalar_to_vector ScalarLoads.SELi16)),
+            (v4i32 (XXSPLTWs (VEXTSH2Ws (LXSIHZX xoaddr:$src)), 1))>;
+  def : Pat<(v2i64 (scalar_to_vector ScalarLoads.SELi16i64)),
+            (v2i64 (XXPERMDIs (VEXTSH2Ds (LXSIHZX xoaddr:$src)), 0))>;
+
+  let Predicates = [IsBigEndian, HasP9Vector] in {
+  // Scalar stores of i8
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 0)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 9), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 1)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 10), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 2)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 11), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 3)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 12), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 4)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 13), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 5)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 14), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 6)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 15), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 7)), xoaddr:$dst),
+            (STXSIBXv $S, xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 8)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 1), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 9)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 2), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 10)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 3), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 11)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 4), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 12)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 5), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 13)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 6), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 14)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 7), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 15)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 8), xoaddr:$dst)>;
+
+  // Scalar stores of i16
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 0)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 10), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 1)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 12), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 2)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 14), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 3)), xoaddr:$dst),
+            (STXSIHXv $S, xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 4)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 2), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 5)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 4), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 6)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 6), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 7)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 8), xoaddr:$dst)>;
+  } // IsBigEndian, HasP9Vector
+
+  let Predicates = [IsLittleEndian, HasP9Vector] in {
+  // Scalar stores of i8
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 0)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 8), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 1)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 7), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 2)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 6), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 3)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 5), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 4)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 4), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 5)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 3), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 6)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 2), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 7)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 1), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 8)), xoaddr:$dst),
+            (STXSIBXv $S, xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 9)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 15), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 10)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 14), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 11)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 13), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 12)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 12), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 13)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 11), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 14)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 10), xoaddr:$dst)>;
+  def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 15)), xoaddr:$dst),
+            (STXSIBXv (VSLDOI $S, $S, 9), xoaddr:$dst)>;
+
+  // Scalar stores of i16
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 0)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 8), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 1)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 6), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 2)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 4), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 3)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 2), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 4)), xoaddr:$dst),
+            (STXSIHXv $S, xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 5)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 14), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 6)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 12), xoaddr:$dst)>;
+  def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 7)), xoaddr:$dst),
+            (STXSIHXv (VSLDOI $S, $S, 10), xoaddr:$dst)>;
+  } // IsLittleEndian, HasP9Vector
+
+  // Vector sign extensions
+  def : Pat<(f64 (PPCVexts f64:$A, 1)),
+            (f64 (COPY_TO_REGCLASS (VEXTSB2Ds $A), VSFRC))>;
+  def : Pat<(f64 (PPCVexts f64:$A, 2)),
+            (f64 (COPY_TO_REGCLASS (VEXTSH2Ds $A), VSFRC))>;
 } // end HasP9Vector, AddedComplexity
 
 let Predicates = [IsISA3_0, HasDirectMove, IsLittleEndian] in {
 def : Pat<(v2i64 (build_vector i64:$rA, i64:$rB)),
           (v2i64 (MTVSRDD $rB, $rA))>;
 def : Pat<(i64 (extractelt v2i64:$A, 0)),
           (i64 (MFVSRLD $A))>;
 }
 
 let Predicates = [IsISA3_0, HasDirectMove, IsBigEndian] in {
 def : Pat<(v2i64 (build_vector i64:$rB, i64:$rA)),
           (v2i64 (MTVSRDD $rB, $rA))>;
 def : Pat<(i64 (extractelt v2i64:$A, 1)),
           (i64 (MFVSRLD $A))>;
 }

lib/Target/PowerPC/PPCMIPeephole.cpp

for (MachineInstr &MI : MBB) {
                 DEBUG(dbgs() << "Optimizing swap/swap => copy: ");
                 DEBUG(MI.dump());
                 BuildMI(MBB, &MI, MI.getDebugLoc(),
                         TII->get(PPC::COPY), MI.getOperand(0).getReg())
                   .addOperand(DefMI->getOperand(1));
                 ToErase = &MI;
                 Simplified = true;
               }
-            }
-          }
+            } else if ((Immed == 0 || Immed == 3) &&
+                       DefMI && DefMI->getOpcode() == PPC::XXPERMDIs) {
+              // Splat fed by another splat - switch the output of the first
+              // and remove the second.
+              DefMI->getOperand(0).setReg(MI.getOperand(0).getReg());
+              ToErase = &MI;
+              Simplified = true;
+              DEBUG(dbgs() << "Removing redundant splat: ");
+              DEBUG(MI.dump());
+            }
+        break;
+      }
+      case PPC::VSPLTB:
+      case PPC::VSPLTH:
+      case PPC::XXSPLTW: {
+        unsigned MyOpcode = MI.getOpcode();
+        unsigned OpNo = MyOpcode == PPC::XXSPLTW ? 1 : 2;
+        unsigned TrueReg = lookThruCopyLike(MI.getOperand(OpNo).getReg());
+        MachineInstr *DefMI = MRI->getVRegDef(TrueReg);
+        if (!DefMI)
+          break;
+        unsigned DefOpcode = DefMI->getOpcode();
+        bool SameOpcode = (MyOpcode == DefOpcode) ||
+          (MyOpcode == PPC::VSPLTB && DefOpcode == PPC::VSPLTBs) ||
+          (MyOpcode == PPC::VSPLTH && DefOpcode == PPC::VSPLTHs) ||
+          (MyOpcode == PPC::XXSPLTW && DefOpcode == PPC::XXSPLTWs);
+        // Splat fed by another splat - switch the output of the first
+        // and remove the second.
+        if (SameOpcode) {
+          DefMI->getOperand(0).setReg(MI.getOperand(0).getReg());
+          ToErase = &MI;
+          Simplified = true;
+          DEBUG(dbgs() << "Removing redundant splat: ");
+          DEBUG(MI.dump());
+        }
+        // Splat fed by a shift. Usually when we align value to splat into
+        // vector element zero.
+        if (DefOpcode == PPC::XXSLDWI) {
+          unsigned ShiftRes = DefMI->getOperand(0).getReg();
+          unsigned ShiftOp1 = DefMI->getOperand(1).getReg();
+          unsigned ShiftOp2 = DefMI->getOperand(2).getReg();
+          unsigned ShiftImm = DefMI->getOperand(3).getImm();
+          unsigned SplatImm = MI.getOperand(2).getImm();
+          if (ShiftOp1 == ShiftOp2) {
+            unsigned NewElem = (SplatImm + ShiftImm) & 0x3;
+            if (MRI->hasOneNonDBGUse(ShiftRes)) {
+              DEBUG(dbgs() << "Removing redundant shift: ");
+              DEBUG(DefMI->dump());
+              ToErase = DefMI;
+            }
+            Simplified = true;
+            DEBUG(dbgs() << "Changing splat immediate from " << SplatImm <<
+                  " to " << NewElem << " in instruction: ");
+            DEBUG(MI.dump());
+            MI.getOperand(1).setReg(ShiftOp1);
+            MI.getOperand(2).setImm(NewElem);
+          }
         }
         break;
       }
       }
     }
 
     // If the last instruction was marked for elimination,
     // remove it now.

lib/Target/PowerPC/PPCRegisterInfo.cpp

unsigned PPCRegisterInfo::getRegPressureLimit(const TargetRegisterClass *RC,
   case PPC::F8RCRegClassID:
   case PPC::F4RCRegClassID:
   case PPC::QFRCRegClassID:
   case PPC::QSRCRegClassID:
   case PPC::QBRCRegClassID:
   case PPC::VRRCRegClassID:
   case PPC::VFRCRegClassID:
   case PPC::VSLRCRegClassID:
-  case PPC::VSHRCRegClassID:
     return 32 - DefaultSafety;
   case PPC::VSRCRegClassID:
   case PPC::VSFRCRegClassID:
   case PPC::VSSRCRegClassID:
     return 64 - DefaultSafety;
   case PPC::CRRCRegClassID:
     return 8 - DefaultSafety;
   }

lib/Target/PowerPC/PPCRegisterInfo.td

 
 let Namespace = "PPC" in {
 def sub_lt : SubRegIndex<1>;
 def sub_gt : SubRegIndex<1, 1>;
 def sub_eq : SubRegIndex<1, 2>;
 def sub_un : SubRegIndex<1, 3>;
 def sub_32 : SubRegIndex<32>;
 def sub_64 : SubRegIndex<64>;
-def sub_128 : SubRegIndex<128>;
 }
 
 
 class PPCReg<string n> : Register<n> {
   let Namespace = "PPC";
 }
 
 // We identify all our registers with a 5-bit ID, for consistency's sake.

 // VSRL - One of the 32 128-bit VSX registers that overlap with the scalar
 // floating-point registers.
 class VSRL<FPR SubReg, string n> : PPCReg<n> {
   let HWEncoding = SubReg.HWEncoding;
   let SubRegs = [SubReg];
   let SubRegIndices = [sub_64];
 }
 
-// VSRH - One of the 32 128-bit VSX registers that overlap with the vector
-// registers.
-class VSRH<VR SubReg, string n> : PPCReg<n> {
-  let HWEncoding{4-0} = SubReg.HWEncoding{4-0};
-  let HWEncoding{5} = 1;
-  let SubRegs = [SubReg];
-  let SubRegIndices = [sub_128];
-}
-
 // CR - One of the 8 4-bit condition registers
 class CR<bits<3> num, string n, list<Register> subregs> : PPCReg<n> {
   let HWEncoding{2-0} = num;
   let SubRegs = subregs;
 }
 
 // CRBIT - One of the 32 1-bit condition register fields
 class CRBIT<bits<5> num, string n> : PPCReg<n> {

 }
 
 // Floating-point registers
 foreach Index = 0-31 in {
   def F#Index : FPR<Index, "f"#Index>,
                 DwarfRegNum<[!add(Index, 32), !add(Index, 32)]>;
 }
 
-// Floating-point vector subregisters (for VSX)
+// 64-bit Floating-point subregisters of Altivec registers
+// Note: the register names are v0-v31 or vs32-vs63 depending on the use.
+//       Custom C++ code is used to produce the correct name and encoding.
 foreach Index = 0-31 in {
-  def VF#Index : VF<Index, "vs" # !add(Index, 32)>;
+  def VF#Index : VF<Index, "v" #Index>,
+                 DwarfRegNum<[!add(Index, 77), !add(Index, 77)]>;
 }
 
 // QPX Floating-point registers
 foreach Index = 0-31 in {
   def QF#Index : QFPR<!cast<FPR>("F"#Index), "q"#Index>,
                  DwarfRegNum<[!add(Index, 32), !add(Index, 32)]>;
 }
 
 // Vector registers
 foreach Index = 0-31 in {
   def V#Index : VR<!cast<VF>("VF"#Index), "v"#Index>,
                 DwarfRegNum<[!add(Index, 77), !add(Index, 77)]>;
 }
 
 // VSX registers
 foreach Index = 0-31 in {
   def VSL#Index : VSRL<!cast<FPR>("F"#Index), "vs"#Index>,
                   DwarfRegAlias<!cast<FPR>("F"#Index)>;
 }
-foreach Index = 0-31 in {
-  def VSH#Index : VSRH<!cast<VR>("V"#Index), "vs" # !add(Index, 32)>,
-                  DwarfRegAlias<!cast<VR>("V"#Index)>;
+
+// Dummy VSX registers, this defines string: "vs32"-"vs63", and is only used for
+// asm printing.
+foreach Index = 32-63 in {
+  def VSX#Index : PPCReg<"vs"#Index>;
 }
 
 // The reprsentation of r0 when treated as the constant 0.
 def ZERO  : GPR<0, "0">,    DwarfRegAlias<R0>;
 def ZERO8 : GP8<ZERO, "0">, DwarfRegAlias<X0>;
 
 // Representations of the frame pointer used by ISD::FRAMEADDR.
 def FP   : GPR<0 /* arbitrary */, "**FRAME POINTER**">;

 // register N is spilled to offset 8 * (32 - N) below the back chain word of the
 // previous stack frame. By allocating non-volatiles in reverse order we make
 // sure that the Floating-point register save area is always as small as
 // possible because there aren't any unused spill slots.
 def F8RC : RegisterClass<"PPC", [f64], 64, (add (sequence "F%u", 0, 13),
                                                 (sequence "F%u", 31, 14))>;
 def F4RC : RegisterClass<"PPC", [f32], 32, (add F8RC)>;
 
-def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v1i128,v4f32], 128,
+def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v2f64,v1i128,v4f32], 128,
                          (add V2, V3, V4, V5, V0, V1, V6, V7, V8, V9, V10, V11,
                              V12, V13, V14, V15, V16, V17, V18, V19, V31, V30,
                              V29, V28, V27, V26, V25, V24, V23, V22, V21, V20)>;
 
 // VSX register classes (the allocation order mirrors that of the corresponding
 // subregister classes).
 def VSLRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
                           (add (sequence "VSL%u", 0, 13),
                                (sequence "VSL%u", 31, 14))>;
-def VSHRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
-                          (add VSH2, VSH3, VSH4, VSH5, VSH0, VSH1, VSH6, VSH7,
-             VSH8, VSH9, VSH10, VSH11, VSH12, VSH13, VSH14,
-                               VSH15, VSH16, VSH17, VSH18, VSH19, VSH31, VSH30,
-                               VSH29, VSH28, VSH27, VSH26, VSH25, VSH24, VSH23,
-                               VSH22, VSH21, VSH20)>;
 def VSRC  : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
-                          (add VSLRC, VSHRC)>;
+                          (add VSLRC, VRRC)>;
 
 // Register classes for the 64-bit "scalar" VSX subregisters.
 def VFRC :  RegisterClass<"PPC", [f64], 64,
                           (add VF2, VF3, VF4, VF5, VF0, VF1, VF6, VF7,
                                VF8, VF9, VF10, VF11, VF12, VF13, VF14,
                                VF15, VF16, VF17, VF18, VF19, VF31, VF30,
                                VF29, VF28, VF27, VF26, VF25, VF24, VF23,
                                VF22, VF21, VF20)>;

lib/Target/PowerPC/PPCVSXCopy.cpp

bool processBlock(MachineBasicBlock &MBB) {
         MachineOperand &DstMO = MI.getOperand(0);
         MachineOperand &SrcMO = MI.getOperand(1);
 
         if ( IsVSReg(DstMO.getReg(), MRI) &&
             !IsVSReg(SrcMO.getReg(), MRI)) {
           // This is a copy *to* a VSX register from a non-VSX register.
           Changed = true;
 
-          const TargetRegisterClass *SrcRC =
-            IsVRReg(SrcMO.getReg(), MRI) ? &PPC::VSHRCRegClass :
-                                           &PPC::VSLRCRegClass;
+          const TargetRegisterClass *SrcRC = &PPC::VSLRCRegClass;
           assert((IsF8Reg(SrcMO.getReg(), MRI) ||
-                  IsVRReg(SrcMO.getReg(), MRI) ||
                   IsVSSReg(SrcMO.getReg(), MRI) ||
                   IsVSFReg(SrcMO.getReg(), MRI)) &&
                  "Unknown source for a VSX copy");
 
           unsigned NewVReg = MRI.createVirtualRegister(SrcRC);
           BuildMI(MBB, MI, MI.getDebugLoc(),
                   TII->get(TargetOpcode::SUBREG_TO_REG), NewVReg)
               .addImm(1) // add 1, not 0, because there is no implicit clearing
                          // of the high bits.
               .addOperand(SrcMO)
-              .addImm(IsVRReg(SrcMO.getReg(), MRI) ? PPC::sub_128
-                                                   : PPC::sub_64);
+              .addImm(PPC::sub_64);
 
           // The source of the original copy is now the new virtual register.
           SrcMO.setReg(NewVReg);
         } else if (!IsVSReg(DstMO.getReg(), MRI) &&
                     IsVSReg(SrcMO.getReg(), MRI)) {
           // This is a copy *from* a VSX register to a non-VSX register.
           Changed = true;
 
-          const TargetRegisterClass *DstRC =
-            IsVRReg(DstMO.getReg(), MRI) ? &PPC::VSHRCRegClass :
-                                           &PPC::VSLRCRegClass;
+          const TargetRegisterClass *DstRC = &PPC::VSLRCRegClass;
           assert((IsF8Reg(DstMO.getReg(), MRI) ||
                   IsVSFReg(DstMO.getReg(), MRI) ||
-                  IsVSSReg(DstMO.getReg(), MRI) ||
-                  IsVRReg(DstMO.getReg(), MRI)) &&
+                  IsVSSReg(DstMO.getReg(), MRI)) &&
                  "Unknown destination for a VSX copy");
 
           // Copy the VSX value into a new VSX register of the correct subclass.
           unsigned NewVReg = MRI.createVirtualRegister(DstRC);
           BuildMI(MBB, MI, MI.getDebugLoc(), TII->get(TargetOpcode::COPY),
                   NewVReg)
               .addOperand(SrcMO);
 
           // Transform the original copy into a subregister extraction copy.
           SrcMO.setReg(NewVReg);
-          SrcMO.setSubReg(IsVRReg(DstMO.getReg(), MRI) ? PPC::sub_128 :
-                                                         PPC::sub_64);
+          SrcMO.setSubReg(PPC::sub_64);
         }
       }
 
       return Changed;
     }
 
 public:
     bool runOnMachineFunction(MachineFunction &MF) override {

test/CodeGen/PowerPC/inline-asm-scalar-to-vector-error.ll

 ; RUN: not llc -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8a -mattr=+altivec %s -o - 2>&1 | FileCheck %s
 
 define hidden void @f(i32 %x) {
   ; CHECK: scalar-to-vector conversion failed, possible invalid constraint for vector type
   tail call void asm sideeffect "nop", "{v1}"(i32 %x) nounwind
 
   ; CHECK: scalar-to-vector conversion failed, possible invalid constraint for vector type
   tail call void asm sideeffect "nop", "{vsl1}"(i32 %x) nounwind
 
-  ; CHECK: scalar-to-vector conversion failed, possible invalid constraint for vector type
-  tail call void asm sideeffect "nop", "{vsh1}"(i32 %x) nounwind
-
   ret void
 }

test/CodeGen/PowerPC/load-v4i8-improved.ll

 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck \
 ; RUN:   -implicit-check-not vmrg -implicit-check-not=vperm %s
 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck \
-; RUN:   -implicit-check-not vmrg -implicit-check-not=vperm %s \
-; RUN:   --check-prefix=CHECK-BE
+; RUN:   -implicit-check-not vmrg -implicit-check-not=vperm %s
 
 define <16 x i8> @test(i32* %s, i32* %t) {
 entry:
   %0 = bitcast i32* %s to <4 x i8>*
   %1 = load <4 x i8>, <4 x i8>* %0, align 4
   %2 = shufflevector <4 x i8> %1, <4 x i8> undef, <16 x i32> <i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3, i32 0, i32 1, i32 2, i32 3>
   ret <16 x i8> %2
 ; CHECK-LABEL: test
-; CHECK: lwz [[GPR:[0-9]+]], 0(3)
-; CHECK: mtvsrd [[VSR:[0-9]+]], [[GPR]]
-; CHECK: xxswapd  [[SWP:[0-9]+]], [[VSR]]
-; CHECK: xxspltw 34, [[SWP]], 3
-; CHECK-BE-LABEL: test
-; CHECK-BE: lwz [[GPR:[0-9]+]], 0(3)
-; CHECK-BE: sldi [[SHL:[0-9]+]], [[GPR]], 32
-; CHECK-BE: mtvsrd [[VSR:[0-9]+]], [[SHL]]
-; CHECK-BE: xxspltw 34, [[VSR]], 0
+; CHECK: lxsiwax 34, 0, 3
+; CHECK: xxspltw 34, 34, 1
 }

test/CodeGen/PowerPC/machine-combiner.ll

 ; CHECK-LABEL: vector_reassociate_adds1:
 ; CHECK:       # BB#0:
 ; CHECK-QPX:       qvfadds [[REG0:[0-9]+]], 1, 2
 ; CHECK-QPX:       qvfadds [[REG1:[0-9]+]], 3, 4
 ; CHECK-QPX:       qvfadds 1, [[REG0]], [[REG1]]
 ; CHECK-PWR:       xvaddsp [[REG0:[0-9]+]], 34, 35
 ; CHECK-PWR:       xvaddsp [[REG1:[0-9]+]], 36, 37
 ; CHECK-PWR:       xvaddsp 34, [[REG0]], [[REG1]]
-; CHECK-PWR:       # kill
 ; CHECK-NEXT:  blr
 
   %t0 = fadd <4 x float> %x0, %x1
   %t1 = fadd <4 x float> %t0, %x2
   %t2 = fadd <4 x float> %t1, %x3
   ret <4 x float> %t2
 }
 
 define <4 x float> @vector_reassociate_adds2(<4 x float> %x0, <4 x float> %x1, <4 x float> %x2, <4 x float> %x3) {
 ; CHECK-LABEL: vector_reassociate_adds2:
 ; CHECK:       # BB#0:
 ; CHECK-QPX:       qvfadds [[REG0:[0-9]+]], 1, 2
 ; CHECK-QPX:       qvfadds [[REG1:[0-9]+]], 3, 4
 ; CHECK-QPX:       qvfadds 1, [[REG0]], [[REG1]]
 ; CHECK-PWR:       xvaddsp [[REG0:[0-9]+]], 34, 35
 ; CHECK-PWR:       xvaddsp [[REG1:[0-9]+]], 36, 37
 ; CHECK-PWR:       xvaddsp 34, [[REG0]], [[REG1]]
-; CHECK-PWR:       # kill
 ; CHECK-NEXT:  blr
 
   %t0 = fadd <4 x float> %x0, %x1
   %t1 = fadd <4 x float> %x2, %t0
   %t2 = fadd <4 x float> %t1, %x3
   ret <4 x float> %t2
 }
 
 define <4 x float> @vector_reassociate_adds3(<4 x float> %x0, <4 x float> %x1, <4 x float> %x2, <4 x float> %x3) {
 ; CHECK-LABEL: vector_reassociate_adds3:
 ; CHECK:       # BB#0:
 ; CHECK-QPX:       qvfadds [[REG0:[0-9]+]], 1, 2
 ; CHECK-QPX:       qvfadds [[REG1:[0-9]+]], 3, 4
 ; CHECK-QPX:       qvfadds 1, [[REG0]], [[REG1]]
 ; CHECK-PWR:       xvaddsp [[REG0:[0-9]+]], 34, 35
 ; CHECK-PWR:       xvaddsp [[REG1:[0-9]+]], 36, 37
 ; CHECK-PWR:       xvaddsp 34, [[REG0]], [[REG1]]
-; CHECK-PWR:       # kill
 ; CHECK-NEXT:  blr
 
   %t0 = fadd <4 x float> %x0, %x1
   %t1 = fadd <4 x float> %t0, %x2
   %t2 = fadd <4 x float> %x3, %t1
   ret <4 x float> %t2
 }
 
 define <4 x float> @vector_reassociate_adds4(<4 x float> %x0, <4 x float> %x1, <4 x float> %x2, <4 x float> %x3) {
 ; CHECK-LABEL: vector_reassociate_adds4:
 ; CHECK:       # BB#0:
 ; CHECK-QPX:       qvfadds [[REG0:[0-9]+]], 1, 2
 ; CHECK-QPX:       qvfadds [[REG1:[0-9]+]], 3, 4
 ; CHECK-QPX:       qvfadds 1, [[REG0]], [[REG1]]
 ; CHECK-PWR:       xvaddsp [[REG0:[0-9]+]], 34, 35
 ; CHECK-PWR:       xvaddsp [[REG1:[0-9]+]], 36, 37
 ; CHECK-PWR:       xvaddsp 34, [[REG0]], [[REG1]]
-; CHECK-PWR:       # kill
 ; CHECK-NEXT:  blr
 
   %t0 = fadd <4 x float> %x0, %x1
   %t1 = fadd <4 x float> %x2, %t0
   %t2 = fadd <4 x float> %x3, %t1
   ret <4 x float> %t2
 }
 

test/CodeGen/PowerPC/p8-scalar_vector_conversions.ll

entry:
   %a.addr = alloca i64, align 8
   store i64 %a, i64* %a.addr, align 8
   %0 = load i64, i64* %a.addr, align 8
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %0, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 ; CHECK: mtvsrd {{[0-9]+}}, 3
 ; CHECK-LE: mtvsrd [[REG1:[0-9]+]], 3
-; CHECK-LE: xxspltd [[REG1]], [[REG1]], 0
+; CHECK-LE: xxspltd 34, [[REG1]], 0
 }
 
 ; Function Attrs: nounwind
 define <4 x float> @buildf(float %a) {
 entry:
   %a.addr = alloca float, align 4
   store float %a, float* %a.addr, align 4
   %0 = load float, float* %a.addr, align 4
   %splat.splatinsert = insertelement <4 x float> undef, float %0, i32 0
   %splat.splat = shufflevector <4 x float> %splat.splatinsert, <4 x float> undef, <4 x i32> zeroinitializer
   ret <4 x float> %splat.splat
-; CHECK: xscvdpspn {{[0-9]+}}, 1
+; CHECK: xscvdpspn [[REG1:[0-9]+]], 1
+; CHECK: xxspltw 34, [[REG1]]
 ; CHECK-LE: xscvdpspn [[REG1:[0-9]+]], 1
-; CHECK-LE: xxsldwi {{[0-9]+}}, [[REG1]], [[REG1]], 1
+; CHECK-LE: xxspltw 34, [[REG1]]
 }
 
 ; The optimization to remove stack operations from PPCDAGToDAGISel::Select
 ; should still trigger for v2f64, producing an lxvdsx.
 ; Function Attrs: nounwind
 define <2 x double> @buildd() #0 {
 entry:
   %0 = load double, double* @d, align 8

test/CodeGen/PowerPC/power9-moves-and-splats.ll

 ; RUN: llc -mcpu=pwr9 -mtriple=powerpc64le-unknown-linux-gnu < %s | FileCheck %s
 ; RUN: llc -mcpu=pwr9 -mtriple=powerpc64-unknown-linux-gnu < %s | FileCheck %s \
 ; RUN:   --check-prefix=CHECK-BE
 
 @Globi = external global i32, align 4
 @Globf = external global float, align 4
 
 define <2 x i64> @test1(i64 %a, i64 %b) {
 entry:
+; The FIXME below is due to the lowering for BUILD_VECTOR needing a re-vamp
+; which will happen in a subsequent patch.
 ; CHECK-LABEL: test1
-; CHECK: mtvsrdd 34, 4, 3
+; FIXME: mtvsrdd 34, 4, 3
+; CHECK: mtvsrd {{[0-9]+}}, 3
+; CHECK: mtvsrd {{[0-9]+}}, 4
+; CHECK: xxmrgld
 ; CHECK-BE-LABEL: test1
-; CHECK-BE: mtvsrdd 34, 3, 4
+; FIXME-BE: mtvsrdd 34, 3, 4
+; CHECK-BE: mtvsrd {{[0-9]+}}, 4
+; CHECK-BE: mtvsrd {{[0-9]+}}, 3
+; CHECK-BE: xxmrghd
   %vecins = insertelement <2 x i64> undef, i64 %a, i32 0
   %vecins1 = insertelement <2 x i64> %vecins, i64 %b, i32 1
   ret <2 x i64> %vecins1
 }
 
 define i64 @test2(<2 x i64> %a) {
 entry:
 ; CHECK-LABEL: test2

test/CodeGen/PowerPC/ppc64-i128-abi.ll

 
 ; CHECK-LE-LABEL: @v1i128_increment_by_one
 ; CHECK-LE: lxvd2x [[VAL:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-LE: xxswapd 35, [[VAL]]
 ; CHECK-LE: vadduqm 2, 2, 3
 ; CHECK-LE: blr
 
 ; CHECK-P9-LABEL: @v1i128_increment_by_one
-; CHECK-P9-DAG: li [[R1:r[0-9]+]], 1
-; CHECK-P9-DAG: li [[R2:r[0-9]+]], 0
-; CHECK-P9: mtvsrdd [[V1:v[0-9]+]], [[R2]], [[R1]]
+; The below FIXME is due to the lowering for BUILD_VECTOR that will be fixed
+; in a subsequent patch.
+; FIXME: li [[R1:r[0-9]+]], 1
+; FIXME: li [[R2:r[0-9]+]], 0
+; FIXME: mtvsrdd [[V1:v[0-9]+]], [[R2]], [[R1]]
+; CHECK-P9: lxvx [[V1:v[0-9]+]]
 ; CHECK-P9: vadduqm v2, v2, [[V1]]
 ; CHECK-P9: blr
 
 ; CHECK-BE-LABEL: @v1i128_increment_by_one
 ; CHECK-BE: lxvd2x 35, {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-BE-NOT: xxswapd 
 ; CHECK-BE: vadduqm 2, 2, 3 
 ; CHECK-BE-NOT: xxswapd 34, {{[0-9]+}}

test/CodeGen/PowerPC/select-i1-vs-i1.ll

 define <4 x float> @testv4floatslt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp slt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
-; FIXME: This test (and the other v4f32 tests) should use the same bclr
-; technique as the v2f64 tests below.
-
 ; CHECK-LABEL: @testv4floatslt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatult(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ult i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatult
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsle(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sle i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsle
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatule(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ule i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatule
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floateq(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp eq i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floateq
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 35, 35
-; CHECK-DAG: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 34, 34
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bc 12, [[REG1]], .LBB[[BB1:[0-9_]+]]
+; CHECK: vor 3, 2, 2
+; CHECK: .LBB[[BB1]]
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsge(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sge i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsge
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatuge(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp uge i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatuge
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsgt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sgt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsgt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatugt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ugt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatugt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatne(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ne i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatne
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define ppc_fp128 @testppc_fp128eq(ppc_fp128 %c1, ppc_fp128 %c2, ppc_fp128 %c3, ppc_fp128 %c4, ppc_fp128 %a1, ppc_fp128 %a2) #0 {
 entry:
   %cmp1 = fcmp oeq ppc_fp128 %c3, %c4
   %cmp3tmp = fcmp oeq ppc_fp128 %c1, %c2
   %cmp3 = icmp eq i1 %cmp3tmp, %cmp1

 
 ; CHECK-LABEL: @testv2doubleeq
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
 ; CHECK: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
 ; CHECK: bc 12, [[REG1]], .LBB[[BB55:[0-9_]+]]
 ; CHECK: vor 3, 2, 2
 ; CHECK: .LBB[[BB55]]
-; CHECK: xxlor 34, 35, 35
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <2 x double> @testv2doublesge(float %c1, float %c2, float %c3, float %c4, <2 x double> %a1, <2 x double> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sge i1 %cmp3tmp, %cmp1

test/CodeGen/PowerPC/sjlj.ll

 ; FIXME: We should be saving VRSAVE on Darwin, but we're not!
 
 ; CHECK: @main
 ; CHECK: std
 ; Make sure that we're not saving VRSAVE on non-Darwin:
 ; CHECK-NOT: mfspr
 
 ; CHECK-DAG: stfd
-; CHECK-DAG: stvx
+; CHECK-DAG: stxvd2x
 
 ; CHECK-DAG: addis [[REG:[0-9]+]], 2, env_sigill@toc@ha
 ; CHECK-DAG: std 31, env_sigill@toc@l([[REG]])
 ; CHECK-DAG: addi [[REGA:[0-9]+]], [[REG]], env_sigill@toc@l
 ; CHECK-DAG: std [[REGA]], [[OFF:[0-9]+]](31)                  # 8-byte Folded Spill
 ; CHECK-DAG: std 1, 16([[REGA]])
 ; CHECK-DAG: std 2, 24([[REGA]])
 ; CHECK: bcl 20, 31, .LBB1_5
 ; CHECK: li 3, 1
 ; CHECK: #EH_SjLj_Setup .LBB1_5
 ; CHECK: b .LBB1_1
 
 ; CHECK: .LBB1_4:
 
 ; CHECK: lfd
-; CHECK: lvx
+; CHECK: lxvd2x
 ; CHECK: ld
 ; CHECK: blr
 
 ; CHECK: .LBB1_5:
 ; CHECK: mflr [[REGL:[0-9]+]]
 ; CHECK: ld [[REG2:[0-9]+]], [[OFF]](31)                   # 8-byte Folded Reload
 ; CHECK: std [[REGL]], 8([[REG2]])
 ; CHECK: li 3, 0
 
 ; CHECK-NOAV: @main
-; CHECK-NOAV-NOT: stvx
+; CHECK-NOAV-NOT: stxvd2x
 ; CHECK-NOAV: bcl
 ; CHECK-NOAV: mflr
 ; CHECK-NOAV: bl foo
-; CHECK-NOAV-NOT: lvx
+; CHECK-NOAV-NOT: lxvd2x
 ; CHECK-NOAV: blr
 }
 
 define signext i32 @main2() #0 {
 entry:
   %a = alloca i8, align 64
   call void @bar(i8* %a)
   %retval = alloca i32, align 4

test/CodeGen/PowerPC/vsx-args.ll

 ; RUN: llc -verify-machineinstrs < %s -mcpu=pwr7 -mattr=+vsx | FileCheck %s
-; RUN: llc -verify-machineinstrs < %s -mcpu=pwr7 -mattr=+vsx -fast-isel -O0 | FileCheck %s
+; RUN: llc -verify-machineinstrs < %s -mcpu=pwr7 -mattr=+vsx -fast-isel -O0 | \
+; RUN:   FileCheck -check-prefix=CHECK-FISL %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 declare <2 x double> @sv(<2 x double>, <2 x i64>, <4 x float>) #0
 
 define <2 x double> @main(<4 x float> %a, <2 x double> %b, <2 x i64> %c) #1 {
 entry:
   %ca = tail call <2 x double> @sv(<2 x double> %b, <2 x i64> %c,  <4 x float> %a)
   %v = fadd <2 x double> %ca, <double 1.0, double 1.0>
   ret <2 x double> %v
 
 ; CHECK-LABEL: @main
 ; CHECK-DAG: vor [[V:[0-9]+]], 2, 2
-; CHECK-DAG: xxlor 34, 35, 35
-; CHECK-DAG: xxlor 35, 36, 36
+; CHECK-DAG: vor 2, 3, 3
+; CHECK-DAG: vor 3, 4, 4
 ; CHECK-DAG: vor 4, [[V]], [[V]]
-; CHECK-DAG: bl sv
-; CHECK-DAG: lxvd2x [[VC:[0-9]+]],
+; CHECK: bl sv
+; CHECK: lxvd2x [[VC:[0-9]+]],
 ; CHECK: xvadddp 34, 34, [[VC]]
 ; CHECK: blr
+
+; CHECK-FISL-LABEL: @main
+; CHECK-FISL: stxvd2x 34
+; CHECK-FISL: vor 2, 3, 3
+; CHECK-FISL: vor 3, 4, 4
+; CHECK-FISL: lxvd2x 36
+; CHECK-FISL: bl sv
+; CHECK-FISL: lxvd2x [[VC:[0-9]+]],
+; CHECK-FISL: xvadddp 34, 34, [[VC]]
+; CHECK-FISL: blr
 }
 
 attributes #0 = { noinline nounwind readnone }
 attributes #1 = { nounwind }
 

test/CodeGen/PowerPC/vsx-infl-copy1.ll

 ; RUN: llc -verify-machineinstrs -mcpu=pwr7 < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 @ub = external global [1024 x i32], align 4
 @uc = external global [1024 x i32], align 4
 
 ; Function Attrs: noinline nounwind
 define <4 x i32> @_Z8example9Pj(<4 x i32>* %addr1, i64 %input1, i64 %input2) #0 {
 entry:
   br label %vector.body
 
 ; CHECK-LABEL: @_Z8example9Pj
-; CHECK: xxlor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
 
 vector.body:                                      ; preds = %vector.body, %entry
   %index = phi i64 [ 0, %entry ], [ %index.next, %vector.body ]
   %vec.phi = phi <4 x i32> [ zeroinitializer, %entry ], [ %43, %vector.body ]
   %vec.phi20 = phi <4 x i32> [ zeroinitializer, %entry ], [ %44, %vector.body ]
   %vec.phi21 = phi <4 x i32> [ zeroinitializer, %entry ], [ %45, %vector.body ]
   %vec.phi23 = phi <4 x i32> [ zeroinitializer, %entry ], [ %46, %vector.body ]
   %vec.phi24 = phi <4 x i32> [ zeroinitializer, %entry ], [ %47, %vector.body ]

test/CodeGen/PowerPC/vsx-p8.ll

define <4 x float> @test32u(<4 x float>* %a) {
   %v = load <4 x float>, <4 x float>* %a, align 8
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test32u
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test32u
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 }
 
 define void @test33u(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 8
   ret void
 
 ; CHECK-REG-LABEL: @test33u
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33u
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 }
 

test/CodeGen/PowerPC/vsx-partword-int-loads-and-stores.ll

+; RUN: llc -mcpu=pwr9 -mtriple=powerpc64le-unknown-unknown < %s | FileCheck %s
+; RUN: llc -mcpu=pwr9 -mtriple=powerpc64-unknown-unknown < %s | FileCheck %s \
+; RUN:   --check-prefix=CHECK-BE
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecucuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %0, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecucuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecucuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <8 x i16> @vecusuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i16
+  %splat.splatinsert = insertelement <8 x i16> undef, i16 %conv, i32 0
+  %splat.splat = shufflevector <8 x i16> %splat.splatinsert, <8 x i16> undef, <8 x i32> zeroinitializer
+  ret <8 x i16> %splat.splat
+; CHECK-LABEL: vecusuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vsplth 2, 2, 3
+; CHECK-BE-LABEL: vecusuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vsplth 2, 2, 3
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecuiuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecuiuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecuiuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @veculuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: veculuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: veculuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecscuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %0, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecscuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecscuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <8 x i16> @vecssuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i16
+  %splat.splatinsert = insertelement <8 x i16> undef, i16 %conv, i32 0
+  %splat.splat = shufflevector <8 x i16> %splat.splatinsert, <8 x i16> undef, <8 x i32> zeroinitializer
+  ret <8 x i16> %splat.splat
+; CHECK-LABEL: vecssuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vsplth 2, 2, 3
+; CHECK-BE-LABEL: vecssuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vsplth 2, 2, 3
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecsiuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecsiuc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecsiuc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @vecsluc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = zext i8 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: vecsluc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: vecsluc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x float> @vecfuc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = uitofp i8 %0 to float
+  %splat.splatinsert = insertelement <4 x float> undef, float %conv, i32 0
+  %splat.splat = shufflevector <4 x float> %splat.splatinsert, <4 x float> undef, <4 x i32> zeroinitializer
+  ret <4 x float> %splat.splat
+; CHECK-LABEL: vecfuc
+; CHECK: lxsibzx [[LD:[0-9]+]], 0, 3
+; CHECK-NEXT: xscvuxdsp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-NEXT: xxspltw 34, [[CONVS]], 0
+; CHECK-BE-LABEL: vecfuc
+; CHECK-BE: lxsibzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: xscvuxdsp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-BE-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-BE-NEXT: xxspltw 34, [[CONVS]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x double> @vecduc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = uitofp i8 %0 to double
+  %splat.splatinsert = insertelement <2 x double> undef, double %conv, i32 0
+  %splat.splat = shufflevector <2 x double> %splat.splatinsert, <2 x double> undef, <2 x i32> zeroinitializer
+  ret <2 x double> %splat.splat
+; CHECK-LABEL: vecduc
+; CHECK: lxsibzx [[LD:[0-9]+]], 0, 3
+; CHECK-NEXT: xscvuxddp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-NEXT: xxspltd  34, [[CONVD]], 0
+; CHECK-BE-LABEL: vecduc
+; CHECK-BE: lxsibzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: xscvuxddp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-BE-NEXT: xxspltd  34, [[CONVD]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecucsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %0, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecucsc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecucsc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecuisc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sext i8 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecuisc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2w 2, 2
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecuisc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2w 2, 2
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @veculsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sext i8 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: veculsc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2d 2, 2
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: veculsc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2d 2, 2
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecscsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %0, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecscsc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecscsc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecsisc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sext i8 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecsisc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2w 2, 2
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecsisc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2w 2, 2
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @vecslsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sext i8 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: vecslsc
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2d 2, 2
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: vecslsc
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2d 2, 2
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x float> @vecfsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sitofp i8 %0 to float
+  %splat.splatinsert = insertelement <4 x float> undef, float %conv, i32 0
+  %splat.splat = shufflevector <4 x float> %splat.splatinsert, <4 x float> undef, <4 x i32> zeroinitializer
+  ret <4 x float> %splat.splat
+; CHECK-LABEL: vecfsc
+; CHECK: lxsibzx
+; CHECK-NEXT: vextsb2d
+; CHECK-NEXT: xscvsxdsp [[CONVD:[0-9]+]],
+; CHECK-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-NEXT: xxspltw 34, [[CONVS]], 0
+; CHECK-BE-LABEL: vecfsc
+; CHECK-BE: lxsibzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: vextsb2d
+; CHECK-BE-NEXT: xscvsxdsp [[CONVD:[0-9]+]],
+; CHECK-BE-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-BE-NEXT: xxspltw 34, [[CONVS]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x double> @vecdsc(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sitofp i8 %0 to double
+  %splat.splatinsert = insertelement <2 x double> undef, double %conv, i32 0
+  %splat.splat = shufflevector <2 x double> %splat.splatinsert, <2 x double> undef, <2 x i32> zeroinitializer
+  ret <2 x double> %splat.splat
+; CHECK-LABEL: vecdsc
+; CHECK: lxsibzx
+; CHECK-NEXT: vextsb2d
+; CHECK-NEXT: xscvsxddp [[CONVD:[0-9]+]],
+; CHECK-NEXT: xxspltd  34, [[CONVD]], 0
+; CHECK-BE-LABEL: vecdsc
+; CHECK-BE: lxsibzx
+; CHECK-BE-NEXT: vextsb2d
+; CHECK-BE-NEXT: xscvsxddp [[CONVD:[0-9]+]],
+; CHECK-BE-NEXT: xxspltd  34, [[CONVD]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecucus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = trunc i16 %0 to i8
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %conv, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecucus
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecucus
+; CHECK-BE: li [[OFFSET:[0-9]+]], 1
+; CHECK-BE-NEXT: lxsibzx 34, 3, [[OFFSET]]
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <8 x i16> @vecusus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %splat.splatinsert = insertelement <8 x i16> undef, i16 %0, i32 0
+  %splat.splat = shufflevector <8 x i16> %splat.splatinsert, <8 x i16> undef, <8 x i32> zeroinitializer
+  ret <8 x i16> %splat.splat
+; CHECK-LABEL: vecusus
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vsplth 2, 2, 3
+; CHECK-BE-LABEL: vecusus
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vsplth 2, 2, 3
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecuius(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = zext i16 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecuius
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecuius
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @veculus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = zext i16 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: veculus
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: veculus
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecscus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = trunc i16 %0 to i8
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %conv, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecscus
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecscus
+; CHECK-BE: li [[OFFSET:[0-9]+]], 1
+; CHECK-BE-NEXT: lxsibzx 34, 3, [[OFFSET]]
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <8 x i16> @vecssus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %splat.splatinsert = insertelement <8 x i16> undef, i16 %0, i32 0
+  %splat.splat = shufflevector <8 x i16> %splat.splatinsert, <8 x i16> undef, <8 x i32> zeroinitializer
+  ret <8 x i16> %splat.splat
+; CHECK-LABEL: vecssus
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vsplth 2, 2, 3
+; CHECK-BE-LABEL: vecssus
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vsplth 2, 2, 3
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecsius(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = zext i16 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecsius
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecsius
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @vecslus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = zext i16 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: vecslus
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: vecslus
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x float> @vecfus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = uitofp i16 %0 to float
+  %splat.splatinsert = insertelement <4 x float> undef, float %conv, i32 0
+  %splat.splat = shufflevector <4 x float> %splat.splatinsert, <4 x float> undef, <4 x i32> zeroinitializer
+  ret <4 x float> %splat.splat
+; CHECK-LABEL: vecfus
+; CHECK: lxsihzx [[LD:[0-9]+]], 0, 3
+; CHECK-NEXT: xscvuxdsp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-NEXT: xxspltw 34, [[CONVS]], 0
+; CHECK-BE-LABEL: vecfus
+; CHECK-BE: lxsihzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: xscvuxdsp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-BE-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-BE-NEXT: xxspltw 34, [[CONVS]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x double> @vecdus(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = uitofp i16 %0 to double
+  %splat.splatinsert = insertelement <2 x double> undef, double %conv, i32 0
+  %splat.splat = shufflevector <2 x double> %splat.splatinsert, <2 x double> undef, <2 x i32> zeroinitializer
+  ret <2 x double> %splat.splat
+; CHECK-LABEL: vecdus
+; CHECK: lxsihzx [[LD:[0-9]+]], 0, 3
+; CHECK-NEXT: xscvuxddp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-NEXT: xxspltd  34, [[CONVD]], 0
+; CHECK-BE-LABEL: vecdus
+; CHECK-BE: lxsihzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: xscvuxddp [[CONVD:[0-9]+]], [[LD]]
+; CHECK-BE-NEXT: xxspltd  34, [[CONVD]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecucss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = trunc i16 %0 to i8
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %conv, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecucss
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecucss
+; CHECK-BE: li [[OFFSET:[0-9]+]], 1
+; CHECK-BE-NEXT: lxsibzx 34, 3, [[OFFSET]]
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecuiss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sext i16 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecuiss
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2w 2, 2
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecuiss
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2w 2, 2
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @veculss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sext i16 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: veculss
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2d 2, 2
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: veculss
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2d 2, 2
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <16 x i8> @vecscss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = trunc i16 %0 to i8
+  %splat.splatinsert = insertelement <16 x i8> undef, i8 %conv, i32 0
+  %splat.splat = shufflevector <16 x i8> %splat.splatinsert, <16 x i8> undef, <16 x i32> zeroinitializer
+  ret <16 x i8> %splat.splat
+; CHECK-LABEL: vecscss
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vspltb 2, 2, 7
+; CHECK-BE-LABEL: vecscss
+; CHECK-BE: li [[OFFSET:[0-9]+]], 1
+; CHECK-BE-NEXT: lxsibzx 34, 3, [[OFFSET]]
+; CHECK-BE-NEXT: vspltb 2, 2, 7
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x i32> @vecsiss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sext i16 %0 to i32
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %conv, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  ret <4 x i32> %splat.splat
+; CHECK-LABEL: vecsiss
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2w 2, 2
+; CHECK-NEXT: xxspltw 34, 34, 1
+; CHECK-BE-LABEL: vecsiss
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2w 2, 2
+; CHECK-BE-NEXT: xxspltw 34, 34, 1
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x i64> @vecslss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sext i16 %0 to i64
+  %splat.splatinsert = insertelement <2 x i64> undef, i64 %conv, i32 0
+  %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
+  ret <2 x i64> %splat.splat
+; CHECK-LABEL: vecslss
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2d 2, 2
+; CHECK-NEXT: xxspltd 34, 34, 0
+; CHECK-BE-LABEL: vecslss
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2d 2, 2
+; CHECK-BE-NEXT: xxspltd 34, 34, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <4 x float> @vecfss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sitofp i16 %0 to float
+  %splat.splatinsert = insertelement <4 x float> undef, float %conv, i32 0
+  %splat.splat = shufflevector <4 x float> %splat.splatinsert, <4 x float> undef, <4 x i32> zeroinitializer
+  ret <4 x float> %splat.splat
+; CHECK-LABEL: vecfss
+; CHECK: lxsihzx
+; CHECK-NEXT: vextsh2d
+; CHECK-NEXT: xscvsxdsp [[CONVD:[0-9]+]],
+; CHECK-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-NEXT: xxspltw 34, [[CONVS]], 0
+; CHECK-BE-LABEL: vecfss
+; CHECK-BE: lxsihzx [[LD:[0-9]+]], 0, 3
+; CHECK-BE-NEXT: vextsh2d
+; CHECK-BE-NEXT: xscvsxdsp [[CONVD:[0-9]+]],
+; CHECK-BE-NEXT: xscvdpspn [[CONVS:[0-9]+]], [[CONVD]]
+; CHECK-BE-NEXT: xxspltw 34, [[CONVS]], 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define <2 x double> @vecdss(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sitofp i16 %0 to double
+  %splat.splatinsert = insertelement <2 x double> undef, double %conv, i32 0
+  %splat.splat = shufflevector <2 x double> %splat.splatinsert, <2 x double> undef, <2 x i32> zeroinitializer
+  ret <2 x double> %splat.splat
+; CHECK-LABEL: vecdss
+; CHECK: lxsihzx
+; CHECK-NEXT: vextsh2d
+; CHECK-NEXT: xscvsxddp [[CONVD:[0-9]+]],
+; CHECK-NEXT: xxspltd  34, [[CONVD]], 0
+; CHECK-BE-LABEL: vecdss
+; CHECK-BE: lxsihzx
+; CHECK-BE-NEXT: vextsh2d
+; CHECK-BE-NEXT: xscvsxddp [[CONVD:[0-9]+]],
+; CHECK-BE-NEXT: xxspltd  34, [[CONVD]], 0
+}
+
+; Function Attrs: norecurse nounwind
+define void @storefsc(float %f, i8* nocapture %ptr) {
+entry:
+  %conv = fptosi float %f to i8
+  store i8 %conv, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storefsc
+; CHECK: xscvdpsxws 0, 1
+; CHECK: stxsibx 0, 0, 4
+; CHECK-BE-LABEL: storefsc
+; CHECK-BE: xscvdpsxws 0, 1
+; CHECK-BE: stxsibx 0, 0, 4
+}
+
+; Function Attrs: norecurse nounwind
+define void @storedsc(double %d, i8* nocapture %ptr) {
+entry:
+  %conv = fptosi double %d to i8
+  store i8 %conv, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storedsc
+; CHECK: xscvdpsxws 0, 1
+; CHECK: stxsibx 0, 0, 4
+; CHECK-BE-LABEL: storedsc
+; CHECK-BE: xscvdpsxws 0, 1
+; CHECK-BE: stxsibx 0, 0, 4
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc0(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 0
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc0
+; CHECK: vsldoi 2, 2, 2, 8
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc0
+; CHECK-BE: vsldoi 2, 2, 2, 9
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc1(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 1
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc1
+; CHECK: vsldoi 2, 2, 2, 7
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc1
+; CHECK-BE: vsldoi 2, 2, 2, 10
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc2(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 2
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc2
+; CHECK: vsldoi 2, 2, 2, 6
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc2
+; CHECK-BE: vsldoi 2, 2, 2, 11
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc3(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 3
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc3
+; CHECK: vsldoi 2, 2, 2, 5
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc3
+; CHECK-BE: vsldoi 2, 2, 2, 12
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc4(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 4
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc4
+; CHECK: vsldoi 2, 2, 2, 4
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc4
+; CHECK-BE: vsldoi 2, 2, 2, 13
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc5(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 5
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc5
+; CHECK: vsldoi 2, 2, 2, 3
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc5
+; CHECK-BE: vsldoi 2, 2, 2, 14
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc6(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 6
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc6
+; CHECK: vsldoi 2, 2, 2, 2
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc6
+; CHECK-BE: vsldoi 2, 2, 2, 15
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc7(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 7
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc7
+; CHECK: vsldoi 2, 2, 2, 1
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc7
+; CHECK-BE: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc8(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 8
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc8
+; CHECK: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc8
+; CHECK-BE: vsldoi 2, 2, 2, 1
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc9(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 9
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc9
+; CHECK: vsldoi 2, 2, 2, 15
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc9
+; CHECK-BE: vsldoi 2, 2, 2, 2
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc10(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 10
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc10
+; CHECK: vsldoi 2, 2, 2, 14
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc10
+; CHECK-BE: vsldoi 2, 2, 2, 3
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc11(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 11
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc11
+; CHECK: vsldoi 2, 2, 2, 13
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc11
+; CHECK-BE: vsldoi 2, 2, 2, 4
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc12(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 12
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc12
+; CHECK: vsldoi 2, 2, 2, 12
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc12
+; CHECK-BE: vsldoi 2, 2, 2, 5
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc13(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 13
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc13
+; CHECK: vsldoi 2, 2, 2, 11
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc13
+; CHECK-BE: vsldoi 2, 2, 2, 6
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc14(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 14
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc14
+; CHECK: vsldoi 2, 2, 2, 10
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc14
+; CHECK-BE: vsldoi 2, 2, 2, 7
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevcsc15(<16 x i8> %v, i8* nocapture %ptr) {
+entry:
+  %vecext = extractelement <16 x i8> %v, i32 15
+  store i8 %vecext, i8* %ptr, align 1
+  ret void
+; CHECK-LABEL: storevcsc15
+; CHECK: vsldoi 2, 2, 2, 9
+; CHECK-NEXT: stxsibx 34, 0, 5
+; CHECK-BE-LABEL: storevcsc15
+; CHECK-BE: vsldoi 2, 2, 2, 8
+; CHECK-BE-NEXT: stxsibx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storefss(float %f, i16* nocapture %ptr) {
+entry:
+  %conv = fptosi float %f to i16
+  store i16 %conv, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storefss
+; CHECK: xscvdpsxws 0, 1
+; CHECK: stxsihx 0, 0, 4
+; CHECK-BE-LABEL: storefss
+; CHECK-BE: xscvdpsxws 0, 1
+; CHECK-BE: stxsihx 0, 0, 4
+}
+
+; Function Attrs: norecurse nounwind
+define void @storedss(double %d, i16* nocapture %ptr) {
+entry:
+  %conv = fptosi double %d to i16
+  store i16 %conv, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storedss
+; CHECK: xscvdpsxws 0, 1
+; CHECK: stxsihx 0, 0, 4
+; CHECK-BE-LABEL: storedss
+; CHECK-BE: xscvdpsxws 0, 1
+; CHECK-BE: stxsihx 0, 0, 4
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss0(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 0
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss0
+; CHECK: vsldoi 2, 2, 2, 8
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss0
+; CHECK-BE: vsldoi 2, 2, 2, 10
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss1(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 1
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss1
+; CHECK: vsldoi 2, 2, 2, 6
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss1
+; CHECK-BE: vsldoi 2, 2, 2, 12
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss2(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 2
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss2
+; CHECK: vsldoi 2, 2, 2, 4
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss2
+; CHECK-BE: vsldoi 2, 2, 2, 14
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss3(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 3
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss3
+; CHECK: vsldoi 2, 2, 2, 2
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss3
+; CHECK-BE: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss4(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 4
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss4
+; CHECK: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss4
+; CHECK-BE: vsldoi 2, 2, 2, 2
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss5(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 5
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss5
+; CHECK: vsldoi 2, 2, 2, 14
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss5
+; CHECK-BE: vsldoi 2, 2, 2, 4
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss6(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 6
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss6
+; CHECK: vsldoi 2, 2, 2, 12
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss6
+; CHECK-BE: vsldoi 2, 2, 2, 6
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind
+define void @storevsss7(<8 x i16> %v, i16* nocapture %ptr) {
+entry:
+  %vecext = extractelement <8 x i16> %v, i32 7
+  store i16 %vecext, i16* %ptr, align 2
+  ret void
+; CHECK-LABEL: storevsss7
+; CHECK: vsldoi 2, 2, 2, 10
+; CHECK-NEXT: stxsihx 34, 0, 5
+; CHECK-BE-LABEL: storevsss7
+; CHECK-BE: vsldoi 2, 2, 2, 8
+; CHECK-BE-NEXT: stxsihx 34, 0, 5
+}
+
+; Function Attrs: norecurse nounwind readonly
+define float @convscf(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sitofp i8 %0 to float
+  ret float %conv
+; CHECK-LABEL: convscf
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2d 2, 2
+; CHECK-NEXT: xscvsxdsp 1, 34
+; CHECK-BE-LABEL: convscf
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2d 2, 2
+; CHECK-BE-NEXT: xscvsxdsp 1, 34
+}
+
+; Function Attrs: norecurse nounwind readonly
+define float @convucf(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = uitofp i8 %0 to float
+  ret float %conv
+; CHECK-LABEL: convucf
+; CHECK: lxsibzx 0, 0, 3
+; CHECK-NEXT: xscvuxdsp 1, 0
+; CHECK-BE-LABEL: convucf
+; CHECK-BE: lxsibzx 0, 0, 3
+; CHECK-BE-NEXT: xscvuxdsp 1, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define double @convscd(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = sitofp i8 %0 to double
+; CHECK-LABEL: convscd
+; CHECK: lxsibzx 34, 0, 3
+; CHECK-NEXT: vextsb2d 2, 2
+; CHECK-NEXT: xscvsxddp 1, 34
+; CHECK-BE-LABEL: convscd
+; CHECK-BE: lxsibzx 34, 0, 3
+; CHECK-BE-NEXT: vextsb2d 2, 2
+; CHECK-BE-NEXT: xscvsxddp 1, 34
+  ret double %conv
+}
+
+; Function Attrs: norecurse nounwind readonly
+define double @convucd(i8* nocapture readonly %ptr) {
+entry:
+  %0 = load i8, i8* %ptr, align 1
+  %conv = uitofp i8 %0 to double
+  ret double %conv
+; CHECK-LABEL: convucd
+; CHECK: lxsibzx 0, 0, 3
+; CHECK-NEXT: xscvuxddp 1, 0
+; CHECK-BE-LABEL: convucd
+; CHECK-BE: lxsibzx 0, 0, 3
+; CHECK-BE-NEXT: xscvuxddp 1, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define float @convssf(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sitofp i16 %0 to float
+  ret float %conv
+; CHECK-LABEL: convssf
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2d 2, 2
+; CHECK-NEXT: xscvsxdsp 1, 34
+; CHECK-BE-LABEL: convssf
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2d 2, 2
+; CHECK-BE-NEXT: xscvsxdsp 1, 34
+}
+
+; Function Attrs: norecurse nounwind readonly
+define float @convusf(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = uitofp i16 %0 to float
+  ret float %conv
+; CHECK-LABEL: convusf
+; CHECK: lxsihzx 0, 0, 3
+; CHECK-NEXT: xscvuxdsp 1, 0
+; CHECK-BE-LABEL: convusf
+; CHECK-BE: lxsihzx 0, 0, 3
+; CHECK-BE-NEXT: xscvuxdsp 1, 0
+}
+
+; Function Attrs: norecurse nounwind readonly
+define double @convssd(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = sitofp i16 %0 to double
+  ret double %conv
+; CHECK-LABEL: convssd
+; CHECK: lxsihzx 34, 0, 3
+; CHECK-NEXT: vextsh2d 2, 2
+; CHECK-NEXT: xscvsxddp 1, 34
+; CHECK-BE-LABEL: convssd
+; CHECK-BE: lxsihzx 34, 0, 3
+; CHECK-BE-NEXT: vextsh2d 2, 2
+; CHECK-BE-NEXT: xscvsxddp 1, 34
+}
+
+; Function Attrs: norecurse nounwind readonly
+define double @convusd(i16* nocapture readonly %ptr) {
+entry:
+  %0 = load i16, i16* %ptr, align 2
+  %conv = uitofp i16 %0 to double
+  ret double %conv
+; CHECK-LABEL: convusd
+; CHECK: lxsihzx 0, 0, 3
+; CHECK-NEXT: xscvuxddp 1, 0
+; CHECK-BE-LABEL: convusd
+; CHECK-BE: lxsihzx 0, 0, 3
+; CHECK-BE-NEXT: xscvuxddp 1, 0
+}

test/CodeGen/PowerPC/vsx-spill-norwstore.ll

 ; RUN: llc -mcpu=pwr7 -verify-machineinstrs < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 @.str1 = external unnamed_addr constant [5 x i8], align 1
 @.str10 = external unnamed_addr constant [9 x i8], align 1
+@.v2f64 = external unnamed_addr constant <2 x double>, align 16
 
 ; Function Attrs: nounwind
 define void @main() #0 {
 ; CHECK-LABEL: @main
 ; Make sure that the stxvd2x passes -verify-machineinstrs
 ; CHECK: stxvd2x
 
 entry:
+  %val = load <2 x double>, <2 x double>* @.v2f64, align 16
   %0 = tail call <8 x i16> @llvm.ppc.altivec.vupkhsb(<16 x i8> <i8 0, i8 -1, i8 -1, i8 0, i8 0, i8 0, i8 -1, i8 0, i8 -1, i8 0, i8 0, i8 -1, i8 -1, i8 -1, i8 0, i8 -1>) #0
   %1 = tail call <8 x i16> @llvm.ppc.altivec.vupklsb(<16 x i8> <i8 0, i8 -1, i8 -1, i8 0, i8 0, i8 0, i8 -1, i8 0, i8 -1, i8 0, i8 0, i8 -1, i8 -1, i8 -1, i8 0, i8 -1>) #0
   br i1 false, label %if.then.i68.i, label %check.exit69.i
 
 if.then.i68.i:                                    ; preds = %entry
   unreachable
 
 check.exit69.i:                                   ; preds = %entry
   br i1 undef, label %if.then.i63.i, label %check.exit64.i
 
 if.then.i63.i:                                    ; preds = %check.exit69.i
-  tail call void (i8*, ...) @printf(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str10, i64 0, i64 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @.str1, i64 0, i64 0)) #0
+  tail call void (i8*, ...) @printf(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str10, i64 0, i64 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @.str1, i64 0, i64 0), <2 x double> %val) #0
   br label %check.exit64.i
 
 check.exit64.i:                                   ; preds = %if.then.i63.i, %check.exit69.i
   %2 = tail call i32 @llvm.ppc.altivec.vcmpequh.p(i32 2, <8 x i16> %0, <8 x i16> <i16 0, i16 -1, i16 -1, i16 0, i16 0, i16 0, i16 -1, i16 0>) #0
   %tobool.i55.i = icmp eq i32 %2, 0
   br i1 %tobool.i55.i, label %if.then.i58.i, label %check.exit59.i
 
 if.then.i58.i:                                    ; preds = %check.exit64.i

test/CodeGen/PowerPC/vsx-vec-spill.ll

+; RUN: llc < %s -march=ppc64 -mattr=+vsx -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=VSX
+; RUN: llc < %s -march=ppc64 -mattr=-vsx -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=NOVSX
+
+define <2 x double> @interleaving_VSX_VMX(
+  <2 x double> %a, <2 x double> %b, <2 x double> %c,
+  <2 x double> %d, <2 x double> %e, <2 x double> %f) {
+entry:
+  tail call void asm sideeffect "# clobbers",
+    "~{v14},~{v15},~{v16},~{v17},~{v18},~{v19},~{v20},~{v21},~{v22},~{v23},~{v24},~{v25},~{v26},~{v27},~{v28},~{v29},~{v30},~{v31}"() nounwind
+  tail call void @goo(<2 x double> %a) nounwind
+  %add = fadd <2 x double> %a, %b
+  %sub = fsub <2 x double> %a, %b
+  %mul = fmul <2 x double> %add, %sub
+  %add1 = fadd <2 x double> %c, %d
+  %sub2 = fsub <2 x double> %c, %d
+  %mul3 = fmul <2 x double> %add1, %sub2
+  %add4 = fadd <2 x double> %mul, %mul3
+  %add5 = fadd <2 x double> %e, %f
+  %sub6 = fsub <2 x double> %e, %f
+  %mul7 = fmul <2 x double> %add5, %sub6
+  %add8 = fadd <2 x double> %add4, %mul7
+  ret <2 x double> %add8
+; VSX-LABEL: interleaving_VSX_VMX
+; VSX-NOT: stvx
+; VSX-NOT: lvx
+
+; NOVSX-LABEL: interleaving_VSX_VMX
+; NOVSX-NOT: stxvd2x
+; NOVSX-NOT: lxvd2x
+}
+
+declare void @goo(<2 x double>)

test/CodeGen/PowerPC/vsx.ll

entry:
   %v = xor <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test5
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test5
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxlxor
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test5
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test6(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = xor <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test6
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test6
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlxor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test6
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test7(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = xor <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test7
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test7
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlxor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test7
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test8(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = or <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test8
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test8
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxlor
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test8
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test9(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = or <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test9
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test9
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test9
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test10(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = or <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test10
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test10
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test10
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test11(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = and <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test11
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test11
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxland
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test11
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test12(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = and <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test12
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test12
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxland 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test12
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test13(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = and <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test13
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test13
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxland 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test13
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test14(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = or <4 x i32> %a, %b
   %w = xor <4 x i32> %v, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %w
 
 ; CHECK-REG-LABEL: @test14
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test14
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: xxlor 0, 37, 36
-; CHECK-FISL: xxlnor 36, 37, 36
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlnor 34, 34, 35
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
 ; CHECK-FISL: stxvd2x 0, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test14
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test15(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = or <8 x i16> %a, %b
   %w = xor <8 x i16> %v, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>
   ret <8 x i16> %w
 
 ; CHECK-REG-LABEL: @test15
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test15
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlnor 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test15
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test16(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = or <16 x i8> %a, %b
   %w = xor <16 x i8> %v, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>
   ret <16 x i8> %w
 
 ; CHECK-REG-LABEL: @test16
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test16
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlnor 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test16
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test17(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %w = xor <4 x i32> %b, <i32 -1, i32 -1, i32 -1, i32 -1>
   %v = and <4 x i32> %a, %w
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test17
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test17
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: xxlnor 36, 36, 36
-; CHECK-FISL: xxland 36, 37, 36
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlnor 35, 35, 35
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test17
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test18(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %w = xor <8 x i16> %b, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>
   %v = and <8 x i16> %a, %w
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test18
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test18
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlandc 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlnor 0, 35, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlandc 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test18
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test19(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %w = xor <16 x i8> %b, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>
   %v = and <16 x i8> %a, %w
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test19
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test19
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlandc 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlnor 0, 35, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlandc 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test19
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test20(<4 x i32> %a, <4 x i32> %b, <4 x i32> %c, <4 x i32> %d) {
 entry:
   %m = icmp eq <4 x i32> %c, %d
   %v = select <4 x i1> %m, <4 x i32> %a, <4 x i32> %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test20
 ; CHECK-REG: vcmpequw {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
-; FIXME: The fast-isel code is pretty miserable for this one.
-
 ; CHECK-FISL-LABEL: @test20
-; CHECK-FISL: vor 0, 5, 5
-; CHECK-FISL: vor 1, 4, 4
-; CHECK-FISL: vor 6, 3, 3
-; CHECK-FISL: vor 7, 2, 2
-; CHECK-FISL: vor 2, 1, 1
-; CHECK-FISL: vor 3, 0, 0
-; CHECK-FISL: vcmpequw 2, 2, 3
-; CHECK-FISL: vor 0, 2, 2
-; CHECK-FISL: xxsel 32, 38, 39, 32
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: vcmpequw {{[0-9]+}}, 4, 5
+; CHECK-FISL: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test20
 ; CHECK-LE: vcmpequw {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 
 define <4 x float> @test21(<4 x float> %a, <4 x float> %b, <4 x float> %c, <4 x float> %d) {
 entry:
   %m = fcmp oeq <4 x float> %c, %d
   %v = select <4 x i1> %m, <4 x float> %a, <4 x float> %b
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test21
 ; CHECK-REG: xvcmpeqsp [[V1:[0-9]+]], 36, 37
 ; CHECK-REG: xxsel 34, 35, 34, [[V1]]
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test21
-; CHECK-FISL: vor 0, 5, 5
-; CHECK-FISL: vor 1, 4, 4
-; CHECK-FISL: vor 6, 3, 3
-; CHECK-FISL: vor 7, 2, 2
-; CHECK-FISL: xvcmpeqsp 32, 33, 32
-; CHECK-FISL: xxsel 32, 38, 39, 32
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: xvcmpeqsp [[V1:[0-9]+]], 36, 37
+; CHECK-FISL: xxsel 34, 35, 34, [[V1]]
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test21
 ; CHECK-LE: xvcmpeqsp [[V1:[0-9]+]], 36, 37
 ; CHECK-LE: xxsel 34, 35, 34, [[V1]]
 ; CHECK-LE: blr
 }
 

 ; CHECK-REG-DAG: xxlnor
 ; CHECK-REG-DAG: xxlnor
 ; CHECK-REG-DAG: xxlor
 ; CHECK-REG-DAG: xxlor
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test22
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 33, 32
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 32, 32
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 33, 33
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 37, 37
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 36, 36
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 36, 37
 ; CHECK-FISL-DAG: xxlnor
 ; CHECK-FISL-DAG: xxlnor
 ; CHECK-FISL-DAG: xxlor
 ; CHECK-FISL-DAG: xxlor
-; CHECK-FISL: xxsel 0, 38, 39, {{[0-9]+}}
+; CHECK-FISL: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test22
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 37, 37
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 36, 36
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 36, 37
 ; CHECK-LE-DAG: xxlnor
 ; CHECK-LE-DAG: xxlnor

 
 ; CHECK-REG-LABEL: @test23
 ; CHECK-REG: vcmpequh {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test23
 ; CHECK-FISL: vcmpequh 4, 4, 5
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: vor 1, 2, 2
-; CHECK-FISL: vor 6, 4, 4
-; CHECK-FISL: xxsel 32, 32, 33, 38
-; CHECK-FISL: vor 2, 0, 
+; CHECK-FISL: xxsel 34, 35, 34, 36
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test23
 ; CHECK-LE: vcmpequh {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test24(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c, <16 x i8> %d) {
 entry:
   %m = icmp eq <16 x i8> %c, %d
   %v = select <16 x i1> %m, <16 x i8> %a, <16 x i8> %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test24
 ; CHECK-REG: vcmpequb {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test24
 ; CHECK-FISL: vcmpequb 4, 4, 5
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: vor 1, 2, 2
-; CHECK-FISL: vor 6, 4, 4
-; CHECK-FISL: xxsel 32, 32, 33, 38
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: xxsel 34, 35, 34, 36
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test24
 ; CHECK-LE: vcmpequb {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 

 
 ; CHECK-REG-LABEL: @test30
 ; CHECK-REG: lxvd2x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test30
 ; CHECK-FISL: lxvd2x 0, 0, 3
 ; CHECK-FISL: xxlor 34, 0, 0
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: vor 2, 3, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test30
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
define <4 x float> @test32(<4 x float>* %a) {
   %v = load <4 x float>, <4 x float>* %a, align 16
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test32
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test32
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test32
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
 define void @test33(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 16
   ret void
 
 ; CHECK-REG-LABEL: @test33
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test33
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
define void @test33u(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 8
   ret void
 
 ; CHECK-REG-LABEL: @test33u
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33u
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test33u
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test34(<4 x i32>* %a) {
   %v = load <4 x i32>, <4 x i32>* %a, align 16
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test34
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test34
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test34
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
 define void @test35(<4 x i32>* %a, <4 x i32> %b) {
   store <4 x i32> %b, <4 x i32>* %a, align 16
   ret void
 
 ; CHECK-REG-LABEL: @test35
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test35
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test35
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
define <2 x i1> @test65(<2 x i64> %a, <2 x i64> %b) {
   %w = icmp eq <2 x i64> %a, %b
   ret <2 x i1> %w
 
 ; CHECK-REG-LABEL: @test65
 ; CHECK-REG: vcmpequw 2, 2, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test65
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: vcmpequw 4, 5, 4
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: vcmpequw 2, 2, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test65
 ; CHECK-LE: vcmpequd 2, 2, 3
 ; CHECK-LE: blr
 }
 
 define <2 x i1> @test66(<2 x i64> %a, <2 x i64> %b) {
   %w = icmp ne <2 x i64> %a, %b
   ret <2 x i1> %w
 
 ; CHECK-REG-LABEL: @test66
 ; CHECK-REG: vcmpequw {{[0-9]+}}, 2, 3
 ; CHECK-REG: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test66
-; CHECK-FISL: vcmpequw {{[0-9]+}}, 5, 4
-; CHECK-FISL: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
+; CHECK-FISL: vcmpequw 2, 2, 3
+; CHECK-FISL: xxlnor 34, 34, 34
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test66
 ; CHECK-LE: vcmpequd {{[0-9]+}}, 2, 3
 ; CHECK-LE: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-LE: blr
 }