[PPC] Implement the heuristic to choose between a X-Form VSX ld/st vs a X-Form FP ld/st.

[jtony]

Summary

The VSX versions have the advantage of a full 64-register target whereas the FP ones have the advantage of lower latency and higher throughput. So what we’re after is using the faster instructions in low register pressure situations and using the larger register file in high register pressure situations.

The heuristic chooses between the following 7 pairs of instructions.

PPC::LXSSPX vs PPC::LFSX
PPC::LXSDX vs PPC::LFDX
PPC::STXSSPX vs PPC::STFSX
PPC::STXSDX vs PPC::STFDX
PPC::LXSIWAX vs PPC::LFIWAX
PPC::LXSIWZX vs PPC::LFIWZX
PPC::STXSIWX vs PPC::STFIWX

[jtony]

lib/Target/PowerPC/PPCInstrVSX.td

3089 ↗

(On Diff #117459)

Do we really need the NoP9Vector here ? If not, I can remove it.

[hfinkel]

lib/Target/PowerPC/PPCInstrVSX.td

3089 ↗

(On Diff #117459)

If we don't have it, won't this pattern compete with the DblToIntLoadP9 pattern on the P9?Maybe the complexity metric will make the latter win, but it probably makes sense to leave the NoP9Vector here so that it's explicit.

Otherwise, this LGTM.

[nemanjai]

test/CodeGen/PowerPC/build-vector-tests.ll

3511 ↗

(On Diff #117459)

Is this just matching because we're emitting lfsx? I really don't see anything in this patch that can convert an X-Form load to a D-Form load. If it is indeed a D-Form we're emitting here, please respond to this comment as to how this is happening.
Furthermore, if we're converting this to a D-Form here, explain why we're not doing the same with the lxsdx -> lfdx change above.

[nemanjai]

lib/Target/PowerPC/PPCInstrInfo.cpp

2080 ↗

(On Diff #117459)

Minor nit: everything you've added below here is just a duplication of how we handle the D-Forms above. Why not just unify the two - after all, the approach is identical:

• Identify the upper and lower opcodes
• Change the opcode of the instruction accordingly

lib/Target/PowerPC/PPCInstrVSX.td

3089 ↗

(On Diff #117459)

I agree with Hal here. Please leave this in, the fact that there is no observable change in behaviour may be just luck for some of the patterns (i.e. if Complexity and CodeSize are equal).
Plus this makes it very nice and clear when looking at the code that we mean to use these on P7 and P8 targets.

[jtony]

Address comments from Nemanja.
(1) Abstract the common part code for handling D-Form and X-Form LD/ST heuristic into a function.
(2) Fix missing predicate guards (HasP8Vector, AddedComplexity=400) problem.

[jtony]

test/CodeGen/PowerPC/build-vector-tests.ll

3511 ↗

(On Diff #117459)

This has been fixed by the new patch. It is caused by not guarding the new Pseudo instructions with proper predicates: HasP8Vector and AddedComplexity=400.

[nemanjai]

Please add test cases for the following:

• An actual use of the VSX instructions (i.e. where the register pressure is high enough that an upper VSX register will be allocated - you should be able to accomplish this with inline asm clobbers lists in a small test case)
• A Power9 test case where we will use an X-Form version (i.e. the offset is large enough not to fit in a displacement field)

lib/Target/PowerPC/PPCInstrInfo.cpp

1992 ↗

(On Diff #118834)

Redundant line.

1993 ↗

(On Diff #118834)

The name is a bit too vague. Perhaps something like expandVSXMemInstr(), to indicate that it does the actual expansion (i.e. changes the opcode of the MI) as well as that it handles VSX memory operation instructions.

lib/Target/PowerPC/PPCInstrVSX.td

2864 ↗

(On Diff #118834)

I really don't think it is a particularly good idea to separate this out here. I'd keep it next to where the definitions of the actual instructions are. This will ensure that you define these in the right scope with all the properties set and will avoid having to figure out what the correct set of let expressions is. For example, as you've defined them here, the mayLoad/mayStore flags won't be set correctly here. These flags are used in various post-ISEL passes to check for side-effects, so it is important to set them correctly.

[jtony]

Address the comments from Nemanja.

[nemanjai]

The remaining comments I have are minor nits. Please feel free to fix them on the commit. Otherwise, LGTM.

lib/Target/PowerPC/PPCInstrVSX.td

134 ↗

(On Diff #120443)

Style nits (of course, this applies not only here but everywhere below as well):

1. No braces when the let expression applies to a single def
2. Just enclose the new def into a block if the same let expression applies to both (i.e. let CodeSize = 3 applies to both LXSDX and XFLOADf64 here. Just enclose the two into a block like you enclosed them into the mayLoad/mayStore block.
3. I find it surprising that we need let isPseudo = 1 on definitions that use Pseudo<>. Are you sure that's needed? Isn't it part of the class definition for Pseudo<>?

test/CodeGen/PowerPC/VSX-XForm-Scalars.ll

12 ↗

(On Diff #120443)

So we don't get this on Power9?

[nemanjai]

test/CodeGen/PowerPC/VSX-XForm-Scalars.ll

12 ↗

(On Diff #120443)

Looks like part of my comment disappeared. What I meant to add here is to ask you to please add checks for the code before and after the inline asm for both Power8 and Power9.

llvm/trunk/lib/Target/PowerPC/P9InstrResources.td

 def : InstRW<[P9_LS_5C, IP_AGEN_1C, DISP_1C, DISP_1C],
       (instrs
     LXSDX,
     LXVD2X,
     LXSIWZX,
     LXV,
     LXVX,
     LXSD,
-    DFLOADf64
+    DFLOADf64,
+    XFLOADf64
 )>;
 
 // 4 Cycle load uses a single slice.
 def : InstRW<[P9_LS_4C, IP_AGEN_1C, DISP_1C, DISP_1C],
       (instrs
     COPY
 )>;
 

 //  operations cannot be done at the same time and so their latencies are added.
 // Full 4 dispatches are required as this is a cracked instruction.
 def : InstRW<[P9_LoadAndALUOp_7C, IP_AGEN_1C, IP_EXEC_1C,
               DISP_1C, DISP_1C, DISP_1C, DISP_1C],
       (instrs
     LXSSPX,
     LXSIWAX,
     LXSSP,
-    DFLOADf32
+    DFLOADf32,
+    XFLOADf32,
+    LIWAX,
+    LIWZX
 )>;
 
 // Cracked Load that requires the PM resource.
 // Since the Load and the PM cannot be done at the same time the latencies are
 //  added. Requires 8 cycles.
 // Since the PM requires the full superslice we need both EXECE, EXECO pipelines
 //  as well as 3 dispatches for the PM. The Load requires the remaining 2
 //  dispatches.
def : InstRW<[P9_LS_1C, IP_EXEC_1C, IP_AGEN_1C, DISP_1C, DISP_1C, DISP_1C],
     STFD,
     STFIWX,
     STFSX,
     STFDX,
     STXSDX,
     STXSSPX,
     STXSIWX,
     DFSTOREf32,
-    DFSTOREf64
+    DFSTOREf64,
+    XFSTOREf32,
+    XFSTOREf64,
+    STIWX
 )>;
 
 // Store operation that requires the whole superslice.
 def : InstRW<[P9_LS_1C, IP_EXECE_1C, IP_EXECO_1C, IP_AGEN_1C,
               DISP_1C, DISP_1C, DISP_1C],
       (instrs
     STXVD2X,
     STXVW4X

llvm/trunk/lib/Target/PowerPC/PPCInstrInfo.h

public:
   decomposeMachineOperandsTargetFlags(unsigned TF) const override;
 
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableDirectMachineOperandTargetFlags() const override;
 
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableBitmaskMachineOperandTargetFlags() const override;
 
+  // Expand VSX Memory Pseudo instruction to either a VSX or a FP instruction.
+  bool expandVSXMemPseudo(MachineInstr &MI) const;
+
   // Lower pseudo instructions after register allocation.
   bool expandPostRAPseudo(MachineInstr &MI) const override;
 
   static bool isVFRegister(unsigned Reg) {
     return Reg >= PPC::VF0 && Reg <= PPC::VF31;
   }
   static bool isVRRegister(unsigned Reg) {
     return Reg >= PPC::V0 && Reg <= PPC::V31;

llvm/trunk/lib/Target/PowerPC/PPCInstrInfo.cpp

PPCInstrInfo::getSerializableBitmaskMachineOperandTargetFlags() const {
   static const std::pair<unsigned, const char *> TargetFlags[] = {
       {MO_PLT, "ppc-plt"},
       {MO_PIC_FLAG, "ppc-pic"},
       {MO_NLP_FLAG, "ppc-nlp"},
       {MO_NLP_HIDDEN_FLAG, "ppc-nlp-hidden"}};
   return makeArrayRef(TargetFlags);
 }
 
-bool PPCInstrInfo::expandPostRAPseudo(MachineInstr &MI) const {
-  auto &MBB = *MI.getParent();
-  auto DL = MI.getDebugLoc();
-  switch (MI.getOpcode()) {
-  case TargetOpcode::LOAD_STACK_GUARD: {
-    assert(Subtarget.isTargetLinux() &&
-           "Only Linux target is expected to contain LOAD_STACK_GUARD");
-    const int64_t Offset = Subtarget.isPPC64() ? -0x7010 : -0x7008;
-    const unsigned Reg = Subtarget.isPPC64() ? PPC::X13 : PPC::R2;
-    MI.setDesc(get(Subtarget.isPPC64() ? PPC::LD : PPC::LWZ));
-    MachineInstrBuilder(*MI.getParent()->getParent(), MI)
-        .addImm(Offset)
-        .addReg(Reg);
-    return true;
-  }
-  case PPC::DFLOADf32:
-  case PPC::DFLOADf64:
-  case PPC::DFSTOREf32:
-  case PPC::DFSTOREf64: {
-    assert(Subtarget.hasP9Vector() &&
-           "Invalid D-Form Pseudo-ops on non-P9 target.");
-    assert(MI.getOperand(2).isReg() && MI.getOperand(1).isImm() &&
-           "D-form op must have register and immediate operands");
+// Expand VSX Memory Pseudo instruction to either a VSX or a FP instruction.
+// The VSX versions have the advantage of a full 64-register target whereas
+// the FP ones have the advantage of lower latency and higher throughput. So
+// what we are after is using the faster instructions in low register pressure
+// situations and using the larger register file in high register pressure
+// situations.
+bool PPCInstrInfo::expandVSXMemPseudo(MachineInstr &MI) const {
     unsigned UpperOpcode, LowerOpcode;
     switch (MI.getOpcode()) {
     case PPC::DFLOADf32:
       UpperOpcode = PPC::LXSSP;
       LowerOpcode = PPC::LFS;
       break;
     case PPC::DFLOADf64:
       UpperOpcode = PPC::LXSD;
       LowerOpcode = PPC::LFD;
       break;
     case PPC::DFSTOREf32:
       UpperOpcode = PPC::STXSSP;
       LowerOpcode = PPC::STFS;
       break;
     case PPC::DFSTOREf64:
       UpperOpcode = PPC::STXSD;
       LowerOpcode = PPC::STFD;
       break;
+    case PPC::XFLOADf32:
+      UpperOpcode = PPC::LXSSPX;
+      LowerOpcode = PPC::LFSX;
+      break;
+    case PPC::XFLOADf64:
+      UpperOpcode = PPC::LXSDX;
+      LowerOpcode = PPC::LFDX;
+      break;
+    case PPC::XFSTOREf32:
+      UpperOpcode = PPC::STXSSPX;
+      LowerOpcode = PPC::STFSX;
+      break;
+    case PPC::XFSTOREf64:
+      UpperOpcode = PPC::STXSDX;
+      LowerOpcode = PPC::STFDX;
+      break;
+    case PPC::LIWAX:
+      UpperOpcode = PPC::LXSIWAX;
+      LowerOpcode = PPC::LFIWAX;
+      break;
+    case PPC::LIWZX:
+      UpperOpcode = PPC::LXSIWZX;
+      LowerOpcode = PPC::LFIWZX;
+      break;
+    case PPC::STIWX:
+      UpperOpcode = PPC::STXSIWX;
+      LowerOpcode = PPC::STFIWX;
+      break;
+    default:
+      llvm_unreachable("Unknown Operation!");
     }
+
     unsigned TargetReg = MI.getOperand(0).getReg();
     unsigned Opcode;
     if ((TargetReg >= PPC::F0 && TargetReg <= PPC::F31) ||
         (TargetReg >= PPC::VSL0 && TargetReg <= PPC::VSL31))
       Opcode = LowerOpcode;
     else
       Opcode = UpperOpcode;
     MI.setDesc(get(Opcode));
     return true;
-  }
+
+bool PPCInstrInfo::expandPostRAPseudo(MachineInstr &MI) const {
+  auto &MBB = *MI.getParent();
+  auto DL = MI.getDebugLoc();
+
+  switch (MI.getOpcode()) {
+  case TargetOpcode::LOAD_STACK_GUARD: {
+    assert(Subtarget.isTargetLinux() &&
+           "Only Linux target is expected to contain LOAD_STACK_GUARD");
+    const int64_t Offset = Subtarget.isPPC64() ? -0x7010 : -0x7008;
+    const unsigned Reg = Subtarget.isPPC64() ? PPC::X13 : PPC::R2;
+    MI.setDesc(get(Subtarget.isPPC64() ? PPC::LD : PPC::LWZ));
+    MachineInstrBuilder(*MI.getParent()->getParent(), MI)
+        .addImm(Offset)
+        .addReg(Reg);
+    return true;
+  }
+  case PPC::DFLOADf32:
+  case PPC::DFLOADf64:
+  case PPC::DFSTOREf32:
+  case PPC::DFSTOREf64: {
+    assert(Subtarget.hasP9Vector() &&
+           "Invalid D-Form Pseudo-ops on Pre-P9 target.");
+    assert(MI.getOperand(2).isReg() && MI.getOperand(1).isImm() &&
+           "D-form op must have register and immediate operands");
+    return expandVSXMemPseudo(MI);
+  }
+  case PPC::XFLOADf32:
+  case PPC::XFSTOREf32:
+  case PPC::LIWAX:
+  case PPC::LIWZX:
+  case PPC::STIWX: {
+    assert(Subtarget.hasP8Vector() &&
+           "Invalid X-Form Pseudo-ops on Pre-P8 target.");
+    assert(MI.getOperand(2).isReg() && MI.getOperand(1).isReg() &&
+           "X-form op must have register and register operands");
+    return expandVSXMemPseudo(MI);
+  }
+  case PPC::XFLOADf64:
+  case PPC::XFSTOREf64: {
+    assert(Subtarget.hasVSX() &&
+           "Invalid X-Form Pseudo-ops on target that has no VSX.");
+    assert(MI.getOperand(2).isReg() && MI.getOperand(1).isReg() &&
+           "X-form op must have register and register operands");
+    return expandVSXMemPseudo(MI);
+  }
   case PPC::SPILLTOVSR_LD: {
     unsigned TargetReg = MI.getOperand(0).getReg();
     if (PPC::VSFRCRegClass.contains(TargetReg)) {
       MI.setDesc(get(PPC::DFLOADf64));
       return expandPostRAPseudo(MI);
     }
     else
       MI.setDesc(get(PPC::LD));

llvm/trunk/lib/Target/PowerPC/PPCInstrVSX.td

let Uses = [RM] in {
   // Load indexed instructions
   let mayLoad = 1, mayStore = 0 in {
     let CodeSize = 3 in
     def LXSDX : XX1Form<31, 588,
                         (outs vsfrc:$XT), (ins memrr:$src),
                         "lxsdx $XT, $src", IIC_LdStLFD,
                         [(set f64:$XT, (load xoaddr:$src))]>;
 
+    // Pseudo instruction XFLOADf64 will be expanded to LXSDX or LFDX later
+    let isPseudo = 1, CodeSize = 3 in
+      def XFLOADf64  : Pseudo<(outs vsfrc:$XT), (ins memrr:$src),
+                              "#XFLOADf64",
+                              [(set f64:$XT, (load xoaddr:$src))]>;
+
     let Predicates = [HasVSX, HasOnlySwappingMemOps] in
     def LXVD2X : XX1Form<31, 844,
                          (outs vsrc:$XT), (ins memrr:$src),
                          "lxvd2x $XT, $src", IIC_LdStLFD,
                          [(set v2f64:$XT, (int_ppc_vsx_lxvd2x xoaddr:$src))]>;
 
     def LXVDSX : XX1Form<31, 332,
                          (outs vsrc:$XT), (ins memrr:$src),
let Uses = [RM] in {
   // Store indexed instructions
   let mayStore = 1, mayLoad = 0 in {
     let CodeSize = 3 in
     def STXSDX : XX1Form<31, 716,
                         (outs), (ins vsfrc:$XT, memrr:$dst),
                         "stxsdx $XT, $dst", IIC_LdStSTFD,
                         [(store f64:$XT, xoaddr:$dst)]>;
 
+    // Pseudo instruction XFSTOREf64  will be expanded to STXSDX or STFDX later
+    let isPseudo = 1, CodeSize = 3 in
+      def XFSTOREf64 : Pseudo<(outs), (ins vsfrc:$XT, memrr:$dst),
+                              "#XFSTOREf64",
+                              [(store f64:$XT, xoaddr:$dst)]>;
+
     let Predicates = [HasVSX, HasOnlySwappingMemOps] in {
     // The behaviour of this instruction is endianness-specific so we provide no
     // pattern to match it without considering endianness.
     def STXVD2X : XX1Form<31, 972,
                          (outs), (ins vsrc:$XT, memrr:$dst),
                          "stxvd2x $XT, $dst", IIC_LdStSTFD,
                          []>;
 
def XXLORC : XX3Form<60, 170,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                        "xxlorc $XT, $XA, $XB", IIC_VecGeneral,
                        [(set v4i32:$XT, (or v4i32:$XA, (vnot_ppc v4i32:$XB)))]>;
 
   // VSX scalar loads introduced in ISA 2.07
   let mayLoad = 1, mayStore = 0 in {
     let CodeSize = 3 in
     def LXSSPX : XX1Form<31, 524, (outs vssrc:$XT), (ins memrr:$src),
-                         "lxsspx $XT, $src", IIC_LdStLFD,
-                         [(set f32:$XT, (load xoaddr:$src))]>;
+                         "lxsspx $XT, $src", IIC_LdStLFD, []>;
     def LXSIWAX : XX1Form<31, 76, (outs vsfrc:$XT), (ins memrr:$src),
-                          "lxsiwax $XT, $src", IIC_LdStLFD,
-                          [(set f64:$XT, (PPClfiwax xoaddr:$src))]>;
+                          "lxsiwax $XT, $src", IIC_LdStLFD, []>;
     def LXSIWZX : XX1Form<31, 12, (outs vsfrc:$XT), (ins memrr:$src),
-                          "lxsiwzx $XT, $src", IIC_LdStLFD,
-                          [(set f64:$XT, (PPClfiwzx xoaddr:$src))]>;
+                          "lxsiwzx $XT, $src", IIC_LdStLFD, []>;
+
+    // Please note let isPseudo = 1 is not part of class Pseudo<>. Missing it
+    // would cause these Pseudos are not expanded in expandPostRAPseudos()
+    let isPseudo = 1 in {
+      // Pseudo instruction XFLOADf32 will be expanded to LXSSPX or LFSX later
+      let CodeSize = 3 in
+      def XFLOADf32  : Pseudo<(outs vssrc:$XT), (ins memrr:$src),
+                              "#XFLOADf32",
+                              [(set f32:$XT, (load xoaddr:$src))]>;
+      // Pseudo instruction LIWAX will be expanded to LXSIWAX or LFIWAX later
+      def LIWAX : Pseudo<(outs vsfrc:$XT), (ins memrr:$src),
+                         "#LIWAX",
+                         [(set f64:$XT, (PPClfiwax xoaddr:$src))]>;
+      // Pseudo instruction LIWZX will be expanded to LXSIWZX or LFIWZX later
+      def LIWZX : Pseudo<(outs vsfrc:$XT), (ins memrr:$src),
+                         "#LIWZX",
+    }
   } // mayLoad
 
   // VSX scalar stores introduced in ISA 2.07
   let mayStore = 1, mayLoad = 0 in {
     let CodeSize = 3 in
     def STXSSPX : XX1Form<31, 652, (outs), (ins vssrc:$XT, memrr:$dst),
-                          "stxsspx $XT, $dst", IIC_LdStSTFD,
-                          [(store f32:$XT, xoaddr:$dst)]>;
+                          "stxsspx $XT, $dst", IIC_LdStSTFD, []>;
     def STXSIWX : XX1Form<31, 140, (outs), (ins vsfrc:$XT, memrr:$dst),
-                          "stxsiwx $XT, $dst", IIC_LdStSTFD,
-                          [(PPCstfiwx f64:$XT, xoaddr:$dst)]>;
+                          "stxsiwx $XT, $dst", IIC_LdStSTFD, []>;
+
+    // Please note let isPseudo = 1 is not part of class Pseudo<>. Missing it
+    // would cause these Pseudos are not expanded in expandPostRAPseudos()
+    let isPseudo = 1 in {
+      // Pseudo instruction XFSTOREf32 will be expanded to STXSSPX or STFSX later
+      let CodeSize = 3 in
+      def XFSTOREf32 : Pseudo<(outs), (ins vssrc:$XT, memrr:$dst),
+                              "#XFSTOREf32",
+                              [(store f32:$XT, xoaddr:$dst)]>;
+      // Pseudo instruction STIWX will be expanded to STXSIWX or STFIWX later
+      def STIWX : Pseudo<(outs), (ins vsfrc:$XT, memrr:$dst),
+                         "#STIWX",
+    }
   } // mayStore
   } // UseVSXReg = 1
 
   def : Pat<(f64 (extloadf32 xoaddr:$src)),
-            (COPY_TO_REGCLASS (LXSSPX xoaddr:$src), VSFRC)>;
+            (COPY_TO_REGCLASS (XFLOADf32 xoaddr:$src), VSFRC)>;
   def : Pat<(f32 (fpround (extloadf32 xoaddr:$src))),
-            (f32 (LXSSPX xoaddr:$src))>;
+            (f32 (XFLOADf32 xoaddr:$src))>;
   def : Pat<(f64 (fpextend f32:$src)),
             (COPY_TO_REGCLASS $src, VSFRC)>;
 
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLT)),
             (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETULT)),
             (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLE)),
let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
   def : Pat<(f32 (PPCfcfids (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 0))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS $S, VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus (PPCmtvsra (i64 (vector_extract v2i64:$S, 1))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   }
   def : Pat<(v4i32 (scalar_to_vector ScalarLoads.Li32)),
-            (v4i32 (XXSPLTWs (LXSIWAX xoaddr:$src), 1))>;
+            (v4i32 (XXSPLTWs (LIWAX xoaddr:$src), 1))>;
 } // AddedComplexity = 400
 } // HasP8Vector
 
 let UseVSXReg = 1, AddedComplexity = 400 in {
 let Predicates = [HasDirectMove] in {
   // VSX direct move instructions
   def MFVSRD : XX1_RS6_RD5_XO<31, 51, (outs g8rc:$rA), (ins vsfrc:$XT),
                               "mfvsrd $rA, $XT", IIC_VecGeneral,
let Predicates = [HasVSX] in {
     def : Pat<(v2i64 (build_vector DblToLong.A, DblToLong.A)),
               (v2i64 (XXPERMDI (COPY_TO_REGCLASS (XSCVDPSXDS $A), VSRC),
                                (COPY_TO_REGCLASS (XSCVDPSXDS $A), VSRC), 0))>;
     def : Pat<(v2i64 (build_vector DblToULong.A, DblToULong.A)),
               (v2i64 (XXPERMDI (COPY_TO_REGCLASS (XSCVDPUXDS $A), VSRC),
                                (COPY_TO_REGCLASS (XSCVDPUXDS $A), VSRC), 0))>;
     def : Pat<(v4i32 (scalar_to_vector FltToIntLoad.A)),
               (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                                (XSCVDPSXWSs (LXSSPX xoaddr:$A)), VSRC), 1))>;
+                                (XSCVDPSXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1))>;
     def : Pat<(v4i32 (scalar_to_vector FltToUIntLoad.A)),
               (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                                (XSCVDPUXWSs (LXSSPX xoaddr:$A)), VSRC), 1))>;
+                                (XSCVDPUXWSs (XFLOADf32 xoaddr:$A)), VSRC), 1))>;
     def : Pat<(v4f32 (build_vector f32:$A, f32:$A, f32:$A, f32:$A)),
               (v4f32 (XXSPLTW (v4f32 (XSCVDPSPN $A)), 0))>;
 
     // Build vectors of floating point converted to i64.
     def : Pat<(v2i64 (build_vector FltToLong.A, FltToLong.A)),
               (v2i64 (XXPERMDIs
                        (COPY_TO_REGCLASS (XSCVDPSXDSs $A), VSFRC), 0))>;
     def : Pat<(v2i64 (build_vector FltToULong.A, FltToULong.A)),
               (v2i64 (XXPERMDIs
                        (COPY_TO_REGCLASS (XSCVDPUXDSs $A), VSFRC), 0))>;
     def : Pat<(v2i64 (scalar_to_vector DblToLongLoad.A)),
               (v2i64 (XVCVDPSXDS (LXVDSX xoaddr:$A)))>;
     def : Pat<(v2i64 (scalar_to_vector DblToULongLoad.A)),
               (v2i64 (XVCVDPUXDS (LXVDSX xoaddr:$A)))>;
   }
 
   let Predicates = [HasVSX, NoP9Vector] in {
-    // Load-and-splat with fp-to-int conversion (using X-Form VSX loads).
+    // Load-and-splat with fp-to-int conversion (using X-Form VSX/FP loads).
     def : Pat<(v4i32 (scalar_to_vector DblToIntLoad.A)),
               (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                                (XSCVDPSXWS (LXSDX xoaddr:$A)), VSRC), 1))>;
+                                (XSCVDPSXWS (XFLOADf64 xoaddr:$A)), VSRC), 1))>;
     def : Pat<(v4i32 (scalar_to_vector DblToUIntLoad.A)),
               (v4i32 (XXSPLTW (COPY_TO_REGCLASS
-                                (XSCVDPUXWS (LXSDX xoaddr:$A)), VSRC), 1))>;
+                                (XSCVDPUXWS (XFLOADf64 xoaddr:$A)), VSRC), 1))>;
     def : Pat<(v2i64 (scalar_to_vector FltToLongLoad.A)),
               (v2i64 (XXPERMDIs (XSCVDPSXDS (COPY_TO_REGCLASS
-                                              (LXSSPX xoaddr:$A), VSFRC)), 0))>;
+                                              (XFLOADf32 xoaddr:$A), VSFRC)), 0))>;
     def : Pat<(v2i64 (scalar_to_vector FltToULongLoad.A)),
               (v2i64 (XXPERMDIs (XSCVDPUXDS (COPY_TO_REGCLASS
-                                              (LXSSPX xoaddr:$A), VSFRC)), 0))>;
+                                              (XFLOADf32 xoaddr:$A), VSFRC)), 0))>;
   }
 
   // Big endian, available on all targets with VSX
   let Predicates = [IsBigEndian, HasVSX] in {
     def : Pat<(v2f64 (build_vector f64:$A, f64:$B)),
               (v2f64 (XXPERMDI
                         (COPY_TO_REGCLASS $A, VSRC),
                         (COPY_TO_REGCLASS $B, VSRC), 0))>;

llvm/trunk/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp

for (MachineInstr &MI : MBB) {
       case PPC::LXVW4X:
         // Permuting loads are marked as both load and swap, and are
         // safe for optimization.
         SwapVector[VecIdx].IsLoad = 1;
         SwapVector[VecIdx].IsSwap = 1;
         break;
       case PPC::LXSDX:
       case PPC::LXSSPX:
+      case PPC::XFLOADf64:
+      case PPC::XFLOADf32:
         // A load of a floating-point value into the high-order half of
         // a vector register is safe, provided that we introduce a swap
         // following the load, which will be done by the SUBREG_TO_REG
         // support.  So just mark these as safe.
         SwapVector[VecIdx].IsLoad = 1;
         SwapVector[VecIdx].IsSwappable = 1;
         break;
       case PPC::STVX:

llvm/trunk/test/CodeGen/PowerPC/VSX-XForm-Scalars.ll

+; RUN: llc < %s -mcpu=pwr8 -mtriple=powerpc64le-unknown-unknown \
+; RUN:   -verify-machineinstrs | FileCheck %s --check-prefix=CHECK-P8
+; RUN: llc < %s -mcpu=pwr9 -mtriple=powerpc64le-unknown-unknown \
+; RUN:   -verify-machineinstrs | FileCheck %s --check-prefix=CHECK-P9
+
+@a = external local_unnamed_addr global <4 x i32>, align 16
+@pb = external local_unnamed_addr global float*, align 8
+
+define void @testExpandPostRAPseudo(i32* nocapture readonly %ptr) {
+; CHECK-P8-LABEL:     testExpandPostRAPseudo:
+; CHECK-P8:           lxsiwax 34, 0, 3
+; CHECK-P8-NEXT:      xxspltw 34, 34, 1
+; CHECK-P8-NEXT:      stvx 2, 0, 4
+; CHECK-P8:           #APP
+; CHECK-P8-NEXT:      #Clobber Rigisters
+; CHECK-P8-NEXT:      #NO_APP
+; CHECK-P8-NEXT:      lis 4, 1024
+; CHECK-P8-NEXT:      lfiwax 0, 0, 3
+; CHECK-P8:           stfsx 0, 3, 4
+; CHECK-P8-NEXT:      blr
+
+; CHECK-P9-LABEL:     testExpandPostRAPseudo:
+; CHECK-P9:           lxvwsx 0, 0, 3
+; CHECK-P9:           stxvx 0, 0, 4
+; CHECK-P9:           #APP
+; CHECK-P9-NEXT:      #Clobber Rigisters
+; CHECK-P9-NEXT:      #NO_APP
+; CHECK-P9-NEXT:      lis 4, 1024
+; CHECK-P9-NEXT:      lfiwax 0, 0, 3
+; CHECK-P9:           stfsx 0, 3, 4
+; CHECK-P9-NEXT:      blr
+
+entry:
+  %0 = load i32, i32* %ptr, align 4
+  %splat.splatinsert = insertelement <4 x i32> undef, i32 %0, i32 0
+  %splat.splat = shufflevector <4 x i32> %splat.splatinsert, <4 x i32> undef, <4 x i32> zeroinitializer
+  store <4 x i32> %splat.splat, <4 x i32>* @a, align 16
+  tail call void asm sideeffect "#Clobber Rigisters", "~{f0},~{f3},~{f4},~{f5},~{f6},~{f7},~{f8},~{f9},~{f10},~{f11},~{f12},~{f13},~{f14},~{f15},~{f16},~{f17},~{f18},~{f19},~{f20},~{f21},~{f22},~{f23},~{f24},~{f25},~{f26},~{f27},~{f28},~{f29},~{f30},~{f31}"()
+  %1 = load i32, i32* %ptr, align 4
+  %conv = sitofp i32 %1 to float
+  %2 = load float*, float** @pb, align 8
+  %add.ptr = getelementptr inbounds float, float* %2, i64 16777216
+  store float %conv, float* %add.ptr, align 4
+  ret void
+}

llvm/trunk/test/CodeGen/PowerPC/build-vector-tests.ll

 ; P9BE-LABEL: spltMemValConvftoi
 ; P9LE-LABEL: spltMemValConvftoi
 ; P8BE-LABEL: spltMemValConvftoi
 ; P8LE-LABEL: spltMemValConvftoi
 ; P9BE: lxvwsx [[REG1:[vs0-9]+]], 0, r3
 ; P9BE: xvcvspsxws v2, [[REG1]]
 ; P9LE: [[REG1:[vs0-9]+]], 0, r3
 ; P9LE: xvcvspsxws v2, [[REG1]]
-; P8BE: lxsspx [[REG1:f[0-9]+]], 0, r3
+; P8BE: lfsx [[REG1:f[0-9]+]], 0, r3
 ; P8BE: xscvdpsxws f[[REG2:[0-9]+]], [[REG1]]
 ; P8BE: xxspltw v2, vs[[REG2]], 1
-; P8LE: lxsspx [[REG1:f[0-9]+]], 0, r3
+; P8LE: lfsx [[REG1:f[0-9]+]], 0, r3
 ; P8LE: xscvdpsxws f[[REG2:[vs0-9]+]], [[REG1]]
 ; P8LE: xxspltw v2, vs[[REG2]], 1
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <4 x i32> @spltCnstConvdtoi() {
 entry:
   ret <4 x i32> <i32 4, i32 4, i32 4, i32 4>

 ; P9BE: lfd
 ; P9BE: xscvdpsxws
 ; P9BE: xxspltw
 ; P9BE: blr
 ; P9LE: lfd
 ; P9LE: xscvdpsxws
 ; P9LE: xxspltw
 ; P9LE: blr
-; P8BE: lxsdx
+; P8BE: lfdx
 ; P8BE: xscvdpsxws
 ; P8BE: xxspltw
 ; P8BE: blr
-; P8LE: lxsdx
+; P8LE: lfdx
 ; P8LE: xscvdpsxws
 ; P8LE: xxspltw
 ; P8LE: blr
 }
 ; Function Attrs: norecurse nounwind readnone
 define <4 x i32> @allZeroui() {
 entry:
   ret <4 x i32> zeroinitializer

 ; P9BE-LABEL: spltMemValConvftoui
 ; P9LE-LABEL: spltMemValConvftoui
 ; P8BE-LABEL: spltMemValConvftoui
 ; P8LE-LABEL: spltMemValConvftoui
 ; P9BE: lxvwsx [[REG1:[vs0-9]+]], 0, r3
 ; P9BE: xvcvspuxws v2, [[REG1]]
 ; P9LE: [[REG1:[vs0-9]+]], 0, r3
 ; P9LE: xvcvspuxws v2, [[REG1]]
-; P8BE: lxsspx [[REG1:f[0-9]+]], 0, r3
+; P8BE: lfsx [[REG1:f[0-9]+]], 0, r3
 ; P8BE: xscvdpuxws f[[REG2:[0-9]+]], [[REG1]]
 ; P8BE: xxspltw v2, vs[[REG2]], 1
-; P8LE: lxsspx [[REG1:f[0-9]+]], 0, r3
+; P8LE: lfsx [[REG1:f[0-9]+]], 0, r3
 ; P8LE: xscvdpuxws f[[REG2:[vs0-9]+]], [[REG1]]
 ; P8LE: xxspltw v2, vs[[REG2]], 1
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <4 x i32> @spltCnstConvdtoui() {
 entry:
   ret <4 x i32> <i32 4, i32 4, i32 4, i32 4>

 ; P9BE: lfd
 ; P9BE: xscvdpuxws
 ; P9BE: xxspltw
 ; P9BE: blr
 ; P9LE: lfd
 ; P9LE: xscvdpuxws
 ; P9LE: xxspltw
 ; P9LE: blr
-; P8BE: lxsdx
+; P8BE: lfdx
 ; P8BE: xscvdpuxws
 ; P8BE: xxspltw
 ; P8BE: blr
-; P8LE: lxsdx
+; P8LE: lfdx
 ; P8LE: xscvdpuxws
 ; P8LE: xxspltw
 ; P8LE: blr
 }
 ; Function Attrs: norecurse nounwind readnone
 define <2 x i64> @allZeroll() {
 entry:
   ret <2 x i64> zeroinitializer

 ; P9BE: xxmrghd
 ; P9BE-NEXT: xvcvdpsxds v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpsxds v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
-; P8BE: lxsspx
+; P8BE: lfsx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpsxds v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
-; P8LE: lxsspx
+; P8LE: lfsx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpsxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemConsDConvftoll(float* nocapture readonly %ptr) {
 entry:

 ; P9BE: xxmrghd
 ; P9BE-NEXT: xvcvdpsxds v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpsxds v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
-; P8BE: lxsspx
+; P8BE: lfsx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpsxds v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
-; P8LE: lxsspx
+; P8LE: lfsx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpsxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemVarAConvftoll(float* nocapture readonly %arr, i32 signext %elem) {
 entry:

 ; P9LE: sldi
 ; P9LE: lfsux
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpsxds v2
 ; P9LE-NEXT: blr
 ; P8BE: sldi
 ; P8BE: lfsux
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpsxds v2
 ; P8BE-NEXT: blr
 ; P8LE: sldi
 ; P8LE: lfsux
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpsxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemVarDConvftoll(float* nocapture readonly %arr, i32 signext %elem) {
 entry:

 ; P9LE: sldi
 ; P9LE: lfsux
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpsxds v2
 ; P9LE-NEXT: blr
 ; P8BE: sldi
 ; P8BE: lfsux
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpsxds v2
 ; P8BE-NEXT: blr
 ; P8LE: sldi
 ; P8LE: lfsux
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpsxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <2 x i64> @spltRegValConvftoll(float %val) {
 entry:

 ; P9BE: lfs
 ; P9BE-NEXT: xscvdpsxds
 ; P9BE-NEXT: xxspltd v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE-NEXT: xscvdpsxds
 ; P9LE-NEXT: xxspltd v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE-NEXT: xscvdpsxds
 ; P8BE-NEXT: xxspltd v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE-NEXT: xscvdpsxds
 ; P8LE-NEXT: xxspltd v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <2 x i64> @spltCnstConvdtoll() {
 entry:

 ; P9BE: xxmrghd
 ; P9BE-NEXT: xvcvdpuxds v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpuxds v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
-; P8BE: lxsspx
+; P8BE: lfsx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpuxds v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
-; P8LE: lxsspx
+; P8LE: lfsx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpuxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemConsDConvftoull(float* nocapture readonly %ptr) {
 entry:

 ; P9BE: xxmrghd
 ; P9BE-NEXT: xvcvdpuxds v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpuxds v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
-; P8BE: lxsspx
+; P8BE: lfsx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpuxds v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
-; P8LE: lxsspx
+; P8LE: lfsx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpuxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemVarAConvftoull(float* nocapture readonly %arr, i32 signext %elem) {
 entry:

 ; P9LE: sldi
 ; P9LE: lfsux
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpuxds v2
 ; P9LE-NEXT: blr
 ; P8BE: sldi
 ; P8BE: lfsux
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpuxds v2
 ; P8BE-NEXT: blr
 ; P8LE: sldi
 ; P8LE: lfsux
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpuxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define <2 x i64> @fromDiffMemVarDConvftoull(float* nocapture readonly %arr, i32 signext %elem) {
 entry:

 ; P9LE: sldi
 ; P9LE: lfsux
 ; P9LE: lfs
 ; P9LE: xxmrghd
 ; P9LE-NEXT: xvcvdpuxds v2
 ; P9LE-NEXT: blr
 ; P8BE: sldi
 ; P8BE: lfsux
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE: xxmrghd
 ; P8BE-NEXT: xvcvdpuxds v2
 ; P8BE-NEXT: blr
 ; P8LE: sldi
 ; P8LE: lfsux
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE: xxmrghd
 ; P8LE-NEXT: xvcvdpuxds v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <2 x i64> @spltRegValConvftoull(float %val) {
 entry:

 ; P9BE: lfs
 ; P9BE-NEXT: xscvdpuxds
 ; P9BE-NEXT: xxspltd v2
 ; P9BE-NEXT: blr
 ; P9LE: lfs
 ; P9LE-NEXT: xscvdpuxds
 ; P9LE-NEXT: xxspltd v2
 ; P9LE-NEXT: blr
-; P8BE: lxsspx
+; P8BE: lfsx
 ; P8BE-NEXT: xscvdpuxds
 ; P8BE-NEXT: xxspltd v2
 ; P8BE-NEXT: blr
-; P8LE: lxsspx
+; P8LE: lfsx
 ; P8LE-NEXT: xscvdpuxds
 ; P8LE-NEXT: xxspltd v2
 ; P8LE-NEXT: blr
 }
 
 ; Function Attrs: norecurse nounwind readnone
 define <2 x i64> @spltCnstConvdtoull() {
 entry:

llvm/trunk/test/CodeGen/PowerPC/direct-move-profit.ll

entry:
   %conv = sitofp i32 %0 to float
   %mul = fmul float %conv, 0x4002916880000000
   %arrayidx2 = getelementptr inbounds float, float* %a, i64 %idxprom
   store float %mul, float* %arrayidx2, align 4, !tbaa !5
   ret void
 
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
-; CHECK: lxsiwax [[REG:[0-9]+]], {{.*}}
+; CHECK: lfiwax [[REG:[0-9]+]], {{.*}}
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
 ; CHECK: xscvsxdsp {{.*}}, [[REG]]
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
 ; CHECK: blr
 
 }
 
 ; Function Attrs: norecurse nounwind readonly
 define float @test2(i32* nocapture readonly %b) #0 {
 
 ; CHECK-LABEL: test2
 
 entry:
   %0 = load i32, i32* %b, align 4, !tbaa !1
   %conv = sitofp i32 %0 to float
   %mul = fmul float %conv, 0x40030A3D80000000
   ret float %mul
 
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
-; CHECK: lxsiwax [[REG:[0-9]+]], {{.*}}
+; CHECK: lfiwax [[REG:[0-9]+]], {{.*}}
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
 ; CHECK: xscvsxdsp {{.*}}, [[REG]]
 ; CHECK-NOT: mtvsrwa
 ; CHECK-NOT: mtfprwa
 ; CHECK: blr
 
 }

llvm/trunk/test/CodeGen/PowerPC/p9-xxinsertw-xxextractuw.ll

 ; CHECK-LABEL: testSameVecEl3LE
 ; CHECK: xxinsertw 34, 34, 0
   %vecins = shufflevector <4 x float> %a, <4 x float> %a, <4 x i32> <i32 0, i32 1, i32 2, i32 6>
   ret <4 x float> %vecins
 }
 define <4 x float> @insertVarF(<4 x float> %a, float %f, i32 %el) {
 entry:
 ; CHECK-LABEL: insertVarF
-; CHECK: stxsspx 1,
+; CHECK: stfsx 1,
 ; CHECK: lxv
 ; CHECK-BE-LABEL: insertVarF
-; CHECK-BE: stxsspx 1,
+; CHECK-BE: stfsx 1,
 ; CHECK-BE: lxv
   %vecins = insertelement <4 x float> %a, float %f, i32 %el
   ret <4 x float> %vecins
 }
 define <4 x i32> @insertVarI(<4 x i32> %a, i32 %i, i32 %el) {
 entry:
 ; CHECK-LABEL: insertVarI
 ; CHECK: stwx

llvm/trunk/test/CodeGen/PowerPC/ppc64le-smallarg.ll

 declare void @test1(%struct.small_arg* sret, %struct.large_arg* byval, %struct.small_arg* byval)
 
 define float @callee2(float %pad1, float %pad2, float %pad3, float %pad4, float %pad5, float %pad6, float %pad7, float %pad8, float %pad9, float %pad10, float %pad11, float %pad12, float %pad13, float %x) {
 entry:
   ret float %x
 }
 ; CHECK: @callee2
 ; CHECK: addi [[TOCREG:[0-9]+]], 1, 136
-; CHECK: lxsspx {{[0-9]+}}, {{[0-9]+}}, [[TOCREG]]
+; CHECK: lfsx {{[0-9]+}}, {{[0-9]+}}, [[TOCREG]]
 ; CHECK: blr
 
 define void @caller2() {
 entry:
   %0 = load float, float* @gf, align 4
   %call = tail call float @test2(float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float 0.000000e+00, float %0)
   ret void
 }
 ; CHECK: @caller2
 ; CHECK: addi [[TOCOFF:[0-9]+]], {{[0-9]+}}, 136
-; CHECK: stxsspx {{[0-9]+}}, 0, [[TOCOFF]]
+; CHECK: stfsx {{[0-9]+}}, 0, [[TOCOFF]]
 ; CHECK: bl test2
 
 declare float @test2(float, float, float, float, float, float, float, float, float, float, float, float, float, float)
 

llvm/trunk/test/CodeGen/PowerPC/pr25157-peephole.ll

L.LB38_2452:
   %0 = load float, float* bitcast (i8* getelementptr inbounds (%struct.BSS38.51.4488.9911.14348.16813.20264.24701.28152.31603.35054.39491.44914.45407.46393.46886.47872.49351.49844.50830.51323.52309.53295.53788.54281.55267.55760.59211.61625, %struct.BSS38.51.4488.9911.14348.16813.20264.24701.28152.31603.35054.39491.44914.45407.46393.46886.47872.49351.49844.50830.51323.52309.53295.53788.54281.55267.55760.59211.61625* @.BSS38, i64 0, i32 0, i64 16) to float*), align 16
   %1 = fpext float %0 to double
   %2 = insertelement <2 x double> undef, double %1, i32 1
   store <2 x double> %2, <2 x double>* bitcast (i8* getelementptr inbounds (%struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626, %struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626* @_main1_2_, i64 0, i32 0, i64 32) to <2 x double>*), align 16
   unreachable
 }
 
 ; CHECK-LABEL: @aercalc_
-; CHECK: lxsspx
+; CHECK: lfsx
 ; CHECK: xxspltd
 ; CHECK: stxvd2x
 ; CHECK-NOT: xxswapd
 
 ; CHECK-P9-LABEL: @aercalc_
 ; CHECK-P9: lfs
 ; CHECK-P9: xxspltd
 ; CHECK-P9: stxv
 ; CHECK-P9-NOT: xxswapd

llvm/trunk/test/CodeGen/PowerPC/pr25157.ll

L.LB38_2452:
   %0 = load float, float* bitcast (i8* getelementptr inbounds (%struct.BSS38.51.4488.9911.14348.16813.20264.24701.28152.31603.35054.39491.44914.45407.46393.46886.47872.49351.49844.50830.51323.52309.53295.53788.54281.55267.55760.59211.61625, %struct.BSS38.51.4488.9911.14348.16813.20264.24701.28152.31603.35054.39491.44914.45407.46393.46886.47872.49351.49844.50830.51323.52309.53295.53788.54281.55267.55760.59211.61625* @.BSS38, i64 0, i32 0, i64 16) to float*), align 16
   %1 = fpext float %0 to double
   %2 = insertelement <2 x double> undef, double %1, i32 1
   store <2 x double> %2, <2 x double>* bitcast (i8* getelementptr inbounds (%struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626, %struct_main1_2_.491.4928.10351.14788.17253.20704.25141.28592.32043.35494.39931.45354.45847.46833.47326.48312.49791.50284.51270.51763.52749.53735.54228.54721.55707.56200.59651.61626* @_main1_2_, i64 0, i32 0, i64 32) to <2 x double>*), align 16
   unreachable
 }
 
 ; CHECK-LABEL: @aercalc_
-; CHECK: lxsspx
+; CHECK: lfsx
 ; CHECK-P9-LABEL: @aercalc_
 ; CHECK-P9: lfs

llvm/trunk/test/CodeGen/PowerPC/select-addrRegRegOnly.ll

 ; RUN: llc -mcpu=pwr8 -mtriple=powerpc64le-unknown-unknown -verify-machineinstrs < %s | FileCheck %s
 ; RUN: llc -mcpu=pwr8 -mtriple=powerpc64-unknown-unknown -verify-machineinstrs < %s | FileCheck %s
 
 ; Function Attrs: norecurse nounwind readonly
 define float @testSingleAccess(i32* nocapture readonly %arr) local_unnamed_addr #0 {
 ; CHECK-LABEL: testSingleAccess:
 ; CHECK:       # BB#0: # %entry
 ; CHECK-NEXT:    addi 3, 3, 8
-; CHECK-NEXT:    lxsiwax 0, 0, 3
+; CHECK-NEXT:    lfiwax 0, 0, 3
 ; CHECK-NEXT:    xscvsxdsp 1, 0
 ; CHECK-NEXT:    blr
 entry:
   %arrayidx = getelementptr inbounds i32, i32* %arr, i64 2
   %0 = load i32, i32* %arrayidx, align 4
   %conv = sitofp i32 %0 to float
   ret float %conv
 }

llvm/trunk/test/CodeGen/PowerPC/select_const.ll

 ; ALL-NEXT:  # BB#1:
 ; ALL-NEXT:    addis 3, 2, .LCPI38_0@toc@ha
 ; ALL-NEXT:    addi 3, 3, .LCPI38_0@toc@l
 ; ALL-NEXT:    lxsdx 1, 0, 3
 ; ALL-NEXT:    blr
 ; ALL-NEXT:  .LBB38_2:
 ; ALL-NEXT:    addis 3, 2, .LCPI38_1@toc@ha
 ; ALL-NEXT:    addi 3, 3, .LCPI38_1@toc@l
-; ALL-NEXT:    lxsspx 1, 0, 3
+; ALL-NEXT:    lfsx 1, 0, 3
 ; ALL-NEXT:    blr
   %sel = select i1 %cond, double -4.0, double 23.3
   %bo = frem double %sel, 5.1
   ret double %bo
 }
 

llvm/trunk/test/CodeGen/PowerPC/vsx_insert_extract_le.ll

define <2 x double> @testi0(<2 x double>* %p1, double* %p2) {
   %v = load <2 x double>, <2 x double>* %p1
   %s = load double, double* %p2
   %r = insertelement <2 x double> %v, double %s, i32 0
   ret <2 x double> %r
 
 ; CHECK-LABEL: testi0
 ; CHECK: lxvd2x 0, 0, 3
 ; CHECK: lxsdx 1, 0, 4
-; CHECK: xxswapd 0, 0
-; CHECK: xxspltd 1, 1, 0
+; CHECK-DAG: xxspltd 1, 1, 0
+; CHECK-DAG: xxswapd 0, 0
 ; CHECK: xxpermdi 34, 0, 1, 1
 
 ; CHECK-P9-LABEL: testi0
 ; CHECK-P9: lfd [[REG1:[0-9]+]], 0(4)
 ; CHECK-P9: lxv [[REG2:[0-9]+]], 0(3)
 ; CHECK-P9: xxspltd [[REG3:[0-9]+]], [[REG1]], 0
 ; CHECK-P9: xxpermdi 34, [[REG2]], [[REG3]], 1
 }
 
 define <2 x double> @testi1(<2 x double>* %p1, double* %p2) {
   %v = load <2 x double>, <2 x double>* %p1
   %s = load double, double* %p2
   %r = insertelement <2 x double> %v, double %s, i32 1
   ret <2 x double> %r
 
 ; CHECK-LABEL: testi1
 ; CHECK: lxvd2x 0, 0, 3
 ; CHECK: lxsdx 1, 0, 4
-; CHECK: xxswapd 0, 0
-; CHECK: xxspltd 1, 1, 0
+; CHECK-DAG: xxspltd 1, 1, 0
+; CHECK-DAG: xxswapd 0, 0
 ; CHECK: xxmrgld 34, 1, 0
 
 ; CHECK-P9-LABEL: testi1
 ; CHECK-P9: lfd [[REG1:[0-9]+]], 0(4)
 ; CHECK-P9: lxv [[REG2:[0-9]+]], 0(3)
 ; CHECK-P9: xxspltd [[REG3:[0-9]+]], [[REG1]], 0
 ; CHECK-P9: xxmrgld 34, [[REG3]], [[REG2]]
 }

llvm/trunk/test/CodeGen/PowerPC/vsx_scalar_ld_st.ll

 entry:
   %ii = alloca i32, align 4
   %0 = load double, double* @d, align 8
   %conv = fptosi double %0 to i32
   store volatile i32 %conv, i32* %ii, align 4
   ret void
 ; CHECK-LABEL: @dblToInt
 ; CHECK: xscvdpsxws [[REGCONV1:[0-9]+]],
-; CHECK: stxsiwx [[REGCONV1]],
+; CHECK: stfiwx [[REGCONV1]],
 }
 
 ; Function Attrs: nounwind
 define void @fltToInt() #0 {
 entry:
   %ii = alloca i32, align 4
   %0 = load float, float* @f, align 4
   %conv = fptosi float %0 to i32
   store volatile i32 %conv, i32* %ii, align 4
   ret void
 ; CHECK-LABEL: @fltToInt
 ; CHECK: xscvdpsxws [[REGCONV2:[0-9]+]],
-; CHECK: stxsiwx [[REGCONV2]],
+; CHECK: stfiwx [[REGCONV2]],
 }
 
 ; Function Attrs: nounwind
 define void @intToDbl() #0 {
 entry:
   %dd = alloca double, align 8
   %0 = load i32, i32* @i, align 4
   %conv = sitofp i32 %0 to double
   store volatile double %conv, double* %dd, align 8
   ret void
 ; CHECK-LABEL: @intToDbl
-; CHECK: lxsiwax [[REGLD1:[0-9]+]],
+; CHECK: lfiwax [[REGLD1:[0-9]+]],
 ; CHECK: xscvsxddp {{[0-9]+}}, [[REGLD1]]
 }
 
 ; Function Attrs: nounwind
 define void @intToFlt() #0 {
 entry:
   %ff = alloca float, align 4
   %0 = load i32, i32* @i, align 4
   %conv = sitofp i32 %0 to float
   store volatile float %conv, float* %ff, align 4
   ret void
 ; CHECK-LABEL: @intToFlt
-; CHECK: lxsiwax [[REGLD2:[0-9]+]],
+; CHECK: lfiwax [[REGLD2:[0-9]+]],
 ; CHECK: xscvsxdsp {{[0-9]}}, [[REGLD2]]
 }
 
 ; Function Attrs: nounwind
 define void @dblToUInt() #0 {
 entry:
   %uiui = alloca i32, align 4
   %0 = load double, double* @d, align 8
   %conv = fptoui double %0 to i32
   store volatile i32 %conv, i32* %uiui, align 4
   ret void
 ; CHECK-LABEL: @dblToUInt
 ; CHECK: xscvdpuxws [[REGCONV3:[0-9]+]],
-; CHECK: stxsiwx [[REGCONV3]],
+; CHECK: stfiwx [[REGCONV3]],
 }
 
 ; Function Attrs: nounwind
 define void @fltToUInt() #0 {
 entry:
   %uiui = alloca i32, align 4
   %0 = load float, float* @f, align 4
   %conv = fptoui float %0 to i32
   store volatile i32 %conv, i32* %uiui, align 4
   ret void
 ; CHECK-LABEL: @fltToUInt
 ; CHECK: xscvdpuxws [[REGCONV4:[0-9]+]],
-; CHECK: stxsiwx [[REGCONV4]],
+; CHECK: stfiwx [[REGCONV4]],
 }
 
 ; Function Attrs: nounwind
 define void @uIntToDbl() #0 {
 entry:
   %dd = alloca double, align 8
   %0 = load i32, i32* @ui, align 4
   %conv = uitofp i32 %0 to double
   store volatile double %conv, double* %dd, align 8
   ret void
 ; CHECK-LABEL: @uIntToDbl
-; CHECK: lxsiwzx [[REGLD3:[0-9]+]],
+; CHECK: lfiwzx [[REGLD3:[0-9]+]],
 ; CHECK: xscvuxddp {{[0-9]+}}, [[REGLD3]]
 }
 
 ; Function Attrs: nounwind
 define void @uIntToFlt() #0 {
 entry:
   %ff = alloca float, align 4
   %0 = load i32, i32* @ui, align 4
   %conv = uitofp i32 %0 to float
   store volatile float %conv, float* %ff, align 4
   ret void
 ; CHECK-LABEL: @uIntToFlt
-; CHECK: lxsiwzx [[REGLD4:[0-9]+]],
+; CHECK: lfiwzx [[REGLD4:[0-9]+]],
 ; CHECK: xscvuxdsp {{[0-9]+}}, [[REGLD4]]
 }
 
 ; Function Attrs: nounwind
 define void @dblToFloat() #0 {
 entry:
   %ff = alloca float, align 4
   %0 = load double, double* @d, align 8
   %conv = fptrunc double %0 to float
   store volatile float %conv, float* %ff, align 4
   ret void
 ; CHECK-LABEL: @dblToFloat
 ; CHECK: lxsdx [[REGLD5:[0-9]+]],
-; CHECK: stxsspx [[REGLD5]],
+; CHECK: stfsx [[REGLD5]],
 ; CHECK-P9-LABEL: @dblToFloat
 ; CHECK-P9: lfd [[REGLD5:[0-9]+]],
 ; CHECK-P9: stfs [[REGLD5]],
 }
 
 ; Function Attrs: nounwind
 define void @floatToDbl() #0 {
 entry:
   %dd = alloca double, align 8
   %0 = load float, float* @f, align 4
   %conv = fpext float %0 to double
   store volatile double %conv, double* %dd, align 8
   ret void
 ; CHECK-LABEL: @floatToDbl
-; CHECK: lxsspx [[REGLD5:[0-9]+]],
+; CHECK: lfsx [[REGLD5:[0-9]+]],
 ; CHECK: stxsdx [[REGLD5]],
 ; CHECK-P9-LABEL: @floatToDbl
 ; CHECK-P9: lfs [[REGLD5:[0-9]+]],
 ; CHECK-P9: stfd [[REGLD5]],
 }