[PowerPC] Simplify fp-to-int store optimization

[qiucf]

Summary

On PowerPC VSX targets, fp-to-int will be transformed into xscv* + mfvsr. When the result is to be stored, mfvsr can be replaced by a direct store.

This patch simplifies the optimization by using existing fp-to-int code, which also helps for:

1. Strict floating point support of this optimization (in another patch)
2. Better codegen from CSE

[shchenz]

llvm/test/CodeGen/PowerPC/store_fptoi.ll

1032

From the case change, I am thinking common the 3 xscvdpsxds sounds like making more sense?

[shchenz]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8075

This change is not enough I think. You extend this function to be used before type legalizer and make this function depend on the caller that will not pass other illegal types except i8 and i16. How about simple types like i2 and i4?

We need a guard for the type in this function.

8076

And a case please for i16 or i8

8089

If we plan to support type MVT::f128, we may need to change the PPCISD node names to generic ones. These names are 1:1 map to PPC instructions and these instructions assume the types are f64.

14878

We may need to following this logic and consider about adding the users back to the work list.

llvm/lib/Target/PowerPC/PPCInstrP10.td

1257

These two patterns are same with below two patterns. We still need to distinguish f64 and f128 input for PPCstore_scal_int_from_vsr.

2203

Same like above, there are duplicated patterns now as f64 and f128 are not distinguished

[shchenz]

If this patch can optimize the redundant xscvdpsxws away for case: https://godbolt.org/z/zWsaae4j4 ?

[qiucf]

In D141473#4260035, @shchenz wrote:

If this patch can optimize the redundant xscvdpsxws away for case: https://godbolt.org/z/zWsaae4j4 ?

Yes, one less xscvdpsxws.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8075

combineStoreFPToInt already rejected illegal types. And codegen for i2 i4 are good:

xscvdpsxws 0, 1
mffprwz	3, 0
clrlwi	3, 3, 28
stb 3, 0(4)
stb 3, 0(5)
stb 3, 0(6)
blr

8089

Thanks for reminding about this. I think that deserves another patch. I propose renaming them into CONV_FP_(I|U)(32|64)

14878

Is that necessary? DAG combiner will add the users into worklist.

[qiucf]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

14878

The root node to be returned is the ST_VSR_SCAL_INT, then FCTIDZ, then FP_EXTEND (if f32). The FP_EXTEND from f32 to f64 will disappear after ISel. The others are PPC-specific nodes so adding them into worklist manually will not help.

[shchenz]

Yes, one less xscvdpsxws.

Good to hear that. This patch looks right to me. Some more comments

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

8075

OK. But if we assume caller will only pass certain types to this function, we may need to add an assertion here? Otherwise when there is a new caller for this function in future, it is not so obvious that this function has assumption for the types. For example, if we pass i2 or i4 or even pass i8 or i16 on a pwr8 target, we get llvm_unreachable("Unhandled FP_TO_INT type in custom expander!");?

8128

Should we use MVT::i64 for DestTy if the target is 64-bit? With this, we should still have accurate round result if the input float is between 2^31 and 2^63 -1 before truncating.

14878

OK, fair. Thanks for checking.

llvm/lib/Target/PowerPC/PPCInstrP10.td

1258

Can you explain why we need to copy to regclass VSRC? The definition of PSTXSDpc seems requiring RC to be VFRC for $src.

llvm/lib/Target/PowerPC/PPCInstrVSX.td

3161

Compared with the lines at 4046, we seems missing some pattern here? This is for [HasVSX, NoP9Vector].

3311

This is for [HasVSX, HasP8Vector]

4046

This is for HasVSX, HasP9Vector.

Except the patterns for STXSIHX and STXSIBX(Power9 specific), all other patterns seems common for these three predicates: [HasVSX, HasP9Vector], [HasVSX, HasP8Vector], [HasVSX, NoP9Vector]. Can we try to refactor these patterns a little bit, there seems some redundancy with current implementation?

[qiucf]

Address comments and further simplify tablegen patterns.

[qiucf]

llvm/lib/Target/PowerPC/PPCInstrP10.td

1258

Thanks, I updated the copied classes to respective regclass of the instructions. This COPY_TO_REGCLASS is needed, otherewise we'll get a HW typeset is empty build failure.

llvm/lib/Target/PowerPC/PPCInstrVSX.td

3161

This is for pwr7/pwr8 only, other patterns are located at HasP8Vector HasVSX section, so that they can be shared with pwr9.

[shchenz]

llvm/lib/Target/PowerPC/PPCInstrP10.td

1258

Yeah, I know we need a COPY_TO_REGCLASS, my confusion is why the dest RC is VSRC, not the original VFRC as the $src for PSTXSDpc requires VFRC.

[qiucf]

llvm/lib/Target/PowerPC/PPCInstrP10.td

1258

I updated them accordingly, they should get missed in last revision.

As of why VSRC also passes, I think the classes will be automatically shortened.

[shchenz]

LGTM. No more comments from me.

Please hold on some days in case other reviewers have concerns. Thanks very much for this improvement.

llvm/lib/Target/PowerPC/PPCInstrP10.td

1258

Thanks, yes, VFRC matchs what I thought.

[shchenz]

And look forward to your refactor patch related to the PPCISD nodes.

[vitalybuka]

Reverting to fix the bot

[vitalybuka]

https://lab.llvm.org/buildbot/#/builders/18/builds/9118

llvm/lib/Target/PowerPC/PPCISelLowering.h

enum NodeType : unsigned {
     FCTIDZ,
     FCTIWZ,
 
     /// Newer FCTI[D,W]UZ floating-point-to-integer conversion instructions for
     /// unsigned integers with round toward zero.
     FCTIDUZ,
     FCTIWUZ,
 
-    /// Floating-point-to-integer conversion instructions
-    FP_TO_UINT_IN_VSR,
-    FP_TO_SINT_IN_VSR,
-
     /// VEXTS, ByteWidth - takes an input in VSFRC and produces an output in
     /// VSFRC that is sign-extended from ByteWidth to a 64-byte integer.
     VEXTS,
 
     /// Reciprocal estimate instructions (unary FP ops).
     FRE,
     FRSQRTE,
 

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

   case PPCISD::FCFID:           return "PPCISD::FCFID";
   case PPCISD::FCFIDU:          return "PPCISD::FCFIDU";
   case PPCISD::FCFIDS:          return "PPCISD::FCFIDS";
   case PPCISD::FCFIDUS:         return "PPCISD::FCFIDUS";
   case PPCISD::FCTIDZ:          return "PPCISD::FCTIDZ";
   case PPCISD::FCTIWZ:          return "PPCISD::FCTIWZ";
   case PPCISD::FCTIDUZ:         return "PPCISD::FCTIDUZ";
   case PPCISD::FCTIWUZ:         return "PPCISD::FCTIWUZ";
-  case PPCISD::FP_TO_UINT_IN_VSR:
-                                return "PPCISD::FP_TO_UINT_IN_VSR,";
-  case PPCISD::FP_TO_SINT_IN_VSR:
-                                return "PPCISD::FP_TO_SINT_IN_VSR";
   case PPCISD::FRE:             return "PPCISD::FRE";
   case PPCISD::FRSQRTE:         return "PPCISD::FRSQRTE";
   case PPCISD::FTSQRT:
     return "PPCISD::FTSQRT";
   case PPCISD::FSQRT:
     return "PPCISD::FSQRT";
   case PPCISD::STFIWX:          return "PPCISD::STFIWX";
   case PPCISD::VPERM:           return "PPCISD::VPERM";

       Cmp = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f64, Cmp);
     return DAG.getNode(PPCISD::FSEL, dl, ResVT, Cmp, FV, TV);
   case ISD::SETOLE:
   case ISD::SETLE:
     Cmp = DAG.getNode(ISD::FSUB, dl, CmpVT, RHS, LHS, Flags);
     if (Cmp.getValueType() == MVT::f32)   // Comparison is always 64-bits
       Cmp = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f64, Cmp);
     return DAG.getNode(PPCISD::FSEL, dl, ResVT, Cmp, TV, FV);
   }
   return Op;
 }
 
 static unsigned getPPCStrictOpcode(unsigned Opc) {
   switch (Opc) {
   default:
     llvm_unreachable("No strict version of this opcode!");
   case PPCISD::FCTIDZ:
     return PPCISD::STRICT_FCTIDZ;
   case PPCISD::FCTIWZ:
     return PPCISD::STRICT_FCTIWZ;
   case PPCISD::FCTIDUZ:
     return PPCISD::STRICT_FCTIDUZ;
   case PPCISD::FCTIWUZ:
     return PPCISD::STRICT_FCTIWUZ;
   case PPCISD::FCFID:
     return PPCISD::STRICT_FCFID;
   case PPCISD::FCFIDU:
     return PPCISD::STRICT_FCFIDU;
   case PPCISD::FCFIDS:
     return PPCISD::STRICT_FCFIDS;
   case PPCISD::FCFIDUS:

     if (IsStrict) {
       Src =
           DAG.getNode(ISD::STRICT_FP_EXTEND, dl,
                       DAG.getVTList(MVT::f64, MVT::Other), {Chain, Src}, Flags);
       Chain = Src.getValue(1);
     } else
       Src = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f64, Src);
   }
-  SDValue Conv;
+  MVT DestTy = Op.getSimpleValueType();
+  if ((DestTy == MVT::i8 || DestTy == MVT::i16) && Subtarget.hasP9Vector())
+    DestTy = MVT::i32;
   unsigned Opc = ISD::DELETED_NODE;
-  switch (Op.getSimpleValueType().SimpleTy) {
+  switch (DestTy.SimpleTy) {
   default: llvm_unreachable("Unhandled FP_TO_INT type in custom expander!");
   case MVT::i32:
     Opc = IsSigned ? PPCISD::FCTIWZ
                    : (Subtarget.hasFPCVT() ? PPCISD::FCTIWUZ : PPCISD::FCTIDZ);
     break;
   case MVT::i64:
     assert((IsSigned || Subtarget.hasFPCVT()) &&
            "i64 FP_TO_UINT is supported only with FPCVT");
     Opc = IsSigned ? PPCISD::FCTIDZ : PPCISD::FCTIDUZ;
   }
+  EVT ConvTy = Src.getValueType() == MVT::f128 ? MVT::f128 : MVT::f64;
+  SDValue Conv;
   if (IsStrict) {
     Opc = getPPCStrictOpcode(Opc);
-    Conv = DAG.getNode(Opc, dl, DAG.getVTList(MVT::f64, MVT::Other),
-                       {Chain, Src}, Flags);
+    Conv = DAG.getNode(Opc, dl, DAG.getVTList(ConvTy, MVT::Other), {Chain, Src},
+                       Flags);
   } else {
-    Conv = DAG.getNode(Opc, dl, MVT::f64, Src);
+    Conv = DAG.getNode(Opc, dl, ConvTy, Src);
   }
   return Conv;
 }
 
 void PPCTargetLowering::LowerFP_TO_INTForReuse(SDValue Op, ReuseLoadInfo &RLI,
                                                SelectionDAG &DAG,
                                                const SDLoc &dl) const {
   SDValue Tmp = convertFPToInt(Op, DAG, Subtarget);

         // a smaller FP_TO_SINT.
         if (IsStrict) {
           SDValue Res = DAG.getNode(PPCISD::STRICT_FADDRTZ, dl,
                                     DAG.getVTList(MVT::f64, MVT::Other),
                                     {Op.getOperand(0), Lo, Hi}, Flags);
           return DAG.getNode(ISD::STRICT_FP_TO_SINT, dl,
                              DAG.getVTList(MVT::i32, MVT::Other),
                              {Res.getValue(1), Res}, Flags);
         } else {

 
   if (ResVT == MVT::f128 && !Subtarget.hasP9Vector())
     return SDValue();
 
   if (ResVT == MVT::ppcf128 || !Subtarget.hasP8Vector() ||
       cast<StoreSDNode>(N)->isTruncatingStore() || !ValidTypeForStoreFltAsInt)
     return SDValue();
 
-  // Extend f32 values to f64
-  if (ResVT.getScalarSizeInBits() == 32) {
-    Val = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f64, Val);
-    DCI.AddToWorklist(Val.getNode());
-  }
-
-  // Set signed or unsigned conversion opcode.
-  unsigned ConvOpcode = (Opcode == ISD::FP_TO_SINT) ?
-                          PPCISD::FP_TO_SINT_IN_VSR :
-                          PPCISD::FP_TO_UINT_IN_VSR;
-
-  Val = DAG.getNode(ConvOpcode,
-                    dl, ResVT == MVT::f128 ? MVT::f128 : MVT::f64, Val);
-  DCI.AddToWorklist(Val.getNode());
+  Val = convertFPToInt(N->getOperand(1), DAG, Subtarget);
 
   // Set number of bytes being converted.
   unsigned ByteSize = Op1VT.getScalarSizeInBits() / 8;
   SDValue Ops[] = { N->getOperand(0), Val, N->getOperand(2),
                     DAG.getIntPtrConstant(ByteSize, dl, false),
                     DAG.getValueType(Op1VT) };
 
   Val = DAG.getMemIntrinsicNode(PPCISD::ST_VSR_SCAL_INT, dl,
           DAG.getVTList(MVT::Other), Ops,
           cast<StoreSDNode>(N)->getMemoryVT(),
           cast<StoreSDNode>(N)->getMemOperand());
 
-  DCI.AddToWorklist(Val.getNode());
   return Val;
 }
 
 static bool isAlternatingShuffMask(const ArrayRef<int> &Mask, int NumElts) {
   // Check that the source of the element keeps flipping
   // (i.e. Mask[i] < NumElts -> Mask[i+i] >= NumElts).
   bool PrevElemFromFirstVec = Mask[0] < NumElts;
   for (int i = 1, e = Mask.size(); i < e; i++) {

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 ]>;
 def SDT_PPCstxsix : SDTypeProfile<0, 3, [
   SDTCisVT<0, f64>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
 ]>;
 def SDT_PPCcv_fp_to_int  : SDTypeProfile<1, 1, [
   SDTCisFP<0>, SDTCisFP<1>
   ]>;
 def SDT_PPCstore_scal_int_from_vsr : SDTypeProfile<0, 3, [
-  SDTCisVT<0, f64>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
+  SDTCisFP<0>, SDTCisPtrTy<1>, SDTCisPtrTy<2>
 ]>;
 def SDT_PPCVexts  : SDTypeProfile<1, 2, [
   SDTCisVT<0, f64>, SDTCisVT<1, f64>, SDTCisPtrTy<2>
 ]>;
 
 def SDT_PPCCallSeqStart : SDCallSeqStart<[ SDTCisVT<0, i32>,
                                            SDTCisVT<1, i32> ]>;
 def SDT_PPCCallSeqEnd   : SDCallSeqEnd<[ SDTCisVT<0, i32>,
def PPCany_fcfidu : PatFrags<(ops node:$op),
                               (PPCstrict_fcfidu node:$op)]>;
 def PPCany_fcfids : PatFrags<(ops node:$op),
                               [(PPCfcfids node:$op),
                                (PPCstrict_fcfids node:$op)]>;
 def PPCany_fcfidus : PatFrags<(ops node:$op),
                               [(PPCfcfidus node:$op),
                                (PPCstrict_fcfidus node:$op)]>;
 
-def PPCcv_fp_to_uint_in_vsr:
-    SDNode<"PPCISD::FP_TO_UINT_IN_VSR", SDT_PPCcv_fp_to_int, []>;
-def PPCcv_fp_to_sint_in_vsr:
-    SDNode<"PPCISD::FP_TO_SINT_IN_VSR", SDT_PPCcv_fp_to_int, []>;
 def PPCstore_scal_int_from_vsr:
    SDNode<"PPCISD::ST_VSR_SCAL_INT", SDT_PPCstore_scal_int_from_vsr,
            [SDNPHasChain, SDNPMayStore]>;
 def PPCstfiwx : SDNode<"PPCISD::STFIWX", SDT_PPCstfiwx,
                        [SDNPHasChain, SDNPMayStore]>;
 def PPClfiwax : SDNode<"PPCISD::LFIWAX", SDT_PPClfiwx,
                        [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 def PPClfiwzx : SDNode<"PPCISD::LFIWZX", SDT_PPClfiwx,

llvm/lib/Target/PowerPC/PPCInstrP10.td

   def : Pat<(atomic_store_16 (PPCmatpcreladdr PCRelForm:$ga), i64:$RS),
             (PSTH8pc $RS, $ga, 0)>;
   def : Pat<(atomic_store_32 (PPCmatpcreladdr PCRelForm:$ga), i64:$RS),
             (PSTW8pc $RS, $ga, 0)>;
   def : Pat<(atomic_store_64 (PPCmatpcreladdr PCRelForm:$ga), i64:$RS),
             (PSTDpc $RS, $ga, 0)>;
 
   // Special Cases For PPCstore_scal_int_from_vsr
-  def : Pat<(PPCstore_scal_int_from_vsr
-              (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)),
-              (PPCmatpcreladdr PCRelForm:$dst), 8),
-            (PSTXSDpc (XSCVDPSXDS f64:$src), $dst, 0)>;
-  def : Pat<(PPCstore_scal_int_from_vsr
-              (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)),
-              (PPCmatpcreladdr PCRelForm:$dst), 8),
-            (PSTXSDpc (COPY_TO_REGCLASS (XSCVQPSDZ f128:$src), VFRC), $dst, 0)>;
-
-  def : Pat<(PPCstore_scal_int_from_vsr
-              (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)),
-              (PPCmatpcreladdr PCRelForm:$dst), 8),
-            (PSTXSDpc (XSCVDPUXDS f64:$src), $dst, 0)>;
-  def : Pat<(PPCstore_scal_int_from_vsr
-              (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)),
-              (PPCmatpcreladdr PCRelForm:$dst), 8),
-            (PSTXSDpc (COPY_TO_REGCLASS (XSCVQPUDZ f128:$src), VFRC), $dst, 0)>;
+  def : Pat<(PPCstore_scal_int_from_vsr f64:$src, (PPCmatpcreladdr PCRelForm:$dst), 8),
+            (PSTXSDpc $src, $dst, 0)>;
+  def : Pat<(PPCstore_scal_int_from_vsr f128:$src, (PPCmatpcreladdr PCRelForm:$dst), 8),
+            (PSTXSDpc (COPY_TO_REGCLASS $src, VSRC), $dst, 0)>;
 
   def : Pat<(v4f32 (PPCldvsxlh (PPCmatpcreladdr PCRelForm:$addr))),
             (SUBREG_TO_REG (i64 1), (PLFDpc $addr, 0), sub_64)>;
 
   // If the PPCmatpcreladdr node is not caught by any other pattern it should be
   // caught here and turned into a paddi instruction to materialize the address.
   def : Pat<(PPCmatpcreladdr PCRelForm:$addr), (PADDI8pc 0, $addr)>;
   // PPCtlsdynamatpcreladdr node is used for TLS dynamic models to materialize

   def : Pat<(v4f32 (load PDForm:$src)), (PLXV memri34:$src)>;
   def : Pat<(store v4f32:$XS, PDForm:$dst), (PSTXV $XS, memri34:$dst)>;
 
   // Load / Store v2f64
   def : Pat<(v2f64 (load PDForm:$src)), (PLXV memri34:$src)>;
   def : Pat<(store v2f64:$XS, PDForm:$dst), (PSTXV $XS, memri34:$dst)>;
 
   // Cases For PPCstore_scal_int_from_vsr
-  def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), PDForm:$dst, 8),
-            (PSTXSD (XSCVDPUXDS f64:$src), PDForm:$dst)>;
-  def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), PDForm:$dst, 8),
-            (PSTXSD (XSCVDPSXDS f64:$src), PDForm:$dst)>;
-  def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), PDForm:$dst, 8),
-            (PSTXSD (COPY_TO_REGCLASS (XSCVQPUDZ f128:$src), VFRC),
-                     PDForm:$dst)>;
-  def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), PDForm:$dst, 8),
-            (PSTXSD (COPY_TO_REGCLASS (XSCVQPSDZ f128:$src), VFRC),
-                     PDForm:$dst)>;
+  def : Pat<(PPCstore_scal_int_from_vsr f64:$src, PDForm:$dst, 8),
+            (PSTXSD $src, PDForm:$dst)>;
+  def : Pat<(PPCstore_scal_int_from_vsr f128:$src, PDForm:$dst, 8),
+            (PSTXSD (COPY_TO_REGCLASS $src, VSRC), PDForm:$dst)>;
 }
 
 let Predicates = [PrefixInstrs] in {
   def : Pat<(i32 imm34:$imm), (PLI (getImmAs64BitInt imm:$imm))>;
   def : Pat<(i64 imm34:$imm), (PLI8 (getImmAs64BitInt imm:$imm))>;
   def : Pat<(v16i8 (int_ppc_vsx_xxpermx v16i8:$A, v16i8:$B, v16i8:$C, timm:$D)),
             (COPY_TO_REGCLASS (XXPERMX (COPY_TO_REGCLASS $A, VSRC),
                                        (COPY_TO_REGCLASS $B, VSRC),
                                        (COPY_TO_REGCLASS $C, VSRC), $D), VSRC)>;

llvm/lib/Target/PowerPC/PPCInstrVSX.td

     def XSCVQPDPO : X_VT5_XO5_VB5_VSFR_Ro<63, 20, 836, "xscvqpdpo",
                                           [(set f64:$RST,
                                           (int_ppc_truncf128_round_to_odd
                                           f128:$RB))]>;
   }
 
   // Truncate & Convert QP -> (Un)Signed (D)Word (dword[1] is set to zero)
   let mayRaiseFPException = 1 in {
-    def XSCVQPSDZ : X_VT5_XO5_VB5<63, 25, 836, "xscvqpsdz", []>;
-    def XSCVQPSWZ : X_VT5_XO5_VB5<63,  9, 836, "xscvqpswz", []>;
-    def XSCVQPUDZ : X_VT5_XO5_VB5<63, 17, 836, "xscvqpudz", []>;
-    def XSCVQPUWZ : X_VT5_XO5_VB5<63,  1, 836, "xscvqpuwz", []>;
+    def XSCVQPSDZ : X_VT5_XO5_VB5<63, 25, 836, "xscvqpsdz",
+        [(set f128:$RST, (PPCany_fctidz f128:$RB))]>;
+    def XSCVQPSWZ : X_VT5_XO5_VB5<63,  9, 836, "xscvqpswz",
+        [(set f128:$RST, (PPCany_fctiwz f128:$RB))]>;
+    def XSCVQPUDZ : X_VT5_XO5_VB5<63, 17, 836, "xscvqpudz",
+        [(set f128:$RST, (PPCany_fctiduz f128:$RB))]>;
+    def XSCVQPUWZ : X_VT5_XO5_VB5<63,  1, 836, "xscvqpuwz",
+        [(set f128:$RST, (PPCany_fctiwuz f128:$RB))]>;
   }
 
   // Convert (Un)Signed DWord -> QP.
   def XSCVSDQP  : X_VT5_XO5_VB5_TyVB<63, 10, 836, "xscvsdqp", vfrc, []>;
   def XSCVUDQP  : X_VT5_XO5_VB5_TyVB<63,  2, 836, "xscvudqp", vfrc, []>;
 
   // (Round &) Convert DP <-> HP
   // Note! xscvdphp's src and dest register both use the left 64 bits, so we use

 def : Pat<(v2f64 (insertelt v2f64:$A, f64:$B, 0)),
           (v2f64 (XXPERMDI $A, (SUBREG_TO_REG (i64 1), $B, sub_64), 0))>;
 def : Pat<(v2f64 (insertelt v2f64:$A, f64:$B, 1)),
           (v2f64 (XXPERMDI (SUBREG_TO_REG (i64 1), $B, sub_64), $A, 1))>;
 } // HasVSX, IsLittleEndian
 
 // Any pre-Power9 VSX subtarget.
 let Predicates = [HasVSX, NoP9Vector] in {
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), ForceXForm:$dst, 8),
-          (STXSDX (XSCVDPSXDS f64:$src), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), ForceXForm:$dst, 8),
-          (STXSDX (XSCVDPUXDS f64:$src), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 8),
+          (STXSDX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 8),
+          (STXSDX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
 
 // Load-and-splat with fp-to-int conversion (using X-Form VSX/FP loads).
 defm : ScalToVecWPermute<
   v4i32, DblToIntLoad.A,
   (XXSPLTW (SUBREG_TO_REG (i64 1), (XSCVDPSXWS (XFLOADf64 ForceXForm:$A)), sub_64), 1),
   (SUBREG_TO_REG (i64 1), (XSCVDPSXWS (XFLOADf64 ForceXForm:$A)), sub_64)>;
 defm : ScalToVecWPermute<
   v4i32, DblToUIntLoad.A,

 // Although XSNEGDP is available in P7, we want to select it starting from P8,
 // so that FNMSUBS can be selected for fneg-fmsub pattern on P7. (VSX version,
 // XSNMSUBASP, is available since P8)
 def : Pat<(f32 (fneg f32:$S)),
           (f32 (COPY_TO_REGCLASS (XSNEGDP
                (COPY_TO_REGCLASS $S, VSFRC)), VSSRC))>;
 
 // Instructions for converting float to i32 feeding a store.
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), ForceXForm:$dst, 4),
-          (STIWX (XSCVDPSXWS f64:$src), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), ForceXForm:$dst, 4),
-          (STIWX (XSCVDPUXWS f64:$src), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 4),
+          (STIWX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 4),
+          (STIWX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
 
 def : Pat<(v2i64 (smax v2i64:$src1, v2i64:$src2)),
           (v2i64 (VMAXSD (COPY_TO_REGCLASS $src1, VRRC),
                          (COPY_TO_REGCLASS $src2, VRRC)))>;
 def : Pat<(v2i64 (umax v2i64:$src1, v2i64:$src2)),
           (v2i64 (VMAXUD (COPY_TO_REGCLASS $src1, VRRC),
                          (COPY_TO_REGCLASS $src2, VRRC)))>;
 def : Pat<(v2i64 (smin v2i64:$src1, v2i64:$src2)),

 def : Pat<(i64 (any_fp_to_uint f128:$src)), (i64 (MFVRD (XSCVQPUDZ $src)))>;
 def : Pat<(i32 (any_fp_to_sint f128:$src)),
           (i32 (MFVSRWZ (COPY_TO_REGCLASS (XSCVQPSWZ $src), VFRC)))>;
 def : Pat<(i32 (any_fp_to_uint f128:$src)),
           (i32 (MFVSRWZ (COPY_TO_REGCLASS (XSCVQPUWZ $src), VFRC)))>;
 
 // Instructions for store(fptosi).
 // The 8-byte version is repeated here due to availability of D-Form STXSD.
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), XForm:$dst, 8),
-          (STXSDX (COPY_TO_REGCLASS (XSCVQPSDZ f128:$src), VFRC),
-                  XForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), DSForm:$dst, 8),
-          (STXSD (COPY_TO_REGCLASS (XSCVQPSDZ f128:$src), VFRC),
-                 DSForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), ForceXForm:$dst, 4),
-          (STXSIWX (COPY_TO_REGCLASS (XSCVQPSWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), ForceXForm:$dst, 2),
-          (STXSIHX (COPY_TO_REGCLASS (XSCVQPSWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), ForceXForm:$dst, 1),
-          (STXSIBX (COPY_TO_REGCLASS (XSCVQPSWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), XForm:$dst, 8),
-          (STXSDX (XSCVDPSXDS f64:$src), XForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), DSForm:$dst, 8),
-          (STXSD (XSCVDPSXDS f64:$src), DSForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), ForceXForm:$dst, 2),
-          (STXSIHX (XSCVDPSXWS f64:$src), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), ForceXForm:$dst, 1),
-          (STXSIBX (XSCVDPSXWS f64:$src), ForceXForm:$dst)>;
-
-// Instructions for store(fptoui).
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), XForm:$dst, 8),
-          (STXSDX (COPY_TO_REGCLASS (XSCVQPUDZ f128:$src), VFRC),
-                  XForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), DSForm:$dst, 8),
-          (STXSD (COPY_TO_REGCLASS (XSCVQPUDZ f128:$src), VFRC),
-                 DSForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), ForceXForm:$dst, 4),
-          (STXSIWX (COPY_TO_REGCLASS (XSCVQPUWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), ForceXForm:$dst, 2),
-          (STXSIHX (COPY_TO_REGCLASS (XSCVQPUWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), ForceXForm:$dst, 1),
-          (STXSIBX (COPY_TO_REGCLASS (XSCVQPUWZ $src), VFRC), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), XForm:$dst, 8),
-          (STXSDX (XSCVDPUXDS f64:$src), XForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), DSForm:$dst, 8),
-          (STXSD (XSCVDPUXDS f64:$src), DSForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), ForceXForm:$dst, 2),
-          (STXSIHX (XSCVDPUXWS f64:$src), ForceXForm:$dst)>;
-def : Pat<(PPCstore_scal_int_from_vsr
-            (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), ForceXForm:$dst, 1),
-          (STXSIBX (XSCVDPUXWS f64:$src), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, XForm:$dst, 8),
+          (STXSDX $src, XForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, DSForm:$dst, 8),
+          (STXSD $src, DSForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 4),
+          (STXSIWX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 2),
+          (STXSIHX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 1),
+          (STXSIBX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, XForm:$dst, 8),
+          (STXSDX $src, XForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, DSForm:$dst, 8),
+          (STXSD $src, DSForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 2),
+          (STXSIHX $src, ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f64:$src, ForceXForm:$dst, 1),
+          (STXSIBX $src, ForceXForm:$dst)>;
+
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, XForm:$dst, 8),
+          (STXSDX (COPY_TO_REGCLASS $src, VSRC), XForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, DSForm:$dst, 8),
+          (STXSD (COPY_TO_REGCLASS $src, VSRC), DSForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 4),
+          (STXSIWX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 2),
+          (STXSIHX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 1),
+          (STXSIBX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, XForm:$dst, 8),
+          (STXSDX (COPY_TO_REGCLASS $src, VSRC), XForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, DSForm:$dst, 8),
+          (STXSD (COPY_TO_REGCLASS $src, VSRC), DSForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 2),
+          (STXSIHX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstore_scal_int_from_vsr f128:$src, ForceXForm:$dst, 1),
+          (STXSIBX (COPY_TO_REGCLASS $src, VSRC), ForceXForm:$dst)>;
 
 // Round & Convert QP -> DP/SP
 def : Pat<(f64 (any_fpround f128:$src)), (f64 (XSCVQPDP $src))>;
 def : Pat<(f32 (any_fpround f128:$src)), (f32 (XSRSP (XSCVQPDPO $src)))>;
 
 // Convert SP -> QP
 def : Pat<(f128 (any_fpextend f32:$src)),
           (f128 (XSCVDPQP (COPY_TO_REGCLASS $src, VFRC)))>;

llvm/test/CodeGen/PowerPC/scalar-double-ldst.ll

   store i8 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_uint8_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_uint8_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsibx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_uint8_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i8 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_int8_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_int8_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsibx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_int8_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i16 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_uint16_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_uint16_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsihx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_uint16_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i16 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_int16_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_int16_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsihx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_int16_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i32 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_uint32_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_uint32_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_uint32_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i32 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_double_int32_t(i64 %ptr, double %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_double_int32_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_double_int32_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

llvm/test/CodeGen/PowerPC/scalar-float-ldst.ll

   store i8 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_uint8_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_uint8_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsibx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_uint8_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i8 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_int8_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_int8_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsibx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_int8_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i16 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_uint16_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_uint16_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsihx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_uint16_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i16 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_int16_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_int16_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stxsihx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_int16_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i32 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_uint32_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_uint32_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpuxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_uint32_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

   store i32 %conv, ptr %0, align 16
   ret void
 }
 
 ; Function Attrs: nofree norecurse nounwind uwtable willreturn writeonly
 define dso_local void @st_not_disjoint64_float_int32_t(i64 %ptr, float %str) {
 ; CHECK-P10-LABEL: st_not_disjoint64_float_int32_t:
 ; CHECK-P10:       # %bb.0: # %entry
+; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    pli r4, 232
 ; CHECK-P10-NEXT:    pli r5, 3567587329
-; CHECK-P10-NEXT:    xscvdpsxws f0, f1
 ; CHECK-P10-NEXT:    rldimi r5, r4, 32, 0
 ; CHECK-P10-NEXT:    or r3, r3, r5
 ; CHECK-P10-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P10-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: st_not_disjoint64_float_int32_t:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    li r4, 29

llvm/test/CodeGen/PowerPC/store_fptoi.ll

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2sdw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2sdw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpsxds 0, 0
-; CHECK-NEXT:    sldi 5, 5, 3
-; CHECK-NEXT:    stxsdx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 3
+; CHECK-NEXT:    stxsdx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2sdw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 3, 5, 3
 ; CHECK-PWR8-NEXT:    xscvdpsxds 0, 0
 ; CHECK-PWR8-NEXT:    stxsdx 0, 4, 3

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2sw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2sw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpsxws 0, 0
-; CHECK-NEXT:    sldi 5, 5, 2
-; CHECK-NEXT:    stfiwx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 2
+; CHECK-NEXT:    stfiwx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2sw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 3, 5, 2
 ; CHECK-PWR8-NEXT:    xscvdpsxws 0, 0
 ; CHECK-PWR8-NEXT:    stfiwx 0, 4, 3

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2shw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2shw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpsxws 0, 0
-; CHECK-NEXT:    sldi 5, 5, 1
-; CHECK-NEXT:    stxsihx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 1
+; CHECK-NEXT:    stxsihx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2shw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 5, 5, 1
 ; CHECK-PWR8-NEXT:    xscvdpsxws 0, 0
 ; CHECK-PWR8-NEXT:    mffprwz 3, 0

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2udw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2udw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpuxds 0, 0
-; CHECK-NEXT:    sldi 5, 5, 3
-; CHECK-NEXT:    stxsdx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 3
+; CHECK-NEXT:    stxsdx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2udw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 3, 5, 3
 ; CHECK-PWR8-NEXT:    xscvdpuxds 0, 0
 ; CHECK-PWR8-NEXT:    stxsdx 0, 4, 3

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2uw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2uw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpuxws 0, 0
-; CHECK-NEXT:    sldi 5, 5, 2
-; CHECK-NEXT:    stfiwx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 2
+; CHECK-NEXT:    stfiwx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2uw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 3, 5, 2
 ; CHECK-PWR8-NEXT:    xscvdpuxws 0, 0
 ; CHECK-PWR8-NEXT:    stfiwx 0, 4, 3

 }
 
 ; Function Attrs: norecurse nounwind
 define void @spConv2uhw_x(ptr nocapture readonly %a, ptr nocapture %b,
 ; CHECK-LABEL: spConv2uhw_x:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    lfs 0, 0(3)
 ; CHECK-NEXT:    xscvdpuxws 0, 0
-; CHECK-NEXT:    sldi 5, 5, 1
-; CHECK-NEXT:    stxsihx 0, 4, 5
+; CHECK-NEXT:    sldi 3, 5, 1
+; CHECK-NEXT:    stxsihx 0, 4, 3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR8-LABEL: spConv2uhw_x:
 ; CHECK-PWR8:       # %bb.0: # %entry
 ; CHECK-PWR8-NEXT:    lfs 0, 0(3)
 ; CHECK-PWR8-NEXT:    sldi 5, 5, 1
 ; CHECK-PWR8-NEXT:    xscvdpsxws 0, 0
 ; CHECK-PWR8-NEXT:    mffprwz 3, 0

   %0 = load float, ptr %a, align 4
   %conv = fptoui float %0 to i8
   %idxprom = zext i32 %idx to i64
   %arrayidx = getelementptr inbounds i8, ptr %b, i64 %idxprom
   store i8 %conv, ptr %arrayidx, align 1
   ret void
 
 
 }
+
+define void @multiple_store_64(double %m, ptr %addr1, ptr %addr2, ptr %addr3) {
+; CHECK-LABEL: multiple_store_64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xscvdpsxds 2, 1
+; CHECK-NEXT:    stxsd 2, 0(4)
+; CHECK-NEXT:    stxsd 2, 0(5)
+; CHECK-NEXT:    stxsd 2, 0(6)
+; CHECK-NEXT:    blr
+;
+; CHECK-PWR8-LABEL: multiple_store_64:
+; CHECK-PWR8:       # %bb.0: # %entry
+; CHECK-PWR8-NEXT:    xscvdpsxds 0, 1
+; CHECK-PWR8-NEXT:    stxsdx 0, 0, 4
+; CHECK-PWR8-NEXT:    stxsdx 0, 0, 5
+; CHECK-PWR8-NEXT:    stxsdx 0, 0, 6
+; CHECK-PWR8-NEXT:    blr
+entry:
+  %conv1 = fptosi double %m to i64
+  store i64 %conv1, ptr %addr1, align 8
+  store i64 %conv1, ptr %addr2, align 8
+  store i64 %conv1, ptr %addr3, align 8
+  ret void
+}
+
+define void @multiple_store_32(double %m, ptr %addr1, ptr %addr2, ptr %addr3) {
+; CHECK-LABEL: multiple_store_32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xscvdpsxws 0, 1
+; CHECK-NEXT:    stfiwx 0, 0, 4
+; CHECK-NEXT:    stfiwx 0, 0, 5
+; CHECK-NEXT:    stfiwx 0, 0, 6
+; CHECK-NEXT:    blr
+;
+; CHECK-PWR8-LABEL: multiple_store_32:
+; CHECK-PWR8:       # %bb.0: # %entry
+; CHECK-PWR8-NEXT:    xscvdpsxws 0, 1
+; CHECK-PWR8-NEXT:    stfiwx 0, 0, 4
+; CHECK-PWR8-NEXT:    stfiwx 0, 0, 5
+; CHECK-PWR8-NEXT:    stfiwx 0, 0, 6
+; CHECK-PWR8-NEXT:    blr
+entry:
+  %conv1 = fptosi double %m to i32
+  store i32 %conv1, ptr %addr1, align 8
+  store i32 %conv1, ptr %addr2, align 8
+  store i32 %conv1, ptr %addr3, align 8
+  ret void
+}
+
+define void @multiple_store_16(double %m, ptr %addr1, ptr %addr2, ptr %addr3) {
+; CHECK-LABEL: multiple_store_16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xscvdpsxws 0, 1
+; CHECK-NEXT:    stxsihx 0, 0, 4
+; CHECK-NEXT:    stxsihx 0, 0, 5
+; CHECK-NEXT:    stxsihx 0, 0, 6
+; CHECK-NEXT:    blr
+;
+; CHECK-PWR8-LABEL: multiple_store_16:
+; CHECK-PWR8:       # %bb.0: # %entry
+; CHECK-PWR8-NEXT:    xscvdpsxws 0, 1
+; CHECK-PWR8-NEXT:    mffprwz 3, 0
+; CHECK-PWR8-NEXT:    sth 3, 0(4)
+; CHECK-PWR8-NEXT:    sth 3, 0(5)
+; CHECK-PWR8-NEXT:    sth 3, 0(6)
+; CHECK-PWR8-NEXT:    blr
+entry:
+  %conv1 = fptosi double %m to i16
+  store i16 %conv1, ptr %addr1, align 8
+  store i16 %conv1, ptr %addr2, align 8
+  store i16 %conv1, ptr %addr3, align 8
+  ret void
+}