[PowerPC] Implement canCombineStoreAndExtract and provide the missing codegen patterns

[nemanjai]

Summary

We have VSX store instructions that will store a single element from a vector without modifying it in any way. Previous generation cores can do this for word-sized elements and Power9 can also do it for half-word and byte-sized elements.
The TableGen patterns for the word-sized versions were missing - this patch adds them.

Furthermore, it provides the information about the cost of such a combine - zero cost when the index is the element number that the instruction stores, cost of 3 for other elements. The cost for the other elements is because a vector permute is needed to shift the element and it's only worthwhile if keeping the value as a vector reduces the cost enough to offset the cost of the permute.

[nemanjai]

Add the missing predicates on the patterns.

[lei]

test/CodeGen/PowerPC/combine-extract-store.ll

9

nit: line past col 80

12

Should these 2 lines be CHECK-DAG since they don't necessarily have to happen in this order?

[nemanjai]

Ping.

test/CodeGen/PowerPC/combine-extract-store.ll

9

I think we generally allow code in test cases to have lines however long clang produces them.

12

They certainly don't need to happen in this order, but I find that changing the CHECK directive produced by the tool makes the test case harder to maintain sometimes.

[nemanjai]

Implemented all the patterns for codegen of extract->store.

[steven.zhang]

lib/Target/PowerPC/PPCISelLowering.cpp

14307

I didn't take deep look at the implementation for this patch. The condition here seems not quite align with the comments. If the bitwidth is 32bit, we will combine the store and extract no matter if it is Power9 or not. I am not sure if this is by intention.

[jsji]

It is a good idea if we can CombineStoreAndExtract to save one instruction.
But looks like to me that it is not always safe and beneficial to do so, especially for floating points.

lib/Target/PowerPC/PPCInstrVSX.td

3557

This looks like not related to "extract+store" exploitation? Maybe we should do it in another patch?

test/CodeGen/PowerPC/combine-extract-store.ll

1

I would be better if we can split this new testcase into new patch, then just show diff due to this change here.

2

How about pwr9? Any change due to this patch?

test/CodeGen/PowerPC/extract-and-store.ll

69

This looks not better than using stfd ? Can we avoid combining this case?

122

Are we sure that the semantic are equivalent here for stfs-> stfiwx?

With stfs: The contents of register FRS are converted to single format (see page 160) and stored into the word in storage addressed by EA.,

With stfiwx: (FRS)32:63 are stored, without conversion, into the word in storage addressed by EA.

So, is it safe to assume that there is not difference due to conversion?

157

Similar to above, are we sure stxsiwx will store the save value as stfs for all single precision values?

test/CodeGen/PowerPC/store_fptoi.ll

21

Looks like not related to "extract+store" exploitation? Maybe we should do it in another patch?

[nemanjai]

Upon closer inspection, this actually almost never fires on PPC so spending any more time on it does not seem useful. Abandoning this patch.

lib/Target/PowerPC/PPCISelLowering.cpp

14307

That is correct. And I feel that the comment does not suggest otherwise. Perhaps if I change the comment to something like:

// Prior to Power9, we only have an instruction that combines a store and extract
// for i32 (STXSIWX). ISA 3.0 (Power9) introduced instructions that do this for
// subword types (i8, i16). There is also no advantage to doing this for i64 on
// any subtarget.

lib/Target/PowerPC/PPCISelLowering.h

private:
     SDValue lowerToVINSERTB(ShuffleVectorSDNode *N, SelectionDAG &DAG) const;
 
     // Return whether the call instruction can potentially be optimized to a
     // tail call. This will cause the optimizers to attempt to move, or
     // duplicate return instructions to help enable tail call optimizations.
     bool mayBeEmittedAsTailCall(const CallInst *CI) const override;
     bool hasBitPreservingFPLogic(EVT VT) const override;
     bool isMaskAndCmp0FoldingBeneficial(const Instruction &AndI) const override;
+    // If the input vector will require a direct-move to extract the element
+    // but the store can be combined into PPC::STIWX, we want to combine it.
+    bool canCombineStoreAndExtract(Type *VectorTy, Value *Idx,
+                                   unsigned &Cost) const override;
   }; // end class PPCTargetLowering
 
   namespace PPC {
 
     FastISel *createFastISel(FunctionLoweringInfo &FuncInfo,
                              const TargetLibraryInfo *LibInfo);
 
   } // end namespace PPC

lib/Target/PowerPC/PPCISelLowering.cpp

bool PPCTargetLowering::mayBeEmittedAsTailCall(const CallInst *CI) const {
   if (!areCallingConvEligibleForTCO_64SVR4(Caller->getCallingConv(),
                                            CI->getCallingConv()))
       return false;
 
   // If the function is local then we have a good chance at tail-calling it
   return getTargetMachine().shouldAssumeDSOLocal(*Caller->getParent(), Callee);
 }
 
+bool PPCTargetLowering::canCombineStoreAndExtract(Type *VectorTy, Value *Idx,
+                                                  unsigned &Cost) const {
+  if (!Subtarget.hasDirectMove() || !Subtarget.hasAltivec() ||
+      !Subtarget.hasVSX())
+    return false;
+
+  // If the index is unknown at compile time, this is very expensive to lower
+  // and it is not possible to combine the store with the extract.
+  ConstantInt *CI = dyn_cast<ConstantInt>(Idx);
+  if (!CI)
+    return false;
+
+  assert(VectorTy->isVectorTy() && "VectorTy is not a vector type");
+  unsigned BitWidth = VectorTy->getScalarSizeInBits();
+
+  // Only have combined stores for sub-word types on Power9.
+  if (BitWidth > 32 || (!Subtarget.hasP9Vector() && BitWidth != 32))
+    return false;
+
+  uint64_t CIdx = CI->getZExtValue();
+  uint64_t NaturalIdx = -1UL;
+  switch (BitWidth) {
+  default: return false;
+  case 8: NaturalIdx = Subtarget.isLittleEndian() ? 8 : 7; break;
+  case 16: NaturalIdx = Subtarget.isLittleEndian() ? 4 : 3; break;
+  case 32: NaturalIdx = Subtarget.isLittleEndian() ? 2 : 1; break;
+  }
+
+  Cost = CIdx == NaturalIdx ? 0 : 3;
+  return true;
+}
+
 bool PPCTargetLowering::hasBitPreservingFPLogic(EVT VT) const {
   if (!Subtarget.hasVSX())
     return false;
   if (Subtarget.hasP9Vector() && VT == MVT::f128)
     return true;
   return VT == MVT::f32 || VT == MVT::f64 ||
     VT == MVT::v4f32 || VT == MVT::v2f64;
 }

lib/Target/PowerPC/PPCInstrVSX.td

let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
 
   // Conversions between vector and scalar single precision
   def XSCVDPSPN : XX2Form<60, 267, (outs vsrc:$XT), (ins vssrc:$XB),
                           "xscvdpspn $XT, $XB", IIC_VecFP, []>;
   def XSCVSPDPN : XX2Form<60, 331, (outs vssrc:$XT), (ins vsrc:$XB),
                           "xscvspdpn $XT, $XB", IIC_VecFP, []>;
   } // UseVSXReg = 1
 
-  let Predicates = [IsLittleEndian] in {
+  let Predicates = [HasP8Vector, IsLittleEndian] in {
   def : Pat<(f32 (PPCfcfids
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 0)))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfids
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 1)))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS
                               (f64 (COPY_TO_REGCLASS $S, VSRC)), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 0)))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 1)))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS
                               (f64 (COPY_TO_REGCLASS $S, VSRC)), VSFRC)))>;
-  }
+  def : Pat<(store (i32 (extractelt v4i32:$A, 2)), xoaddr:$src),
+            (STIWX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  def : Pat<(store (f32 (extractelt v4f32:$A, 2)), xoaddr:$src),
+            (STIWX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  foreach Idx = [ [0,2], [1,1], [3,3] ] in {
+    def : Pat<(store (i32 (extractelt v4i32:$A, !head(Idx))), xoaddr:$src),
+              (STIWX (EXTRACT_SUBREG (XXSLDWI $A, $A, !head(!tail(Idx))),
+                                     sub_64), xoaddr:$src)>;
+    def : Pat<(store (f32 (extractelt v4f32:$A, !head(Idx))), xoaddr:$src),
+              (STIWX (EXTRACT_SUBREG (XXSLDWI $A, $A, !head(!tail(Idx))),
+                                     sub_64), xoaddr:$src)>;
+  }
+  } // HasP8Vector, IsLittleEndian
+
+  let Predicates = [HasP8Vector, IsLittleEndian, NoP9Vector] in {
+  def : Pat<(store (i64 (extractelt v2i64:$A, 0)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2), sub_64), xoaddr:$src)>;
+  def : Pat<(store (f64 (extractelt v2f64:$A, 0)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2), sub_64), xoaddr:$src)>;
+  def : Pat<(store (i64 (extractelt v2i64:$A, 1)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  def : Pat<(store (f64 (extractelt v2f64:$A, 1)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  } // HasP8Vector, IsLittleEndian, NoP9Vector
 
-  let Predicates = [IsBigEndian] in {
+  let Predicates = [HasP8Vector, IsBigEndian] in {
   def : Pat<(f32 (PPCfcfids
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 0)))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS $S, VSFRC)))>;
   def : Pat<(f32 (PPCfcfids
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 1)))))),
             (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 0)))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS $S, VSFRC)))>;
   def : Pat<(f32 (PPCfcfidus
                    (f64 (PPCmtvsra (i64 (vector_extract v2i64:$S, 1)))))),
             (f32 (XSCVUXDSP (COPY_TO_REGCLASS (XXPERMDI $S, $S, 2), VSFRC)))>;
+  def : Pat<(store (i32 (extractelt v4i32:$A, 1)), xoaddr:$src),
+            (STIWX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  def : Pat<(store (f32 (extractelt v4f32:$A, 1)), xoaddr:$src),
+            (STIWX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  foreach Idx = [ [0,3], [2,1], [3,2] ] in {
+    def : Pat<(store (i32 (extractelt v4i32:$A, !head(Idx))), xoaddr:$src),
+              (STIWX (EXTRACT_SUBREG (XXSLDWI $A, $A, !head(!tail(Idx))),
+                                     sub_64), xoaddr:$src)>;
+    def : Pat<(store (f32 (extractelt v4f32:$A, !head(Idx))), xoaddr:$src),
+              (STIWX (EXTRACT_SUBREG (XXSLDWI $A, $A, !head(!tail(Idx))),
+                                     sub_64), xoaddr:$src)>;
+  }
+  } // HasP8Vector, IsBigEndian
+
+  let Predicates = [HasP8Vector, IsBigEndian, NoP9Vector] in {
+  def : Pat<(store (i64 (extractelt v2i64:$A, 0)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  def : Pat<(store (f64 (extractelt v2f64:$A, 0)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG $A, sub_64), xoaddr:$src)>;
+  def : Pat<(store (i64 (extractelt v2i64:$A, 1)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2), sub_64), xoaddr:$src)>;
+  def : Pat<(store (f64 (extractelt v2f64:$A, 1)), xoaddr:$src),
+            (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2), sub_64), xoaddr:$src)>;
   }
 
   // Instructions for converting float to i64 feeding a store.
   let Predicates = [NoP9Vector] in {
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), xoaddr:$dst, 8),
             (STXSDX (XSCVDPSXDS f64:$src), xoaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
// of all 64 VSX registers.
     (COPY_TO_REGCLASS (XFLOADf64 xaddr:$src), VSRC), 2))>;
 
       def : Pat<(v2f64 (scalar_to_vector (f64 (load ixaddr:$src)))),
                 (v2f64 (XXPERMDIs
                 (COPY_TO_REGCLASS (DFLOADf64 ixaddr:$src), VSRC), 2))>;
       def : Pat<(v2f64 (scalar_to_vector (f64 (load xaddr:$src)))),
                 (v2f64 (XXPERMDIs
                 (COPY_TO_REGCLASS (XFLOADf64 xaddr:$src), VSRC), 2))>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 0)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                        sub_64), xaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 0)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                         sub_64), xaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 1)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG $A, sub_64), xaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 1)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG $A, sub_64), xaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 0)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                            sub_64), ixaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 0)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2), sub_64),
+                            ixaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 1)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG $A, sub_64), ixaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 1)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG $A, sub_64), ixaddr:$src)>;
+
     }
 
     let Predicates = [IsBigEndian, HasP9Vector] in {
       def : Pat<(v2i64 (scalar_to_vector (i64 (load ixaddr:$src)))),
                 (v2i64 (COPY_TO_REGCLASS (DFLOADf64 ixaddr:$src), VSRC))>;
       def : Pat<(v2i64 (scalar_to_vector (i64 (load xaddr:$src)))),
                 (v2i64 (COPY_TO_REGCLASS (XFLOADf64 xaddr:$src), VSRC))>;
 
       def : Pat<(v2f64 (scalar_to_vector (f64 (load ixaddr:$src)))),
                 (v2f64 (COPY_TO_REGCLASS (DFLOADf64 ixaddr:$src), VSRC))>;
       def : Pat<(v2f64 (scalar_to_vector (f64 (load xaddr:$src)))),
                 (v2f64 (COPY_TO_REGCLASS (XFLOADf64 xaddr:$src), VSRC))>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 0)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                        sub_64), xaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 0)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                        sub_64), xaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 1)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG $A, sub_64), xaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 1)), xaddr:$src),
+                (STXSDX (EXTRACT_SUBREG $A, sub_64), xaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 0)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                        sub_64), ixaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 0)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG (XXPERMDI $A, $A, 2),
+                        sub_64), ixaddr:$src)>;
+      def : Pat<(store (i64 (extractelt v2i64:$A, 1)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG $A, sub_64), ixaddr:$src)>;
+      def : Pat<(store (f64 (extractelt v2f64:$A, 1)), ixaddr:$src),
+                (DFSTOREf64 (EXTRACT_SUBREG $A, sub_64), ixaddr:$src)>;
     }
   }
 
   let Predicates = [IsBigEndian, HasP9Vector] in {
 
     // (Un)Signed DWord vector extract -> QP
     def : Pat<(f128 (sint_to_fp (i64 (extractelt v2i64:$src, 0)))),
               (f128 (XSCVSDQP (COPY_TO_REGCLASS $src, VFRC)))>;

   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f128:$src)), xoaddr:$dst, 1),
             (STXSIBX (COPY_TO_REGCLASS (XSCVQPSWZ $src), VFRC), xoaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), xaddr:$dst, 8),
             (STXSDX (XSCVDPSXDS f64:$src), xaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), ixaddr:$dst, 8),
-            (STXSD (XSCVDPSXDS f64:$src), ixaddr:$dst)>;
+            (DFSTOREf64 (XSCVDPSXDS f64:$src), ixaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), xoaddr:$dst, 2),
             (STXSIHX (XSCVDPSXWS f64:$src), xoaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_sint_in_vsr f64:$src)), xoaddr:$dst, 1),
             (STXSIBX (XSCVDPSXWS f64:$src), xoaddr:$dst)>;
 
   // Instructions for store(fptoui).

   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_uint_in_vsr f128:$src)), xoaddr:$dst, 1),
             (STXSIBX (COPY_TO_REGCLASS (XSCVQPUWZ $src), VFRC), xoaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), xaddr:$dst, 8),
             (STXSDX (XSCVDPUXDS f64:$src), xaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), ixaddr:$dst, 8),
-            (STXSD (XSCVDPUXDS f64:$src), ixaddr:$dst)>;
+            (DFSTOREf64 (XSCVDPUXDS f64:$src), ixaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), xoaddr:$dst, 2),
             (STXSIHX (XSCVDPUXWS f64:$src), xoaddr:$dst)>;
   def : Pat<(PPCstore_scal_int_from_vsr
               (f64 (PPCcv_fp_to_uint_in_vsr f64:$src)), xoaddr:$dst, 1),
             (STXSIBX (XSCVDPUXWS f64:$src), xoaddr:$dst)>;
 
   // Round & Convert QP -> DP/SP

test/CodeGen/PowerPC/combine-extract-store.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mcpu=pwr8 -mtriple=powerpc64le-unkknown-unknown \
+; RUN:   -verify-machineinstrs -O2 < %s | FileCheck %s
+; RUN: llc -mcpu=pwr8 -mtriple=powerpc64-unkknown-unknown \
+; RUN:   -verify-machineinstrs -O2 < %s | FileCheck %s --check-prefix=CHECK-BE
+
+; Function Attrs: norecurse nounwind
+define void @test(<4 x i32>* noalias nocapture readonly %VP, <4 x i32>* noalias nocapture %VP2, i32* noalias nocapture %IP) local_unnamed_addr #0 {
+; CHECK-LABEL: test:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vspltisw 2, 4
+; CHECK-NEXT:    lvx 3, 0, 3
+; CHECK-NEXT:    vadduwm 2, 3, 2
+; CHECK-NEXT:    stxsiwx 34, 0, 5
+; CHECK-NEXT:    stvx 3, 0, 4
+; CHECK-NEXT:    blr
+entry:
+  %0 = load <4 x i32>, <4 x i32>* %VP, align 16
+  %vecext = extractelement <4 x i32> %0, i32 2
+  %add = add nsw i32 %vecext, 4
+  store i32 %add, i32* %IP, align 4
+  store <4 x i32> %0, <4 x i32>* %VP2, align 16
+  ret void
+}
+
+; Function Attrs: norecurse nounwind
+define void @testf(<4 x float>* noalias nocapture readonly %VP, <4 x float>* noalias nocapture %VP2, float* noalias nocapture %IP) local_unnamed_addr #0 {
+; CHECK-LABEL: testf:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    addis 6, 2, .LCPI1_0@toc@ha
+; CHECK-NEXT:    lvx 2, 0, 3
+; CHECK-NEXT:    addi 6, 6, .LCPI1_0@toc@l
+; CHECK-NEXT:    lvx 3, 0, 6
+; CHECK-NEXT:    stvx 2, 0, 4
+; CHECK-NEXT:    xvaddsp 0, 34, 35
+; CHECK-NEXT:    stfiwx 0, 0, 5
+; CHECK-NEXT:    blr
+entry:
+  %0 = load <4 x float>, <4 x float>* %VP, align 16
+  %vecext = extractelement <4 x float> %0, i32 2
+  %add = fadd float %vecext, 4.000000e+00
+  store float %add, float* %IP, align 4
+  store <4 x float> %0, <4 x float>* %VP2, align 16
+  ret void
+}
+
+; Function Attrs: norecurse nounwind
+define void @testBE(<4 x i32>* noalias nocapture readonly %VP, <4 x i32>* noalias nocapture %VP2, i32* noalias nocapture %IP) local_unnamed_addr #0 {
+; CHECK-BE-LABEL: testBE:
+; CHECK-BE:       # %bb.0: # %entry
+; CHECK-BE-NEXT:    vspltisw 2, 4
+; CHECK-BE-NEXT:    lxvw4x 35, 0, 3
+; CHECK-BE-NEXT:    vadduwm 2, 3, 2
+; CHECK-BE-NEXT:    stxsiwx 34, 0, 5
+; CHECK-BE-NEXT:    stxvw4x 35, 0, 4
+; CHECK-BE-NEXT:    blr
+entry:
+  %0 = load <4 x i32>, <4 x i32>* %VP, align 16
+  %vecext = extractelement <4 x i32> %0, i32 1
+  %add = add nsw i32 %vecext, 4
+  store i32 %add, i32* %IP, align 4
+  store <4 x i32> %0, <4 x i32>* %VP2, align 16
+  ret void
+}
+
+; Function Attrs: norecurse nounwind
+define void @testBEf(<4 x float>* noalias nocapture readonly %VP, <4 x float>* noalias nocapture %VP2, float* noalias nocapture %IP) local_unnamed_addr #0 {
+; CHECK-BE-LABEL: testBEf:
+; CHECK-BE:       # %bb.0: # %entry
+; CHECK-BE-NEXT:    addis 6, 2, .LCPI3_0@toc@ha
+; CHECK-BE-NEXT:    lxvw4x 0, 0, 3
+; CHECK-BE-NEXT:    addi 6, 6, .LCPI3_0@toc@l
+; CHECK-BE-NEXT:    lxvw4x 1, 0, 6
+; CHECK-BE-NEXT:    stxvw4x 0, 0, 4
+; CHECK-BE-NEXT:    xvaddsp 1, 0, 1
+; CHECK-BE-NEXT:    stfiwx 1, 0, 5
+; CHECK-BE-NEXT:    blr
+entry:
+  %0 = load <4 x float>, <4 x float>* %VP, align 16
+  %vecext = extractelement <4 x float> %0, i32 1
+  %add = fadd float %vecext, 4.000000e+00
+  store float %add, float* %IP, align 4
+  store <4 x float> %0, <4 x float>* %VP2, align 16
+  ret void
+}

test/CodeGen/PowerPC/extract-and-store.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mcpu=pwr8 -mtriple=powerpc64le-unkknown-unknown \
 ; RUN:   -ppc-asm-full-reg-names -verify-machineinstrs -O2 < %s | FileCheck %s
 ; RUN: llc -mcpu=pwr8 -mtriple=powerpc64-unkknown-unknown \
 ; RUN:   -ppc-asm-full-reg-names -verify-machineinstrs -O2 < %s | FileCheck %s \
 ; RUN:   --check-prefix=CHECK-BE
 ; RUN: llc -mcpu=pwr9 -mtriple=powerpc64le-unkknown-unknown \
 ; RUN:   -ppc-asm-full-reg-names -verify-machineinstrs -O2 < %s | FileCheck %s \
 ; RUN:   --check-prefix=CHECK-P9
 ; Function Attrs: norecurse nounwind writeonly
 define <2 x i64> @testll0(<2 x i64> returned %a, <2 x i64> %b, i64* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testll0:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xxswapd vs0, vs34
-; CHECK-NEXT:    mfvsrd r3, f0
-; CHECK-NEXT:    std r3, 24(r7)
+; CHECK-NEXT:    addi r3, r7, 24
+; CHECK-NEXT:    stxsdx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testll0:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    mfvsrd r3, vs34
-; CHECK-BE-NEXT:    std r3, 24(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 24
+; CHECK-BE-NEXT:    stxsdx vs34, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testll0:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mfvsrld r3, vs34
-; CHECK-P9-NEXT:    std r3, 24(r7)
+; CHECK-P9-NEXT:    xxswapd vs0, vs34
+; CHECK-P9-NEXT:    stfd f0, 24(r7)
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <2 x i64> %a, i32 0
   %arrayidx = getelementptr inbounds i64, i64* %ap, i64 3
   store i64 %vecext, i64* %arrayidx, align 8
   ret <2 x i64> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <2 x i64> @testll1(<2 x i64> returned %a, i64 %b, i64* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testll1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    mfvsrd r3, vs34
-; CHECK-NEXT:    std r3, 24(r6)
+; CHECK-NEXT:    addi r3, r6, 24
+; CHECK-NEXT:    stxsdx vs34, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testll1:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxswapd vs0, vs34
-; CHECK-BE-NEXT:    mfvsrd r3, f0
-; CHECK-BE-NEXT:    std r3, 24(r6)
+; CHECK-BE-NEXT:    addi r3, r6, 24
+; CHECK-BE-NEXT:    stxsdx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testll1:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mfvsrd r3, vs34
-; CHECK-P9-NEXT:    std r3, 24(r6)
+; CHECK-P9-NEXT:    stxsd v2, 24(r6)
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <2 x i64> %a, i32 1
   %arrayidx = getelementptr inbounds i64, i64* %ap, i64 3
   store i64 %vecext, i64* %arrayidx, align 8
   ret <2 x i64> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <2 x double> @testd0(<2 x double> returned %a, <2 x double> %b, double* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testd0:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xxswapd vs0, vs34
-; CHECK-NEXT:    stfd f0, 24(r7)
+; CHECK-NEXT:    addi r3, r7, 24
+; CHECK-NEXT:    stxsdx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testd0:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    addi r3, r7, 24
 ; CHECK-BE-NEXT:    stxsdx vs34, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;

 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    addi r3, r7, 24
 ; CHECK-NEXT:    stxsdx vs34, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testd1:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxswapd vs0, vs34
-; CHECK-BE-NEXT:    stfd f0, 24(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 24
+; CHECK-BE-NEXT:    stxsdx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testd1:
 ; CHECK-P9:       # %bb.0: # %entry
 ; CHECK-P9-NEXT:    stxsd v2, 24(r7)
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <2 x double> %a, i32 1
   %arrayidx = getelementptr inbounds double, double* %ap, i64 3
   store double %vecext, double* %arrayidx, align 8
   ret <2 x double> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x float> @testf0(<4 x float> returned %a, <4 x float> %b, float* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testf0:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 3
-; CHECK-NEXT:    xscvspdpn f0, vs0
-; CHECK-NEXT:    stfs f0, 12(r7)
+; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testf0:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    xscvspdpn f0, vs34
-; CHECK-BE-NEXT:    stfs f0, 12(r7)
+; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 3
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testf0:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 3
-; CHECK-P9-NEXT:    xscvspdpn f0, vs0
-; CHECK-P9-NEXT:    stfs f0, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x float> %a, i32 0
   %arrayidx = getelementptr inbounds float, float* %ap, i64 3
   store float %vecext, float* %arrayidx, align 4
   ret <4 x float> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x float> @testf1(<4 x float> returned %a, <4 x float> %b, float* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testf1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xxswapd vs0, vs34
-; CHECK-NEXT:    xscvspdpn f0, vs0
-; CHECK-NEXT:    stfs f0, 12(r7)
+; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 1
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testf1:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 1
-; CHECK-BE-NEXT:    xscvspdpn f0, vs0
-; CHECK-BE-NEXT:    stfs f0, 12(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testf1:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    xxswapd vs0, vs34
-; CHECK-P9-NEXT:    xscvspdpn f0, vs0
-; CHECK-P9-NEXT:    stfs f0, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 1
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x float> %a, i32 1
   %arrayidx = getelementptr inbounds float, float* %ap, i64 3
   store float %vecext, float* %arrayidx, align 4
   ret <4 x float> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x float> @testf2(<4 x float> returned %a, <4 x float> %b, float* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testf2:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 1
-; CHECK-NEXT:    xscvspdpn f0, vs0
-; CHECK-NEXT:    stfs f0, 12(r7)
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testf2:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    xxswapd vs0, vs34
-; CHECK-BE-NEXT:    xscvspdpn f0, vs0
-; CHECK-BE-NEXT:    stfs f0, 12(r7)
+; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 1
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testf2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 1
-; CHECK-P9-NEXT:    xscvspdpn f0, vs0
-; CHECK-P9-NEXT:    stfs f0, 12(r7)
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x float> %a, i32 2
   %arrayidx = getelementptr inbounds float, float* %ap, i64 3
   store float %vecext, float* %arrayidx, align 4
   ret <4 x float> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x float> @testf3(<4 x float> returned %a, <4 x float> %b, float* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testf3:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xscvspdpn f0, vs34
-; CHECK-NEXT:    stfs f0, 12(r7)
+; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 3
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testf3:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 3
-; CHECK-BE-NEXT:    xscvspdpn f0, vs0
-; CHECK-BE-NEXT:    stfs f0, 12(r7)
+; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testf3:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    xscvspdpn f0, vs34
-; CHECK-P9-NEXT:    stfs f0, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 3
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x float> %a, i32 3
   %arrayidx = getelementptr inbounds float, float* %ap, i64 3
   store float %vecext, float* %arrayidx, align 4
   ret <4 x float> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x i32> @testi0(<4 x i32> returned %a, <4 x i32> %b, i32* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testi0:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xxswapd vs0, vs34
-; CHECK-NEXT:    mfvsrwz r3, f0
-; CHECK-NEXT:    stw r3, 12(r7)
+; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testi0:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 3
-; CHECK-BE-NEXT:    mfvsrwz r3, f0
-; CHECK-BE-NEXT:    stw r3, 12(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi0:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    li r3, 0
-; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
-; CHECK-P9-NEXT:    stw r3, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x i32> %a, i32 0
   %arrayidx = getelementptr inbounds i32, i32* %ap, i64 3
   store i32 %vecext, i32* %arrayidx, align 4
   ret <4 x i32> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x i32> @testi1(<4 x i32> returned %a, <4 x i32> %b, i32* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testi1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 1
-; CHECK-NEXT:    mfvsrwz r3, f0
-; CHECK-NEXT:    stw r3, 12(r7)
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testi1:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    mfvsrwz r3, vs34
-; CHECK-BE-NEXT:    stw r3, 12(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi1:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    li r3, 4
-; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
-; CHECK-P9-NEXT:    stw r3, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 1
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x i32> %a, i32 1
   %arrayidx = getelementptr inbounds i32, i32* %ap, i64 3
   store i32 %vecext, i32* %arrayidx, align 4
   ret <4 x i32> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x i32> @testi2(<4 x i32> returned %a, <4 x i32> %b, i32* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testi2:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    mfvsrwz r3, vs34
-; CHECK-NEXT:    stw r3, 12(r7)
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testi2:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 1
-; CHECK-BE-NEXT:    mfvsrwz r3, f0
-; CHECK-BE-NEXT:    stw r3, 12(r7)
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi2:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    mfvsrwz r3, vs34
-; CHECK-P9-NEXT:    stw r3, 12(r7)
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stxsiwx vs34, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x i32> %a, i32 2
   %arrayidx = getelementptr inbounds i32, i32* %ap, i64 3
   store i32 %vecext, i32* %arrayidx, align 4
   ret <4 x i32> %a
 }
 
 ; Function Attrs: norecurse nounwind writeonly
 define <4 x i32> @testi3(<4 x i32> returned %a, <4 x i32> %b, i32* nocapture %ap) local_unnamed_addr #0 {
 ; CHECK-LABEL: testi3:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xxsldwi vs0, vs34, vs34, 3
-; CHECK-NEXT:    mfvsrwz r3, f0
-; CHECK-NEXT:    stw r3, 12(r7)
+; CHECK-NEXT:    addi r3, r7, 12
+; CHECK-NEXT:    stfiwx f0, 0, r3
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: testi3:
 ; CHECK-BE:       # %bb.0: # %entry
-; CHECK-BE-NEXT:    xxswapd vs0, vs34
-; CHECK-BE-NEXT:    mfvsrwz r3, f0
-; CHECK-BE-NEXT:    stw r3, 12(r7)
+; CHECK-BE-NEXT:    xxsldwi vs0, vs34, vs34, 2
+; CHECK-BE-NEXT:    addi r3, r7, 12
+; CHECK-BE-NEXT:    stfiwx f0, 0, r3
 ; CHECK-BE-NEXT:    blr
 ;
 ; CHECK-P9-LABEL: testi3:
 ; CHECK-P9:       # %bb.0: # %entry
-; CHECK-P9-NEXT:    li r3, 12
-; CHECK-P9-NEXT:    vextuwrx r3, r3, v2
-; CHECK-P9-NEXT:    stw r3, 12(r7)
+; CHECK-P9-NEXT:    xxsldwi vs0, vs34, vs34, 3
+; CHECK-P9-NEXT:    addi r3, r7, 12
+; CHECK-P9-NEXT:    stfiwx f0, 0, r3
 ; CHECK-P9-NEXT:    blr
 entry:
   %vecext = extractelement <4 x i32> %a, i32 3
   %arrayidx = getelementptr inbounds i32, i32* %ap, i64 3
   store i32 %vecext, i32* %arrayidx, align 4
   ret <4 x i32> %a
 }

test/CodeGen/PowerPC/scalar_vector_test_2.ll

 define void @test_liwzx1(<1 x float>* %A, <1 x float>* %B, <1 x float>* %C) {
 ; P9LE-LABEL: test_liwzx1:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    lfiwzx f0, 0, r3
 ; P9LE-NEXT:    lfiwzx f1, 0, r4
 ; P9LE-NEXT:    xxpermdi vs0, f0, f0, 2
 ; P9LE-NEXT:    xxpermdi vs1, f1, f1, 2
 ; P9LE-NEXT:    xvaddsp vs0, vs0, vs1
-; P9LE-NEXT:    xxsldwi vs0, vs0, vs0, 3
-; P9LE-NEXT:    xscvspdpn f0, vs0
-; P9LE-NEXT:    stfs f0, 0(r5)
+; P9LE-NEXT:    xxsldwi vs0, vs0, vs0, 2
+; P9LE-NEXT:    stfiwx f0, 0, r5
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: test_liwzx1:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    lfiwzx f0, 0, r3
 ; P9BE-NEXT:    lfiwzx f1, 0, r4
 ; P9BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P9BE-NEXT:    xxsldwi vs1, f1, f1, 1
 ; P9BE-NEXT:    xvaddsp vs0, vs0, vs1
-; P9BE-NEXT:    xscvspdpn f0, vs0
-; P9BE-NEXT:    stfs f0, 0(r5)
+; P9BE-NEXT:    xxsldwi vs0, vs0, vs0, 3
+; P9BE-NEXT:    stfiwx f0, 0, r5
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: test_liwzx1:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    lfiwzx f0, 0, r3
 ; P8LE-NEXT:    lfiwzx f1, 0, r4
 ; P8LE-NEXT:    xxpermdi vs0, f0, f0, 2
 ; P8LE-NEXT:    xxpermdi vs1, f1, f1, 2
 ; P8LE-NEXT:    xvaddsp vs0, vs0, vs1
-; P8LE-NEXT:    xxsldwi vs0, vs0, vs0, 3
-; P8LE-NEXT:    xscvspdpn f0, vs0
-; P8LE-NEXT:    stfsx f0, 0, r5
+; P8LE-NEXT:    xxsldwi vs0, vs0, vs0, 2
+; P8LE-NEXT:    stfiwx f0, 0, r5
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: test_liwzx1:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    lfiwzx f1, 0, r4
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE-NEXT:    xxsldwi vs1, f1, f1, 1
 ; P8BE-NEXT:    xvaddsp vs0, vs0, vs1
-; P8BE-NEXT:    xscvspdpn f0, vs0
-; P8BE-NEXT:    stfsx f0, 0, r5
+; P8BE-NEXT:    xxsldwi vs0, vs0, vs0, 3
+; P8BE-NEXT:    stfiwx f0, 0, r5
 ; P8BE-NEXT:    blr
   %a = load <1 x float>, <1 x float>* %A
   %b = load <1 x float>, <1 x float>* %B
   %X = fadd <1 x float> %a, %b
   store <1 x float> %X, <1 x float>* %C
   ret void
 }
 
 define <1 x float>* @test_liwzx2(<1 x float>* %A, <1 x float>* %B, <1 x float>* %C) {
 ; P9LE-LABEL: test_liwzx2:
 ; P9LE:       # %bb.0:
 ; P9LE-NEXT:    lfiwzx f0, 0, r3
 ; P9LE-NEXT:    lfiwzx f1, 0, r4
 ; P9LE-NEXT:    mr r3, r5
 ; P9LE-NEXT:    xxpermdi vs0, f0, f0, 2
 ; P9LE-NEXT:    xxpermdi vs1, f1, f1, 2
 ; P9LE-NEXT:    xvsubsp vs0, vs0, vs1
-; P9LE-NEXT:    xxsldwi vs0, vs0, vs0, 3
-; P9LE-NEXT:    xscvspdpn f0, vs0
-; P9LE-NEXT:    stfs f0, 0(r5)
+; P9LE-NEXT:    xxsldwi vs0, vs0, vs0, 2
+; P9LE-NEXT:    stfiwx f0, 0, r5
 ; P9LE-NEXT:    blr
 
 ; P9BE-LABEL: test_liwzx2:
 ; P9BE:       # %bb.0:
 ; P9BE-NEXT:    lfiwzx f0, 0, r3
 ; P9BE-NEXT:    lfiwzx f1, 0, r4
 ; P9BE-NEXT:    mr r3, r5
 ; P9BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P9BE-NEXT:    xxsldwi vs1, f1, f1, 1
 ; P9BE-NEXT:    xvsubsp vs0, vs0, vs1
-; P9BE-NEXT:    xscvspdpn f0, vs0
-; P9BE-NEXT:    stfs f0, 0(r5)
+; P9BE-NEXT:    xxsldwi vs0, vs0, vs0, 3
+; P9BE-NEXT:    stfiwx f0, 0, r5
 ; P9BE-NEXT:    blr
 
 ; P8LE-LABEL: test_liwzx2:
 ; P8LE:       # %bb.0:
 ; P8LE-NEXT:    lfiwzx f0, 0, r3
 ; P8LE-NEXT:    lfiwzx f1, 0, r4
 ; P8LE-NEXT:    mr r3, r5
 ; P8LE-NEXT:    xxpermdi vs0, f0, f0, 2
 ; P8LE-NEXT:    xxpermdi vs1, f1, f1, 2
 ; P8LE-NEXT:    xvsubsp vs0, vs0, vs1
-; P8LE-NEXT:    xxsldwi vs0, vs0, vs0, 3
-; P8LE-NEXT:    xscvspdpn f0, vs0
-; P8LE-NEXT:    stfsx f0, 0, r5
+; P8LE-NEXT:    xxsldwi vs0, vs0, vs0, 2
+; P8LE-NEXT:    stfiwx f0, 0, r5
 ; P8LE-NEXT:    blr
 
 ; P8BE-LABEL: test_liwzx2:
 ; P8BE:       # %bb.0:
 ; P8BE-NEXT:    lfiwzx f0, 0, r3
 ; P8BE-NEXT:    lfiwzx f1, 0, r4
 ; P8BE-NEXT:    mr r3, r5
 ; P8BE-NEXT:    xxsldwi vs0, f0, f0, 1
 ; P8BE-NEXT:    xxsldwi vs1, f1, f1, 1
 ; P8BE-NEXT:    xvsubsp vs0, vs0, vs1
-; P8BE-NEXT:    xscvspdpn f0, vs0
-; P8BE-NEXT:    stfsx f0, 0, r5
+; P8BE-NEXT:    xxsldwi vs0, vs0, vs0, 3
+; P8BE-NEXT:    stfiwx f0, 0, r5
 ; P8BE-NEXT:    blr
   %a = load <1 x float>, <1 x float>* %A
   %b = load <1 x float>, <1 x float>* %B
   %X = fsub <1 x float> %a, %b
   store <1 x float> %X, <1 x float>* %C
   ret <1 x float>* %C
 }

test/CodeGen/PowerPC/store_fptoi.ll

entry:
   %0 = load double, double* %a, align 8
   %conv = fptosi double %0 to i64
   store i64 %conv, i64* %b, align 8
   ret void
 
 ; CHECK-LABEL: dpConv2sdw
 ; CHECK: lfd [[LD:[0-9]+]], 0(3)
 ; CHECK: xscvdpsxds [[CONV:[0-9]+]], [[LD]]
-; CHECK-NEXT: stxsd [[CONV]], 0(4)
+; CHECK-NEXT: stfd [[CONV]], 0(4)
 ; CHECK-NEXT: blr
 
 ; CHECK-PWR8-LABEL: dpConv2sdw
 ; CHECK-PWR8: lfdx [[LD:[0-9]+]], 0, 3
 ; CHECK-PWR8-NEXT: xscvdpsxds [[CONV:[0-9]+]], [[LD]]
 ; CHECK-PWR8-NEXT: stxsdx [[CONV]], 0, 4
 ; CHECK-PWR8-NEXT: blr
 }
entry:
   %0 = load float, float* %a, align 4
   %conv = fptosi float %0 to i64
   store i64 %conv, i64* %b, align 8
   ret void
 
 ; CHECK-LABEL: spConv2sdw
 ; CHECK: lfs [[LD:[0-9]+]], 0(3)
 ; CHECK-NEXT: xscvdpsxds [[CONV:[0-9]+]], [[LD]]
-; CHECK-NEXT: stxsd [[CONV]], 0(4)
+; CHECK-NEXT: stfd [[CONV]], 0(4)
 ; CHECK-NEXT: blr
 
 ; CHECK-PWR8-LABEL: spConv2sdw
 ; CHECK-PWR8: lfsx [[LD:[0-9]+]], 0, 3
 ; CHECK-PWR8-NEXT: xscvdpsxds [[CONV:[0-9]+]], [[LD]]
 ; CHECK-PWR8-NEXT: stxsdx [[CONV]], 0, 4
 ; CHECK-PWR8-NEXT: blr
 }
entry:
   %0 = load double, double* %a, align 8
   %conv = fptoui double %0 to i64
   store i64 %conv, i64* %b, align 8
   ret void
 
 ; CHECK-LABEL: dpConv2udw
 ; CHECK: lfd [[LD:[0-9]+]], 0(3)
 ; CHECK: xscvdpuxds [[CONV:[0-9]+]], [[LD]]
-; CHECK-NEXT: stxsd [[CONV]], 0(4)
+; CHECK-NEXT: stfd [[CONV]], 0(4)
 ; CHECK-NEXT: blr
 
 ; CHECK-PWR8-LABEL: dpConv2udw
 ; CHECK-PWR8: lfdx [[LD:[0-9]+]], 0, 3
 ; CHECK-PWR8-NEXT: xscvdpuxds [[CONV:[0-9]+]], [[LD]]
 ; CHECK-PWR8-NEXT: stxsdx [[CONV]], 0, 4
 ; CHECK-PWR8-NEXT: blr
 }
entry:
   %0 = load float, float* %a, align 4
   %conv = fptoui float %0 to i64
   store i64 %conv, i64* %b, align 8
   ret void
 
 ; CHECK-LABEL: spConv2udw
 ; CHECK: lfs [[LD:[0-9]+]], 0(3)
 ; CHECK-NEXT: xscvdpuxds [[CONV:[0-9]+]], [[LD]]
-; CHECK-NEXT: stxsd [[CONV]], 0(4)
+; CHECK-NEXT: stfd [[CONV]], 0(4)
 ; CHECK-NEXT: blr
 
 ; CHECK-PWR8-LABEL: spConv2udw
 ; CHECK-PWR8: lfsx [[LD:[0-9]+]], 0, 3
 ; CHECK-PWR8-NEXT: xscvdpuxds [[CONV:[0-9]+]], [[LD]]
 ; CHECK-PWR8-NEXT: stxsdx [[CONV]], 0, 4
 ; CHECK-PWR8-NEXT: blr
 }