[SelectionDAG][PowerPC] Memset reuse vector element for tail store

[tingwang]

Summary

On PPC there are instructions to store element from vector(e.g. stxsdx/stxsiwx), and these instructions can be leveraged to avoid tail constant in memset and constant splat array initialization.

This patch tries to explore these opportunities.

[nemanjai]

I think this is useful, but we should ensure we can get rid of the swap that this introduces (in a separate patch).

llvm/include/llvm/CodeGen/TargetLowering.h

672 ↗

(On Diff #478424)

Do we need this? Can canCombineStoreAndExtract() suffice for this purpose?

llvm/test/CodeGen/PowerPC/memset-tail.ll

195

Why do we now get the redundant swap for the vector store that we didn't get before? Was it eliminated by the swap elimination before and now it is not because we have a use of the partial vector?

[nemanjai]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

7277 ↗

(On Diff #478424)

Can we reduce the nesting here by converting this to else if?

[tingwang]

Update according to comments:
(1) Use existing canCombineStoreAndExtract() instead of creating new.
(2) Nest the else statement properly.
(3) Saw two cases changed due to (1).

[tingwang]

llvm/include/llvm/CodeGen/TargetLowering.h

672 ↗

(On Diff #478424)

Thank you! I did a try and updated patch. Saw two cases changed. I will look into the detail tomorrow.

llvm/test/CodeGen/PowerPC/memset-tail.ll

195

Debug-only ppc-vsx-swaps shows "Web 0 rejected for physreg, partial reg, or not swap[pable]". I will look into it and probably post another patch to fix the issue. Thank you!

[tingwang]

Realized I need to use PPCTTIImpl::getVectorInstrCost() API to determine the cost of instructions. I'm working on it now.

[tingwang]

Changes in this update:
(1) I was trying to use TTI.getVectorInstrCost() to query instruction cost in PPCTargetLowering::canCombineStoreAndExtract(). However not able to reach TTI, and didn't find any reference to do that in SDAG. Given that the original implementation of canCombineStoreAndExtract() on ARM implemented its own logic to calculate Cost, followed the approach and implemented logic by referring to PPCTTIImpl::getVectorInstrCost().

(2) Refactored logic inside getMemsetStores(). For PPC CombineStoreAndExtract is beneficial on some specific element index according to endianness and instruction (StoreAndExtract elements on other indexes requires vector permutation, which makes the whole idea less attractive). Since this is platform independent logic, I'm querying the cost for indexes, and pick the least cost to do the combine.

[tingwang]

llvm/test/CodeGen/PowerPC/p10-fi-elim.ll

31 ↗

(On Diff #478808)

Instruction sequence change in PowerPC/p10-fi-elim.ll is result of CodeGenPrepare::optimizeExtractElementInst() now can generate combined pattern since we enabled canCombineStoreAndExtract(). Seems we can avoid two mfvsrd instructions.

[tingwang]

llvm/test/CodeGen/ARM/memset-align.ll

21 ↗

(On Diff #478808)

Hello @dmgreen @asavonic. This patch tries to reuse vector element for the tail store in memset by implementing canCombineStoreAndExtract() on PPC. This changed introduced test case change on ARM in llvm/test/CodeGen/ARM/memset-align.ll. Could you please help me check if the change looks good or not? Thank you!

Looked into the scenario on ARM, if the i8 fill value of memset is zero, it creates vector for the initial 16B, and constant tail for the remaining bytes, which exactly hit this patch's scenario. For other values, it creates i32 for memset and will not be impacted by this patch.

[tingwang]

Add memset-tail.ll changes.

[lkail]

I would expect not only memset, some consecutive stores could also reuse the result of vector split, see https://godbolt.org/z/77aMvncb4.
For

void foo(long a[3]) {
    a[0] = 12;
    a[1] = 12;
    a[2] = 12;
}

foo(long*):                               # @foo(long*)
        .quad   .Lfunc_begin0
        .quad   .TOC.@tocbase
        .quad   0
.Lfunc_begin0:
        xxlxor 0, 0, 0
        li 4, 12
        xxsplti32dx 0, 1, 12
        std 4, 16(3)
        stxv 0, 0(3)
        blr
        .long   0
        .quad   0

We don't reuse the result of xxsplti32dx.

[tingwang]

In D138883#3962255, @lkail wrote:

I would expect not only memset, some consecutive stores could also reuse the result of vector split, see https://godbolt.org/z/77aMvncb4.
For

void foo(long a[3]) {
    a[0] = 12;
    a[1] = 12;
    a[2] = 12;
}

foo(long*):                               # @foo(long*)
        .quad   .Lfunc_begin0
        .quad   .TOC.@tocbase
        .quad   0
.Lfunc_begin0:
        xxlxor 0, 0, 0
        li 4, 12
        xxsplti32dx 0, 1, 12
        std 4, 16(3)
        stxv 0, 0(3)
        blr
        .long   0
        .quad   0

We don't reuse the result of xxsplti32dx.

Sure. The posted IR could be handled by DAGCombiner::mergeConsecutiveStores(), and I agree similar combine can be applied there. But for this case, memset stores are volatile, and DAGCombiner::getStoreMergeCandidates() does not accept volatile store currently.

[asavonic]

llvm/test/CodeGen/ARM/memset-align.ll

21 ↗

(On Diff #478808)

This looks fine to me. VST1 and scalar STR seem equivalent in this case, if I'm reading the docs right.

[tingwang]

llvm/test/CodeGen/ARM/memset-align.ll

21 ↗

(On Diff #478808)

This looks fine to me. VST1 and scalar STR seem equivalent in this case, if I'm reading the docs right.

Thank you for the confirm!

[tingwang]

Test case update.

[tingwang]

In D138883#3955904, @nemanjai wrote:

I think this is useful, but we should ensure we can get rid of the swap that this introduces (in a separate patch).

According to my test case, there are two kinds of swap in LE: (1) swap after vector splat immediate; (2) swap after load from constant-pool. Submitted two patches D139193 and D139491 to address them separately.

[tingwang]

Gentle ping.

[tingwang]

Update patch as following test case pattern changed:
llvm/test/CodeGen/ARM/memset-align.ll

[tingwang]

Gentle ping.

[tingwang]

Gentle ping.

[tingwang]

Rebase && Gentle ping.

[nemanjai]

While I am not principally against this approach, it doesn't really give me a great feeling going in this direction. The issue is more widespread than just memset/memcpy/memmove as Kai's example illustrates.
I wonder if it would be a more complete solution to add a DAG combine that looks up the chain from the store to see if a store of a splat of the same value exists. That should certainly cover both examples.

[tingwang]

I will try to get similar results by DAG combine. Thanks to Nemanja and Kai's insight!

[tingwang]

Redo the implementation, and now both memset and constant splat array initialization get changed.

[tingwang]

Plan to continue improve the patch...

[tingwang]

(1) Format code to follow coding style guidance.
(2) Fix SplatValue check.
(3) Remaining redundant instructions like mtfprd will be fixed in separate patches.

[tingwang]

(1) Update element type ElemTy which now matches the type expected by both STFIWX and STXSIX PPCISD nodes.
(2) Add missing match pattern for PPCstxsix.

[tingwang]

Update StoreSizeInBits check to skip on PowerOf2 bit size less than 8.

[tingwang]

Attempt to push the logic into DAGCombiner::mergeConsecutiveStores()...

[nemanjai]

Sorry, I had unsubmitted comments. Not sure if they still apply.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15034

Nit: you don't need the name of the function here.

15048

What if dyn_cast returns null (i.e. if operand 1 is not a constant)?

15057–15059

We don't need to construct an APInt just to check whether it is a power of 2. You can just use isPowerOf2_64() from MathExtras.h.

[tingwang]

In D138883#4181453, @nemanjai wrote:

Sorry, I had unsubmitted comments. Not sure if they still apply.

Hi Nemanja,

Appreciate your help! I planned to change due to the reason that DAGCombiner::getStoreMergeCandidates() already walks through chain of stores, and I realized that it could be a better place to find candidate for this opportunity. By the way, maybe the criteria of splat of constant could be relieved to just match the subsection that is extracted by the target store, and I would like to have a try.

I hope next version will be final for review. Thank you again for taking time looking into this!

[tingwang]

Since canCombineStoreAndExtract() target hook looks not good on PPC, reconsider use this approach to do the combine.

Change in this version:
(1) Address comments from previous review.
(2) Add check to make sure do not combine on truncated stores or those stores that return value.

[tingwang]

Gentle ping. Since the alternative path (D146602/D146610) looks not good, shall we take this approach forward? Any comments are welcome. Thank you!

[tingwang]

Minor update:
(1) Add bitwidth is multiple of check for isSplat() call.
(2) Reduce MaxSearchNodes from 4 to 3, this is the minimum setting to allow target patterns in test cases.

And ping...

[tingwang]

Rebase and added some comments.

Hi @nemanjai, I accepted and addressed your previous comments. Do you have any more concerns on the approach that is implemented here? Thank you!

[nemanjai]

In D138883#4479380, @tingwang wrote:

Rebase and added some comments.

Hi @nemanjai, I accepted and addressed your previous comments. Do you have any more concerns on the approach that is implemented here? Thank you!

I am really sorry about the delay...
While I am not completely opposed to this, it seems like a fair bit of machinery to add for something that we could solve more simply with unaligned stores (i.e. the same way we would codegen a memset where the tail is *not* a power of 2).

I don't think that

xxspltib 0, 165
li 4, 16
stxsibx 0, 3, 4
stxv 0, 0(3)

is any better than

xxspltib 0, 165
li 4, 1
stxvx 0, 3, 4
stxv 0, 0(3)

Is it possible to do something like that without walking the chain and with existing capabilities?

[tingwang]

Hi @nemanjai, appreciate your time looking into this patch. Thank you!

I agree with you, and I think walking the chain is burning CPU cycles without achieving anything. I realized it is difficult for me to take both targets (the original memset case, and the one @lkail mentioned) in this patch, so I would like to drop the second target, in order to focus on the first one.

I like the idea to use unaligned store, and quickly tested that to see if any potential issue. Created memset.c with multiple memset(p, 0xXY, 24); lines to stress the performance. According to my numbers from Power10, use extract-and-store (https://reviews.llvm.org/D138883?id=493461) got 17% faster than baseline, whereas unaligned store got about 30% slower than baseline.

From performance perspective, I think I should pursuit the original approach. However since canCombineStoreAndExtract target hook has been proved not beneficial (https://reviews.llvm.org/D146602) on PPC, I probably need to create one for PPC only at this moment.

Let me know if any comments. I will post patch shortly.

memset.c92 KBDownload

[tingwang]

Return to the initial proposal after exploring different approaches. Since canCombineStoreAndExtract() is not beneficial to PPC, created another filter for PPC.

[tingwang]

llvm/test/CodeGen/PowerPC/memset-tail.ll

195

Will be eliminated by https://reviews.llvm.org/D139193.

237

Plan to address this pattern in separate patch.

245

Will be eliminated by https://reviews.llvm.org/D139193.

260

Plan to address this pattern in separate patch.

297

Will be eliminated by https://reviews.llvm.org/D139193.

351

Will be eliminated by https://reviews.llvm.org/D139193.

713

Plan to address this pattern in separate patch.

736

Plan to address this pattern in separate patch.

[tingwang]

(1) Rebase after commit D139193.
(2) Add two P10 patterns to match extract-and-store.

Now test case is clean. Gentle ping.

[tingwang]

Rebase && Ping.

[tingwang]

Gentle ping...

[tingwang]

Gentle ping...

[shchenz]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

17151

If using stfd is allowed for tail size 5/6/7, then can we use stfd for tail size 3/4 too? (I assume the change here impacts cases memsetTailV1B3 and memsetTailV1B4?)

llvm/test/CodeGen/PowerPC/memset-tail.ll

244

This seems a legacy issue because I also found same issue in case memsetTailV1B12 and also from the left side of this case.

Is it safe to extend the store length from 23 bytes to 32(or 24) bytes here? There is no clue saying that memory after (char *)p + 7 is writable by the user?

The related logic is in allowsMisalignedMemoryAccesses(). But is it correct that we can safely assume this memset can write more memory even this memset handles aligned memory?

What do you think? @nemanjai

[shchenz]

llvm/test/CodeGen/PowerPC/memset-tail.ll

244

Sorry, please ignore this comment. I didn't realize that the two stores stxsdx and stxvd2x have overlaps. So the real write size is not extended.

[tingwang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

17151

It seems TargetLowering::findOptimalMemOpLowering() decides the type of each store. I guess if we change the type for the size 3/4 case from i32 to i64, then it will result in stfd.

[shchenz]

This LGTM with some nits.

Let's first target for the memset cases as this is the common case where splat values happens.

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

7644 ↗

(On Diff #546725)

Nit: this comment needs update?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1637

nit: We may need comments here why we don't try to extract constant for ElemSizeInBits 8/16. (I guess the reason is we don't have benefit as we need li to load the index and this li can also be used to load the 8/16 bit imm?

17151

Thanks. Better to add some comment here why we need to set the type to MVT::v8i16

[tingwang]

In D138883#4639161, @shchenz wrote:

This LGTM with some nits.

Let's first target for the memset cases as this is the common case where splat values happens.

Thank you! I will address the remaining comments in the commit.

llvm/lib/Target/PowerPC/PPCISelLowering.h

public:
     /// type with new values built out of custom code.
     ///
     void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue>&Results,
                             SelectionDAG &DAG) const override;
 
     SDValue expandVSXLoadForLE(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue expandVSXStoreForLE(SDNode *N, DAGCombinerInfo &DCI) const;
 
+    SDValue findAndReuseSplatForConst(SDNode *N, DAGCombinerInfo &DCI) const;
+
     SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
     SDValue BuildSDIVPow2(SDNode *N, const APInt &Divisor, SelectionDAG &DAG,
                           SmallVectorImpl<SDNode *> &Created) const override;
 
     Register getRegisterByName(const char* RegName, LLT VT,
                                const MachineFunction &MF) const override;
 

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

 }
 
 const char *PPCTargetLowering::getTargetNodeName(unsigned Opcode) const {
   switch ((PPCISD::NodeType)Opcode) {
   case PPCISD::FIRST_NUMBER:    break;
   case PPCISD::FSEL:            return "PPCISD::FSEL";
   case PPCISD::XSMAXC:          return "PPCISD::XSMAXC";
   case PPCISD::XSMINC:          return "PPCISD::XSMINC";
   case PPCISD::FCFID:           return "PPCISD::FCFID";
   case PPCISD::FCFIDU:          return "PPCISD::FCFIDU";
   case PPCISD::FCFIDS:          return "PPCISD::FCFIDS";
   case PPCISD::FCFIDUS:         return "PPCISD::FCFIDUS";
   case PPCISD::FCTIDZ:          return "PPCISD::FCTIDZ";
   case PPCISD::FCTIWZ:          return "PPCISD::FCTIWZ";
   case PPCISD::FCTIDUZ:         return "PPCISD::FCTIDUZ";
   case PPCISD::FCTIWUZ:         return "PPCISD::FCTIWUZ";
   case PPCISD::FP_TO_UINT_IN_VSR:
SDValue PPCTargetLowering::expandVSXStoreForLE(SDNode *N,
   SDValue StoreOps[] = { Chain, Swap, Base };
   SDValue Store = DAG.getMemIntrinsicNode(PPCISD::STXVD2X, dl,
                                           DAG.getVTList(MVT::Other),
                                           StoreOps, VecTy, MMO);
   DCI.AddToWorklist(Store.getNode());
   return Store;
 }
 
+// Search for opportunity that store constant int/fp shares the same chain with
+// another store constant vector, which is a splat of first store's constant.
+// Convert the first store into store vector element.
+SDValue
+PPCTargetLowering::findAndReuseSplatForConst(SDNode *N,
+                                             DAGCombinerInfo &DCI) const {
+  StoreSDNode *ST = cast<StoreSDNode>(N);
+  // Truncated stores are likely simple constants. Stores that return values are
+  // out-of-scope. Do not bother reuse.
+  if (ST->isTruncatingStore() || ST->getNumValues() != 1)
+    return SDValue();
+
+  EVT StoreVT = ST->getValue().getValueType();
+  if (!StoreVT.isSimple() || StoreVT.isVector())
+    return SDValue();
+  if (!(StoreVT.isScalarInteger() || StoreVT.isFloatingPoint()))
+    return SDValue();
+  // Expect Byte/Halfword/Word/Doubleword size, and check for supported
+  // subtarget.
+  auto StoreSizeInBits = ST->getMemoryVT().getSizeInBits();
+  if (!llvm::isPowerOf2_64(StoreSizeInBits) || StoreSizeInBits > 64 ||
+      StoreSizeInBits < 8)
+    return SDValue();
+  if (StoreSizeInBits == 32 && !Subtarget.hasP8Vector())
+    return SDValue();
+  if ((StoreSizeInBits == 16 || StoreSizeInBits == 8) &&
+      !Subtarget.hasP9Vector())
+    return SDValue();
+
+  APInt ConstVal;
+  if (auto *Const = dyn_cast<ConstantSDNode>(N->getOperand(1))) {
+    ConstVal = Const->getAPIntValue();
+  } else if (auto *Const = dyn_cast<ConstantFPSDNode>(N->getOperand(1))) {
+    ConstVal = Const->getValueAPF().bitcastToAPInt();
+  } else
+    return SDValue();
+  if (ConstVal.isZero() || ConstVal.isAllOnes())
+    return SDValue();
+  if (ConstVal.getBitWidth() != StoreSizeInBits)
+    return SDValue();
+
+  SelectionDAG &DAG = DCI.DAG;
+  SDLoc dl(N);
+
+  SDValue Chain = ST->getChain();
+  SDValue Base = ST->getBasePtr();
+  MachineMemOperand *MMO = ST->getMemOperand();
+
+  unsigned NumNodesExplored = 0;
+  const unsigned MaxSearchNodes = 4;
+  SDNode *RootNode = Chain.getNode();
+  for (auto I = RootNode->use_begin(), E = RootNode->use_end();
+       I != E && NumNodesExplored < MaxSearchNodes; ++I, ++NumNodesExplored) {
+    if (*I == N)
+      continue;
+
+    auto *OtherStore = dyn_cast<StoreSDNode>(*I);
+    if (!OtherStore)
+      continue;
+
+    SDValue OtherStoredVal = peekThroughBitcasts(OtherStore->getValue());
+    APInt SplatVal;
+    if (!isTypeLegal(OtherStoredVal.getValueType()) ||
+        !ISD::isConstantSplatVector(OtherStoredVal.getNode(), SplatVal))
+      continue;
+
+    // Val should be a splat of SubVal to allow reuse constant from vector.
+    APInt SubVal =
+        (SplatVal.getBitWidth() < ConstVal.getBitWidth()) ? SplatVal : ConstVal;
+    APInt Val =
+        (SplatVal.getBitWidth() < ConstVal.getBitWidth()) ? ConstVal : SplatVal;
+    if (Val.trunc(SubVal.getBitWidth()) != SubVal)
+      continue;
+
+    if (!Val.isSplat(SubVal.getBitWidth()))
+      continue;
+
+    // Element type shall be f64 as expected by STXSIX and STFIWX, and should be
+    // reversed for extract_vector_elt from v2*64 types to allow
+    // extract_vector_elt node.
+    EVT ElemTy = EVT::getFloatingPointVT(64);
+    if (StoreSizeInBits == 64 && StoreVT.isFloatingPoint())
+      ElemTy = EVT::getIntegerVT(*DAG.getContext(), 64);
+
+    EVT VecTy = EVT::getVectorVT(*DAG.getContext(), ElemTy, 2);
+    SDValue Tmp1 = DAG.getNode(ISD::BITCAST, dl, VecTy, OtherStore->getValue());
+    if (Tmp1.getOpcode() != ISD::BITCAST)
+      continue;
+
+    unsigned ElemIdx = Subtarget.isLittleEndian() ? 1 : 0;
+    SDValue Tmp2 = DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, ElemTy, Tmp1,
+                               DAG.getVectorIdxConstant(ElemIdx, dl));
+    if (Tmp2.getOpcode() != ISD::EXTRACT_VECTOR_ELT)
+      continue;
+
+    SDValue Store;
+    if (StoreSizeInBits == 32) {
+      SDValue StoreOps[] = {Chain, Tmp2, Base};
+      Store =
+          DAG.getMemIntrinsicNode(PPCISD::STFIWX, dl, DAG.getVTList(MVT::Other),
+                                  StoreOps, StoreVT, MMO);
+    } else if (StoreSizeInBits == 64) {
+      Store = DAG.getStore(Chain, dl, Tmp2, Base, MMO);
+    } else {
+      SDValue StoreOps[] = {
+          Chain, Tmp2, Base,
+          DAG.getConstant((StoreSizeInBits / 8), dl, MVT::i32)};
+      Store =
+          DAG.getMemIntrinsicNode(PPCISD::STXSIX, dl, DAG.getVTList(MVT::Other),
+                                  StoreOps, StoreVT, MMO);
+    }
+    DCI.AddToWorklist(Store.getNode());
+    return Store;
+  }
+  return SDValue();
+}
+
 // Handle DAG combine for STORE (FP_TO_INT F).
 SDValue PPCTargetLowering::combineStoreFPToInt(SDNode *N,
                                                DAGCombinerInfo &DCI) const {
 
   SelectionDAG &DAG = DCI.DAG;
   SDLoc dl(N);
   unsigned Opcode = N->getOperand(1).getOpcode();
 
case ISD::STORE: {
     // Not needed on ISA 3.0 based CPUs since we have a non-permuting store.
     if (Op1VT.isSimple()) {
       MVT StoreVT = Op1VT.getSimpleVT();
       if (Subtarget.needsSwapsForVSXMemOps() &&
           (StoreVT == MVT::v2f64 || StoreVT == MVT::v2i64 ||
            StoreVT == MVT::v4f32 || StoreVT == MVT::v4i32))
         return expandVSXStoreForLE(N, DCI);
     }
+
+    // Find opportunity to reuse constant from ConstantSplatVector
+    if (Subtarget.hasVSX() && isa<ConstantSDNode>(N->getOperand(1)))
+      if (SDValue Val = findAndReuseSplatForConst(N, DCI))
+        return Val;
+
     break;
   }
   case ISD::LOAD: {
     LoadSDNode *LD = cast<LoadSDNode>(N);
     EVT VT = LD->getValueType(0);
 
     // For little endian, VSX loads require generating lxvd2x/xxswapd.
     // Not needed on ISA 3.0 based CPUs since we have a non-permuting load.
if (Subtarget.hasAltivec() && Op.size() >= 16 &&
         (Op.isAligned(Align(16)) ||
          ((Op.isMemset() && Subtarget.hasVSX()) || Subtarget.hasP8Vector())))
       return MVT::v4i32;
   }
 
   if (Subtarget.isPPC64()) {
     return MVT::i64;
   }
 
   return MVT::i32;
 }
 
 /// Returns true if it is beneficial to convert a load of a constant
 /// to just the constant itself.
 bool PPCTargetLowering::shouldConvertConstantLoadToIntImm(const APInt &Imm,
                                                           Type *Ty) const {
   assert(Ty->isIntegerTy());

llvm/lib/Target/PowerPC/PPCInstrVSX.td

 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 12)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 5)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 13)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 6)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 14)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 7)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 15)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 8)), VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstxsix (f64 (vector_extract v2f64:$S, 0)), ForceXForm:$dst, 1),
+          (STXSIBXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 
 // Scalar stores of i16
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 0)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 10)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 1)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 12)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 2)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 14)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 3)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 4)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 2)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 5)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 4)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 6)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 6)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 7)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 8)), VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstxsix (f64 (vector_extract v2f64:$S, 0)), ForceXForm:$dst, 2),
+          (STXSIHXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 } // HasVSX, HasP9Vector, IsBigEndian
 
 // Big endian 64Bit Power9 subtarget.
 let Predicates = [HasVSX, HasP9Vector, IsBigEndian, IsPPC64] in {
 def : Pat<(v2i64 (scalar_to_vector (i64 (load DSForm:$src)))),
           (v2i64 (SUBREG_TO_REG (i64 1), (DFLOADf64 DSForm:$src), sub_64))>;
 def : Pat<(v2i64 (scalar_to_vector (i64 (load XForm:$src)))),
           (v2i64 (SUBREG_TO_REG (i64 1), (XFLOADf64 XForm:$src), sub_64))>;

 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 12)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 12)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 13)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 11)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 14)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 10)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei8 (i32 (vector_extract v16i8:$S, 15)), ForceXForm:$dst),
           (STXSIBXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 9)), VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstxsix (f64 (vector_extract v2f64:$S, 1)), ForceXForm:$dst, 1),
+          (STXSIBXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 
 // Scalar stores of i16
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 0)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 8)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 1)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 6)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 2)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 4)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 3)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 2)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 4)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 5)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 14)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 6)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 12)), VSRC), ForceXForm:$dst)>;
 def : Pat<(truncstorei16 (i32 (vector_extract v8i16:$S, 7)), ForceXForm:$dst),
           (STXSIHXv (COPY_TO_REGCLASS (v16i8 (VSLDOI $S, $S, 10)), VSRC), ForceXForm:$dst)>;
+def : Pat<(PPCstxsix (f64 (vector_extract v2f64:$S, 1)), ForceXForm:$dst, 2),
+          (STXSIHXv (COPY_TO_REGCLASS $S, VSRC), ForceXForm:$dst)>;
 
 defm : ScalToVecWPermute<
   v2i64, (i64 (load DSForm:$src)),
   (XXPERMDIs (DFLOADf64 DSForm:$src), 2),
   (SUBREG_TO_REG (i64 1), (DFLOADf64 DSForm:$src), sub_64)>;
 defm : ScalToVecWPermute<
   v2i64, (i64 (load XForm:$src)),
   (XXPERMDIs (XFLOADf64 XForm:$src), 2),

llvm/test/CodeGen/PowerPC/const-splat-array-init.ll

 ; P8-BE-NEXT:    sth 4, 16(3)
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo1:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C0(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    li 4, 3333
-; P9-BE-NEXT:    sth 4, 16(3)
+; P9-BE-NEXT:    li 4, 16
 ; P9-BE-NEXT:    stxv 0, 0(3)
+; P9-BE-NEXT:    stxsihx 0, 3, 4
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo1:
 ; P10-BE:       # %bb.0: # %entry
 ; P10-BE-NEXT:    xxspltiw 0, 218434821
-; P10-BE-NEXT:    li 4, 3333
-; P10-BE-NEXT:    sth 4, 16(3)
+; P10-BE-NEXT:    li 4, 16
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stxsihx 0, 3, 4
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo1:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI0_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI0_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
 ; P8-LE-NEXT:    li 4, 3333
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    sth 4, 16(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo1:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI0_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI0_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    li 4, 3333
-; P9-LE-NEXT:    sth 4, 16(3)
+; P9-LE-NEXT:    li 4, 16
 ; P9-LE-NEXT:    stxv 0, 0(3)
+; P9-LE-NEXT:    stxsihx 0, 3, 4
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo1:
 ; P10-LE:       # %bb.0: # %entry
 ; P10-LE-NEXT:    xxspltiw 0, 218434821
-; P10-LE-NEXT:    li 4, 3333
-; P10-LE-NEXT:    sth 4, 16(3)
+; P10-LE-NEXT:    li 4, 16
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stxsihx 0, 3, 4
 ; P10-LE-NEXT:    blr
 entry:
   store <8 x i16> <i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333>, ptr %a, align 2
   %arrayidx8 = getelementptr inbounds i16, ptr %a, i64 8
   store i16 3333, ptr %arrayidx8, align 2
   ret void
 }
 
 define dso_local void @foo2(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo2:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C1(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 3333
-; P8-BE-NEXT:    ori 4, 4, 3333
-; P8-BE-NEXT:    stw 4, 16(3)
+; P8-BE-NEXT:    li 4, 16
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
+; P8-BE-NEXT:    stfiwx 0, 3, 4
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo2:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C1(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 3333
-; P9-BE-NEXT:    ori 4, 4, 3333
-; P9-BE-NEXT:    stw 4, 16(3)
+; P9-BE-NEXT:    li 4, 16
 ; P9-BE-NEXT:    stxv 0, 0(3)
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo2:
 ; P10-BE:       # %bb.0: # %entry
 ; P10-BE-NEXT:    xxspltiw 0, 218434821
-; P10-BE-NEXT:    pli 4, 218434821
-; P10-BE-NEXT:    stw 4, 16(3)
+; P10-BE-NEXT:    li 4, 16
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo2:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI1_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI1_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 3333
-; P8-LE-NEXT:    ori 4, 4, 3333
+; P8-LE-NEXT:    li 4, 16
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    stw 4, 16(3)
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo2:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI1_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI1_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 3333
-; P9-LE-NEXT:    ori 4, 4, 3333
-; P9-LE-NEXT:    stw 4, 16(3)
+; P9-LE-NEXT:    li 4, 16
 ; P9-LE-NEXT:    stxv 0, 0(3)
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo2:
 ; P10-LE:       # %bb.0: # %entry
 ; P10-LE-NEXT:    xxspltiw 0, 218434821
-; P10-LE-NEXT:    pli 4, 218434821
-; P10-LE-NEXT:    stw 4, 16(3)
+; P10-LE-NEXT:    li 4, 16
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    blr
 entry:
   store <8 x i16> <i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333>, ptr %a, align 2
   %arrayidx8 = getelementptr inbounds i16, ptr %a, i64 8
   store i16 3333, ptr %arrayidx8, align 2
   %arrayidx9 = getelementptr inbounds i16, ptr %a, i64 9
   store i16 3333, ptr %arrayidx9, align 2
   ret void
 }
 
 define dso_local void @foo3(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo3:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C2(2) # %const.0
+; P8-BE-NEXT:    li 5, 3333
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 3333
-; P8-BE-NEXT:    ori 4, 4, 3333
-; P8-BE-NEXT:    stw 4, 16(3)
-; P8-BE-NEXT:    li 4, 3333
+; P8-BE-NEXT:    li 4, 16
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
-; P8-BE-NEXT:    sth 4, 20(3)
+; P8-BE-NEXT:    stfiwx 0, 3, 4
+; P8-BE-NEXT:    sth 5, 20(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo3:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C2(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 3333
-; P9-BE-NEXT:    ori 4, 4, 3333
-; P9-BE-NEXT:    stw 4, 16(3)
-; P9-BE-NEXT:    li 4, 3333
+; P9-BE-NEXT:    li 4, 16
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    sth 4, 20(3)
+; P9-BE-NEXT:    stfiwx 0, 3, 4
+; P9-BE-NEXT:    li 4, 20
+; P9-BE-NEXT:    stxsihx 0, 3, 4
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo3:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 218434821
 ; P10-BE-NEXT:    xxspltiw 0, 218434821
-; P10-BE-NEXT:    stw 4, 16(3)
-; P10-BE-NEXT:    li 4, 3333
+; P10-BE-NEXT:    li 4, 16
 ; P10-BE-NEXT:    stxv 0, 0(3)
-; P10-BE-NEXT:    sth 4, 20(3)
+; P10-BE-NEXT:    stfiwx 0, 3, 4
+; P10-BE-NEXT:    li 4, 20
+; P10-BE-NEXT:    stxsihx 0, 3, 4
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo3:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI2_0@toc@ha
 ; P8-LE-NEXT:    li 5, 3333
 ; P8-LE-NEXT:    addi 4, 4, .LCPI2_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 3333
-; P8-LE-NEXT:    ori 4, 4, 3333
+; P8-LE-NEXT:    li 4, 16
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    stw 4, 16(3)
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    sth 5, 20(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo3:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI2_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI2_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 3333
-; P9-LE-NEXT:    ori 4, 4, 3333
-; P9-LE-NEXT:    stw 4, 16(3)
-; P9-LE-NEXT:    li 4, 3333
+; P9-LE-NEXT:    li 4, 16
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    sth 4, 20(3)
+; P9-LE-NEXT:    stfiwx 0, 3, 4
+; P9-LE-NEXT:    li 4, 20
+; P9-LE-NEXT:    stxsihx 0, 3, 4
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo3:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 218434821
 ; P10-LE-NEXT:    xxspltiw 0, 218434821
-; P10-LE-NEXT:    stw 4, 16(3)
-; P10-LE-NEXT:    li 4, 3333
+; P10-LE-NEXT:    li 4, 16
 ; P10-LE-NEXT:    stxv 0, 0(3)
-; P10-LE-NEXT:    sth 4, 20(3)
+; P10-LE-NEXT:    stfiwx 0, 3, 4
+; P10-LE-NEXT:    li 4, 20
+; P10-LE-NEXT:    stxsihx 0, 3, 4
 ; P10-LE-NEXT:    blr
 entry:
   store <8 x i16> <i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333>, ptr %a, align 2
   %arrayidx8 = getelementptr inbounds i16, ptr %a, i64 8
   store i16 3333, ptr %arrayidx8, align 2
   %arrayidx9 = getelementptr inbounds i16, ptr %a, i64 9
   store i16 3333, ptr %arrayidx9, align 2
   %arrayidx10 = getelementptr inbounds i16, ptr %a, i64 10
   store i16 3333, ptr %arrayidx10, align 2
   ret void
 }
 
 define dso_local void @foo4(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo4:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C3(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 3333
-; P8-BE-NEXT:    ori 4, 4, 3333
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
-; P8-BE-NEXT:    std 4, 16(3)
+; P8-BE-NEXT:    stfd 0, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo4:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C3(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 3333
-; P9-BE-NEXT:    ori 4, 4, 3333
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    std 4, 16(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo4:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 218434821
 ; P10-BE-NEXT:    xxspltiw 0, 218434821
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-BE-NEXT:    stxv 0, 0(3)
-; P10-BE-NEXT:    std 4, 16(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo4:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI3_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI3_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 3333
-; P8-LE-NEXT:    ori 4, 4, 3333
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    std 4, 16(3)
+; P8-LE-NEXT:    stfd 0, 16(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo4:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI3_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI3_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 3333
-; P9-LE-NEXT:    ori 4, 4, 3333
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    std 4, 16(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo4:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 218434821
 ; P10-LE-NEXT:    xxspltiw 0, 218434821
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-LE-NEXT:    stxv 0, 0(3)
-; P10-LE-NEXT:    std 4, 16(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   store <8 x i16> <i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333, i16 3333>, ptr %a, align 2
   %arrayidx8 = getelementptr inbounds i16, ptr %a, i64 8
   store i16 3333, ptr %arrayidx8, align 2
   %arrayidx9 = getelementptr inbounds i16, ptr %a, i64 9
   store i16 3333, ptr %arrayidx9, align 2
   %arrayidx10 = getelementptr inbounds i16, ptr %a, i64 10
   store i16 3333, ptr %arrayidx10, align 2
   %arrayidx11 = getelementptr inbounds i16, ptr %a, i64 11
   store i16 3333, ptr %arrayidx11, align 2
   ret void
 }
 
 define dso_local void @foo5(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo5:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C4(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 5
-; P8-BE-NEXT:    ori 4, 4, 5653
-; P8-BE-NEXT:    stw 4, 16(3)
+; P8-BE-NEXT:    li 4, 16
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
+; P8-BE-NEXT:    stfiwx 0, 3, 4
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo5:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C4(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 5
-; P9-BE-NEXT:    ori 4, 4, 5653
-; P9-BE-NEXT:    stw 4, 16(3)
+; P9-BE-NEXT:    li 4, 16
 ; P9-BE-NEXT:    stxv 0, 0(3)
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo5:
 ; P10-BE:       # %bb.0: # %entry
 ; P10-BE-NEXT:    xxspltiw 0, 333333
-; P10-BE-NEXT:    pli 4, 333333
-; P10-BE-NEXT:    stw 4, 16(3)
+; P10-BE-NEXT:    li 4, 16
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo5:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI4_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI4_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 5
-; P8-LE-NEXT:    ori 4, 4, 5653
+; P8-LE-NEXT:    li 4, 16
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    stw 4, 16(3)
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo5:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI4_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI4_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 5
-; P9-LE-NEXT:    ori 4, 4, 5653
-; P9-LE-NEXT:    stw 4, 16(3)
+; P9-LE-NEXT:    li 4, 16
 ; P9-LE-NEXT:    stxv 0, 0(3)
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo5:
 ; P10-LE:       # %bb.0: # %entry
 ; P10-LE-NEXT:    xxspltiw 0, 333333
-; P10-LE-NEXT:    pli 4, 333333
-; P10-LE-NEXT:    stw 4, 16(3)
+; P10-LE-NEXT:    li 4, 16
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    blr
 entry:
   store <4 x i32> <i32 333333, i32 333333, i32 333333, i32 333333>, ptr %a, align 4
   %arrayidx4 = getelementptr inbounds i32, ptr %a, i64 4
   store i32 333333, ptr %arrayidx4, align 4
   ret void
 }
 
 define dso_local void @foo6(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo6:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C5(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 5
-; P8-BE-NEXT:    ori 4, 4, 5653
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
-; P8-BE-NEXT:    std 4, 16(3)
+; P8-BE-NEXT:    stfd 0, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo6:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C5(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 5
-; P9-BE-NEXT:    ori 4, 4, 5653
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    std 4, 16(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo6:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 333333
 ; P10-BE-NEXT:    xxspltiw 0, 333333
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-BE-NEXT:    stxv 0, 0(3)
-; P10-BE-NEXT:    std 4, 16(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo6:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI5_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI5_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 5
-; P8-LE-NEXT:    ori 4, 4, 5653
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    std 4, 16(3)
+; P8-LE-NEXT:    stfd 0, 16(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo6:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI5_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI5_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 5
-; P9-LE-NEXT:    ori 4, 4, 5653
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    std 4, 16(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo6:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 333333
 ; P10-LE-NEXT:    xxspltiw 0, 333333
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-LE-NEXT:    stxv 0, 0(3)
-; P10-LE-NEXT:    std 4, 16(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   store <4 x i32> <i32 333333, i32 333333, i32 333333, i32 333333>, ptr %a, align 4
   %arrayidx4 = getelementptr inbounds i32, ptr %a, i64 4
   store i32 333333, ptr %arrayidx4, align 4
   %arrayidx5 = getelementptr inbounds i32, ptr %a, i64 5
   store i32 333333, ptr %arrayidx5, align 4
   ret void
 }
 
 define dso_local void @foo7(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo7:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C6(2) # %const.0
 ; P8-BE-NEXT:    lxvd2x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 508
-; P8-BE-NEXT:    ori 4, 4, 41045
-; P8-BE-NEXT:    std 4, 16(3)
 ; P8-BE-NEXT:    stxvd2x 0, 0, 3
+; P8-BE-NEXT:    stfd 0, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo7:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C6(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 508
-; P9-BE-NEXT:    ori 4, 4, 41045
-; P9-BE-NEXT:    std 4, 16(3)
 ; P9-BE-NEXT:    stxv 0, 0(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo7:
 ; P10-BE:       # %bb.0: # %entry
 ; P10-BE-NEXT:    xxlxor 0, 0, 0
-; P10-BE-NEXT:    pli 4, 33333333
 ; P10-BE-NEXT:    xxsplti32dx 0, 1, 33333333
-; P10-BE-NEXT:    std 4, 16(3)
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo7:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI6_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI6_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 508
-; P8-LE-NEXT:    ori 4, 4, 41045
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    std 4, 16(3)
+; P8-LE-NEXT:    stfd 0, 16(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo7:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI6_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI6_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 508
-; P9-LE-NEXT:    ori 4, 4, 41045
-; P9-LE-NEXT:    std 4, 16(3)
 ; P9-LE-NEXT:    stxv 0, 0(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo7:
 ; P10-LE:       # %bb.0: # %entry
 ; P10-LE-NEXT:    xxlxor 0, 0, 0
-; P10-LE-NEXT:    pli 4, 33333333
 ; P10-LE-NEXT:    xxsplti32dx 0, 1, 33333333
-; P10-LE-NEXT:    std 4, 16(3)
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   store <2 x i64> <i64 33333333, i64 33333333>, ptr %a, align 8
   %arrayidx2 = getelementptr inbounds i64, ptr %a, i64 2
   store i64 33333333, ptr %arrayidx2, align 8
   ret void
 }
 
 define dso_local void @foo8(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo8:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C7(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 16469
-; P8-BE-NEXT:    ori 4, 4, 7864
-; P8-BE-NEXT:    stw 4, 16(3)
+; P8-BE-NEXT:    li 4, 16
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
+; P8-BE-NEXT:    stfiwx 0, 3, 4
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo8:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C7(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 16469
-; P9-BE-NEXT:    ori 4, 4, 7864
-; P9-BE-NEXT:    stw 4, 16(3)
+; P9-BE-NEXT:    li 4, 16
 ; P9-BE-NEXT:    stxv 0, 0(3)
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo8:
 ; P10-BE:       # %bb.0: # %entry
 ; P10-BE-NEXT:    xxspltiw 0, 1079320248
-; P10-BE-NEXT:    pli 4, 1079320248
-; P10-BE-NEXT:    stw 4, 16(3)
+; P10-BE-NEXT:    li 4, 16
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo8:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI7_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI7_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 16469
-; P8-LE-NEXT:    ori 4, 4, 7864
+; P8-LE-NEXT:    li 4, 16
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    stw 4, 16(3)
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo8:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI7_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI7_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 16469
-; P9-LE-NEXT:    ori 4, 4, 7864
-; P9-LE-NEXT:    stw 4, 16(3)
+; P9-LE-NEXT:    li 4, 16
 ; P9-LE-NEXT:    stxv 0, 0(3)
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo8:
 ; P10-LE:       # %bb.0: # %entry
 ; P10-LE-NEXT:    xxspltiw 0, 1079320248
-; P10-LE-NEXT:    pli 4, 1079320248
-; P10-LE-NEXT:    stw 4, 16(3)
+; P10-LE-NEXT:    li 4, 16
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    blr
 entry:
   store <4 x float> <float 0x400AA3D700000000, float 0x400AA3D700000000, float 0x400AA3D700000000, float 0x400AA3D700000000>, ptr %a, align 4
   %arrayidx4 = getelementptr inbounds float, ptr %a, i64 4
   store float 0x400AA3D700000000, ptr %arrayidx4, align 4
   ret void
 }
 
entry:
   ret void
 }
 
 define dso_local void @foo10(ptr nocapture noundef writeonly %a) local_unnamed_addr {
 ; P8-BE-LABEL: foo10:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C9(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, 16469
-; P8-BE-NEXT:    ori 4, 4, 7864
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
-; P8-BE-NEXT:    std 4, 16(3)
+; P8-BE-NEXT:    stfd 0, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: foo10:
 ; P9-BE:       # %bb.0: # %entry
 ; P9-BE-NEXT:    ld 4, L..C9(2) # %const.0
 ; P9-BE-NEXT:    lxv 0, 0(4)
-; P9-BE-NEXT:    lis 4, 16469
-; P9-BE-NEXT:    ori 4, 4, 7864
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    std 4, 16(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: foo10:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 1079320248
 ; P10-BE-NEXT:    xxspltiw 0, 1079320248
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-BE-NEXT:    stxv 0, 0(3)
-; P10-BE-NEXT:    std 4, 16(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: foo10:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI9_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI9_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, 16469
-; P8-LE-NEXT:    ori 4, 4, 7864
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
-; P8-LE-NEXT:    std 4, 16(3)
+; P8-LE-NEXT:    stfd 0, 16(3)
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: foo10:
 ; P9-LE:       # %bb.0: # %entry
 ; P9-LE-NEXT:    addis 4, 2, .LCPI9_0@toc@ha
 ; P9-LE-NEXT:    addi 4, 4, .LCPI9_0@toc@l
 ; P9-LE-NEXT:    lxv 0, 0(4)
-; P9-LE-NEXT:    lis 4, 16469
-; P9-LE-NEXT:    ori 4, 4, 7864
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    std 4, 16(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: foo10:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 1079320248
 ; P10-LE-NEXT:    xxspltiw 0, 1079320248
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
 ; P10-LE-NEXT:    stxv 0, 0(3)
-; P10-LE-NEXT:    std 4, 16(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   store <4 x float> <float 0x400AA3D700000000, float 0x400AA3D700000000, float 0x400AA3D700000000, float 0x400AA3D700000000>, ptr %a, align 4
   %arrayidx4 = getelementptr inbounds float, ptr %a, i64 4
   store float 0x400AA3D700000000, ptr %arrayidx4, align 4
   %arrayidx5 = getelementptr inbounds float, ptr %a, i64 5
   store float 0x400AA3D700000000, ptr %arrayidx5, align 4
   ret void
 }

llvm/test/CodeGen/PowerPC/memset-tail.ll

entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 25, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B8(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B8:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    vspltisb 2, 15
-; P8-BE-NEXT:    lis 4, 3855
-; P8-BE-NEXT:    ori 4, 4, 3855
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
+; P8-BE-NEXT:    li 4, 16
+; P8-BE-NEXT:    stxsdx 34, 3, 4
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
-; P8-BE-NEXT:    std 4, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B8:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, 3855
 ; P9-BE-NEXT:    xxspltib 0, 15
-; P9-BE-NEXT:    ori 4, 4, 3855
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
-; P9-BE-NEXT:    std 4, 16(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B8:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 252645135
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
-; P10-BE-NEXT:    std 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 15
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B8:
 ; P8-LE:       # %bb.0: # %entry
-; P8-LE-NEXT:    lis 4, 3855
 ; P8-LE-NEXT:    vspltisb 2, 15
-; P8-LE-NEXT:    ori 4, 4, 3855
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
-; P8-LE-NEXT:    std 4, 16(3)
-; P8-LE-NEXT:    stxvd2x 34, 0, 3
+; P8-LE-NEXT:    li 4, 16
+; P8-LE-NEXT:    xxswapd 0, 34
+; P8-LE-NEXT:    stxsdx 34, 3, 4
+; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B8:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, 3855
 ; P9-LE-NEXT:    xxspltib 0, 15
-; P9-LE-NEXT:    ori 4, 4, 3855
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
-; P9-LE-NEXT:    std 4, 16(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B8:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 252645135
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
-; P10-LE-NEXT:    std 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 15
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 24, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B7(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B7:
 ; P8-BE:       # %bb.0: # %entry
-; P8-BE-NEXT:    lis 4, 3855
 ; P8-BE-NEXT:    vspltisb 2, 15
-; P8-BE-NEXT:    li 5, 15
-; P8-BE-NEXT:    ori 4, 4, 3855
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
-; P8-BE-NEXT:    stdx 4, 3, 5
+; P8-BE-NEXT:    li 4, 15
+; P8-BE-NEXT:    stxsdx 34, 3, 4
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B7:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, 3855
-; P9-BE-NEXT:    li 5, 15
-; P9-BE-NEXT:    ori 4, 4, 3855
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
-; P9-BE-NEXT:    stdx 4, 3, 5
 ; P9-BE-NEXT:    xxspltib 0, 15
+; P9-BE-NEXT:    stfd 0, 15(3)
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B7:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 252645135
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
-; P10-BE-NEXT:    pstd 4, 15(3), 0
 ; P10-BE-NEXT:    xxspltib 0, 15
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    pstfd 0, 15(3), 0
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B7:
 ; P8-LE:       # %bb.0: # %entry
-; P8-LE-NEXT:    lis 4, 3855
 ; P8-LE-NEXT:    vspltisb 2, 15
-; P8-LE-NEXT:    li 5, 15
-; P8-LE-NEXT:    ori 4, 4, 3855
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
-; P8-LE-NEXT:    stdx 4, 3, 5
-; P8-LE-NEXT:    stxvd2x 34, 0, 3
+; P8-LE-NEXT:    li 4, 15
+; P8-LE-NEXT:    xxswapd 0, 34
+; P8-LE-NEXT:    stxsdx 34, 3, 4
+; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B7:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, 3855
-; P9-LE-NEXT:    li 5, 15
-; P9-LE-NEXT:    ori 4, 4, 3855
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
-; P9-LE-NEXT:    stdx 4, 3, 5
 ; P9-LE-NEXT:    xxspltib 0, 15
+; P9-LE-NEXT:    stfd 0, 15(3)
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B7:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 252645135
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
-; P10-LE-NEXT:    pstd 4, 15(3), 0
 ; P10-LE-NEXT:    xxspltib 0, 15
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    pstfd 0, 15(3), 0
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 23, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B4(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B4:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    vspltisb 2, 15
-; P8-BE-NEXT:    lis 4, 3855
-; P8-BE-NEXT:    ori 4, 4, 3855
-; P8-BE-NEXT:    stw 4, 16(3)
+; P8-BE-NEXT:    li 4, 16
+; P8-BE-NEXT:    stxsiwx 34, 3, 4
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B4:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, 3855
-; P9-BE-NEXT:    ori 4, 4, 3855
-; P9-BE-NEXT:    stw 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 15
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B4:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 252645135
-; P10-BE-NEXT:    stw 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 15
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B4:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    vspltisb 2, 15
-; P8-LE-NEXT:    lis 4, 3855
-; P8-LE-NEXT:    ori 4, 4, 3855
-; P8-LE-NEXT:    stw 4, 16(3)
-; P8-LE-NEXT:    stxvd2x 34, 0, 3
+; P8-LE-NEXT:    li 4, 16
+; P8-LE-NEXT:    xxswapd 0, 34
+; P8-LE-NEXT:    stxsiwx 34, 3, 4
+; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B4:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, 3855
-; P9-LE-NEXT:    ori 4, 4, 3855
-; P9-LE-NEXT:    stw 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 15
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B4:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 252645135
-; P10-LE-NEXT:    stw 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 15
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i32(ptr %p, i8 15, i32 20, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B3(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B3:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    vspltisb 2, 15
-; P8-BE-NEXT:    lis 4, 3855
-; P8-BE-NEXT:    ori 4, 4, 3855
+; P8-BE-NEXT:    li 4, 15
+; P8-BE-NEXT:    stxsiwx 34, 3, 4
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
-; P8-BE-NEXT:    stw 4, 15(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B3:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, 3855
-; P9-BE-NEXT:    ori 4, 4, 3855
-; P9-BE-NEXT:    stw 4, 15(3)
 ; P9-BE-NEXT:    xxspltib 0, 15
+; P9-BE-NEXT:    li 4, 15
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B3:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 252645135
-; P10-BE-NEXT:    stw 4, 15(3)
 ; P10-BE-NEXT:    xxspltib 0, 15
+; P10-BE-NEXT:    li 4, 15
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B3:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    vspltisb 2, 15
-; P8-LE-NEXT:    lis 4, 3855
-; P8-LE-NEXT:    ori 4, 4, 3855
-; P8-LE-NEXT:    stw 4, 15(3)
-; P8-LE-NEXT:    stxvd2x 34, 0, 3
+; P8-LE-NEXT:    li 4, 15
+; P8-LE-NEXT:    xxswapd 0, 34
+; P8-LE-NEXT:    stxsiwx 34, 3, 4
+; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B3:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, 3855
-; P9-LE-NEXT:    ori 4, 4, 3855
-; P9-LE-NEXT:    stw 4, 15(3)
 ; P9-LE-NEXT:    xxspltib 0, 15
+; P9-LE-NEXT:    li 4, 15
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B3:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 252645135
-; P10-LE-NEXT:    stw 4, 15(3)
 ; P10-LE-NEXT:    xxspltib 0, 15
+; P10-LE-NEXT:    li 4, 15
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 19, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B2(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B2:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    vspltisb 2, 15
 ; P8-BE-NEXT:    li 4, 3855
 ; P8-BE-NEXT:    sth 4, 16(3)
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B2:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    li 4, 3855
-; P9-BE-NEXT:    sth 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 15
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stxsihx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B2:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    li 4, 3855
-; P10-BE-NEXT:    sth 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 15
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stxsihx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B2:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    vspltisb 2, 15
 ; P8-LE-NEXT:    li 4, 3855
 ; P8-LE-NEXT:    sth 4, 16(3)
 ; P8-LE-NEXT:    stxvd2x 34, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B2:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    li 4, 3855
-; P9-LE-NEXT:    sth 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 15
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stxsihx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B2:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    li 4, 3855
-; P10-LE-NEXT:    sth 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 15
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stxsihx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 18, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV1B1(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memsetTailV1B1:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    vspltisb 2, 15
 ; P8-BE-NEXT:    li 4, 15
 ; P8-BE-NEXT:    stb 4, 16(3)
 ; P8-BE-NEXT:    stxvw4x 34, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memsetTailV1B1:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    li 4, 15
-; P9-BE-NEXT:    stb 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 15
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stxsibx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memsetTailV1B1:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    li 4, 15
-; P10-BE-NEXT:    stb 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 15
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stxsibx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memsetTailV1B1:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    vspltisb 2, 15
 ; P8-LE-NEXT:    li 4, 15
 ; P8-LE-NEXT:    stb 4, 16(3)
 ; P8-LE-NEXT:    stxvd2x 34, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memsetTailV1B1:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    li 4, 15
-; P9-LE-NEXT:    stb 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 15
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stxsibx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memsetTailV1B1:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    li 4, 15
-; P10-LE-NEXT:    stb 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 15
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stxsibx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 15, i64 17, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B15(ptr nocapture noundef writeonly %p) local_unnamed_addr {
entry:
   ret void
 }
 
 define dso_local void @memset2TailV1B8(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B8:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C3(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, -23131
-; P8-BE-NEXT:    ori 4, 4, 42405
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
+; P8-BE-NEXT:    stfd 0, 16(3)
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
-; P8-BE-NEXT:    std 4, 16(3)
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B8:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, -23131
 ; P9-BE-NEXT:    xxspltib 0, 165
-; P9-BE-NEXT:    ori 4, 4, 42405
 ; P9-BE-NEXT:    stxv 0, 0(3)
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
-; P9-BE-NEXT:    std 4, 16(3)
+; P9-BE-NEXT:    stfd 0, 16(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B8:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 2779096485
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
-; P10-BE-NEXT:    std 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 165
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    stfd 0, 16(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B8:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI12_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI12_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, -23131
-; P8-LE-NEXT:    ori 4, 4, 42405
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
-; P8-LE-NEXT:    std 4, 16(3)
+; P8-LE-NEXT:    stfd 0, 16(3)
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B8:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, -23131
 ; P9-LE-NEXT:    xxspltib 0, 165
-; P9-LE-NEXT:    ori 4, 4, 42405
 ; P9-LE-NEXT:    stxv 0, 0(3)
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
-; P9-LE-NEXT:    std 4, 16(3)
+; P9-LE-NEXT:    stfd 0, 16(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B8:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 2779096485
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
-; P10-LE-NEXT:    std 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 165
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    stfd 0, 16(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 165, i64 24, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B7(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B7:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C4(2) # %const.0
-; P8-BE-NEXT:    lis 5, -23131
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    ori 4, 5, 42405
-; P8-BE-NEXT:    li 5, 15
-; P8-BE-NEXT:    rldimi 4, 4, 32, 0
-; P8-BE-NEXT:    stdx 4, 3, 5
+; P8-BE-NEXT:    stfd 0, 15(3)
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B7:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, -23131
-; P9-BE-NEXT:    li 5, 15
-; P9-BE-NEXT:    ori 4, 4, 42405
-; P9-BE-NEXT:    rldimi 4, 4, 32, 0
-; P9-BE-NEXT:    stdx 4, 3, 5
 ; P9-BE-NEXT:    xxspltib 0, 165
+; P9-BE-NEXT:    stfd 0, 15(3)
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B7:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, 2779096485
-; P10-BE-NEXT:    rldimi 4, 4, 32, 0
-; P10-BE-NEXT:    pstd 4, 15(3), 0
 ; P10-BE-NEXT:    xxspltib 0, 165
 ; P10-BE-NEXT:    stxv 0, 0(3)
+; P10-BE-NEXT:    pstfd 0, 15(3), 0
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B7:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI13_0@toc@ha
-; P8-LE-NEXT:    lis 5, -23131
 ; P8-LE-NEXT:    addi 4, 4, .LCPI13_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    ori 4, 5, 42405
-; P8-LE-NEXT:    li 5, 15
-; P8-LE-NEXT:    rldimi 4, 4, 32, 0
-; P8-LE-NEXT:    stdx 4, 3, 5
+; P8-LE-NEXT:    stfd 0, 15(3)
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B7:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, -23131
-; P9-LE-NEXT:    li 5, 15
-; P9-LE-NEXT:    ori 4, 4, 42405
-; P9-LE-NEXT:    rldimi 4, 4, 32, 0
-; P9-LE-NEXT:    stdx 4, 3, 5
 ; P9-LE-NEXT:    xxspltib 0, 165
+; P9-LE-NEXT:    stfd 0, 15(3)
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B7:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, 2779096485
-; P10-LE-NEXT:    rldimi 4, 4, 32, 0
-; P10-LE-NEXT:    pstd 4, 15(3), 0
 ; P10-LE-NEXT:    xxspltib 0, 165
 ; P10-LE-NEXT:    stxv 0, 0(3)
+; P10-LE-NEXT:    pstfd 0, 15(3), 0
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 165, i64 23, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B4(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B4:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C5(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, -23131
-; P8-BE-NEXT:    ori 4, 4, 42405
-; P8-BE-NEXT:    stw 4, 16(3)
+; P8-BE-NEXT:    li 4, 16
+; P8-BE-NEXT:    stfiwx 0, 3, 4
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B4:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, -23131
-; P9-BE-NEXT:    ori 4, 4, 42405
-; P9-BE-NEXT:    stw 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 165
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B4:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, -1515870811
-; P10-BE-NEXT:    stw 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 165
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B4:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI14_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI14_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, -23131
-; P8-LE-NEXT:    ori 4, 4, 42405
-; P8-LE-NEXT:    stw 4, 16(3)
+; P8-LE-NEXT:    li 4, 16
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B4:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, -23131
-; P9-LE-NEXT:    ori 4, 4, 42405
-; P9-LE-NEXT:    stw 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 165
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B4:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, -1515870811
-; P10-LE-NEXT:    stw 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 165
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i32(ptr %p, i8 165, i32 20, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B3(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B3:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C6(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
-; P8-BE-NEXT:    lis 4, -23131
-; P8-BE-NEXT:    ori 4, 4, 42405
-; P8-BE-NEXT:    stw 4, 15(3)
+; P8-BE-NEXT:    li 4, 15
+; P8-BE-NEXT:    stfiwx 0, 3, 4
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B3:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    lis 4, -23131
-; P9-BE-NEXT:    ori 4, 4, 42405
-; P9-BE-NEXT:    stw 4, 15(3)
 ; P9-BE-NEXT:    xxspltib 0, 165
+; P9-BE-NEXT:    li 4, 15
+; P9-BE-NEXT:    stfiwx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B3:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    pli 4, -1515870811
-; P10-BE-NEXT:    stw 4, 15(3)
 ; P10-BE-NEXT:    xxspltib 0, 165
+; P10-BE-NEXT:    li 4, 15
+; P10-BE-NEXT:    stfiwx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B3:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI15_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI15_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
-; P8-LE-NEXT:    lis 4, -23131
-; P8-LE-NEXT:    ori 4, 4, 42405
-; P8-LE-NEXT:    stw 4, 15(3)
+; P8-LE-NEXT:    li 4, 15
+; P8-LE-NEXT:    stfiwx 0, 3, 4
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B3:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    lis 4, -23131
-; P9-LE-NEXT:    ori 4, 4, 42405
-; P9-LE-NEXT:    stw 4, 15(3)
 ; P9-LE-NEXT:    xxspltib 0, 165
+; P9-LE-NEXT:    li 4, 15
+; P9-LE-NEXT:    stfiwx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B3:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    pli 4, -1515870811
-; P10-LE-NEXT:    stw 4, 15(3)
 ; P10-LE-NEXT:    xxspltib 0, 165
+; P10-LE-NEXT:    li 4, 15
+; P10-LE-NEXT:    stfiwx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 165, i64 19, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B2(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B2:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C7(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
 ; P8-BE-NEXT:    li 4, -23131
 ; P8-BE-NEXT:    sth 4, 16(3)
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B2:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    li 4, -23131
-; P9-BE-NEXT:    sth 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 165
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stxsihx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B2:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    li 4, -23131
-; P10-BE-NEXT:    sth 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 165
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stxsihx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B2:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI16_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI16_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
 ; P8-LE-NEXT:    li 4, -23131
 ; P8-LE-NEXT:    sth 4, 16(3)
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B2:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    li 4, -23131
-; P9-LE-NEXT:    sth 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 165
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stxsihx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B2:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    li 4, -23131
-; P10-LE-NEXT:    sth 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 165
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stxsihx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 165, i64 18, i1 false)
   ret void
 }
 
 define dso_local void @memset2TailV1B1(ptr nocapture noundef writeonly %p) local_unnamed_addr {
 ; P8-BE-LABEL: memset2TailV1B1:
 ; P8-BE:       # %bb.0: # %entry
 ; P8-BE-NEXT:    ld 4, L..C8(2) # %const.0
 ; P8-BE-NEXT:    lxvw4x 0, 0, 4
 ; P8-BE-NEXT:    li 4, -91
 ; P8-BE-NEXT:    stb 4, 16(3)
 ; P8-BE-NEXT:    stxvw4x 0, 0, 3
 ; P8-BE-NEXT:    blr
 ;
 ; P9-BE-LABEL: memset2TailV1B1:
 ; P9-BE:       # %bb.0: # %entry
-; P9-BE-NEXT:    li 4, -91
-; P9-BE-NEXT:    stb 4, 16(3)
 ; P9-BE-NEXT:    xxspltib 0, 165
+; P9-BE-NEXT:    li 4, 16
+; P9-BE-NEXT:    stxsibx 0, 3, 4
 ; P9-BE-NEXT:    stxv 0, 0(3)
 ; P9-BE-NEXT:    blr
 ;
 ; P10-BE-LABEL: memset2TailV1B1:
 ; P10-BE:       # %bb.0: # %entry
-; P10-BE-NEXT:    li 4, -91
-; P10-BE-NEXT:    stb 4, 16(3)
 ; P10-BE-NEXT:    xxspltib 0, 165
+; P10-BE-NEXT:    li 4, 16
+; P10-BE-NEXT:    stxsibx 0, 3, 4
 ; P10-BE-NEXT:    stxv 0, 0(3)
 ; P10-BE-NEXT:    blr
 ;
 ; P8-LE-LABEL: memset2TailV1B1:
 ; P8-LE:       # %bb.0: # %entry
 ; P8-LE-NEXT:    addis 4, 2, .LCPI17_0@toc@ha
 ; P8-LE-NEXT:    addi 4, 4, .LCPI17_0@toc@l
 ; P8-LE-NEXT:    lxvd2x 0, 0, 4
 ; P8-LE-NEXT:    li 4, -91
 ; P8-LE-NEXT:    stb 4, 16(3)
 ; P8-LE-NEXT:    stxvd2x 0, 0, 3
 ; P8-LE-NEXT:    blr
 ;
 ; P9-LE-LABEL: memset2TailV1B1:
 ; P9-LE:       # %bb.0: # %entry
-; P9-LE-NEXT:    li 4, -91
-; P9-LE-NEXT:    stb 4, 16(3)
 ; P9-LE-NEXT:    xxspltib 0, 165
+; P9-LE-NEXT:    li 4, 16
+; P9-LE-NEXT:    stxsibx 0, 3, 4
 ; P9-LE-NEXT:    stxv 0, 0(3)
 ; P9-LE-NEXT:    blr
 ;
 ; P10-LE-LABEL: memset2TailV1B1:
 ; P10-LE:       # %bb.0: # %entry
-; P10-LE-NEXT:    li 4, -91
-; P10-LE-NEXT:    stb 4, 16(3)
 ; P10-LE-NEXT:    xxspltib 0, 165
+; P10-LE-NEXT:    li 4, 16
+; P10-LE-NEXT:    stxsibx 0, 3, 4
 ; P10-LE-NEXT:    stxv 0, 0(3)
 ; P10-LE-NEXT:    blr
 entry:
   tail call void @llvm.memset.p0.i64(ptr %p, i8 165, i64 17, i1 false)
   ret void
 }
 
 define dso_local void @memsetTailV0B15(ptr nocapture noundef writeonly %p) local_unnamed_addr {