[PowerPC] Fix spilling of vector registers in PEI of EH aware functions

[nemanjai]

Summary

On little endian targets prior to Power9, we spill vector registers using a
swapping store (i.e. stdxvd2x saves the vector with the two doublewords in
big endian order regardless of endianness). This is generally not a problem
since we restore them using the corresponding swapping load (lxvd2x). However
if the restore is done by the unwinder, the vector register contains data in
the incorrect order.

This patch fixes that by using Altivec loads/stores for vector saves and
restores in PEI (which keep the order correct) under those specific conditions:

• Little endian target
• EH aware function
• Subtarget prior to Power9 (which has non-swapping loads/stores)

[merge_guards_bot]

Unit tests: fail. 62335 tests passed, 1 failed and 838 were skipped.

failed: libc++.std/containers/sequences/array/array_creation/to_array.fail.cpp

clang-tidy: fail. clang-tidy found 0 errors and 3 warnings. 0 of them are added as review comments below (why?).

clang-format: pass.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

[nemanjai]

The unit test failure is due to an unrelated commit to libc++. I'm not sure why it is showing up here, but presumably when I upload any new updates and the bot re-runs the tests, that should go away.

[stefanp]

Couple of nits but overall LGTM.

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

2248

nit:
This condition here in spillCalleeSavedRegisters needs to be identical to the one in restoreCalleeSavedRegisters. To avoid possible future bugs I would create a function to test for this and call it in both places. That would avoid a situation where someone adds a condition here and forgets to add it to restoreCalleeSavedRegisters.

Something like?

bool PPCFrameLowering::mustPreserveElementOrder(MachineFunction *MF) {
  return Subtarget.isLittleEndian() && !MF->getFunction().hasFnAttribute(Attribute::NoUnwind) && !Subtarget.hasP9Vector();
}

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

1321

nit:
We can probably use RC = updatedRC(RC); like we do in PPCInstrInfo::storeRegToStackSlot.

[NeHuang]

llvm/lib/Target/PowerPC/PPCInstrInfo.h

299

nit: IMHO it could be better to add descriptions/comments on NoUpd for the two new functions storeRegToStackSlotNoUpd and loadRegFromStackSlotNoUpd?

314

same as 293

[jsji]

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

2248

Agree.

llvm/lib/Target/PowerPC/PPCInstrInfo.h

299

Nit: NoUpd here is referring to No updatedRC? It might be confusing at first glance, as store with update may refer to store with index update?

Why we want to create a new interface here? Why not extending storeRegToStackSlot with a default arg?
storeRegToStackSlot(... , bool mustPreserveElementOrder=true)
loadRegFromStackSlot(...., bool mustPreserveElementOrder=true)

[NeHuang]

The unit test failure should not relate to this patch. Saw the same failure in https://reviews.llvm.org/D68237. (it passed in the latest run today)

[nemanjai]

Thanks for the comments. I'll update the patch prior to committing.

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

2248

Ah yes, great suggestion. Thanks.

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

1321

Yeah. I didn't change this though. That's how it was.

llvm/lib/Target/PowerPC/PPCInstrInfo.h

299

I can't really say that I'm up on my knowledge of how virtual function signatures are allowed or not allowed to change between the base class and the derived class. But I can try it and see if the compiler complains.

[nemanjai]

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

2248

Turns out we have a function that tests a couple of these already. I don't know why I forgot it since it was me who originally added it :)

The condition now is probably to simple to pull out into a separate function:

if (Subtarget.needsSwapsForVSXMemOps() &&
    !MF->getFunction().hasFnAttribute(Attribute::NoUnwind))
  TII.storeRegToStackSlotNoUpd(MBB, MI, Reg, !IsLiveIn,
                               CSI[i].getFrameIdx(), RC, TRI);
else
  TII.storeRegToStackSlot(MBB, MI, Reg, !IsLiveIn, CSI[i].getFrameIdx(),
                          RC, TRI);

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

1321

Decided to change it anyway.

llvm/lib/Target/PowerPC/PPCInstrInfo.h

299

OK, tried it and apparently you can't change the signature of virtual functions you override. Which I initially assumed without trying it, now I've confirmed it.

[jsji]

llvm/lib/Target/PowerPC/PPCInstrInfo.h

299

Yes, my original proposal is not to change it when override it. But to change the original virtual functions as well. But anyway, this can be a nice to have NFC.

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

if (IsCRField) {
         NumPESpillVSR++;
         BuildMI(MBB, MI, DL, TII.get(PPC::MTVSRD), CSI[i].getDstReg())
           .addReg(Reg, getKillRegState(true));
       } else {
         const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
         // Use !IsLiveIn for the kill flag.
         // We do not want to kill registers that are live in this function
         // before their use because they will become undefined registers.
-        TII.storeRegToStackSlot(MBB, MI, Reg, !IsLiveIn,
+        // Functions without NoUnwind need to preserve the order of elements in
+        // saved vector registers.
+        if (Subtarget.needsSwapsForVSXMemOps() &&
+            !MF->getFunction().hasFnAttribute(Attribute::NoUnwind))
+          TII.storeRegToStackSlotNoUpd(MBB, MI, Reg, !IsLiveIn,
                                        CSI[i].getFrameIdx(), RC, TRI);
+        else
+          TII.storeRegToStackSlot(MBB, MI, Reg, !IsLiveIn, CSI[i].getFrameIdx(),
+                                  RC, TRI);
       }
     }
   }
   return true;
 }
 
 static void
 restoreCRs(bool isPPC64, bool is31,
if (Reg == PPC::CR2) {
       if (CSI[i].isSpilledToReg()) {
         DebugLoc DL;
         NumPEReloadVSR++;
         BuildMI(MBB, I, DL, TII.get(PPC::MFVSRD), Reg)
             .addReg(CSI[i].getDstReg(), getKillRegState(true));
       } else {
        // Default behavior for non-CR saves.
         const TargetRegisterClass *RC = TRI->getMinimalPhysRegClass(Reg);
+
+        // Functions without NoUnwind need to preserve the order of elements in
+        // saved vector registers.
+        if (Subtarget.needsSwapsForVSXMemOps() &&
+            !MF->getFunction().hasFnAttribute(Attribute::NoUnwind))
+          TII.loadRegFromStackSlotNoUpd(MBB, I, Reg, CSI[i].getFrameIdx(), RC,
+                                        TRI);
+        else
           TII.loadRegFromStackSlot(MBB, I, Reg, CSI[i].getFrameIdx(), RC, TRI);
+
         assert(I != MBB.begin() &&
                "loadRegFromStackSlot didn't insert any code!");
       }
     }
 
     // Insert in reverse order.
     if (AtStart)
       I = MBB.begin();

llvm/lib/Target/PowerPC/PPCInstrInfo.h

void copyPhysReg(MachineBasicBlock &MBB, MachineBasicBlock::iterator I,
                    bool KillSrc) const override;
 
   void storeRegToStackSlot(MachineBasicBlock &MBB,
                            MachineBasicBlock::iterator MBBI,
                            Register SrcReg, bool isKill, int FrameIndex,
                            const TargetRegisterClass *RC,
                            const TargetRegisterInfo *TRI) const override;
 
+  // Emits a register spill without updating the register class for vector
+  // registers. This ensures that when we spill a vector register the
+  // element order in the register is the same as it was in memory.
+  void storeRegToStackSlotNoUpd(MachineBasicBlock &MBB,
+                                MachineBasicBlock::iterator MBBI,
+                                unsigned SrcReg, bool isKill, int FrameIndex,
+                                const TargetRegisterClass *RC,
+                                const TargetRegisterInfo *TRI) const;
+
   void loadRegFromStackSlot(MachineBasicBlock &MBB,
                             MachineBasicBlock::iterator MBBI,
                             Register DestReg, int FrameIndex,
                             const TargetRegisterClass *RC,
                             const TargetRegisterInfo *TRI) const override;
 
+  // Emits a register reload without updating the register class for vector
+  // registers. This ensures that when we reload a vector register the
+  // element order in the register is the same as it was in memory.
+  void loadRegFromStackSlotNoUpd(MachineBasicBlock &MBB,
+                                 MachineBasicBlock::iterator MBBI,
+                                 unsigned DestReg, int FrameIndex,
+                                 const TargetRegisterClass *RC,
+                                 const TargetRegisterInfo *TRI) const;
+
   unsigned getStoreOpcodeForSpill(unsigned Reg,
                                   const TargetRegisterClass *RC = nullptr) const;
 
   unsigned getLoadOpcodeForSpill(unsigned Reg,
                                  const TargetRegisterClass *RC = nullptr) const;
 
   bool
   reverseBranchCondition(SmallVectorImpl<MachineOperand> &Cond) const override;

llvm/lib/Target/PowerPC/PPCInstrInfo.cpp

void PPCInstrInfo::StoreRegToStackSlot(
 
   if (PPC::VRSAVERCRegClass.hasSubClassEq(RC))
     FuncInfo->setSpillsVRSAVE();
 
   if (isXFormMemOp(Opcode))
     FuncInfo->setHasNonRISpills();
 }
 
-void PPCInstrInfo::storeRegToStackSlot(MachineBasicBlock &MBB,
-                                       MachineBasicBlock::iterator MI,
-                                       Register SrcReg, bool isKill,
-                                       int FrameIdx,
-                                       const TargetRegisterClass *RC,
-                                       const TargetRegisterInfo *TRI) const {
+void PPCInstrInfo::storeRegToStackSlotNoUpd(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MI, unsigned SrcReg,
+    bool isKill, int FrameIdx, const TargetRegisterClass *RC,
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr *, 4> NewMIs;
 
-  // We need to avoid a situation in which the value from a VRRC register is
-  // spilled using an Altivec instruction and reloaded into a VSRC register
-  // using a VSX instruction. The issue with this is that the VSX
-  // load/store instructions swap the doublewords in the vector and the Altivec
-  // ones don't. The register classes on the spill/reload may be different if
-  // the register is defined using an Altivec instruction and is then used by a
-  // VSX instruction.
-  RC = updatedRC(RC);
-
   StoreRegToStackSlot(MF, SrcReg, isKill, FrameIdx, RC, NewMIs);
 
   for (unsigned i = 0, e = NewMIs.size(); i != e; ++i)
     MBB.insert(MI, NewMIs[i]);
 
   const MachineFrameInfo &MFI = MF.getFrameInfo();
   MachineMemOperand *MMO = MF.getMachineMemOperand(
       MachinePointerInfo::getFixedStack(MF, FrameIdx),
       MachineMemOperand::MOStore, MFI.getObjectSize(FrameIdx),
       MFI.getObjectAlignment(FrameIdx));
   NewMIs.back()->addMemOperand(MF, MMO);
 }
 
+void PPCInstrInfo::storeRegToStackSlot(MachineBasicBlock &MBB,
+                                       MachineBasicBlock::iterator MI,
+                                       Register SrcReg, bool isKill,
+                                       int FrameIdx,
+                                       const TargetRegisterClass *RC,
+                                       const TargetRegisterInfo *TRI) const {
+  // We need to avoid a situation in which the value from a VRRC register is
+  // spilled using an Altivec instruction and reloaded into a VSRC register
+  // using a VSX instruction. The issue with this is that the VSX
+  // load/store instructions swap the doublewords in the vector and the Altivec
+  // ones don't. The register classes on the spill/reload may be different if
+  // the register is defined using an Altivec instruction and is then used by a
+  // VSX instruction.
+  RC = updatedRC(RC);
+  storeRegToStackSlotNoUpd(MBB, MI, SrcReg, isKill, FrameIdx, RC, TRI);
+}
+
 void PPCInstrInfo::LoadRegFromStackSlot(MachineFunction &MF, const DebugLoc &DL,
                                         unsigned DestReg, int FrameIdx,
                                         const TargetRegisterClass *RC,
                                         SmallVectorImpl<MachineInstr *> &NewMIs)
                                         const {
   unsigned Opcode = getLoadOpcodeForSpill(PPC::NoRegister, RC);
   NewMIs.push_back(addFrameReference(BuildMI(MF, DL, get(Opcode), DestReg),
                                      FrameIdx));
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
 
   if (PPC::CRRCRegClass.hasSubClassEq(RC) ||
       PPC::CRBITRCRegClass.hasSubClassEq(RC))
     FuncInfo->setSpillsCR();
 
   if (PPC::VRSAVERCRegClass.hasSubClassEq(RC))
     FuncInfo->setSpillsVRSAVE();
 
   if (isXFormMemOp(Opcode))
     FuncInfo->setHasNonRISpills();
 }
 
-void
-PPCInstrInfo::loadRegFromStackSlot(MachineBasicBlock &MBB,
-                                   MachineBasicBlock::iterator MI,
-                                   Register DestReg, int FrameIdx,
-                                   const TargetRegisterClass *RC,
-                                   const TargetRegisterInfo *TRI) const {
+void PPCInstrInfo::loadRegFromStackSlotNoUpd(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MI, unsigned DestReg,
+    int FrameIdx, const TargetRegisterClass *RC,
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr*, 4> NewMIs;
   DebugLoc DL;
   if (MI != MBB.end()) DL = MI->getDebugLoc();
 
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
   FuncInfo->setHasSpills();
 
-  // We need to avoid a situation in which the value from a VRRC register is
-  // spilled using an Altivec instruction and reloaded into a VSRC register
-  // using a VSX instruction. The issue with this is that the VSX
-  // load/store instructions swap the doublewords in the vector and the Altivec
-  // ones don't. The register classes on the spill/reload may be different if
-  // the register is defined using an Altivec instruction and is then used by a
-  // VSX instruction.
-  if (Subtarget.hasVSX() && RC == &PPC::VRRCRegClass)
-    RC = &PPC::VSRCRegClass;
-
   LoadRegFromStackSlot(MF, DL, DestReg, FrameIdx, RC, NewMIs);
 
   for (unsigned i = 0, e = NewMIs.size(); i != e; ++i)
     MBB.insert(MI, NewMIs[i]);
 
   const MachineFrameInfo &MFI = MF.getFrameInfo();
   MachineMemOperand *MMO = MF.getMachineMemOperand(
       MachinePointerInfo::getFixedStack(MF, FrameIdx),
       MachineMemOperand::MOLoad, MFI.getObjectSize(FrameIdx),
       MFI.getObjectAlignment(FrameIdx));
   NewMIs.back()->addMemOperand(MF, MMO);
 }
 
+void PPCInstrInfo::loadRegFromStackSlot(MachineBasicBlock &MBB,
+                                        MachineBasicBlock::iterator MI,
+                                        Register DestReg, int FrameIdx,
+                                        const TargetRegisterClass *RC,
+                                        const TargetRegisterInfo *TRI) const {
+  // We need to avoid a situation in which the value from a VRRC register is
+  // spilled using an Altivec instruction and reloaded into a VSRC register
+  // using a VSX instruction. The issue with this is that the VSX
+  // load/store instructions swap the doublewords in the vector and the Altivec
+  // ones don't. The register classes on the spill/reload may be different if
+  // the register is defined using an Altivec instruction and is then used by a
+  // VSX instruction.
+  RC = updatedRC(RC);
+
+  loadRegFromStackSlotNoUpd(MBB, MI, DestReg, FrameIdx, RC, TRI);
+}
+
 bool PPCInstrInfo::
 reverseBranchCondition(SmallVectorImpl<MachineOperand> &Cond) const {
   assert(Cond.size() == 2 && "Invalid PPC branch opcode!");
   if (Cond[1].getReg() == PPC::CTR8 || Cond[1].getReg() == PPC::CTR)
     Cond[0].setImm(Cond[0].getImm() == 0 ? 1 : 0);
   else
     // Leave the CR# the same, but invert the condition.
     Cond[0].setImm(PPC::InvertPredicate((PPC::Predicate)Cond[0].getImm()));

llvm/test/CodeGen/PowerPC/CSR-fit.ll

 ; CHECK-PWR8-NEXT:    mflr r0
 ; CHECK-PWR8-NEXT:    std r0, 16(r1)
 ; CHECK-PWR8-NEXT:    stdu r1, -240(r1)
 ; CHECK-PWR8-NEXT:    .cfi_def_cfa_offset 240
 ; CHECK-PWR8-NEXT:    .cfi_offset lr, 16
 ; CHECK-PWR8-NEXT:    .cfi_offset v20, -192
 ; CHECK-PWR8-NEXT:    .cfi_offset v21, -176
 ; CHECK-PWR8-NEXT:    li r5, 48
-; CHECK-PWR8-NEXT:    stxvd2x v20, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v20, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    li r5, 64
-; CHECK-PWR8-NEXT:    stxvd2x v21, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v21, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    #APP
 ; CHECK-PWR8-NEXT:    add r3, r3, r4
 ; CHECK-PWR8-NEXT:    #NO_APP
 ; CHECK-PWR8-NEXT:    extsw r3, r3
 ; CHECK-PWR8-NEXT:    bl callee
 ; CHECK-PWR8-NEXT:    nop
 ; CHECK-PWR8-NEXT:    li r4, 64
-; CHECK-PWR8-NEXT:    lxvd2x v21, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v21, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    li r4, 48
-; CHECK-PWR8-NEXT:    lxvd2x v20, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v20, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    addi r1, r1, 240
 ; CHECK-PWR8-NEXT:    ld r0, 16(r1)
 ; CHECK-PWR8-NEXT:    mtlr r0
 ; CHECK-PWR8-NEXT:    blr
 ;
 ; CHECK-PWR9-LABEL: caller3:
 ; CHECK-PWR9:       # %bb.0: # %entry
 ; CHECK-PWR9-NEXT:    mflr r0

 ; CHECK-PWR8-NEXT:    mflr r0
 ; CHECK-PWR8-NEXT:    std r0, 16(r1)
 ; CHECK-PWR8-NEXT:    stdu r1, -240(r1)
 ; CHECK-PWR8-NEXT:    .cfi_def_cfa_offset 240
 ; CHECK-PWR8-NEXT:    .cfi_offset lr, 16
 ; CHECK-PWR8-NEXT:    .cfi_offset v20, -192
 ; CHECK-PWR8-NEXT:    .cfi_offset v21, -176
 ; CHECK-PWR8-NEXT:    li r5, 48
-; CHECK-PWR8-NEXT:    stxvd2x v20, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v20, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    li r5, 64
-; CHECK-PWR8-NEXT:    stxvd2x v21, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v21, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    #APP
 ; CHECK-PWR8-NEXT:    add r3, r3, r4
 ; CHECK-PWR8-NEXT:    #NO_APP
 ; CHECK-PWR8-NEXT:    extsw r3, r3
 ; CHECK-PWR8-NEXT:    bl callee
 ; CHECK-PWR8-NEXT:    nop
 ; CHECK-PWR8-NEXT:    li r4, 64
-; CHECK-PWR8-NEXT:    lxvd2x v21, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v21, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    li r4, 48
-; CHECK-PWR8-NEXT:    lxvd2x v20, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v20, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    addi r1, r1, 240
 ; CHECK-PWR8-NEXT:    ld r0, 16(r1)
 ; CHECK-PWR8-NEXT:    mtlr r0
 ; CHECK-PWR8-NEXT:    blr
 ;
 ; CHECK-PWR9-LABEL: caller4:
 ; CHECK-PWR9:       # %bb.0: # %entry
 ; CHECK-PWR9-NEXT:    mflr r0

 ; CHECK-PWR8-NEXT:    .cfi_offset lr, 16
 ; CHECK-PWR8-NEXT:    .cfi_offset r14, -288
 ; CHECK-PWR8-NEXT:    .cfi_offset f14, -144
 ; CHECK-PWR8-NEXT:    .cfi_offset v20, -480
 ; CHECK-PWR8-NEXT:    .cfi_offset v21, -464
 ; CHECK-PWR8-NEXT:    li r5, 48
 ; CHECK-PWR8-NEXT:    std r14, 240(r1) # 8-byte Folded Spill
 ; CHECK-PWR8-NEXT:    stfd f14, 384(r1) # 8-byte Folded Spill
-; CHECK-PWR8-NEXT:    stxvd2x v20, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v20, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    li r5, 64
-; CHECK-PWR8-NEXT:    stxvd2x v21, r1, r5 # 16-byte Folded Spill
+; CHECK-PWR8-NEXT:    stvx v21, r1, r5 # 16-byte Folded Spill
 ; CHECK-PWR8-NEXT:    #APP
 ; CHECK-PWR8-NEXT:    add r3, r3, r4
 ; CHECK-PWR8-NEXT:    #NO_APP
 ; CHECK-PWR8-NEXT:    extsw r3, r3
 ; CHECK-PWR8-NEXT:    bl callee
 ; CHECK-PWR8-NEXT:    nop
 ; CHECK-PWR8-NEXT:    li r4, 64
 ; CHECK-PWR8-NEXT:    lfd f14, 384(r1) # 8-byte Folded Reload
 ; CHECK-PWR8-NEXT:    ld r14, 240(r1) # 8-byte Folded Reload
-; CHECK-PWR8-NEXT:    lxvd2x v21, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v21, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    li r4, 48
-; CHECK-PWR8-NEXT:    lxvd2x v20, r1, r4 # 16-byte Folded Reload
+; CHECK-PWR8-NEXT:    lvx v20, r1, r4 # 16-byte Folded Reload
 ; CHECK-PWR8-NEXT:    addi r1, r1, 528
 ; CHECK-PWR8-NEXT:    ld r0, 16(r1)
 ; CHECK-PWR8-NEXT:    mtlr r0
 ; CHECK-PWR8-NEXT:    blr
 ;
 ; CHECK-PWR9-LABEL: caller_mixed:
 ; CHECK-PWR9:       # %bb.0: # %entry
 ; CHECK-PWR9-NEXT:    mflr r0

llvm/test/CodeGen/PowerPC/reg-scavenging.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mcpu=pwr8 -mtriple=powerpc64le-unknown-unknown \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names < %s | FileCheck %s
 
 define dso_local signext i32 @caller(i32 signext %a, i32 signext %b) local_unnamed_addr {
 ; CHECK-LABEL: caller:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    mflr r0
 ; CHECK-NEXT:    std r0, 16(r1)
 ; CHECK-NEXT:    stdu r1, -240(r1)
 ; CHECK-NEXT:    .cfi_def_cfa_offset 240
 ; CHECK-NEXT:    .cfi_offset lr, 16
 ; CHECK-NEXT:    .cfi_offset v20, -192
 ; CHECK-NEXT:    li r5, 48
-; CHECK-NEXT:    stxvd2x v20, r1, r5 # 16-byte Folded Spill
+; CHECK-NEXT:    stvx v20, r1, r5 # 16-byte Folded Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    add r3, r3, r4
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    extsw r3, r3
 ; CHECK-NEXT:    bl callee
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    li r4, 48
-; CHECK-NEXT:    lxvd2x v20, r1, r4 # 16-byte Folded Reload
+; CHECK-NEXT:    lvx v20, r1, r4 # 16-byte Folded Reload
 ; CHECK-NEXT:    addi r1, r1, 240
 ; CHECK-NEXT:    ld r0, 16(r1)
 ; CHECK-NEXT:    mtlr r0
 ; CHECK-NEXT:    blr
 entry:
   %0 = tail call i32 asm "add $0, $1, $2", "=r,r,r,~{v20}"(i32 %a, i32 %b)
   %call = tail call signext i32 @callee(i32 signext %0)
   ret i32 %call
 }
 
 declare signext i32 @callee(i32 signext) local_unnamed_addr
 

llvm/test/CodeGen/PowerPC/vsxD-Form-spills.ll

 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr9 < %s | FileCheck -check-prefix=CHECK-PWR9 %s
 
 define <4 x i32> @testSpill(<4 x i32> %a, <4 x i32> %b) {
 
 ; CHECK-LABEL: testSpill:
 ; CHECK-DAG:    li [[REG48:[0-9]+]], 48
 ; CHECK-DAG:    li [[REG64:[0-9]+]], 64
 ; CHECK-DAG:    li [[REG80:[0-9]+]], 80
 ; CHECK-DAG:    li [[REG96:[0-9]+]], 96
-; CHECK-DAG:    stxvd2x 60, 1, [[REG48]] # 16-byte Folded Spill
-; CHECK-DAG:    stxvd2x 61, 1, [[REG64]] # 16-byte Folded Spill
-; CHECK-DAG:    stxvd2x 62, 1, [[REG80]] # 16-byte Folded Spill
-; CHECK-DAG:    stxvd2x 63, 1, [[REG96]] # 16-byte Folded Spill
+; CHECK-DAG:    stvx 28, 1, [[REG48]] # 16-byte Folded Spill
+; CHECK-DAG:    stvx 29, 1, [[REG64]] # 16-byte Folded Spill
+; CHECK-DAG:    stvx 30, 1, [[REG80]] # 16-byte Folded Spill
+; CHECK-DAG:    stvx 31, 1, [[REG96]] # 16-byte Folded Spill
 ; CHECK:        .LBB0_3
 ; CHECK-DAG:    li [[REG96_LD:[0-9]+]], 96
 ; CHECK-DAG:    li [[REG80_LD:[0-9]+]], 80
 ; CHECK-DAG:    li [[REG64_LD:[0-9]+]], 64
 ; CHECK-DAG:    li [[REG48_LD:[0-9]+]], 48
-; CHECK-DAG:    lxvd2x 63, 1, [[REG96_LD]] # 16-byte Folded Reload
-; CHECK-DAG:    lxvd2x 62, 1, [[REG80_LD]] # 16-byte Folded Reload
-; CHECK-DAG:    lxvd2x 61, 1, [[REG64_LD]] # 16-byte Folded Reload
-; CHECK-DAG:    lxvd2x 60, 1, [[REG48_LD]] # 16-byte Folded Reload
+; CHECK-DAG:    lvx 31, 1, [[REG96_LD]] # 16-byte Folded Reload
+; CHECK-DAG:    lvx 30, 1, [[REG80_LD]] # 16-byte Folded Reload
+; CHECK-DAG:    lvx 29, 1, [[REG64_LD]] # 16-byte Folded Reload
+; CHECK-DAG:    lvx 28, 1, [[REG48_LD]] # 16-byte Folded Reload
 ; CHECK:    mtlr 0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR9-LABEL: testSpill:
 ; CHECK-PWR9-DAG:    stxv 62, 64(1) # 16-byte Folded Spill
 ; CHECK-PWR9-DAG:    stxv 63, 80(1) # 16-byte Folded Spill
 ; CHECK-PWR9-DAG:    stxv 60, 32(1) # 16-byte Folded Spill
 ; CHECK-PWR9-DAG:    stxv 61, 48(1) # 16-byte Folded Spill