[PowerPC] Try to move the stack pointer update instruction later in the prologue and earlier in the epilogue

[stefanp]

Summary

This should provide a small performance boost in functions that are able to do this.
I've kept this change rather conservative at the moment and functions with frame pointers or base pointers will not try to move the stack pointer update.

Performance numbers on Power 8 for SPEC 2006
429.mcf - 1.5 % improvement
462.libquantum - 1.0 % improvement
464.h264ref - 3.0 % improvement
471.omnetpp - 1.0 % improvement
453.povray - 1.5 % improvement

All other benchmarks changed less than 0.2 % which I consider as "no change".
I have found no degradations in SPEC2006.

[sfertile]

lib/Target/PowerPC/PPCFrameLowering.cpp

827

wil -> will

832

Can we use !isLargeFrame here instead of isInt<16>(FrameSize) ?

[stefanp]

lib/Target/PowerPC/PPCFrameLowering.cpp

832

No, they are not actually the same thing unfortunately. In the prologue we use this definition:
bool isLargeFrame = !isInt<16>(NegFrameSize);
In the epilogue we use this definition.
bool isLargeFrame = !isInt<16>(FrameSize);
But I need the two conditions to be identical in the two guards that I've set up. So, unfortunately, I can't use the isLargeFrame in the prologue.

[stefanp]

Fixed a typo and a missing set of brackets.

[echristo]

Just to verify, this is effectively hand scheduling the prologue and epilogue?

-eric

[stefanp]

Yes, we are hand scheduling the prologue and epilogue.
However, this is a little more than moving instructions around. Since we are moving the stack pointer update instruction, spills that use the stack pointer may have to be updated with new offsets.

[echristo]

One inline comment and nit, but then I'm ok with this. Should elaborate about why we're doing this in the commit message.

lib/Target/PowerPC/PPCFrameLowering.cpp

1216

No braces here. Also comment.

[nemanjai]

These comments are not an indication that another review is necessary, but please address them on the commit.

lib/Target/PowerPC/PPCFrameLowering.cpp

832

It would probably be good to put in a quick comment explaining that so someone doesn't change it in the future.

833

Nit: no spaces in the template parameter list when there's a single one. No space after the ampersand either. Also, fix up your naming convention here and elsewhere - variables start with a capital.

1365

I think more importantly - it'll allow the CSR restores to hide the latency of the move-from-LR as they're between the mflr and blr.

[hfinkel]

lib/Target/PowerPC/PPCFrameLowering.cpp

830

movingStackUpdateDown -> MovingStackUpdateDown

831

Please explain in the comment above why these particular things are the validity condition for this optimization.

1367

movingStackUpdateUp -> MovingStackUpdateUp

lib/Target/PowerPC/PPCFrameLowering.cpp

void PPCFrameLowering::emitPrologue(MachineFunction &MF,
 
   // Frames of 32KB & larger require special handling because they cannot be
   // indexed into with a simple STDU/STWU/STD/STW immediate offset operand.
   bool isLargeFrame = !isInt<16>(NegFrameSize);
 
   assert((isPPC64 || !MustSaveCR) &&
          "Prologue CR saving supported only in 64-bit mode");
 
+  // Check if we can move the stack update instruction (stdu) down the prologue
+  //  past the callee saves. Hopefully this wil avoid the situation where the
+  //  saves are waiting for the update on the store with update to complete.
+  MachineBasicBlock::iterator StackUpdateLoc = MBBI;
+  bool movingStackUpdateDown = false;
+  if (FrameSize && !FI->hasFastCall() && !FI->usesPICBase() && !HasFP &&
+      !HasBP && isInt<16>(FrameSize)) {
+    const std::vector< CalleeSavedInfo > & info = MFI.getCalleeSavedInfo();
+    for (int i=0; i<info.size(); i++) {
+      int fi = info[i].getFrameIdx();
+      if (fi < 0) {
+        if (MFI.isFixedObjectIndex(fi) && MFI.getObjectOffset(fi) < 0)
+          MFI.setObjectOffset(fi, MFI.getObjectOffset(fi) + NegFrameSize);
+          StackUpdateLoc++;
+          movingStackUpdateDown = true;
+      }
+    }
+  }
+
   // If we need to spill the CR and the LR but we don't have two separate
   // registers available, we must spill them one at a time
   if (MustSaveCR && SingleScratchReg && MustSaveLR) {
     // In the ELFv2 ABI, we are not required to save all CR fields.
     // If only one or two CR fields are clobbered, it is more efficient to use
     // mfocrf to selectively save just those fields, because mfocrf has short
     // latency compares to mfcr.
     unsigned MfcrOpcode = PPC::MFCR8;
if (HasRedZone) {
     if (HasBP)
       BuildMI(MBB, MBBI, dl, StoreInst)
         .addReg(BPReg)
         .addImm(BPOffset)
         .addReg(SPReg);
   }
 
   if (MustSaveLR)
-    BuildMI(MBB, MBBI, dl, StoreInst)
+    BuildMI(MBB, StackUpdateLoc, dl, StoreInst)
       .addReg(ScratchReg, getKillRegState(true))
       .addImm(LROffset)
       .addReg(SPReg);
 
   if (MustSaveCR &&
       !(SingleScratchReg && MustSaveLR)) { // will only occur for PPC64
     assert(HasRedZone && "A red zone is always available on PPC64");
     BuildMI(MBB, MBBI, dl, TII.get(PPC::STW8))
if (HasBP && MaxAlign > 1) {
 
     BuildMI(MBB, MBBI, dl, StoreUpdtIdxInst, SPReg)
       .addReg(SPReg, RegState::Kill)
       .addReg(SPReg)
       .addReg(ScratchReg);
     HasSTUX = true;
 
   } else if (!isLargeFrame) {
-    BuildMI(MBB, MBBI, dl, StoreUpdtInst, SPReg)
+    BuildMI(MBB, StackUpdateLoc, dl, StoreUpdtInst, SPReg)
       .addReg(SPReg)
       .addImm(NegFrameSize)
       .addReg(SPReg);
 
   } else {
     BuildMI(MBB, MBBI, dl, LoadImmShiftedInst, ScratchReg)
       .addImm(NegFrameSize >> 16);
     BuildMI(MBB, MBBI, dl, OrImmInst, ScratchReg)
for (unsigned I = 0, E = CSI.size(); I != E; ++I) {
         unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::createOffset(
             nullptr, MRI->getDwarfRegNum(CRReg, true), 8));
         BuildMI(MBB, MBBI, dl, TII.get(TargetOpcode::CFI_INSTRUCTION))
             .addCFIIndex(CFIIndex);
         continue;
       }
 
       int Offset = MFI.getObjectOffset(CSI[I].getFrameIdx());
+      if (movingStackUpdateDown) {
+        Offset -= NegFrameSize;
+      }
       unsigned CFIIndex = MF.addFrameInst(MCCFIInstruction::createOffset(
           nullptr, MRI->getDwarfRegNum(Reg, true), Offset));
       BuildMI(MBB, MBBI, dl, TII.get(TargetOpcode::CFI_INSTRUCTION))
           .addCFIIndex(CFIIndex);
     }
   }
 }
 
void PPCFrameLowering::emitEpilogue(MachineFunction &MF,
   // them. In such case, the final update of SP will be to add the frame
   // size to it.
   // To simplify the code, set RBReg to the base register used to restore
   // values from the stack, and set SPAdd to the value that needs to be added
   // to the SP at the end. The default values are as if red zone was present.
   unsigned RBReg = SPReg;
   unsigned SPAdd = 0;
 
+  // Check if we can move the stack update instruction up the epilogue
+  //  past the callee saves. This will allow restoring the lr sooner.
+  MachineBasicBlock::iterator StackUpdateLoc = MBBI;
+  bool movingStackUpdateUp = false;
+  if (FrameSize && !FI->hasFastCall() && !FI->usesPICBase() && !HasFP &&
+      !HasBP && !isLargeFrame) {
+    const std::vector< CalleeSavedInfo > & info = MFI.getCalleeSavedInfo();
+    for (int i=0; i<info.size(); i++) {
+      int fi = info[i].getFrameIdx();
+      if (fi < 0) {
+        if (MFI.isFixedObjectIndex(fi) && MFI.getObjectOffset(fi) < 0) {
+          StackUpdateLoc--;
+          movingStackUpdateUp = true;
+        }
+      }
+    }
+  }
+
   if (FrameSize) {
     // In the prologue, the loaded (or persistent) stack pointer value is
     // offset by the STDU/STDUX/STWU/STWUX instruction. For targets with red
     // zone add this offset back now.
 
     // If this function contained a fastcc call and GuaranteedTailCallOpt is
     // enabled (=> hasFastCall()==true) the fastcc call might contain a tail
     // call which invalidates the stack pointer value in SP(0). So we use the
if (FI->hasFastCall()) {
           .addImm(FrameSize & 0xFFFF);
         BuildMI(MBB, MBBI, dl, AddInst)
           .addReg(RBReg)
           .addReg(FPReg)
           .addReg(ScratchReg);
       }
     } else if (!isLargeFrame && !HasBP && !MFI.hasVarSizedObjects()) {
       if (HasRedZone) {
-        BuildMI(MBB, MBBI, dl, AddImmInst, SPReg)
+        BuildMI(MBB, StackUpdateLoc, dl, AddImmInst, SPReg)
           .addReg(SPReg)
           .addImm(FrameSize);
       } else {
         // Make sure that adding FrameSize will not overflow the max offset
         // size.
         assert(FPOffset <= 0 && BPOffset <= 0 && PBPOffset <= 0 &&
                "Local offsets should be negative");
         SPAdd = FrameSize;
         FPOffset += FrameSize;
         BPOffset += FrameSize;
         PBPOffset += FrameSize;
       }
     } else {
       // We don't want to use ScratchReg as a base register, because it
       // could happen to be R0. Use FP instead, but make sure to preserve it.
       if (!HasRedZone) {
         // If FP is not saved, copy it to ScratchReg.
         if (!HasFP)
           BuildMI(MBB, MBBI, dl, OrInst, ScratchReg)
             .addReg(FPReg)
             .addReg(FPReg);
         RBReg = FPReg;
       }
-      BuildMI(MBB, MBBI, dl, LoadInst, RBReg)
+      BuildMI(MBB, StackUpdateLoc, dl, LoadInst, RBReg)
         .addImm(0)
         .addReg(SPReg);
     }
   }
   assert(RBReg != ScratchReg && "Should have avoided ScratchReg");
   // If there is no red zone, ScratchReg may be needed for holding a useful
   // value (although not the base register). Make sure it is not overwritten
   // too early.
void PPCFrameLowering::emitEpilogue(MachineFunction &MF,
   }
 
   // Delay restoring of the LR if ScratchReg is needed. This is ok, since
   // LR is stored in the caller's stack frame. ScratchReg will be needed
   // if RBReg is anything other than SP. We shouldn't use ScratchReg as
   // a base register anyway, because it may happen to be R0.
   bool LoadedLR = false;
   if (MustSaveLR && RBReg == SPReg && isInt<16>(LROffset+SPAdd)) {
-    BuildMI(MBB, MBBI, dl, LoadInst, ScratchReg)
+    BuildMI(MBB, StackUpdateLoc, dl, LoadInst, ScratchReg)
       .addImm(LROffset+SPAdd)
       .addReg(RBReg);
     LoadedLR = true;
   }
 
   if (MustSaveCR && !(SingleScratchReg && MustSaveLR)) {
     // This will only occur for PPC64.
     assert(isPPC64 && "Expecting 64-bit mode");
void PPCFrameLowering::emitEpilogue(MachineFunction &MF,
 
   if (MustSaveCR &&
       !(SingleScratchReg && MustSaveLR)) // will only occur for PPC64
     for (unsigned i = 0, e = MustSaveCRs.size(); i != e; ++i)
       BuildMI(MBB, MBBI, dl, TII.get(PPC::MTOCRF8), MustSaveCRs[i])
         .addReg(TempReg, getKillRegState(i == e-1));
 
   if (MustSaveLR)
-    BuildMI(MBB, MBBI, dl, MTLRInst).addReg(ScratchReg);
+    BuildMI(MBB, StackUpdateLoc, dl, MTLRInst).addReg(ScratchReg);
 
   // Callee pop calling convention. Pop parameter/linkage area. Used for tail
   // call optimization
   if (IsReturnBlock) {
     unsigned RetOpcode = MBBI->getOpcode();
     if (MF.getTarget().Options.GuaranteedTailCallOpt &&
         (RetOpcode == PPC::BLR || RetOpcode == PPC::BLR8) &&
         MF.getFunction().getCallingConv() == CallingConv::Fast) {

test/CodeGen/PowerPC/MCSE-caller-preserved-reg.ll

 %class.CC = type { %struct.SS }
 %struct.SS = type { void ()* }
 
 @_ZN2CC2ccE = external thread_local global %"struct.CC::TT", align 8
 
 define noalias i8* @_ZN2CC3funEv(%class.CC* %this) {
 ; CHECK-LABEL: _ZN2CC3funEv:
 ; CHECK:    mflr 0
-; CHECK-NEXT:    std 0, 16(1)
-; CHECK-NEXT:    stdu 1, -48(1)
 ; CHECK-NEXT:    .cfi_def_cfa_offset 48
 ; CHECK-NEXT:    .cfi_offset lr, 16
 ; CHECK-NEXT:    .cfi_offset r30, -16
-; CHECK-NEXT:    std 30, 32(1)
+; CHECK-NEXT:    std 30, -16(1)
+; CHECK-NEXT:    std 0, 16(1)
+; CHECK-NEXT:    stdu 1, -48(1)
 ; CHECK-NEXT:    mr 30, 3
 ; CHECK-NEXT:    ld 12, 0(30)
 ; CHECK-NEXT:    std 2, 24(1)
 ; CHECK-NEXT:    mtctr 12
 ; CHECK-NEXT:    bctrl
 ; CHECK-NEXT:    ld 2, 24(1)
 ; CHECK-NEXT:    addis 3, 2, _ZN2CC2ccE@got@tlsgd@ha
 ; CHECK-NEXT:    addi 3, 3, _ZN2CC2ccE@got@tlsgd@l
 ; CHECK-NEXT:    bl __tls_get_addr(_ZN2CC2ccE@tlsgd)
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    ld 4, 0(3)
 ; CHECK-NEXT:    cmpldi 4, 0
 ; CHECK-NEXT:    beq 0, .LBB0_2
 ; CHECK:    addi 4, 3, 8
 ; CHECK-NEXT:    mr 3, 30
 ; CHECK-NEXT:    bl _ZN2CC3barEPi
 ; CHECK-NEXT:    nop
-; CHECK:    ld 30, 32(1)
-; CHECK-NEXT:    li 3, 0
+; CHECK:    li 3, 0
 ; CHECK-NEXT:    addi 1, 1, 48
 ; CHECK-NEXT:    ld 0, 16(1)
 ; CHECK-NEXT:    mtlr 0
+; CHECK:    ld 30, -16(1)
 ; CHECK-NEXT:    blr
 entry:
   %foo = getelementptr inbounds %class.CC, %class.CC* %this, i64 0, i32 0, i32 0
   %0 = load void ()*, void ()** %foo, align 8
   tail call void %0()
   %1 = load i64, i64* getelementptr inbounds (%"struct.CC::TT", %"struct.CC::TT"* @_ZN2CC2ccE, i64 0, i32 0)
   %tobool = icmp eq i64 %1, 0
   br i1 %tobool, label %if.end, label %if.then

test/CodeGen/PowerPC/ppc-redzone-alignment-bug.ll

 ; stdu 1, -544(1)
 ; std 15, 264(1)
 ; stfd 14, 400(1)
 ; stdu 1, -560(1)
 ; std 15, 280(1)
 ; stfd 14, 416(1)
 
 ; After the fix by patch D34337:
+; CHECK-LE:std 15, -280(1)
+; CHECK-LE:stfd 14, -144(1)
 ; CHECK-LE: stdu 1, -528(1)
-; CHECK-LE:std 15, 248(1)
-; CHECK-LE:stfd 14, 384(1)
+; CHECK-BE:std 15, -280(1)
+; CHECK-BE:stfd 14, -144(1)
 ; CHECK-BE: stdu 1, -544(1)
-; CHECK-BE:std 15, 264(1)
-; CHECK-BE:stfd 14, 400(1)
 }
 
 define signext i32 @foo() {
 entry:
   %call = tail call signext i32 @bar(i32 signext 5)
   ret i32 %call
 }
 

test/CodeGen/PowerPC/ppc-shrink-wrapping.ll

 ;
 ; DISABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; DISABLE: slwi 3, 4, 1
 ; DISABLE: .[[EPILOG_BB]]: # %if.end
 ;
 ; Epilogue code.
 ; CHECK: mtlr {{[0-9]+}}
-; CHECK-NEXT: blr
+; CHECK: blr
 ;
 ; ENABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; ENABLE: slwi 3, 4, 1
 ; ENABLE-NEXT: blr
 define i32 @freqSaveAndRestoreOutsideLoop(i32 %cond, i32 %N) {
 entry:
   %tobool = icmp eq i32 %cond, 0

 ; CHECK-DAG: addi [[IV]], [[IV]], -1
 ; CHECK-DAG: add [[SUM]], 3, [[SUM]] 
 ; CHECK-NEXT: cmplwi [[IV]], 0
 ; CHECK-NEXT: bne 0, .[[LOOP]]
 ;
 ; Next BB
 ; CHECK: %for.end
 ; CHECK: mtlr {{[0-9]+}}
-; CHECK-NEXT: blr
+; CHECK: blr
 define i32 @freqSaveAndRestoreOutsideLoop2(i32 %cond) {
 entry:
   br label %for.preheader
 
 for.preheader:
   tail call void asm "nop", ""()
   br label %for.body
 

 ;
 ; ENABLE: cmplwi 0, 3, 0
 ; ENABLE-NEXT: beq 0, .[[ELSE_LABEL:LBB[0-9_]+]]
 ;
 ; Prologue code.
 ; Make sure we save the link register 
 ; CHECK: mflr {{[0-9]+}}
 ;
-; DISABLE: cmplwi 0, 3, 0
-; DISABLE-NEXT: std
+; DISABLE: std
 ; DISABLE-NEXT: std
+; DISABLE: cmplwi 0, 3, 0
 ; DISABLE-NEXT: beq 0, .[[ELSE_LABEL:LBB[0-9_]+]]
 ;
 ; Loop preheader
 ; CHECK-DAG: li [[SUM:[0-9]+]], 0
 ; CHECK-DAG: li [[IV:[0-9]+]], 10
 ; 
 ; Loop body
 ; CHECK: .[[LOOP:LBB[0-9_]+]]: # %for.body

 ;
 ; DISABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; DISABLE: slwi 3, 4, 1
 ;
 ; DISABLE: .[[EPILOG_BB]]: # %if.end
 ; Epilog code
 ; CHECK: mtlr {{[0-9]+}}
-; CHECK-NEXT: blr
+; CHECK: blr
 ; 
 ; ENABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; ENABLE: slwi 3, 4, 1
 ; ENABLE-NEXT: blr
 define i32 @loopInfoSaveOutsideLoop(i32 %cond, i32 %N) {
 entry:
   %tobool = icmp eq i32 %cond, 0

 ;
 ; ENABLE: cmplwi 0, 3, 0
 ; ENABLE-NEXT: beq 0, .[[ELSE_LABEL:LBB[0-9_]+]]
 ;
 ; Prologue code.
 ; Make sure we save the link register
 ; CHECK: mflr {{[0-9]+}}
 ;
-; DISABLE: cmplwi 0, 3, 0
-; DISABLE-NEXT: std
+; DISABLE: std
 ; DISABLE-NEXT: std
+; DISABLE: cmplwi 0, 3, 0
 ; DISABLE-NEXT: beq 0, .[[ELSE_LABEL:LBB[0-9_]+]]
 ;
 ; CHECK: bl somethingElse
 ;
 ; Loop preheader
 ; CHECK-DAG: li [[SUM:[0-9]+]], 0
 ; CHECK-DAG: li [[IV:[0-9]+]], 10
 ; 

 ;
 ; DISABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; DISABLE: slwi 3, 4, 1
 ; DISABLE: .[[EPILOG_BB]]: # %if.end
 ;
 ; Epilogue code.
 ; CHECK: mtlr {{[0-9]+}}
-; CHECK-NEXT: blr
+; CHECK: blr
 ;
 ; ENABLE: .[[ELSE_LABEL]]: # %if.else
 ; Shift second argument by one and store into returned register.
 ; ENABLE: slwi 3, 4, 1
 ; ENABLE-NEXT: blr
 define i32 @loopInfoRestoreOutsideLoop(i32 %cond, i32 %N) #0 {
 entry:
   %tobool = icmp eq i32 %cond, 0

test/CodeGen/PowerPC/tls_get_addr_clobbers.ll

 ; RUN: llc -verify-machineinstrs -mtriple="powerpc64le-unknown-linux-gnu" -relocation-model=pic < %s | FileCheck %s
 
 @a = thread_local global i32* null, align 8
 
 define void @test_foo(i32* nocapture %x01, i32* nocapture %x02, i32* nocapture %x03, i32* nocapture %x04, i32* nocapture %x05, i32* nocapture %x06, i32* nocapture %x07, i32* nocapture %x08) #0 {
 entry:
 
 ; CHECK-LABEL: test_foo:
-; CHECK: stdu 1, {{-?[0-9]+}}(1)
+; CHECK-DAG: stdu 1, {{-?[0-9]+}}(1)
 ; CHECK-DAG: mr [[BACKUP_3:[0-9]+]], 3
 ; CHECK-DAG: mr [[BACKUP_4:[0-9]+]], 4
 ; CHECK-DAG: mr [[BACKUP_5:[0-9]+]], 5
 ; CHECK-DAG: mr [[BACKUP_6:[0-9]+]], 6
 ; CHECK-DAG: mr [[BACKUP_7:[0-9]+]], 7
 ; CHECK-DAG: mr [[BACKUP_8:[0-9]+]], 8
 ; CHECK-DAG: mr [[BACKUP_9:[0-9]+]], 9
 ; CHECK-DAG: mr [[BACKUP_10:[0-9]+]], 10
-; CHECK-DAG: std [[BACKUP_3]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_4]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_5]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_6]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_7]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_8]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_9]], {{[0-9]+}}(1)
-; CHECK-DAG: std [[BACKUP_10]], {{[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_3]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_4]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_5]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_6]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_7]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_8]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_9]], {{-?[0-9]+}}(1)
+; CHECK-DAG: std [[BACKUP_10]], {{-?[0-9]+}}(1)
 ; CHECK: bl __tls_get_addr
 ; CHECK-DAG: stw 3, 0([[BACKUP_3]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_4]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_5]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_6]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_7]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_8]])
 ; CHECK-DAG: stw 3, 0([[BACKUP_9]])

test/CodeGen/PowerPC/vsxD-Form-spills.ll

 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr9 < %s | FileCheck -check-prefix=CHECK-PWR9 %s
 
 define <4 x i32> @testSpill(<4 x i32> %a, <4 x i32> %b) {
 
 ; CHECK-LABEL: testSpill:
-; CHECK:    li 11, 80
-; CHECK:    li 12, 96
-; CHECK:    li 3, 48
-; CHECK:    li 10, 64
-; CHECK:    stxvd2x 62, 1, 11 # 16-byte Folded Spill
-; CHECK:    stxvd2x 63, 1, 12 # 16-byte Folded Spill
-; CHECK:    stxvd2x 60, 1, 3 # 16-byte Folded Spill
-; CHECK:    stxvd2x 61, 1, 10 # 16-byte Folded Spill
-; CHECK:    li 9, 96
-; CHECK:    li 10, 80
-; CHECK:    li 11, 64
-; CHECK:    li 12, 48
-; CHECK:    lxvd2x 63, 1, 9 # 16-byte Folded Reload
-; CHECK:    lxvd2x 62, 1, 10 # 16-byte Folded Reload
-; CHECK:    lxvd2x 61, 1, 11 # 16-byte Folded Reload
-; CHECK:    lxvd2x 60, 1, 12 # 16-byte Folded Reload
+; CHECK-DAG:    li [[REG64:[0-9]+]], -64
+; CHECK-DAG:    li [[REG48:[0-9]+]], -48
+; CHECK-DAG:    li [[REG32:[0-9]+]], -32
+; CHECK-DAG:    li [[REG16:[0-9]+]], -16
+; CHECK-NOT:    li
+; CHECK-DAG:    stxvd2x 60, 1, [[REG64]] # 16-byte Folded Spill
+; CHECK-DAG:    stxvd2x 61, 1, [[REG48]] # 16-byte Folded Spill
+; CHECK-DAG:    stxvd2x 62, 1, [[REG32]] # 16-byte Folded Spill
+; CHECK-DAG:    stxvd2x 63, 1, [[REG16]] # 16-byte Folded Spill
+; CHECK:    std 0, 16(1)
+; CHECK-DAG:    li [[REG16:[0-9]+]], -16
+; CHECK-DAG:    li [[REG32:[0-9]+]], -32
+; CHECK-DAG:    li [[REG48:[0-9]+]], -48
+; CHECK-DAG:    li [[REG64:[0-9]+]], -64
 ; CHECK:    mtlr 0
+; CHECK-DAG:    lxvd2x 63, 1, [[REG16]] # 16-byte Folded Reload
+; CHECK-DAG:    lxvd2x 62, 1, [[REG32]] # 16-byte Folded Reload
+; CHECK-DAG:    lxvd2x 61, 1, [[REG48]] # 16-byte Folded Reload
+; CHECK-DAG:    lxvd2x 60, 1, [[REG64]] # 16-byte Folded Reload
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-PWR9-LABEL: testSpill:
-; CHECK-PWR9:    stxv 62, 80(1) # 16-byte Folded Spill
-; CHECK-PWR9:    stxv 63, 96(1) # 16-byte Folded Spill
-; CHECK-PWR9:    stxv 60, 48(1) # 16-byte Folded Spill
-; CHECK-PWR9:    stxv 61, 64(1) # 16-byte Folded Spill
-; CHECK-PWR9:    lxv 63, 96(1) # 16-byte Folded Reload
-; CHECK-PWR9:    lxv 62, 80(1) # 16-byte Folded Reload
-; CHECK-PWR9:    lxv 61, 64(1) # 16-byte Folded Reload
-; CHECK-PWR9:    lxv 60, 48(1) # 16-byte Folded Reload
+; CHECK-PWR9-DAG:    stxv 60, -64(1) # 16-byte Folded Spill
+; CHECK-PWR9-DAG:    stxv 61, -48(1) # 16-byte Folded Spill
+; CHECK-PWR9-DAG:    stxv 62, -32(1) # 16-byte Folded Spill
+; CHECK-PWR9-DAG:    stxv 63, -16(1) # 16-byte Folded Spill
 ; CHECK-PWR9:    mtlr 0
+; CHECK-PWR9-DAG:    lxv 63, -16(1) # 16-byte Folded Reload
+; CHECK-PWR9-DAG:    lxv 62, -32(1) # 16-byte Folded Reload
+; CHECK-PWR9-DAG:    lxv 61, -48(1) # 16-byte Folded Reload
+; CHECK-PWR9-DAG:    lxv 60, -64(1) # 16-byte Folded Reload
 ; CHECK-PWR9-NEXT:    blr
 
 entry:
   %0 = tail call i32 @llvm.ppc.altivec.vcmpgtsw.p(i32 2, <4 x i32> %a, <4 x i32> %b)
   %tobool = icmp eq i32 %0, 0
   br i1 %tobool, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry